A/B/A Testing of Audio Equipment

[DTB300]

While out surfing the net, I ran across a review from "6 Moons" and as part of this article the reviewer stated:

"A quick word on A/B/As. If done too rapidly and for too short a duration per interval, you may hear no difference at all. That's because the list of differentiators is far longer than can be ascertained in 15-second sound bytes. There's frequency response, harmonic distortion, noise floor, timing, dynamics, resolution and spatial soundstaging cues. If differences were restricted to just one of these areas, it'll take you a while to sort through your list and cross off individual attributes until you hit upon the one (or two or three) that actually differ and know where to look for them. "

It was interesting to read. While trying to determine all the points he listed does take time (frequency resp, noise floor, etc.), I have always thought that the longer you wait between tests, the more your brain "forgets" what it just heard. For instance, if you are testing out Power Amps, and you have to shut it down, re-cable, and power up, there tends to be a great amount of time between these switch-outs. If there are small changes between the amps, they wil then be harder to distinguish. Now if there are GREAT changes, these "should" be heard. Maybe? Possibly?

What are some of your thoughts on this type of testing?

Dan

 

[DavidG]

Live with it

I've always found it best to audition at home, where you can listen to something for a few days, rather than hours or even minutes.

It's only by using something with many different records and at varying volumes that you start to get a true picture of what any given item is really like. For speaker cables, I took over 2 weeks with just 4 sets - but at the end I had a really good picture of the properties and sounds of each different set.

Of course, you need an understanding and well equipped dealer to run a session like this!!

Cheers,

David

 

[garmtz]

I also don't believe in quick A/B tests. Having the equipment at home and 'living' with it for some days wil give you a much better picture of the pros and cons and will make you hear more the more you listen.

 

[DTB300]

I also don't believe in quick A/B tests. Having the equipment at home and 'living' with it for some days wil give you a much better picture of the pros and cons and will make you hear more the more you listen.

I would agree that "long term" listening will really give you an idea of how something sounds - especially when you bring it home to audition as you state. You get the chance to listen to many different types of music this way.

I have heard of other testing, where they have a button in hand, which allows for quick switching between components - but they never stated the listening time period - but the button makes me think it is quicker rather than longer.

Dan

 

[aliveatfive]

IMHO you really have to "live" with a component to understand what it does in your system.

 

[Robin]

I also don't believe in quick A/B tests. Having the equipment at home and 'living' with it for some days wil give you a much better picture of the pros and cons and will make you hear more the more you listen.

While, I understand that, "In-Home", A/B/A testing is idea. It is my view, that it is not always possible in the real world... Let me explain, as you guys know, my home, is in Calistoga, California, which is at least, two hours from the nearest, high-end audio equipment shops, (other than big box stores like, Magnolia HI-Fi). It has been my experience, that living in, an out-of-the-way, small town (like Calistoga), I have been forced to just audition and listen to (A/B/A test) equipment / cables, "In-Store" exclusively. I have had to rely and use my imagination, as to what, a given piece of audio equipment, might sound like on my system, in my home. It is the trade off, I have choosen to make, to live in the small town, verses living in or near a big city.

After reading all of the wonderful, in-home, audition recommendations, which has been written on this forum... I am, now, going to request, of these high-end stores (in San Francisco), to adution their expensive amplifier and / or cable gear, they are selling, in my home.
In piont of fact, though, I really do not expect, these stores to grant my requests or accomidate me. The reality is: #1). They don't know me, from Adam. #2). I live far away. #3). It puts their store at financial risk.
Bottom line, I do not expect they will agree to let me audition (A/B/A test) their expensive audio equipment in my home. It's just too risky to the retail store.

I do agree, with your basic premise, that the best way to audition or A/B/A test, audio equipment, especially before a possible purchase, is to take it home and try it out, first. It is just simply, the best case scenario... I hope, I am wrong, but I will be shocked and awed, if any high-end audio store in the city of San Francisco, accually agrees, to let me take home, to Calistoga, a $4000 - $5000 USD, amplifier or thousands of USD's of audio cable.... just so that I can audition it, in my home.... However, after having said, all that, I am still an optimist, and I am going to ask, anyway...

-Robin

 

[DTB300]

I hope, I am wrong, but I will be shocked and awed, if any high-end audio store in the city of San Francisco, accually agrees, to let me take home, to Calistoga, a $4000 - $5000 USD, amplifier or thousands of USD's of audio cable.... just so that I can audition it, in my home.... However, after having said, all that, I am still an optimist, and I am going to ask, anyway...

If you have a Credit Card to cover the costs (in case you leave town with them), and you visit a store that really wants your business, and the removed components do not cause the store to limit their business, then they should do it for you. If not keep looking until you find one - hopefully not too far away.

The limiting business means - Say they are closed on Sunday, so you stop by on Saturday near closing time to pickup the equipment and you then bring it back on Monday or Tuesday morning when they open. This way you get to hear it at home for at least at day, and you do not put the store in a position where other customers do not have the ability to hear the equipment. This has worked for me and the store I visit. You may be able to have them longer but it depends on the store.

Dan

 

[garmtz]

However, after having said, all that, I am still an optimist, and I am going to ask, anyway...

Good for you! Good luck! And there is still the experience from guys here, there is a lot of experience here, at least with ML speakers!

 

[Muad'Dib]

It depends on the purpose of the study. If you are looking for a match for personal equipment then A/B/A in a short term fashion may not be ideal; however, you will notice upfront differences immediately. Anything long term and you lose your initial comparison and then it's not an A/B/A anymore. Then it is just audition time - of which there is nothing wrong with this.

A/B/A is supposed to be short, usually no longer than a minute (and typically under 30 seconds), comparisons to determine differences from a reference to a deviation and back to a reference condition. The initial purpose of the A/B/A comparison was used as an objective measure of fidelity for three primary transducers: loudspeakers, headphones, and hearing aids. Often TRUE A/B/A comparisons are used for measuring the audibility of distortion levels in context.

For further information see works by Gabrielsson and Toole primarily. Work on the subject by Killion may be easier to come by.

It's an interesting topic, although I think in the high-end world "A/B/A" is a term that is used quite loosely and rarely as it is supposed to be. I've read articles which describe listening to one set of somethings, changing them with another and after "EQ'ing" the second somethings, listening to music and then repeating the process with the initial somethings. This is not an A/B/A comparison. This is more of an absolute subjective (i.e.: adjective) rating with a tinge of relative subjective only in that a preference is made between the two somethings.

Hope this helps. And please don't flame me for contradicting the previous posts - this unfortunately happens to be the subject of my almost finished thesis.

-D

***EDIT***
Oh yeah, Robin - go for it and enjoy!

 

[aliveatfive]

Robin

I don't know if stores in NoCal have a policy of lending equipment, but many in New York do. As was mentioned in a previous post, a credit card number is usually sufficient to allow that kind of transaction to proceed. Give it a try. You have nothing to lose!

 

[roberto]

a/b/a then b/a/b it is much better...

Hola chicos...yes, sometimes it is very difficult to listen a difference. First of all, you must take you time to do the testing, and try to make the test "A" longer than "B", then go back to "A"...rest for a while, then listen longer "B", then "A" and again "B"...this is much easer and it works!. Always select a piece of a song that you like it very much, and with not too many instruments at the same time...again, the clue here is to listen a particular musical instrument, than several, because you can get confused. Also, you have to be with your system, unless the test that you are going to do, is too evident the difference between A/B. Trust your ears!!! and choose the one that you like most, regarless of specifications or look!!!
Hope this can help!
Regards and happy listening,
Roberto.

 

[risabet]

Any serious store . . .

should let you audition at home w/ the credit card proviso. I sold high-end for a couple of years and customers were always offered the chance to audition gear at home and usually for more than 1 day. This included speakers such as the Apogee Scintilla and the Klipsch K-horns.

 

[DTB300]

It depends on the purpose of the study. If you are looking for a match for personal equipment then A/B/A in a short term fashion may not be ideal; however, you will notice upfront differences immediately. Anything long term and you lose your initial comparison and then it's not an A/B/A anymore. Then it is just audition time - of which there is nothing wrong with this.

This is what I had thought, but not knowing enough about it, and seeing the article I thought would bring up some good disucssion on it and a chance to learn more about it. This in turn could help each of us when auditioning new equipment.

Hope this helps. And please don't flame me for contradicting the previous posts - this unfortunately happens to be the subject of my almost finished thesis.

Please let us know more about it if there is more to know than what you have already stated. Knowing the difference between live and recorded music is one thing and I think easy to discern, but learning how to listen for changes between A & B equipment, is something that I do not think is covererd very often.

Thanks for the post.

Dan