It has been my experience that the vast majority of audiophiles base their opinions on invalid subjective testing that is fraught with error and which leads to them making false assumptions about the causes of what they hear that differentiates tested components.
Basic protocols, such as level matching devices under test or ensuring testing is conducted blind and in a timely fashion, are rarely, if ever, implemented by audiophiles. Ask yourself .... is this you? The reality is that, sadly, only a select few audio journalists, reviewers and retailers bother to follow such basic protocol either. And they opine on audio matters negatively influencing readers and buyers in the process.
Heck, we can’t even reach consensus on definitions for the host of subjective terms that are bandied about in this industry.
There really is nothing quite like seeing audiophiles, retailers and reviewers using words with different meanings to describe differences they hear in components that haven’t been properly subjectively tested.
If that’s not enough ……. given that not one of us has, in place, even the most basic comparative fundamentals of (identical systems, setup in the same room, with the same music being played at the same volume levels) any subjective comparison on what any us can or can't hear, even if subjectively tested properly as per above, lacks a true reference and is nothing more than an exercise in futility.
So, what is the answer?
In the absence of such reference I think doing a valid ABX * test, or even a null test, at least brings some commonality to the testing process in an attempt to eliminate the issues above and bring into the fold some element of objectivity which can be referenced by everyone. Even if these processes, themselves, have some issues (and they do) …. surely they are a vast improvement over the highly ineffective subjective processes noted above?
Better yet is to do objective measurements. Yeah, this isn’t for everyone but should it not be? If only to avoid the subjective mess spoken about above! I mean, we all spend oodles of time talking amongst ourselves, in audio clubs and in forums such as these, all in an attempt to find out all we can about gear that we may one day wish to buy. Once content with what we have been told, we spend thousands, many tens of thousands of dollars even, going out there and buying such gear (hopefully not based on the subjective mess spoken about above). Yet barely any of us thinks it wise to spend some portion of our audio $ and buy measurement gear and, if needed, take the time, energy and effort to learn how to measure what it is that we are, or are not, hearing so that the purchases we make are sensible. This is one of the many things that boggles my mind about many of us audiophiles.
As an added bonus. What objective measurement can allow for is … very accurate analysis of what distortions and/or frequency response may be responsible for the subjective preference(s) you may enjoy, or even detest. You could then administer a process which allows you to include components which output these preferred errors so that you are listening to the distortion/response errors that your ears prefer.
Anyhow, for me it boils down to this:
1. All devices under test that objectively perform with measured errors in distortion, noise and frequency response that are below human hearing threshold ** will sound identical to any listener who does a valid listening test.
2. If devices under test are shown to sound different (in a valid listening test) then such sound differences can clearly be demonstrated by using instruments which allow for taking proper objective measurements so that you can visually analyze the results.
* For clarity sake my proper ABX test of electronics is one that is done blind using my switchbox which allows instantaneous switching between compared devices that are level matched to around 0.1 dB.
** There are numerous references to the finite hearing ability of humans insofar as hearing limits. Giving human hearing the benefit of the doubt .... the consensus is that human hearing threshold is as follows:
• Distortions (individual harmonic, alias, modulation, & crosstalk) all below –90 dBFS with their total sum below –80 dBFS (0.01%)
• All noises below –110 dB with total sum below –100 dBFS
• All jitters below –110 dB with total sum below -100 dBFS
• All over a linear fr (20Hz – 19 kHz within +/- 0.1 dB)
Obviously, I am presumptuous and believe that current measurement processes and instruments can measure all things that determine sound quality ***. I freely acknowledge the possibility that components and elements of psychoacoustics (a study of science I have only cursory knowledge of) may well alter & impact hearing in ways that the above criteria can’t explain but, in the absence of science, or even 1 measly controlled listening proof, that refutes the limits, I am certainly not in a position to question it. Not when the overwhelming evidence supports that there is no demonstrated audible difference that isn’t measurable. Could there come a time when this could change? Sure, but that's more a philosophical question which could be debated till the cows come home.
*** applicable only to voltages of electronic devices (none clipping amps, CDPs, DACs, etc) and not so of transducers since speakers output sound waves which can’t be accurately measured in a room.
Stated another way:
• If I can hear it, and measurements suggest I shouldn’t be able to hear it, then I believe that what I hear may be real or imaginary but I side with imaginary and wonder if I should go see my shrink.
• If I can hear it, and measurements agree that I should be able to hear it, then I believe that what I hear is real and I grab another drink.
• If I can't hear it, and measurements suggest I should be able to hear it, then I change up my system.
• If I can't hear it, and measurements suggest I shouldn’t be able to hear it, then I grab another drink.
Cheers ....................
