Let me explain why that test fails.
This was a biased test guaranteed to show a difference. It was statistically very likely that a winner would emerge given the large number of cables vs. number of test runs.
1. They were testing too many cables.
2. They needed more tests of the same cables.
3. They were completely missing any baseline tests.
#2 To validate any measurement is MUST be reproduced. A vs B needed to occur more than once. B vs. A needed to occur more than once. Their test had ZERO averaging.
#3 They didn't included an A vs. A, B vs. B, C vs. C etc.. These tests should have shown 100% no preference.
To have any statistical merit they could have tested with only 2 cables and run the following 4 tests at least twice in different order.
A vs. B
A vs. A
B vs. A
B vs. B
What they did guaranteed to generate an outcome like they saw. There is no validation of any preference given with the testing they conducted.
This was a biased test guaranteed to show a difference. It was statistically very likely that a winner would emerge given the large number of cables vs. number of test runs.
1. They were testing too many cables.
2. They needed more tests of the same cables.
3. They were completely missing any baseline tests.
#2 To validate any measurement is MUST be reproduced. A vs B needed to occur more than once. B vs. A needed to occur more than once. Their test had ZERO averaging.
#3 They didn't included an A vs. A, B vs. B, C vs. C etc.. These tests should have shown 100% no preference.
To have any statistical merit they could have tested with only 2 cables and run the following 4 tests at least twice in different order.
A vs. B
A vs. A
B vs. A
B vs. B
What they did guaranteed to generate an outcome like they saw. There is no validation of any preference given with the testing they conducted.
Last edited: