Audio Science Review Review

First and foremost, any scientific claim has to be reproducible in order for it to be considered credible, hence the inclusion of a section on methodology (or materials and methods) in any credible scientific journal. People have lost careers over their results being irreproducible by others, which often revealed instances of scientific fraud. Obfuscation of how any data is produced is, like you said, the antithesis of science.

 

@purr1n this explains all the false claims of grounding issues https://www.whatsbestforum.com/threads/oh-no-not-again-amir.23670/

 

Latest fiasco: https://www.audiosciencereview.com/forum/index.php?threads/magnepan-lrs-speaker-review.16068/. Now even ASR members are chiming in to point out the flawed measurement techniques.

 

Read the review, too confusing.

Why? No overview of placement of the speakers and no details on the distance between the measuring microphone and the actual speaker in its environment. When I look at speaker reviews I want to know on thing first: despite the flaws the speakers might have; what is the sweet spot (if there is one)?

The graphs might show something interesting but are too confusing to read. The legend is sloppy and even after reading two times, I still feel I need more context.

I might not be his core audience. I do not have to be. That review style does not work.

 

This seems like another example of the OP point #5: not taking into consideration the intent of the design; but in this case far more egregious, because despite claiming to be some sort of audio 'expert' he doesn't seem to get that the dipole design needs to be ON THE FLOOR to avoid bass cancellation (other room considerations aside).

Mind you, it took josh358 some time to spot that too. Who is that guy?

 

Amir actually founded WBF too.

 

Knowing little about ASR beyond what I've read here, I went to take a look for myself. Something that hasn't been mentioned, I don't think - but that's intriguing - is the apparent lack of headphones measurements over there. This is all the more striking when SBAF has, as of this post, 11,003 messages (in 173 discussions) for measurements of headphones vs. 991 messages (in 35 discussions) for headphone amps' measurements. I wonder what accounts for this difference?

 

Looks like topping doesn't make headphones, so nothing to shill, and no need to intentionally bork measurements of competitors products.

 

I think the difference is that SBAF prefers to listen to music rather than look at plots and mindlessly praise whoever created them. That's the weirdest thing about visting ASR. It's like they don't even really listen to music. They seem to just buy the flavor-of-the-month SINAD champion and then never provide any impressions about how it sounds. I think it's because they've made up their minds that amps, DACs, etc., don't have an effect on what they're hearing, provided that a given piece of gear meets Amir's made-up criteria. So the idea of providing listening impressions just never occurs to them. It's the exact opposite here. Theory crafting is generally looked down upon. Just listen, use your ears, and tell us what you hear. It's not voodoo. Listen to what your senses are telling you, and trust your impressions. This goes for basically anything. The key is to keep things in proportion. Subjective impressions can go off the rails when people make mountains out of mole hills (e.g., digital cables). This isn't to say that things like that can't make a difference, but it's important to put things in perspective. But like Marv says, if you know what to listen for, everything has a sound. For the ASR crowd, they're not even trying to learn about their audio preferences, because their dogma tells them that they shouldn't have any.

 

It was probably part of a long-term strategy to discredit SBAF for having an open baffle following.

Mostly joking, of course.

 

I've heard a lot of objective-minded people say that anything that isn't a sealed box with lots of damping is unacceptable.

The thing is they love citing people like Linkwitz and he had determined a long time ago that open baffle is the only way to go for the home environment.

 

The thing with ASR is that they use science for very human needs.

1. Simplification of the infinitely complex - SINAD/Spinorama/Speaker Preference Index uber alles
2. Virtue signaling - we're so rational in our purchases, that we buy flowers depending on their price per pound!
3. Insulation of belief systems - if something isn't measurable, then it can't exist, taking existing research as gospel (static measurements uber alles, time domain doesn't matter, Spinorama can't lie)
4. Total dismissal of qualia, unless expressed by authorities or in accordance with measurement gospel
5. Community building with those who adher to scientism values and stay in line with them

 

That’s not science, it’s dogmatism. A scientific approach is always open to new data, refinement of experimentation techniques, and replication of results from peers.

 

There's an overview in a different post about how the measurement system that they're using works, but the tldr is that it takes a whole bunch of near-field measurements, and then uses math to extrapolate that out into more meaningful measurements.

"Amir is sloppy and doesn't really understand how to use the expensive tools he has" is a common theme in this thread, and I feel like that's very likely what's going on with the LRS review. A dipole radiating line source like the LRS is just a fundamentally very different thing than a forward-radiating point source driver. Given how much mathy analysis that Klippel system does to construct its output from the raw near-field measurements, it strikes me as extremely plausible that it might need different techniques or settings to accurately measure a speaker like that.

Do I know that for a fact? No, I have no idea how to operate that thing. But I know enough to know that if I measured something and got shitty data, my first thought would be "hmm, did I screw this up somehow" rather than "lol these porkers have no idea what they're doing, and this popular line of speakers that they've been making in this form for decades is so obviously unbelievably terrible in ways that contradict many people's firsthand experience, welp, that's just science facts, let's publish."

And I don't mean this to say that Magnepan speakers are immune from criticism. "A speaker like this is inevitably going to be very sensitive to its room, and placement will be super-finicky with tons of variation and bad outcomes possible" is a legitimate criticism that is extremely in-line with everyone's takes on these speakers since forever. But "this measurement shows they're bad, and then with this measurement in my head and no real care given to setup, I listened to them and heard what the measurement said I should hear, so anyone who has ever heard them better is obviously a fool" isn't a conclusion you can come to from this review.

 

Made me think...

It's been a long time, but ISTR that Stereophile publishes measurements with its reviews? I seem to also recall that their subjective review of a piece of gear is done by a different person to the measurements guy. Whilst this may be simply a matter of convenience, technical competence, availability of equipment and lots of etceteras that are not a desire to be objective, it compares well with the Amir situation as you describe it (and I'm sure you're right). Which is one of absolute bias.

 

I find Stereophile reviews to be a bit schizophrenic at times. Often gross flaws found by JA will be completely missed by the writer of the listening part. And I'm not talking about small nits like SINAD not being 120dB. As an editor for the magazine it would make me doubt the expertise of either the listening or measuring party.

 

Ironically, Stereophile has also measured the LRS and, surprise surprise, got different results than Amir: https://www.stereophile.com/content/magnepan-lrs-loudspeaker-measurements

John Atkinson also has his biases, as has been discussed elsewhere on SBAF. However, I think his take on the LRS was more balanced, especially as evidenced by his conclusion:

At least he recognizes the difficulty in measuring them and drawing helpful conclusions from such measurements, vs. Amir’s illogical conclusions noted above.

No one in their right mind would ever argue that a Magnepan speaker is bass heavy. But someone would have to be grossly incompetent at setup or be legitimately deaf to think that their sound actually rolls off as quickly under 300 Hz as Amir’s borked measurements show.

Other Stereophile Magnepan measurements I’m aware of:

• 1.6QR (what I own)
• MG 2.5
• MG 2.6
• MG-IIIA
• MG-20

 

How people measure and perceive sound waves is a discussion that can go on endlessly. Do not get me started. Pandora's box will unleash the worst of the worst from my beloved hobby.

Here is my problem: a reviewer reviews speakers, measures characteristics and can give subjective impressions and I am okay with that; I am not okay with a sloppy review that omits critical details that can give you basic to good understanding of how the audio system operates.

I read the review two times and tried to make sense of it. It reminded me of Hype-Fi in the worst ways. If you want to convince me that your review is worthy of my time, do not confuse me. I need topic, the audio-system, structure, methodology and a conclusion. Do you know who taught me that? Tyll and Solderdude.

This Amir dude might be a nice person, fine. His review style really does not work for me, I am out.

P.S.

Here is why I love S.B.A.F. Sometimes despite the best efforts to map out, explain the whole "topic, audio-system, structure, methodology and a conclusion" the people I trust here simply say: I tried to make it work but I cannot live with this, just no.

If you want to avoid a sloppy mess keep it simple. "I tried to make it work but I cannot live with this, just no."

One such line of how something functions for you as an individual conveys a clearer message than infinite graphs with all the colours of the rainbow. When you understand that, we can talk. Nothing against A.S.R. or Amir, just keep it simple if you cannot make it complicated and understandable.

 

Yes. I remember when the measurement guy (John Atkinson) started doing those measurements.

It was already known that components with the best single-value distortion measurements didn't always sound best, so the idea was to measure a whole bunch of other quantities and correlate those measurements with the results of subjective reviews, to try to find a measurement (or combination of measurements) that DID predict good sound -- or at least see whether each reviewer had an individual combination that correlated with good reviews.