Blind Testing

It's probably not better at all. Massdrop just sent it out for the friendly review hype train.
EDIT: The truth hurts hehe
EDIT 2: All comparison claims to other DACs should be done in a blind test if you're posting as a legitimate product review. Don't care who you are.
EDIT 3: Lol I know I must be a masochist here, but seriously what's wrong with removing inherent biases by blind testing when comparing one audio product to another in what's supposed to be a legit product review?

 

My listening (preference) tests are done using a hardware-assisted blind approach.

It wouldn't pass muster as a true full double-blind test, but it is level matched, switching is managed in hardware with a suitable hardware random number generator, and other than knowing which two units are under test I do not know which I am listening to at a given moment.

When I did that test with Modi MB vs. Bifrost MB it took a fair amount of sighted listening practice before I could reliably tell those two units apart. Other comparisons not so much.

 

Appreciate the clarification, props for doing it the right way!

 

Please see this

 

This is dope, how all professional review/comparisons should be done!

 

Why do people keep thinking we're against ABX? Because we happen to give a shit about subjective reviews too?

 

Never said I was qualified, technically, fiscally or anyway really lol. I listen to a lot of music though and it's clear that blind tests with audio equipment (especially DACs and amps) is the way to go when trying to write a professional style review (do you disagree?). Otherwise I think there's so many biases at play that have nothing to do with sound preference/signature that results aren't accurate. It's just my gripe with the audio community in general (not just sbaf) that a lot of definitive statements are made about equipment without the appropriate measures to back up those claims. Torq backed it up though and I give him mad props for that. I'd be interested to hear someone who doesn't think blind tests are the right way to do it when reviewing gear, honestly.

 

It's also another bias imo to assume that blind testing does not induce newer biases. Plus, who pay the cost for such costly testing? It definitely links to 1) expensive rigs, 2) more fatigue and less enjoyment, 3) not suitable for long listening

Some part of audio listening is inherently impossible to do blindly. Reviewers may use some tricks if they really enjoy doing so. In general, such methods cannot motivate reviewers well.

In my case even very simple ABX foobar testing was very frustrating. Because it cannot maintain my pleasure as much as non-blind listening.

TL;DR - Human being is more complicated than lab animals.

 

I really like the wine analogy.

Learning to tell good audio gear from bad is inherently human*, but learning to tell the good from the great is a learned skill much like being able to discern the differences between various grades of not-crap wines. Being able to evaluate tonality is basic stuff, I've been able to do that since high school (while I may be Asian, I'm not one of those kids who've been studying an instrument from youth, not that my parents didn't try). Learning to evaluate things like imaging and soundstage was only marginally more difficult, while timbre, micro/macrodynamics, are still very much a WIP, and I've been self-training for a few years now.

The long and the schlong short of it is that you can't just take impressions from some random person on reddit who's not put in the time to learn necessary gear evaluation skills and call them good. I used to do that back when I was just getting into this hobby in my mid-teens roughly a decade ago, and ended up with some very poor purchases as a result. Because of that, I now get irritated as is unbecoming of my age whenever I hear someone on reddit say that there is no good reason to get a better amp for, say, an HD650 (which scales like a salmon up a waterfall) when a Fiio E10K is enough, or that all SS amps are all just a matter of amplifying signal to sound louder, with no motherfucking effect on signal quality.

.... I should maybe avoid reddit, or at least r/headphones since I rather like the new Harry Potter: Hogwarts Mystery sub. Yes I play that game, Hufflepuff FTW.



*and for those that think (reportedly) bad stuff like the Topping D30 is good, well, they are perhaps less than human

 

Look, if you don't believe that blind tests provide more accurate results than non-blind tests, I guess the burden of proof is on me to provide with research that says otherwise.

On Some Biases Encountered in Modern Listening Tests (https://iosr.uk/projects/ias/papers/Zielinski.pdf)
Hearing is Believing vs. Believing is Hearing: Blind vs. Sighted Listening Tests, and Other Interesting Things (http://www.aes.org/e-lib/browse.cfm?elib=6338)
SEVEN KEYSTONES OF ACCURATE AUDIO BLIND TESTING (https://headfonics.com/2018/01/seven-keystones-accurate-audio-blind-testing/)

There's a lot more research out there, both related to audio and related to blind/double-blind testing in general that says it's the most objective way to go about measuring (albeit subjective) results. This isn't to say I don't believe that you can tell dacs and amps apart without a blind test purr1n. I truly believe you can and you have a lot of experience with it. I think yours and others opinions are absolutely valid and accurate and you don't need to blind test something to post opinions on a forum. But if someone is writing a "professional" review comparing the equipment to something else, that then gets posted on... say... the Massdrop page for the product, I'd hope it was done with a blind test no matter how golden the reviewers ears are. Because it's often not even about your ears, but about the other biases like confirmation bias, visual cues, etc. that come into play. I know I came off sarcastic/argumentative-for-no-reason in previous posts, but I do feel passionately about the validity of blind tests being the best way to go about the "proper" reviewing of audio gear.

 

I am not sure current rdac reviews are paid reviews or not. (Assumed not)

Even if they are paid, again, nobody is willing to pay the (monetary, emotional, and convenience) costs of blind testing.

Massdrop not
Reviewers not
Sbaf friends (as readers) not

If you really want rigorously conducted blindly tested reviews, do pay the right fee (device costs + labor costs). If the money is large enough to motivate, someone is willing to participate. That's how capitalism works.

 

Oh goodie, more hot double blind test talk, to try and invalidate someone's argument. "Hmmm, I can't win here, so I'll bash the established person's lack of using double blind, and maybe go after their grammar if that doesn't work too..."

Nothing makes listening to music less fun, then those kind of tests....

 

I'd like to add that blind testing is just another tool in the toolbox, just like measurements, sighted listening, production work (such as dialog or music editing) and at the end of the day, low fatigue listening enjoyment. That last one is my final arbiter for whether a system remains in rotation or is punted.

 

Soren @soekris, good to see you defending your products here. Good on ya, mate.

 

Yes except I have no opinion here other than the FACT that blind tests (not necessarily double blind) are better suited for a "professional" review than non-blind tests. Putting the costs aside, if you don't agree with that then I don't know what to tell you.