DAC Blind Test: Modi2 vs ODAC

Discussion in 'Blind Testing and Psychoacoustics' started by purr1n, Nov 17, 2015.

  1. Xen

    Xen Friend

    Friend
    Joined:
    Sep 30, 2015
    Likes Received:
    205
    Dislikes Received:
    1
    Trophy Points:
    43
    Location:
    Houston, TX
    @purrin I wonder if just a non-sighted, leisurely test would be more palatable. Just ask your wife to flip a coin (Head = Modi2; Tails = ODAC), she logs which one she hooks up. You listen for the day and take notes at your leisure. Repeat for a week. Then compare the log with your notes. Instead of trying to determine which DAC is which, just see if you can note sound characteristics that seem to follow a DAC. Takes way longer, but observational biologist basically do this for years.

    EDIT: Of course, you are blind to the DAC you are listening to, but you are not trying to directly compared them. You will compare them only through your notes.
     
  2. purr1n

    purr1n Finding his inner redneck

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    64,190
    Dislikes Received:
    108
    Trophy Points:
    113
    Location:
    Antarctica
    Let's volume match to 0.01db (Ignore the spurs, I'm using the SE probes just to get levels right. Also, there appears to be some drift. the DACs were off 0.01db a minute ago, now it's 0.00db). I think this will make it easier since I won't have to deal with tweaking volume anymore. BTW, the match was pure luck with the parts I had around. This rarely happens.

    VM Modi2.png VM ODAC.png
     
    MrTie likes this.
  3. BioniclePhile

    BioniclePhile The Terminal Man - Friend

    Friend
    Joined:
    Oct 25, 2015
    Likes Received:
    345
    Dislikes Received:
    0
    Trophy Points:
    63
    Location:
    Admudnson-Scott Antarctic Facility (Soledad Cali)
    Home Page:
    "The amp is something I cannot say at this point other than it is a good amp, but isn't near TOTL or anything like that. Basically the real reason is that I'm too lazy to setup the ZD next to my desk."

    So, the Studio Jr. Mk. 47. Hopefully we'll hear more about this little bugger in the future. Good enough not to warrant the xanadu, but still "a good amp". Is Cavalli's Carbon in danger Purrin?! :p
     
  4. purr1n

    purr1n Finding his inner redneck

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    64,190
    Dislikes Received:
    108
    Trophy Points:
    113
    Location:
    Antarctica
    Nothing that expensive. I'm expecting it to come in somewhere between $0 and $600. I know I'm being vague. I think you can consider it more along the lines of entry level.

    BTW. Six more blind tests conducted. All level matched this time. As I suspected, level matched is much easier or either I'm getting good at this through practice.
     
    Last edited: Nov 18, 2015
    Luckbad likes this.
  5. BioniclePhile

    BioniclePhile The Terminal Man - Friend

    Friend
    Joined:
    Oct 25, 2015
    Likes Received:
    345
    Dislikes Received:
    0
    Trophy Points:
    63
    Location:
    Admudnson-Scott Antarctic Facility (Soledad Cali)
    Home Page:
    Possibility of being $1 is interesting. I'm looking forward to it regardless.

    Have you done any comparisons with odac or Modi with any of the multibit options? Just so long as you aren't getting laid, might as well ruin music by turning it into a chore.
     
  6. Priidik

    Priidik Friend

    Friend BWC
    Joined:
    Sep 27, 2015
    Likes Received:
    1,651
    Dislikes Received:
    3
    Trophy Points:
    93
    Location:
    Estonia
    A good amp is somewhat alarming to me.
    I tried to A-B Yggdrasil - Soekris Dam - BMC Puredac from a Apex Pinnacle and pretty much failed consistently.
    I picked up BMC occasionally because of (s-d / Sabre) tonality. In short the Pinnacle is nearly immune to dacs.
    As noted by yourself some time ago Sennheiser HDVA is quilty of these same sins and I agree.
    My mother could probably pick up differences b/w dacs with EC MK2 or Ehha though.
    Cans were HD650?
     
  7. purr1n

    purr1n Finding his inner redneck

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    64,190
    Dislikes Received:
    108
    Trophy Points:
    113
    Location:
    Antarctica
    Cans are my modded HD650. Don't need hyper-resolving HD800s or TOTL amps. I wanted to make this test challenging. I can't say more about most of the equipment until end of year. :p

    Did two more tests. Perfect 8/8. I'm done tonight. Julie getting sick of changing wires around for me.

    Here are post-test level matching results to double check - just in case something was moved or fell off. Drifted back to 0.01db difference. It seemed to me that the ODAC was subjectively slightly louder or more compressed toward loudness. Science says that I must had been imagining such things 8 out of 8 times. Or maybe the DAC which was 0.00-0.01db louder sounded worse.

    I am done here and have nothing left to prove to myself. Really, I'm not going to bother with Yggdrasil vs. ODAC. Tonight was cake. So much easier to tell differences with volume matching since it was one less thing I had to adjust or worry about.

    ODAC R -3.78dBFS
    VM2 Modi2.png

    Modi2 R -3.79dBFS
    VM2 ODAC.png

    ODAC L -3.78dBFS
    VM2 Modi2 L.png

    Modi2 L -3.79dBFS
    VM2 ODAC L.png
     
    Last edited: Nov 19, 2015
    Griffon, Xen, Psalmanazar and 5 others like this.
  8. purr1n

    purr1n Finding his inner redneck

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    64,190
    Dislikes Received:
    108
    Trophy Points:
    113
    Location:
    Antarctica
    LOL, seriously. I would hope that the members understand your use of verbal irony by now!
     
  9. lm4der

    lm4der A very good sport - Friend

    Friend
    Joined:
    Sep 26, 2015
    Likes Received:
    1,442
    Dislikes Received:
    1
    Trophy Points:
    113
    Location:
    Seattle, WA
    Thank you for doing this. You did it, you proved that even DACs this close in performance can be blind A/B'd. I am not aware of anyone (that is, anyone qualified) actually going to the effort to do this experiment, at least not with such rigor.

    Finally, this means a lot to me. I think it puts to rest some of the nwavguy / ha arguments.

    I mean, do you guys remember when Tom's Hardware published an article where they had people try to blindly discriminate 4 dacs, from laptop-out up to a $2000 one, and the result was no one could tell the difference? Without revisiting the problems with their test, this is the kind of thing that we are used to hearing about.

    Your test has a lot of value.
     
    Last edited: Nov 19, 2015
  10. NoStream

    NoStream Acquaintance

    Joined:
    Sep 27, 2015
    Likes Received:
    61
    Dislikes Received:
    0
    Trophy Points:
    18
    Location:
    Oakland, CA
    That's some pretty impressive stuff. From a statistical standpoint, that's 1/(2^8) or .3% chance of being correct if you were guessing. So it seems very likely you were able to pick up differences. (I certainly believe you could.)

    The two obvious questions are (1) could others also pick up these differences in your setup (you could give them a "handicap" like HD800s + a hyper-resolving SS amp) and (2) could you do the same for higher-end comparisons. Something like a high-end DS vs. high-end R2R A/B could be very interesting.
     
  11. TMoney

    TMoney Shits on SBAF over at Head-Case to be cool

    Joined:
    Sep 27, 2015
    Likes Received:
    780
    Dislikes Received:
    2
    Trophy Points:
    93
    Location:
    Sausalito, CA
    This is already one of my favorite threads on SBAF.
     
    lm4der likes this.
  12. Thad E Ginathom

    Thad E Ginathom Friend

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,908
    Dislikes Received:
    13
    Trophy Points:
    113
    Location:
    India
    Possibly, possibly not. Each test subject would bring their own ears and their own experience. An AB[X] test only counts for me/here/now. It doesn't indicate anything about the same person with the same gear the next day, or other people on any day.

    For the individual, it might be useful (especially if one wants to talk about the results) to establish repeatability. For the rest, the statistics would stack up as they stacked up.

    There are some differences that might just never hit some of us at all. I think that some of the smaller digital-artifact stuff comes under that heading. Maybe it doesn't have that much to do with every-day listening, however vital it might be to the lossy compression developer and tweaker.
     
  13. NoStream

    NoStream Acquaintance

    Joined:
    Sep 27, 2015
    Likes Received:
    61
    Dislikes Received:
    0
    Trophy Points:
    18
    Location:
    Oakland, CA
    Yeah, exactly. It's still surprising/exciting to see someone pass a blind dac a/b since such a thing is pretty much unheard of. If it's repeatable, then it'll be interesting to see just how repeatable it is.

    And for me, there's a certain psychological barrier when AB testing DACs is considered "impossible." If it's difficult but doable in some cases, there's probably less predisposition to just say "eh, they sound the same."


    I ran into similar things when testing mp3 compression blind several years ago. Above a certain point around, both files sounded equally "good," but with effort, I could distinguish the roll-off on the mp3 file's cymbals. But at least we can prove that in certain circumstances certain individuals can hear objective differences.
     
  14. mkozlows

    mkozlows Friend

    Friend
    Joined:
    Oct 6, 2015
    Likes Received:
    420
    Dislikes Received:
    1
    Trophy Points:
    63
    One thing you will find is, if you look around for negative blind tests on DACs, there are very few credible ones. The most notable ones are the Big Sound ones, where they're comparing two very expensive high end DACs. It seems plausible to me that maybe those two DACs are audibly indistinguishable, without saying anything about other DACs.
     
  15. NoStream

    NoStream Acquaintance

    Joined:
    Sep 27, 2015
    Likes Received:
    61
    Dislikes Received:
    0
    Trophy Points:
    18
    Location:
    Oakland, CA
    Well the problem is that HA folks think that ABing DACs is laughable and pointless and HF folks think it's unnecessary since differences are so obvious... but reality's somewhere between those two extremes.

    If individuals here who are able to do this can actually establish a body of comparisons, it'll help people navigate purchases much more easily.
     
    MrTie likes this.
  16. purr1n

    purr1n Finding his inner redneck

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    64,190
    Dislikes Received:
    108
    Trophy Points:
    113
    Location:
    Antarctica
    Confirmation bias can go both ways. Maybe people who have the preconception that DACs with linear and non-linear distortion below the thresholds of human hearing sound the same (which is pretty much any decent DAC on the planet), will set up tests where differences will be indistinguishable. Maybe people who hear a difference will set up tests were differences will be distinguishable, including the allowance of dry runs, practice sessions, extended exposure to gear and recordings to be used beforehand, etc. I cannot overemphasize how guessing incorrectly during the practice sessions or even the test sessions themselves helped me hone my ability to discern the differences. The biggest thing I had to overcome with this test was my preconception that one DAC would be better than the other in every circumstance and every aspect.

    The Tom's Hardware test was rigged so the participants would fail:
    1. Instead of A/B, it was A/B/C/D where the participate would have to guess which device it was. This adds a lot of workload to the participant.
    2. Each session was restricted to eight runs (two per devices in random positions) in sequence. The participant was not allowed the freedom choose the device to be listened to at any time at his leisure. The participant was not allowed to select different tracks during each run within the session.
    3. It's unclear how much time was allowed for each participant to become familiar with the recordings or the equipment.
    4. It's unclear how much practice each participant was allowed.
    5. Results for listener B "a more serious enthusiast" were absent for the A/B/C/D tests. It appears listener B only participated in the DSD vs PCM and 174k vs 44.1k tests.
    6. Most of those setups would be considered roughly equivalent in terms of sound quality by the ears here with the exception of the motherboard output being significantly worse than the others.
     
    Last edited: Nov 19, 2015
    magicscreen, SKiring, schiit and 2 others like this.
  17. Thad E Ginathom

    Thad E Ginathom Friend

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,908
    Dislikes Received:
    13
    Trophy Points:
    113
    Location:
    India
    Disclosure: I visit and occasionally post on Hydrogenaud.io. I find it provides some balance in life! I generally agree with "their" point of view on purely sighted testing and "their" dismissals of the criticisms of sighted testing. I have watched the entry level to stereo sound drop over four or five decades, and, for a very long time now, entirely decent (I avoid using the words high fidelity) sound has been available to many more than the high-spending elite. I suppose, then, that reasonable to better-than-reasonable audio electronics is or has become trivial to achieve.

    I do baulk at everything sounds the same. Because it might not be designed to sound the same. It even might be designed to sound worse!

    Every test should be designed so that the subject is comfortable.

    If it is one of those tests, where the object is to debunk some person or piece of equipment, then that is even more important, as the debunkee is obviously going to squirm. Familiarity with music; familiarity with sighted testing; breaks for sighted testing --- all this and more can be built in.
     
  18. chakku

    chakku Friend

    Friend
    Joined:
    Oct 13, 2015
    Likes Received:
    678
    Dislikes Received:
    2
    Trophy Points:
    93
    Location:
    New Zealand
    Home Page:
    I've noticed a number of times people have mentioned they can't stand the "Sabre" sound of a Sabre DAC chip and thus the preference for products without them.

    Would it therefore be plausible to have a blind ABX test with two completely mystery DACs which may or may not have different DAC chips and may or may not be using a specific brand or model of chip and identify which DAC chip is being used in DAC A and DAC B?
     
  19. NoStream

    NoStream Acquaintance

    Joined:
    Sep 27, 2015
    Likes Received:
    61
    Dislikes Received:
    0
    Trophy Points:
    18
    Location:
    Oakland, CA
    There's no need for the test to be that stressful. For a start, you could just have a test of a known "Sabre-sounding" DAC like the Oppo HA-1 vs. a known smoother DAC - say a PCM1704 or other R2R DAC. Forcing people to ID which is which adds unnecessary complication.
     
  20. chakku

    chakku Friend

    Friend
    Joined:
    Oct 13, 2015
    Likes Received:
    678
    Dislikes Received:
    2
    Trophy Points:
    93
    Location:
    New Zealand
    Home Page:
    Surely it wouldn't be a problem if this Sabre sound is so obvious and undesirable though? Why would you even have to know what DAC chips the DAC is using. Not trying to criticize anyone, I'm genuinely curious here and seek more knowledge.

    Or maybe you could just have the single, mystery DAC and ask participants to identify whether it is Sabre or not?
     

Share This Page