Oh shit! SINAD can suck my ****s

Discussion in 'Measurement Techniques Discussion' started by Marvey, Jul 2, 2021.

  1. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    With the recent acquisition of an AP analyzer, my intention was to leverage its automation capabilities to offer an alternative more developed set of measurements to simply SINAD, specifically SINAD at 1kHz into a 300-ohm load at 2Vrms and sometimes 4V RMS with a bandwidth of 20kHz. It's not that I do not believe in "SINAD", but rather in how it as a single number was effectively being used as a dogmatic litmus test and yardstick for sonic transparency by Audio Science Review. My intention was to present a related but alternative to SINAD, that is THD mapped to a matrix of low, midrange, and high frequencies vs. voltage levels more reflective of actual use with popular headphones. (2Vrms / 4Vrms is downright destructive to many headphones, not to mention our hearing). The idea was the make things a bit more complex as things in the real world are complex, while keeping things simple enough for us to digest and relate to.

    The first such testbed for this approach was the Topping L30 amplifier reviewed here. Not surprisingly, the L30 blew by this "THD matrix" with flying colors. (Well, sort of, until I discovered that one channel was borked for levels past 2Vrms for 32-ohm loads.) Out of curiosity, a member asked if there was any reason I why I could not perform the measurements using a real headphone as a load instead of a dummy load. Now the reason we use dummy resistor loads is because we don't destroy headphones if we push the headphones too loudly (or if we accidentally press or punch in the wrong numbers, which happens more often than you think). Realizing that I would be testing headphones at just a bit past reasonable listening (voltage) levels (the target was 100db SPL at the peaks), I didn't see any reason why not.

    The first rationale for this testing using real headphones was to ascertain the effects of back-EMF with respect to global negative feedback. A resistor load will not provide back-EMF. However a headphone driver really isn't that much different from a microphone. If we shout loud enough into a headphone driver and measure at the phono jack, we will see voltage. The second rationale was to ascertain the effects of back-EMF with respect to current capability. It was cited that USB dongle AIOs exhibited sloppier more one-note bass than desktop headphone amplifiers.

    Lo and behold, I most certainly did not expect this! A real headphone load (Sennheiser HD650) resulted in up to 50db worse THD than a dummy 300 ohm load at 37Hz with the Topping L30 amplifier!

    Dummy 300 ohm resistor load, again note results taken from the good L channel.
    764 mVrms, 37Hz, unity gain
    [​IMG]
    [​IMG]

    764 mVrms, 37Hz, Sennheiser HD650 as load. Unity gain.
    [​IMG]
    [​IMG]
     
    • Like / Agree Like / Agree x 24
    • Epic Epic x 21
    • List
    Last edited: Jul 2, 2021
  2. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    So what does this mean? Well, not necessarily anything if all (or even most) amplifiers exhibited similar behavior. Except this wasn't the case. I immediately found an amplifier where the THD of the 37Hz signal didn't get worse at all!

    Jotunheim 1. Low Gain. Sennheiser HD650 as load. Balanced output.
    [​IMG]
    [​IMG]
     
    • Like / Agree Like / Agree x 22
    • Epic Epic x 9
    • List
    Last edited: Jul 2, 2021
  3. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    THIS IS HUGE!!!

    If I believed that purchases should be made on the basis on measurements, I would choose the Jotunheim 1 over the L30 on the basis that the THD at 37Hz for a Sennheiser HD650 far exceeded those of the Topping L30, even though the midrange and treble (1kHz and 9.3kHz) THD were slightly worse.

    I don't know how many of you guys can appreciate how huge this could be. It basically means all those SINAD numbers and Excel bar graphs lining up best to worst on Audio Science Review are suspect and only tell us a limited part of the story. To be honest, I am super happy that with your help, I stumbled upon this. Basically any audiophile with any kind of brain or ears could tell you that those SINAD those did not correlate well to sonic transparency or sound quality much, unless the numbers were very poor.

    Audio Science Review has been a bane in the audio world with its SINAD dogma. For the past two years, ASR has essentially been the Eye or Sauron. If you are a manufacturer, you would hope that the Eye does not gaze upon you, lest it unleash it's minions (lead by the Pink Panther), controlled by the One Ring ("SINAD")*, upon you. The result of this "One Ring to Rule Them All" resulted in a newfound popularity of opamp based headphone amplifiers, since this is the easiest way to get awesome SINAD numbers. Nothing gets us better 1kHz THD+N (1/SINAD) numbers than massive amounts of global negative feedback. Opamps with their massive gain allow us to do this. Such amps are also very easy to design.

    *SINAD per Audio Science Review: 1kHz Sine, 300 ohm load, 2V or 4Vrms, 20kHz bandwidth.

    What we've done here is demonstrate that SINAD (per ASR) alone may not be enough. That it misses a big part of the picture - and when it misses, it misses badly. Again, it's not that I do not believe in SINAD. I do believe that the typical 1kHz SINAD is very useful. Just that I know it's limits.

    Anyway, the One Ring is vanquished.
    [​IMG]
    F the Eye of Sauron.
     
    • Epic Epic x 42
    • Like / Agree Like / Agree x 32
    • List
    Last edited: Jul 5, 2021
  4. SSL

    SSL Friend

    Friend
    Joined:
    Nov 12, 2015
    Likes Received:
    1,277
    Trophy Points:
    93
    Would be interesting to see if this manifests in the time domain somehow. Could have even bigger implications for audibility.
     
  5. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    This "discovery" does lead to more questions than answers. This is why I wanted to create a separate thread.
    1. Why does this happen at the 37Hz measurement and not at 1kHz or 9.3kHz? Is this really the effect of back-EMF? Are lower frequencies affected? If so, when does the effect start?
    2. Why did the Jotunheim amp not exhibit the 50db THD loss at 37Hz as seen in the Topping L30. 50db is a BIG BIG difference.
    3. As such, do balanced amps fare inherently better. Perhaps try the Magnius which uses the same TPA6120 output device as the L30, but in balanced output.
    4. What about negative feedback? One test so far with the Magni 3+ seem to indicate that negative feedback was bad with respect to the 37Hz THD measurement (but as expected better for the 1kHz and 9.3 kHz THD).
    5. What about Grados?
    6. Does this only happen with dynamic headphones with voice coils. What about orthos?
    7. What about OPT tube amps?
    8. Tangentially related: perhaps we should use the volume control on amps to set the right output level using a common input 2Vrms signal as opposed to turning up the volume all the way and letting our analyzer to set input voltage. This goes along with the "realism" theme. That measurements should reflect actual usage by people as much as possible. The downside of course is more work on the part of the person taking the measurement.
    Did I miss anything? Please provide suggestions.
     
    • Like / Agree Like / Agree x 16
    • Epic Epic x 2
    • List
    Last edited: Jul 2, 2021
  6. Merrick

    Merrick A lidless ear

    Friend
    Joined:
    Jan 6, 2016
    Likes Received:
    7,973
    Trophy Points:
    113
    Location:
    Portland, OR
    It would be interesting to see if differential balanced amps were more immune to this phenomenon in general. Also what about low impedance dynamics? Let’s put that 325x to use!
     
  7. zerodeefex

    zerodeefex Grumpiest admin

    Staff Member Friend BWC
    Joined:
    Sep 25, 2015
    Likes Received:
    12,726
    Trophy Points:
    113
    Holy shit, thread of the year
     
    • Like / Agree Like / Agree x 9
    • List
  8. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    Put on the list!

    Words cannot express how super excited and geeked out I am. I have a ton of product to review, but exploring this avenue will take priority! Again this is HUGE. HUGE. HUGE. And the funny thing is that it was sitting in our faces all this time.

    I had actually done tests using Sennheisers as the load, but that was with hefty desktop amps with gain, so I never noticed anything off.

    To me, the APx555 has already paid for itself! By the way, if you have not already donated, please consider doing so regularly: https://www.patreon.com/SBAF I quit my high paying job to focus more time on content and SBAF. I still have a family with growing kids to take care of. Discovering stuff like this is only possible because of the generosity of others, so it's appreciated. There is also a PayPal link at the bottom right of the Home page for one time donations if that works better for you. I don't do my bega-thons on every review like Amir, so please consider this.
     
    • Like / Agree Like / Agree x 21
    • List
    Last edited: Jul 2, 2021
  9. Beefy

    Beefy Almost "Made"

    Joined:
    Apr 10, 2021
    Likes Received:
    256
    Trophy Points:
    63
    Location:
    Canada
    Establishing appropriate levels is going to be so critical to both realism and technical excellence in any measurements.

    With the HD650, my loudest listening level with music only comes up as about 0.25 VRMS when fed a sine wave. Even 0.775 VRMS makes we wince with energetic music, and 2 VRMS would be enough to make my ears ring for a month.

    I think if you really want to do this 'properly', then a quiet, medium, and deafening set of measurements is surely justified.

    But, I do appreciate how easy it is for me to say this, seeing as it's not me doing the work! :D
     
    • Like / Agree Like / Agree x 3
    • List
  10. crenca

    crenca Friend

    Friend
    Joined:
    May 26, 2017
    Likes Received:
    2,400
    Trophy Points:
    93
    Location:
    Southern New Mexico
    Is there any reason why OPT tube amps would be any different than chip amp/discreet in reference to current availability, back EMF? In other words, as they scale up in current capability, would they not in theory display similar results?
     
    • Like / Agree Like / Agree x 1
    • List
  11. zerodeefex

    zerodeefex Grumpiest admin

    Staff Member Friend BWC
    Joined:
    Sep 25, 2015
    Likes Received:
    12,726
    Trophy Points:
    113

    This goes back to the conversation we had many, many years ago about developing a measurement methodology that emulates how people actually USE products. It would be meaningfully interesting for consumers AND the industry to developer a standardized way of measuring audio performance in-context.
     
    • Like / Agree Like / Agree x 7
    • List
  12. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,284
    Trophy Points:
    93
    Added to the discovery list!

    It would be prudent to first identify the circumstances which cause the 37Hz THD to take a dump (or not take a dump). It would also be a good idea to see the at what lower frequencies things start to look bad for amps (and headphones) which exhibit this behavior.

    Lots and lots to discover and learn. Science is fun.
     
    • Like / Agree Like / Agree x 4
    • List
    Last edited: Jul 2, 2021
  13. Biodegraded

    Biodegraded Friend

    Friend
    Joined:
    May 28, 2017
    Likes Received:
    5,402
    Trophy Points:
    113
    Location:
    Vancouver BC
    IEMs, both dynamic and BA. Do small light dynamic drivers produce a similar effect at low frequencies as big headphone drivers (so is it primarily electrical or mechanical)? And related, as @Merrick asks, do low (and not too variable) impedance dynamic headphones do it to the same extent as Senns? Ah, ok - I see you got Grados. Do BA IEMs with impedance peaks outside the bass region do it in the bass, or near their impedance peaks, or at all?

    Edit: in red.
     
    • Like / Agree Like / Agree x 7
    • List
    Last edited: Jul 2, 2021
  14. Beefy

    Beefy Almost "Made"

    Joined:
    Apr 10, 2021
    Likes Received:
    256
    Trophy Points:
    63
    Location:
    Canada
    Shit, maybe it's all in the cables. I would have to eat a lot of crow.
     
  15. Ox Cart

    Ox Cart Facebook Friend

    Joined:
    Mar 13, 2018
    Likes Received:
    134
    Trophy Points:
    33
    Location:
    Austin, TX
    It would be interesting to see measurements at and approaching the driver's resonance frequency to see if this effect relates to that.
     
  16. Ox Cart

    Ox Cart Facebook Friend

    Joined:
    Mar 13, 2018
    Likes Received:
    134
    Trophy Points:
    33
    Location:
    Austin, TX
    Exciting stuff! This might also explain the scalability of high ohm Sennheisers. On the wrong amp, that low frequency distortion may both cause the bass to sound a bit loose and also mask higher frequency information, causing a loss of clarity/detail.
     
    • Like / Agree Like / Agree x 6
    • List
  17. k1arg

    k1arg Facebook Friend

    Contributor
    Joined:
    Oct 12, 2015
    Likes Received:
    145
    Trophy Points:
    43
    Location:
    London, UK
    As an interesting amp to try this experiment on, I nominate the ECP Copenhagen.
     
  18. YMO

    YMO it's not drinking alone if you're on Zoom

    Friend
    Joined:
    Apr 1, 2018
    Likes Received:
    7,089
    Trophy Points:
    113
    Location:
    DUVALLL-904
    Reading this thread while eating my Steak is so worth it.
     
    • Like / Agree Like / Agree x 2
    • List
  19. cameng318

    cameng318 Friend

    Friend
    Joined:
    Jun 14, 2020
    Likes Received:
    208
    Trophy Points:
    33
    Location:
    NYC
    One of the possibility is the nonlinear output impedance of the opa16xx opamps. I haven't followed up with any Topping gears, so correct me if I'm wrong, I guess the L30 is using one of opa16xx for the input stage. If you check the figure 28 of opa1612's datasheet, it's output impedance is approaching 1kohm around 37 Hz. It could reduce the open loop gain in the bass region, especially if the following stage has a really low input impedance (e.g. inverting opamp amplifier, it could be the 2nd stage of the nested loop in L30. It takes reverse engineering to confirm).

    I remember the designer of the opa16xx series discussed the rising output impedance toward the bass region in a post at diyaudio.com. I can't dig it up at the moment. Basically they found it helpful to stabilize the opamp, so they could achieve higher open loop gain and lower distortion in the mids. I guess it's sorta a local feedback at the output stage.

    Edit: found it in this post
    So it's actually a side effect of implementing rail to rail output stage in CMOS technology.
     
    • Like / Agree Like / Agree x 5
    • List
    Last edited: Jul 2, 2021
  20. Tachikoma

    Tachikoma Facebook Friend

    Contributor
    Joined:
    Oct 4, 2015
    Likes Received:
    220
    Trophy Points:
    43
    The difference between LF and MF/HF SINAD also changes with input level. Measuring the SINAD delta versus input level might yield interesting results.

    OTOH, it looks like the SINAD delta its going to be dominated by the degradation in LF SINAD anyway.
     
    • Like / Agree Like / Agree x 1
    • List
    Last edited: Jul 2, 2021

Share This Page