Why SINAD doesn't matter

Discussion in 'Blind Testing and Psychoacoustics' started by purr1n, Aug 28, 2021.

  1. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Recently a member posted a link to an article to headphones.com titled evaluating-sinad-why-its-not-important.

    This thread was shut down by the mods (I didn't do it but I approved) because these types of threads end up as the usual shit slinging. In the end, no one will change their mind, and there will be little talk about science or where the reality sits. There are a lot of good reasons why SINAD doesn't matter (it matters a little, but not much, and once past a certain point, it really has little correlation to sound quality). The headphones.com author cited masking and audibility thresholds. This is one of them which I have already cited, but in a much simpler way. In retrospect, I do think it's wise to keep repeating SINAD does matter. Perhaps every month I should post one reason why it doesn't matter? Well, for this month, I will make one simple argument:

    SINAD does not matter because people cannot hear it.
    By this, I specifically mean that Amir at Audio Science Review cannot hear bad SINAD.

    Take for instance ASR's glowing review of the JBL LSR305:
    https://www.audiosciencereview.com/...mkii-and-control-1-pro-monitors-review.10811/

    Subjective Listening
    The JBL 305P Mark ii is a delightful little speaker. It gives a smooth, very realistic sound impression, beating many "hi-fi" ...

    Review Conclusions
    It is hard to believe such great performance can be had for so little money in the form of JBL 305P Mark ii. We get it because of research and measurements like I have been showing that resulted in the XXX. So I can definitely recommend the 305P Mark ii. Get a pair so that you can get calibrated on what good sound is like, and correlation between that and the measurements I am showing.

    Well, all this looks all good and dandy. The LSR305 mark ii does measure decently in frequency response, nevermind that Amir seem to ignore the dip the lower mids to middle mids (the monitor is slightly bright). However, this is the SINAD performance of the cheap plate amp inside the JBL LSR305 mark ii. Here I have used the same criteria that Amir uses in measuring amplifiers. 5W into 4-ohms. Here is the result:

    JBL 3* plate amp
    1kHz 5W into 4-ohms
    [​IMG]

    The effective SINAD of this amp worse than 35db. Not only that, the distortion is full of very high odd harmonics. Why did Amir not hear this distortion? So what is this "correlation of this SINAD to sound quality" that Amir speaks of (using Amir's same measurement criteria)? THERE IS NONE: Audio Science Review is basically full of shit.

    SINAD does mean something, but if means far less than most people think it does.

    P.S.
    I've owned several LSR3* series throughout the years and still have a pair. I love them. They are great for what they do for how little they cost.
     
    • Like Like x 24
    • Epic Epic x 14
    • List
    Last edited: Aug 28, 2021
  2. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,345
    Trophy Points:
    113
    ^
    The JBL 305 are the dialog editing speakers of choice in the production industry. Their flaws make it cake and they’re so cheap, they’re disposable. Nothing else this cheap is close. The much studier Yamahas start at 400 a pair and the bigger cheap JBLs flat out suck.

    Now the JBL 305 is a useless piece of shit for eqing the low mid junk out of instrumental tracks and sitting vocals. All the dudes online lavishly exalting it have never had to use it.
     
  3. gepardcv

    gepardcv Almost "Made"

    Contributor
    Joined:
    Nov 20, 2015
    Likes Received:
    408
    Trophy Points:
    63
    Location:
    Terra, Sol System
    Betraying my utter ignorance here: how does this chart show that SINAD is worse than than 35 dB?
     
  4. Tekker

    Tekker Facebook Friend

    Banned
    Joined:
    Aug 1, 2020
    Likes Received:
    133
    Trophy Points:
    43
    Location:
    Netherlands
    Amir the type of audioscientist that thinks the HD650 and a random chi-fi heaphone tuned to the Harman curve sound the same in the upper-mids, because they measure the same there in FR-response.
     
  5. Philimon

    Philimon Friend

    Pyrate Contributor Banned
    Joined:
    May 6, 2016
    Likes Received:
    2,092
    Trophy Points:
    113
    Location:
    Portland, OR
    Link to quotes and random chi-fi headphone?
     
  6. ajaxender

    ajaxender New

    Joined:
    Dec 4, 2019
    Likes Received:
    21
    Trophy Points:
    13
    Location:
    New Zealand
    I thought that headphones.com article was pretty good at explaining the components of SINAD and some basic audio science.

    I think you're right though, that it's just overcomplicating the question of why it matters.
     
  7. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    SINAD is THD+N expressed in db.

    THD is all of the harmonics added up (noting that we need to convert them to Vrms). Then we add noise. And with respect to how instrumentation (AP or Aver) does the calculations, also everything else that isn't signal. In essence, the main contributors to THDN is all those -60db harmonic spikes "added" up and maybe that hump in the lows (probably AC mains not sufficiently resolved with too low an FFT window).

    This is why when we extend the bandwidth of the measurement, THDN goes up sometimes. If the distortion doesn't end in the audio band, but goes on, extending the bandwidth will make THDN / SINAD worse. The AP manual says to always state bandwidth whenever SINAD is mentioned.

    The headphones.com article was very good. It was explained very well. I'm coming at it from another angle: that Prophet of Audio Science is full of it; that he recommends gear powered from crappy amplifiers that he would otherwise crucify. The fact is, he couldn't hear the craptastic SINAD of the JBL monitors. Huge disconnect from reality here.
     
    • Like Like x 8
    • Epic Epic x 1
    • List
    Last edited: Aug 28, 2021
  8. Pancakes

    Pancakes Friend

    Pyrate Contributor
    Joined:
    Aug 13, 2020
    Likes Received:
    1,417
    Trophy Points:
    93
    Location:
    Atl
    Aha! I had no idea. I'm looking at the graph thinking "nothing is higher than ~55db".
     
  9. Vtory

    Vtory Audiophile™

    Pyrate MZR
    Joined:
    Sep 12, 2016
    Likes Received:
    10,831
    Trophy Points:
    113
    Location:
    East Coast
    Maybe it's just me. I kinda hate the idea to sum noise and harmonics together. I get where the idea came (i.e., signal vs everything else -- I never think lumping 'everything else' makes any sense..) though. Separating noise from harmonics and looking at all numbers/charts in a comprehensive set (not to mention variation by amplitude and frequency) would be way more beneficial than ranking gears purely by SINAD.

    Even if so, I barely believe we can distinguish bad gears from good ones by metrics itself (because we can't hear individual one per se). The best way to use them is to characterize gears in multi-dimensional context. It must be also worth questioning 'why does sth measure so?' when unusual patterns show up.
     
  10. netforce

    netforce MOT: Headphones.com

    Pyrate
    Joined:
    Aug 1, 2016
    Likes Received:
    3,111
    Trophy Points:
    93
    Nice, Andrew (Resolve) was working a long time on the headphones.com article and also the video to go with it you wanted to give that a watch:


    It will be very similar/sameish as the article component but it was a big labor on Andrew's part. ASR definitely noticed since it was posted there. So it certainly stirred up Amir so he went after Andrew quite personally in the DCA Stealth thread.

    My general take on SINAD having worked in retail for years now, is that the consumers have come to equate higher SINAD to be better quality. And seen first hand, people talking to me about a DAC with high SINAD being better than DACs worth thousands more.

    And the reason they would cite is they saw the ASR review and see how much was written. All these complex and sciencey things and then, see all those bar graphs with all the SINAD scores.
     
  11. soekris

    soekris MOT - Soekris Engineering

    Pyrate
    Joined:
    Mar 25, 2016
    Likes Received:
    954
    Trophy Points:
    93
    Location:
    Denmark
    I consider SINAD useless, taking two normally different measurement, Noise and Distortion and making one number out of them don't make sense to me. SINAD came from communication measurements and don't make sense for Music reproduction.
     
    • Epic Epic x 9
    • Like Like x 5
    • List
  12. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,345
    Trophy Points:
    113
    THD+N at 1khz is from the 1920s or something. Easiest measurement they could do
     
  13. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,345
    Trophy Points:
    113
    ASR don’t realize imd and the analog circuitry rising distortion in treble is more important than thd at 1khz for most people. It either needs to sound good or have a ton of open loop gain to be clean in the treble. Cheap gear cannot afford to be super clean. Schiit and MOTU are about as clean as you get. Steinberg might use the most tank parts but the treble is ugh in those cheap interfaces.

    ASR has poo pooed or sidelined major manufacturers of blank slate clean gear in favor of stuff with measurable and audible switcher noise. Not even 10 dollar AD opamps or 4 dollar TI ones perfectly stabilized can clean that up. They just let it all through. As we’ve heard with a lauded manufacturer of supposedly clean gear that continually gets complaints about the sound from people with ears, year after year and product after product.

    The STA microelectronics stuff is inexcusable and limits the lifespan of the JBLs. When the amps cook, the drivers get more distortion, and they will blow. That button dome can barely take Van Halen guitar tones. There’s no way it will survive a 24 hour straight recording session. If they used even the plate amps Sennheiser and Genelec do (also shitty but by no means as bad), they would cost a little more and last a lot longer.

    Meanwhile it’s pretty much impossible to kill the Yamaha HS series. And if you do, they will fix it or give you the service manual to do it yourself. Yet those are 400-800 dollars a pair and too real sounding for Reddit. You can’t crank a bass shelf into the NS10 with a port that is the HS5. You’ll just hit the internal limiter even harder and it will sound like shit instead of the small midrangey monitor it is.
     
  14. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    This is exactly why my GONAD panel (below) only takes into account THD. It's not THD+N. Noise can be measured separately and across the frequency range. Noise (even AC mains) doesn't seem to correlate to the sound of a piece of gear (unless its random spurs in the highs which could be an indicator of craptastic sound). THD however, specifically the distribution of harmonics (regardless of level), does seem to tell us a bit about the sound, e.g. second order to third order to higher orders, etc.

    [​IMG]

    Also, note how a single SINAD or THD number approach could be insufficient considering the protentional differences with output levels into different loads and at different frequencies. Real life is complex. It's not a single number. Even then, the above GONAD panel still consider insufficient - it's really for academic / curiosity purposes.

    But this a topic for another today. The topic for this thread is how Amir could not hear the really bad SINAD of the amp in his very positively reviewed JBL LSR305ii, and yet condemn via a SINAD litmus test many amplifiers which exhibited far better SINAD, in many cases very reasonable SINAD numbers, but didn't hit 100+db.

    P.S. As an aside, the JBL LSR 3 series is noisy as heck from those plate amps. Depending upon the roll of the dice and ambient levels, one can hear hiss from 1-3 feet away. That's how bad the SINAD of the JBL LSR305ii actually is. I don't know how Amir missed this during his (initial) listening tests.
     
    Last edited: Aug 29, 2021
  15. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    The ONE SINAD (as done by ASR) approach provides for exactly TWO kinds of headphone amps:
    1. Two stage amplifier using chips (OPA1612, OPA1656, TPA6120) with local feedback and global feedback loops unity gain.
    2. THX three stage with FF from first stage to output stage, local feedback, global feedback, all chips unity gain.
    And ONE kind of DAC:
    1. Delta-sigma chip, i.e. ESS Sabre* + chip IV + chip out / LPF.
    *Note AKM is not in the picture anymore.


    The audio world would be really shitty if ASR got their way. All we would have is shit like Topping L30, D50, THX789, Schiit Heresy, Modius, etc. and nothing else.
     
    Last edited: Aug 29, 2021
  16. Qildail

    Qildail Friend

    Pyrate Contributor
    Joined:
    Aug 6, 2021
    Likes Received:
    2,285
    Trophy Points:
    93
    Location:
    Iowa
    I am probably projecting here, but I wonder how much of it is the pendulum swinging back hard against decades of audio (and salesperson) snake oil in a toxic swirl with fewer opportunities to audition an exponential multiple gear choices prior to purchase. The only way I'll ever be able to tell whether a Yggdrasil is better than a Bifrost is better than a Modi (for me personally) is to buy all of them and have them here. That's a lot of money out and back and shipping and restocking, etc. etc. I can see how it ends up being much easier to buy the one with the best APx555 report number inside the budget and call it a day.

    But I guess this is where it goes sideways for me sitting here in the cheap seats. Instead of SINAD being a measurement, it becomes the measurement. If something isn't -120 dB or better, it's dismissed by some completely out of hand. Failed. Try again. Next.

    This puts a bow on it for me. Measurements will (sometimes) call out really bad (or unsafe) engineering. It tells one chapter in a story.

    I wrote and rewrote this three times in an honest attempt to not be hostile/toxic/insulting to anyone. At the end of the day, I want to enjoy my music in peace and genuinely want others be able to do the same. If a -120 dB SINAD headphone amp is what does that best for you, go press "play" and be happy.

    If this were truly a solved problem, though, we'd all be listening to the same gear.
     
    • Like Like x 15
    • Epic Epic x 3
    • List
  17. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    A lot of expensive shitty sounding hi-fi gear is a big part of the SINAD movement. I think most of us who have been around for the past 10-15 years can tell many stories of bad sounding gear of which we've paid thousands of dollars.

    You are absolutely right that instead of SINAD being a measurement, it's become the measurement. Even then, I dislike to say SINAD. It's more like SINAD per ASR methodology (1kHz at 2Vrms into 300-ohms), which if we think about it is very specific and limited in scope.

    SINAD when it is used properly can tell us quite a bit. It's only when it is misapplied or used as a GPU FPS benchmark that we get into trouble. Personally, I think a SINAD of 70db-80db is fine. It's good enough. SINAD isn't a race for state-of-the-art. None of this audio stuff is state-of-the-art. SINAD is only good for telling us is there if something amiss.

    Interestingly enough, for amps, SINAD over 100db+ raises eyebrows. The only to achieve this is via massive open-loop gain so that massive negative feedback can be applied. This method sounds kills the soul of music. You can talk to all the old audio gear designers (still alive): Nelson Pass, Jason Stoddard, Craig Uthus, Mike Moffat, John Curl, Andy Hefley and they will all agree. Obviously Amir knows more than them - but that is keeping his cult of personality.
     
  18. Beefy

    Beefy Friend

    Pyrate
    Joined:
    Apr 10, 2021
    Likes Received:
    1,738
    Trophy Points:
    93
    Location:
    Canada
    I completely agree here.

    Snake oil peddlers end up being a massive barrier to entry in the hobby. Very intimidating to newbies. And being a scientist, I always wanted more objectivity. But then NwAvGuy, and later Amir came along, and it was somehow far, far worse. The problem being, of course, is that they are not scientists, and are not seeking greater insight into why one product might sound better than another. They are just making a sales pitch to switch people from 'Snake oil' over to 'Serpentes Petroleum Distillate'. Same shit, different pile - albeit credit to them for helping grow the hobby/market massively.

    Methinks the pendulum swings a bit too far in both directions.

    And what a boring world that would be.
     
    • Like Like x 7
    • Epic Epic x 1
    • List
  19. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    I wouldn't go so far as saying snake peddlers. The fact is, audio is very subjective. One man's trash is another man's treasure.

    Take for instance the Leben CS300, much heralded as the amp on Head-Fi once upon a time. These were my impressions among others: http://www.changstar.com/www.changstar.com/index.php/topic,115.0.html

    Or this: the Audio-GD Singularity 19 (which was supposed to be their super awesome discrete ladder DAC after the PCM1704 ran out): https://www.superbestaudiofriends.o...y-19-dac-measurements.4135/page-4#post-130278

    Two those pieces of gear sounded authentically bad, no other way to put it. Not different, but bad. I never did take measurements of the Leben, but I did take measurements of the A-GD S19, which were bad - and a magnitude (10db+) worse than what A-GD had posted on their website.

    So sometimes, many times, a 1kHz spectrum (note, NOT SINAD) will give us a reality check. It's just a quick check. Like checking if there is oil in the engine from the dipstick or metal shavings after the oil is drained.

    And again:

    REALITY CHECK <> RACE
    (SINAD isn't anything like GPU FPS).

    P.S. The plate amps in the JBL measured worse than the S19 DAC! This tell us how difficult it is for people to hear distortion / noise / SINAD. (FWIW, I don't think Amir's ears are very well-trained or as well-trained as he says; but you know, cult of personality, managed 1000s of people at M$, etc.)
     
    Last edited: Aug 29, 2021
  20. gepardcv

    gepardcv Almost "Made"

    Contributor
    Joined:
    Nov 20, 2015
    Likes Received:
    408
    Trophy Points:
    63
    Location:
    Terra, Sol System
    Why are we still discussing ASR at all? The now-locked “ASR Review” thread has some solid arguments that the main reviewer there has about as much to do with science as scientology.

    This thread, OTOH, contains valuable information about distortion measurements all on its own.

    Here’s a question. Why do the JBL 3* series amps suck so bad? I can confirm that the pair I bought for some DAW work hisses like a nest of snakes, audible when listening at low levels. I thought the whole point of class D/H amplifiers was: (1) small size for their power output, and (2) crazy good measurements.
     

Share This Page