Comparison of several audio analysis systems - technical measurements

Discussion in 'Measurement Setups, Systems, and Standards' started by atomicbob, May 29, 2017.

  1. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    Man with one clock always know the time. Man with two clocks never certain.
    See Segal’s Law.

    This is an incomplete comparison of two measurements between several audio analysis systems. THD and THD+N will be compared at 999.023 Hz and 0dBV (1Vrms).

    Measurement system comparison setup:

    1. PrismSound dScope III audio analyzer
    2. Quantasylum QA401 audio analyzer
    3. Picoscope 5243b oscilloscope / spectrum analyzer
    4. Hewlett-Packard 339A audio analyzer
    5. Tektronix SG502 audio oscillator
    6. Atten ATF20B function generator
    7. Fluke 189 DMM
    8. AV Access U2EX50 USB cat5e 50M extender (galvanic isolation from laptop)
    9. USB input – Audioquest Forest 0.75 meter
    10. Tecnec 1505-B-B 75 ohm RG59 BNC cables
    11. Shielded 16 AWG 5-15P IEC60320 C13 power cables
    12. Signal sources set to 999.023 Hz, 0dBV (or as close as possible for analog signal generators)
    13. 16384 bin FFT with Hann window function
    14. 12 averages except where noted

    Pictures of test setup:

    Measurement System Comparison Setup.jpg
    Tektronix SG502 - Atten ATF20B
    Quantasylum QA401 - Picoscope 5243b - dScope
    Laptop currently displaying dScope system

    Measurement System Comparison Setup 2.jpg
    Quantasylum QA401 - Picoscope 5243b - dScope - HP339A - Fluke 189

    Measurement System Comparison Setup - connections.jpg Main, digital and analog signal management - Separate shielded mains for SMPS powered devices. Cat5e USB extender for galvanic isolation from laptop.

    20170529 SG502 HP339A 1KHz 0dBV THD.jpg Tektronix SG502 THD observation on HP 339A

    20170529 SG502 HP339A 1KHz 0dBV THD meter closeup.jpg
    Closeup - SG502 THD observation minimizing parallax error

    The numbers:
    20 170529 Audio Analyzer Systems Comparison THD % chart.png

    Editorial: For the most part I hate describing audio system performance with numbers. Graphs contain far more extensive and detailed information providing a better picture. Many people, not technically knowledgeable, would like a simplistic "goodness meter" and latch onto numbers as a way to compare two audio devices. This is misguided at best; often misleading, propagating the blind leading the blind. See the "obsolete" 20 bit converter discussion concerning Stereophile, DAVE and Yggdrasil on several forums for a very good example of how easy it is to fall into the trap. Hint: there are NO DACs providing analog output resolving to an equivalent of 32 bits digital representation as an example.

    Some observations on the numbers acquired:

    1. 5243b THD+N is just wrong - not ruling out operator error yet

    2. dScope THD+N at the top of the respective screenshots is a continuous measurement as opposed to the THD readings below which are averaged over 12 readings. It is possible for a given moment to indicate a LOWER THD+N than the THD. Over time the higher number is the one used. Very hard to synchronize a screen capture to a particular reading so there will be a few discrepancies between graphs and the chart of numbers.

    3. The QA401 has an unusual signal generator output with signal amplitude ramping up and down. Measurement by other instruments expecting a continuous amplitude for asynchronous measurements are challenged to obtain a useful reading. Thus the dScope may indicate a number higher than that obtained by the QA401 which is synchronized to its own generator.


    Editorial Conclusion (warning - personal opinion ahead):

    The dScope offers a much greater set of measurements, pre-made measurement scripts, higher resolution, speed of measurements, etc. as well it should for the price. But given how well the QA401 performs within the constraints of what it was designed to measure, it represents an incredible value. If I were designing audio amps to retail, on a constrained budget, I would opt to own a QA401 and rent time as necessary for the larger set of measurements utilizing an AP, dScope, SRS, Rohde & Schwarz, or equivalent. I believe the QA401 should be part of any serious audio lab bench.

     
    Last edited: May 29, 2017
  2. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    ATF20B function generator as measured by dScope
    01 20170528 ATF20B 1KHz 0dBV dScope.png

    ATF20B function generator as measured by QA401
    02 20170528 ATF20B 1KHz 0dBV QA401.png

    ATF20B function generator as measured by 5243b
    03 20170528 ATF20B 1KHz 0dBV 5243b.png

    SG502 audio oscillator as measured by dScope
    04 20170528 SG502 1KHz 0dBV dScope.png

    SG502 audio oscillator as measured by QA401
    05 20170528 SG502 1KHz 0dBV QA401.png

    SG502 audio oscillator as measured by 5243b
    06 20170528 SG502 1KHz 0dBV 5243b.png

    5243b signal generator as measured by dScope
    07 20170528 5243b 1KHz 0dBV dScope.png

    5243b signal generator as measured by QA401
    08 20170528 5243b 1KHz 0dBV QA401.png

    5243b signal generator as measured by 5243b
    09 20170528 5243b 1KHz 0dBV 5243b.png

     
    Last edited: May 29, 2017
    Dzerh, Changeling and ultrabike like this.
  3. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    QA401 signal generator as measured by dScope - single sweep - no averaging
    10 20170528 QA401 1KHz 0dBV dScope.png

    QA401 signal generator as measured by QA401 - note advantages of synchronization
    11 20170528 QA401 1KHz 0dBV QA401.png

    QA401 signal generator as measured by 5243b - single sweep - no averaging
    additionally it was necessary to limit FFT to 4096 bins to obtain useful data

    12 20170528 QA401 1KHz 0dBV 5243b - 4096 bins only.png

    dScope signal generator as measured by dScope
    13 20170528 dScope 1KHz 0dBV dscope SE.png

    dScope signal generator as measured by QA401
    14 20170528 dScope 1KHz 0dBV QA401.png

    dScope signal generator as measured by 5243b
    15a 20170529 dScope 1KHz 0dBV 5243b with external ground.png

    HP339A signal generator as measured by dScope
    16 20170529 HP339 1KHz 0dBV dScope.png

    HP339A signal generator as measured by QA401
    17d 20170529 HP339A 1KHz 0dBV QA401 t5 - use this one.png

    HP339A signal generator as measured by 5243b
    18 20170529 HP339A 1KHz 0dBV 5243b.png
     
    Last edited: May 29, 2017
  4. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    All devices had been powered on for several hours before measurements. However I noted that the HP339A THD was lower after 4 additional hours.

    HP339A signal generator as measured by dScope after several hours
    16 20170529 HP339 1KHz 0dBV dScope.png

    HP339A signal generator as measured by dScope with an additional 4 hours warmup
    16b 20170529 HP339 1KHz 0dBV dScope 4 hrs warm-up.png

    These lower numbers were not recorded in the main comparison chart as they were acquired inconsistent with the other devices. They are presented as an interesting anecdote supporting warmup over long periods of time benefiting audio devices.

    Last thought - those engineers at HP were what I considered Real Engineers(TM). Seeing a 35+ year old device still performing to spec is enlightening. The 339 input level RMS readings were compared to a recently calibrated Fluke 189 DMM as an example. Dead on for every signal source Vrms measured.



     
    Last edited: May 29, 2017
    Dzerh, SSL, ultrabike and 1 other person like this.
  5. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved 5/10.
     
    Last edited: Jun 20, 2017
  6. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved - coming soon - HW vs SW + Audio I/F measurement system comparisons. Nothing to see here yet. Move along folks. Delays will occur to accommodate addition of RME ADI-2 Pro as front end to SW measurement systems.

    20180615 edit
    It's been a year now. Maybe this summer I'll find some time to finally work on this project again.
     
    Last edited: Jun 15, 2018
    JK47 likes this.
  7. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved 7/10.
     
  8. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved 8/10.
     
  9. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved 9/10.
     
  10. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    reserved 10/10.
     
  11. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    7,079
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    Irvine CA
    QA401 is offering ASIO "unofficial" support. That opens the door to ARTA and REW. Furthermore, there is support for C#.

    I'm getting really tempted to get this now. Check the ARTA test they ran:

    https://quantasylum.com/blogs/news/asio-and-hypermiling-thd

    I think their FW release says that only the left channel support the attenuator through ASIO or something like that:

    https://github.com/QuantAsylum/QA401/releases

    But still, that is awesome!!!

    EDIT: They said the ASIO was a bit tricky in their hypermiling blog post. However the recent release (like 4 days ago) is very encouraging:

    "Added ASIO driver. Only left channel 48K atten active is functional right now. See "readme" in driver directory for instructions on how to install as its not installed by default. This driver won't be actively supported by us,, but we'd love to hear from you if it works as provided. With this driver, you should be able to move back and forth between Arta and the QA401 software. Note, though, that both apps cannot be running at the same time. if you have an Asio app open and try to open the QA401 app (or vice versa), the USB open will fail."
     
    Last edited: Jun 14, 2018
    Vtory and atomicbob like this.
  12. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    Just for a point of reference, here is the same measurement using stock v1.51 SW (no ASIO) with my unit:
    20180614 QA401 hypermiling verification - display only.png

    A little more 60 Hz noise. 2nd and 3rd harmonics appear to be approximately 10 dB higher than the ASIO version on the blog. HW or SW? Could be my particular box.
     
    ultrabike likes this.
  13. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    7,079
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    Irvine CA
    @atomicbob, I think you got the same results they got (or very close). The hypermiling discussion is all about the different THD and THD+N loopback numbers the unit gets depending on the input and output ports they use.

    In their test, the worst set of numbers happen when they loopback R+ to R+ (-112.4 dB THD). Second worse is L- to L+ (-113.8 dB THD). However, they get the best numbers with R- to R+ (-119.8 dB THD). They encourage to select the best loopback ports to get the best results when doing a measurement.

    Your THD+N is roughly the same (loopback is dominated by noise in the given BW): -90.5 or so.

    They also argue that using a Hanning window degrades performance by like 1 dB or so, and they used Rectangular. With windows AFAIK is always a trade off with side lobes and main lobe width. They get what they get I guess.

    So I think the unit you got is about right. It maybe possible to get an even better measurement channel by using a different combination of I/O.

    If you want to give it a go, try switching to Rectangular window and check the loopback that yields the best results. I think they that the best I/O combo is different from unit to unit.
     
    Dzerh likes this.
  14. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    7,079
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    Irvine CA
  15. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    Repeated with
    1. rect FFT window function
    2. one meter clearance in all directions
    3. Vaunix Lab Brick powered EMI shielded USB HUB
    4. lowest noise gen - analyzer input ports used.

    20180614 QA401 hypermiling verification - display only - 100cm clearance.png

    Still, the point being this is pretty impressive for an inexpensive analyzer as was mentioned in the first post. One of the reasons it is often used in my lab for tasks not requiring additional capabilities of the dScope.
     
    Dzerh, Cspirou and ultrabike like this.
  16. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    7,079
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    Irvine CA
    LOL! Oh well. Thanks @atomicbob!

    I will likely buy it depending on the maturity of their ASIO driver. They seem to be working on a QA402 so it may be best to wait and see.
     
    Last edited: Jun 14, 2018
    Ardacer likes this.
  17. Senorx12562

    Senorx12562 Case of the mondays

    Friend
    Joined:
    Oct 26, 2015
    Likes Received:
    1,260
    Dislikes Received:
    43
    Trophy Points:
    93
    Location:
    Limbo
    Rarely have I seen something so far above my paygrade.
     
  18. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    7,079
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    Irvine CA
    To measure high power amplifiers, any recommendations for attenuators?
     
  19. atomicbob

    atomicbob dScope Yoda

    Friend
    Joined:
    Sep 27, 2015
    Likes Received:
    7,075
    Dislikes Received:
    2
    Trophy Points:
    113
    Location:
    On planet
    ultrabike likes this.

Share This Page