New portable headphone measuring setup

Discussion in 'Measurement Setups, Systems, and Standards' started by Crinacle, Dec 13, 2018.

Tags:
  1. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    Straight to the point, I'll be attempting to head into the headphone measuring game. A bit of new territory for me so looking for some additional pointers/advice from the measuring heads here (@Hands, @purr1n, @cskippy, @ultrabike etc. feel free to chime in).

    Current setup with HD800:
    01b7c6b94e0e560359fa359d8e64e136a402bef3fc.jpg

    Reusing my IEC microphone to mimic the input/transfer impedance of the inner ear per Bruel & Kjaer's researched recommendations. Can't find a ITU-T P.57 compliant ear (or relevant clone) so I've resorted to using the MiniDSP EARS for now. A bit of a placeholder in that regard, at least until I can find something more accurate.

    Additional purpose is to be portable hence the foldability:
    014322b143fe7926ab9c4045eec52b01e7893f83ec.jpg

    Procedures and settings are similar to that of my IEM measurements (also shameless plug to my subreddit where I'm updating measurements/impressions more frequently) though I will be running the headphones off of a iFi Micro iDSD BL for power and SPL reasons. Due to the equipment (literally a phone) and the environment, I'll only be taking frequency response measurements. Also, probably only right-channel measurements due to ear used.

    FR graphs acquired, processed from raw data gathered by an iPhone 7 Plus on the AudioTools app and subsequently generated with REW:

    HD600.png
    Sennheiser HD600

    HD650.png
    Sennheiser HD650

    Sennheiser HD800 with SDR

    The data should be closer to that of Innerfidelity's raws, Rinchoi's raws, Stereophile's data, Clarity Fidelity's rig or RTing's stuff. Looks pretty good with the HD650, though the HD800 graph has me scratching my head a little.

    I'll be heading out tomorrow to get more measurements of other headphones for testing and calibration (feel free to request any) but once all my hardware and settings are finalised, I'll go out full-force with the data collection.

    Raw data can be provided if needed, just ask.
     
  2. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Are you mounting the microphone flush with the EARS or using some of the canal? Taking a wild guess here that the microphone you are using is mounted flush.

    The HD800 measurement looks like the microphone may have shifted. The EARS when used with the included microphone as-is in the canal has a high Q resonance around 4.5kHz. When the microphone is flush mounted, this resonance goes away (per @Hands' results). I would expect that if the microphone fell back a little creating a canal, but of shorter length, the resonance would come back at a higher frequency, i.e. 6kHz per your HD800 results.
     
  3. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    I did some extra measurements if anyone needs a reference point for where I'm going with this. Some are unusual/niche models since I just wanted to try them, but again just request for something and I'll get right to it.

    Audio Technica ATH-ADX5000
    ADX5000.png

    Final Audio D8000
    D8000.png

    Focal Utopia
    Utopia.png

    Klipsch HP-3
    HP3.png

    Meze Empyrean
    Empyrean.png

    Sennheiser HD800
    HD800.png

    Sennheiser HD800S
    HD800S.png

    Stax SR-L700
    L700.png

    @purr1n I am using the IEC-provided external canal simulator so the EARS canal is effectively bypassed in this case. The only "component" of the EARS I'm using is its artificial pinna. The 6k spike on the HD800 measurement doesn't seem to be a consistent resonant artifact of the system (as shown in the re-measurement above) and should be part of the transducer. Also seems to be consistent with current reports of the HD800S having a lower 6k spike, per the comparison plot I made below:

    HD800 vs HD800S
    HD800vHD800S.png
     
  4. Superexchanger

    Superexchanger Friend

    Pyrate
    Joined:
    Jul 11, 2019
    Likes Received:
    1,309
    Trophy Points:
    93
    Location:
    Flavortown, USA
    Thanks for providing these data for our review. If you'll indulge a basic question, can you take a stab at what's responsible for the variance of the response in certain frequency bands only depending on placement (c.f. Empyrean varaince 4k-10k, 800S 10k-20k, & the near absence of variance in D8000 over the whole sweep).

    My uneducated guess is standing wave behavior modulated by the pinna/pad volume, giving each model some characteristic range of position-impacted frequencies, but fails to explain why the D8000 would be so immune.

    Not really an issue since averages are given, but the physical reason for this type of thing is interesting to me.
     
    Last edited: Aug 21, 2019
  5. cskippy

    cskippy Creamy warmpoo

    Pyrate MZR
    Joined:
    Oct 14, 2015
    Likes Received:
    3,999
    Trophy Points:
    113
    Location:
    Tempe, Arizona
    That notch at ~2.5kHz looks like concha gain to me. Try removing it with a calibration curve and updating your measurements. I feel a lot if your measurements have excess energy in this area.
     
  6. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    Update: just received the full GRAS 43AG-7 set, so now I'm all set to start building my headphones measurement database. This is the setup:

    [​IMG]
    GRAS KB500X artificial pinna > GRAS RA0402 pre-polarised "high resolution" IEC60318-4 coupler > B&K 1704-C-102 CCP signal conditioner (powered by separate USB bus) > Motu M2 audio interface > PC (REW)

    The foldable mounting system in the first post has been repurposed to accomodate the RA0314 cheek plate to keep everything as portable as possible. That said, the whole setup doesn't look too portable as it is right now, but after a bit of velcro and tetris-ing each of the components into a half-decent singular block, I should be ready for heavy-duty data collection.

    I'll be replacing the laptop with a smaller 8" one as well, one with more than one USB hub (curse you Microsoft) so that I can power the signal conditioner off the laptop itself rather than requiring a separate power source. Yes, the Micro Black Label in the pic above is being used as a glorified powerbank. Don't judge.

    Measurements obtained from this setup will be compatible with most academic target curves, most notably the Harman Target and KEMAR-based diffuse field measurements. That said, I'll be using my own "proprietary" neutral target curve when publishing these graphs (basically flat in the bass till 1kHz, then Harman OE for the rest), though compensations to other targets will be made available in the future on my website's graph comparison tool.

    Measurement procedure:
    1. CCP conditioner at x1 gain, interface at 50% gain
    2. Levels normalised to -20 dBFS
    3. 1M sequence length @ 192kHz sample rate, 2 sweep average
    4. Each channel is measured 3 times, each time taking the headphone off the rig and putting it back on. Positions are not forced to be "intentionally different" so as to simulate natural placements.
    5. All .mdat data is saved, though due to the noisy measurement environments the other non-FR metrics may not be very useful and so will not be published.
    Here are 6 preliminary graphs of the Sennheiser HD600, HD650, HD800S, Focal Clear, Audio Technica ATH-M50X and Hifiman HE1000 V1 after 6 hours of testing and trial-and-error. The first few months with a new rig are always the toughest...

    HD600.jpg
    HD650.jpg
    HD800S.jpg
    Clear.jpg
    M50X.jpg
    HE1000 V1.jpg

    Will gather more data soon, so stay tuned.
     
  7. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
  8. nishan99

    nishan99 Friend

    Pyrate
    Joined:
    Dec 1, 2019
    Likes Received:
    1,617
    Trophy Points:
    93
    Location:
    Layla
    Any idea why the Verite measurements are this inaccurate in the treble region?

    • 5k peak should be 6.2k
    • 11k peak should be 8k instead of the big dip
    • 18k peak should be 14k

    Am I missing something here? because that's what I am hearing with my pair.

    Is it the positioning of the headphones that can cause those shifts or is it something else?

    [​IMG]
     
  9. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    Perceived peaks and dips in a headphone are dependent on your pinna structure (also canal structure).

    I don't claim to have accurate measurements even on this industry-standard rig (no rig, not even the new B&K 5128, will ever get 100% accurate measurements for the reason that they typically approximate a human head), but rather I focus on precision, repeatability and comparability between each measured headphone. Hence, the creation of the graph comparison tool.

    So while you may not agree that the Verite graph I generated has the exact peaks and dips that you hear (for instance, the KB5000/KB5001 artificial pinna has a known null at 8-9kHz, something that may not be the case for your specific ear structure), you can at least infer that the Verite would be relatively recessed in the 1-4kHz regions in comparison to a HD600, and also have a greater 5-6kHz response in the same comparison.
     
  10. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    chifigrasclone.png
    Dropped $400 on a Chinese 43AG clone (single channel, swappable pinna). About double the price of a MiniDSP EARS, but in theory all the required components will be properly simulated on this rig.

    Will gather some data on it, hopefully create a good calibration profile for it so it can be used with existing Diffuse Field/Free Field/Harman targets. At this point it's less than 10% the cost of a real 43AG (with extra pinna) so as long as it gets "close enough", it should be far superior to EARS and flatplates.

    Potential issues I've seen so far:
    • The pinna is Shore 20-A, which is about 60+ on the Shore OO scale (higher = harder) and so headphones that come in contact with the ear will not be as accurately represented in final measurements. For reference, here are the existing GRAS pinnas and their hardness ratings:
      • KB006X: Shore 55-OO (non-anthropometric)
      • KB106X: Shore 35-OO (non-anthropometric)
      • KB500X: Shore 35-OO (anthropometric)
    • Variations in the occluded-ear simulator will be "amplified" when used for headphone measurements, relative to when being used for IEM measurements. This means that while the occluded-ear is simulated in this particular rig, the final result would probably be significantly out-of-spec with IEC60318-7. I've done some experiments with different 711 clones on the same pinna to see what are the effects of the occluded-ear sim on the whole rig (cheek plate, headphone and pinna are made constant):
    711diffheadphones.jpg
    • 99% chance that this Chinese 43AG clone will need its own custom calibration for existing targets. But it shouldn't be too much of an issue, after all even Tyll's HMS II. 3 differs pretty significantly from GRAS' HATSes.
    All in all, probably a significant leap forward for the headphone hobby, much like how the existence of cheap 711s were for the IEM hobby. We'll see how it goes.
     
  11. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    It's awesome to see that this stuff is affordable now. This is exactly what I'm been looking for. Those results actually look really good. Compensation can easily be achieved to arrive at the "real" thing like Jude's rig because the Q of the peak for the ear (pinna, concha, etc.) is much lower that of the miniDSP EARs. All EARs measurements have a big spike near 4.6kHz and a smaller one at 8.5kHz . While I've done what I could to compensate for these spikes, they still concern me. That EARS spike at 4.6kHz is a resonance in the time domain making the impulse responses not so usable to derive CSDs, distortion, burst/decay envelope, etc. The GRAS isn't perfect either with the narrow dip at 9.5kHz, but I would rather have this.

    Here are some raw measurements I took from a GRAS45?, compared to EARS

    GRAS (ORA) vs EARS (GRY)
    Sennheiser HD650
    RAW FR
    upload_2020-11-4_7-51-31.png

    GRAS (ORA) vs EARS (GRY)
    Focal Clear
    RAW FR
    upload_2020-11-4_7-48-45.png
     
  12. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    The problem is not only modelling a human head, it's also who's head? While we can argue that our brains' inverse transfer functions will "undo" the gain effects of our pinna and concha, the other problem is that the interaction between the headphone and the ear will vary depending upon the headphone.

    Looking at the range of the variation in ear gain among different people (there was an excerpt from a paper posted by @james444 or @shotgunshane where scientists did some experiments), we know that IEM measurements, even if the coupler was perfect (there isn't any), can only be "accurate" for an individual because IEMs bypass the outer ear structures. Likewise headphones suffer a similar issue but to a lesser and varying extent. I've observed planars, probably because of their planar wavefront, to be more immune to differences in the outer ear structure, other headphones like the HD800 with the drivers suspended outward are super sensitive to the ear structure, on-ears like Grados are another story, et. al.

    Bottom line is that 2-5kHz (with less up to 7kHz) will vary moderately in terms of "accuracy" for IEMs, and for headphones, vary somewhat, depending upon headphone. There is already anecdotal evidence to support this given how your neutral target and the SBAF neutral target (derived via consensus from a few folks here who probably hear similarly) differ slightly.
     
    Last edited: Nov 4, 2020
  13. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    As an aside, one thing we should know about dips in frequency response is that they are often peaks in disguise. This is where CSDs can come in handy. If that dip in @Crinacle's measurement is interpreted as a peak, then we are suddenly a bit closer to your subjective observations.

    Your post demonstrates the very issues I described in my prior post. Maybe it's possible that your ears are more similar to the artificial ears on the miniDSP EARS than on the GRAS?

    Verite FR from miniDSP EARS (SBAF neutral comp)
    upload_2020-11-4_8-22-42.png

    Ultimately, all of this is problematic because I don't always find the EARS to be reflective of what I hear. Sometimes I think that "Scraps", the flat plate coupler, is more representative; other times, it's the pseudo "free-air" coupler. What I've observed between the different type couplers is a shift in the peaks. Here is the flat plate coupler result. Two peaks got shifted down and one peak got shifted up relative to the EARS.

    Verite CSD from flat plate coupler (no ears)
    upload_2020-11-4_8-43-23.png

    @Serious has mentioned that he's preferred the original V1 pseudo "free-air" coupler for the highs because ear structures simply wreak too much havoc on the response in that region. @ultrabike has argued that different ear structures (whether measurement ears or people ears) will result in radically different measurement results, therefore it may be desirable not to deal with ears at all.

    The downsides of not having any ear is that we will miss out on the varying effects of the headphone-ear interaction. A problem with the pseudo-free air measurement is that the headphone becomes relatively undamped. We know that damping will change the frequency and amplitude of peaks, especially if the headphone driver has high Qts, that is poor electrical and mechanical damping.

    Per above, that depends. The flat plates have another advantage which is huge immunity to variances in placement. I've been able to go back and measure the same headphone years later and get so so close the same result. On rigs with ears, not so much, or at least it's harder.
     
    Last edited: Nov 4, 2020
  14. Crinacle

    Crinacle Friend

    Staff Member Pyrate IEMW MZR
    Joined:
    Dec 12, 2016
    Likes Received:
    812
    Trophy Points:
    93
    Location:
    SG
    Home Page:
    Kind of true in a way, but I wouldn't go as far as to say that the solution to variations in ear structure is to eliminate it entirely. As you've said, the headphone-ear interaction is significant and important, so while one may be trading off accuracy for precision in a flatplate system, I'd argue that it's going too far in that direction.

    The other problem with eliminating the pinna/occluded-ear simulation is that the relative differences between headphones are also affected. Using a single calibration profile would not compensate all headphones equally (which defeats the purpose of a calibration, assuming that one is calibrating a flatplate to mimic a simulated rig), which is why I've been never really gone beyond surface-level experimentation on a flatplate system (or the MiniDSP EARS, for that matter).

    But with high-quality clones now on the market, it would be nice if there's more widespread adoption and so raise the bar for hobbyist measurements. The price is on the steep side for now (though I doubt hardcore enthusiasts would bat an eye spending $400 on a headphone) but I wouldn't have imagined that we'd have viable, cheaper alternatives to industry-standard HATSes just a mere few years ago.
     
  15. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Oh no, I wasn't proposing not using an ear simulator. I was simply stating that we have different tools to arrive at supplementary data sets. For example, a flat plate or free air system could provide alternate or better data in the higher frequencies, say past 8kHz, where there is no longer any gain from pinna or concha.

    Now as far as a single calibration profile for a flat plate system not able to compensate correctly for all headphones, this statement would still be true for systems with different artificial ears. The example I posted with the EARs measurements which happen to align with @nishan99's ears is evidence of this. Also @Hands RANDY (I have one myself) where the left and right ears are so slightly different, has also demonstrated that a even dedicated calibration profiles (one for each ear) cannot work with consistent accuracy: https://www.superbestaudiofriends.o...-the-randy-headphone-measurement-system.6973/

    All this being said, I see myself building a GRAS clone system soon. The benefits of a consistent standardized measurement system across hobbyists would be tremendous for obvious reasons. However, my concern has always been the "one true god" effect. Not everyone's ears respond or are shaped like the ears on the GRAS. We've already seen horrible V-shaped consumer compensation targets as a result of this "one true god" effect.
     
    Last edited: Nov 4, 2020

Share This Page