Community measurement site + removing SBAF compensation from Purr1ns measurements

Discussion in 'Audio Science' started by Suuup, Dec 2, 2020.

Tags:
  1. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    Hi everyone,
    I have created a new measurement website with the help from Oratory1990. To goal is to have a place for the community to easily share measurements of their headphones, etc. It can be found here https://headphonedatabase.com/grapher?ids=30,31

    I've talked to Purr1n, and he has given me the go ahead to upload his measurements from hpdb.io. My site takes in raw measurements from EARS rigs (can also take in data from other rigs), and uses the SN from the EARS rig to fetch the HPN compensation directly from MiniDSP. On top of the HPN compensation, we have developed an estimate at a Harman target response, as well as what we call "Optimum", which is our desired target response.

    Now, in order to read in Purr1ns measurements, I have to remove the compensation. To do this, I've taken the SBAF comp from https://docs.google.com/spreadsheet...NOkjApvMKAIL3IP8Y1At_ldng/edit#gid=2116629201 (green rectangle) after talking to Purr1n, and subtracted it from the measurements. What I get looks like this [​IMG]
    The blue line is the raw output of HD800 from another rig, and the orange line is my attempt at removing the SBAF compensation from Purr1ns measurements.
    The same thing for HD600 looks like this
    [​IMG]

    The difference between the known raw and my attempt at a raw by removing the SBAF comp looks like this
    [​IMG]

    Now I would say that is quite a substantial difference. What I've done is take a bunch of these differences, from many samples and different headphones, correcting from Purr1ns rig to the other known rig, meaning I should get the same results from Purr1ns measurements as on the known rig.

    I'm looking for any comments wrt if this is a valid approach or not. I've uploaded a small handful of Purr1ns measurements to the website, you can see them there.

    Cheers,
    /Suuup
     
    sorrodje, m8o, spoony and 25 others like this.
  2. Lyander

    Lyander Too sensitive for SBAF

    Friend
    Joined:
    Feb 25, 2017
    Likes Received:
    7,398
    Dislikes Received:
    28
    Trophy Points:
    113
    Location:
    Pasig, Philippines
    I'm a total noob when it comes to measurements but as far as I can tell placement may be a massive contributor to differences between results, esp. on rigs with pinnae like the EARS. given the greatest discrepancies are around the concha gain region with the angled HD800 having more of a difference around 1-2kHz this just makes sense to me.

    Could be variability in EARS rigs and headphones as well, cuz I don't think I can logic out why your approach to the problem didn't pan out presuming competence of operation.

    P.S.
    For whatever reason when I and others have attempted to apply the SBAF comp to HEQ files we've found there's something of a shelf between channels. @Josh83, @Denonic in the same thread below, maybe @Biodegraded too? I know @Philimon's acted that way when I applied the comp to his files, so maybe the unit the comps were based on had something funky going on with it?

    Ex: https://www.superbestaudiofriends.o...nsations-from-minidsp-files.7067/#post-236457
     
    Philimon and Biodegraded like this.
  3. E_Schaaf

    E_Schaaf MOT: Upscale Audio; recovering HFM addict

    Friend BWC
    Joined:
    Mar 28, 2017
    Likes Received:
    5,425
    Dislikes Received:
    7
    Trophy Points:
    93
    Location:
    Pasadena, CA
    Can confirm - have driven myself mad trying to fix channel imbalances via mods that weren't actually there for this reason. Mine deviate a bit in the 4-8k area too. Positioning and seal can make a massive difference too. I wish there was a flat-plate coupler by MiniDSP for headphones that was as easy to set up as EARS. I like the idea of the pinnae but ultimately it leads to inconsistent results, and also messes up CSDs and distortion readings.
     
    Philimon and Lyander like this.
  4. Lyander

    Lyander Too sensitive for SBAF

    Friend
    Joined:
    Feb 25, 2017
    Likes Received:
    7,398
    Dislikes Received:
    28
    Trophy Points:
    113
    Location:
    Pasig, Philippines
    Just be careful with the mics if you try to convert yours into a FPC, haha. The wiring on the capsules can be... delicate. Agreed that it sucks for CSD and distortion.
     
    E_Schaaf likes this.
  5. E_Schaaf

    E_Schaaf MOT: Upscale Audio; recovering HFM addict

    Friend BWC
    Joined:
    Mar 28, 2017
    Likes Received:
    5,425
    Dislikes Received:
    7
    Trophy Points:
    93
    Location:
    Pasadena, CA
    To be quite honest I actually appreciate the inherently flawed and slightly inaccurate results I have with EARS when it comes to my modding ventures. Forces me to rely more on my ears.
     
    Lyander likes this.
  6. cameng318

    cameng318 Facebook Friend

    Contributor
    Joined:
    Jun 14, 2020
    Likes Received:
    163
    Dislikes Received:
    1
    Trophy Points:
    33
    Location:
    U.S.
    My rig also has the 6 kHz dip for about 6 dB. It's probably an early batch, the serial number is 1080. The channels are mismatched on all compensations, so I only use its right channel. You could convert it into a simple FPC by taking off the 4 screws and taking off the silicon ear. It's a little too reflective around the mic, but it's great for matching channels when modding headphones. Maybe using the EARs as FPC can yield more consistency across units, it takes more measurements to confirm it.
     
    Philimon likes this.
  7. Cooper32

    Cooper32 Rando

    Contributor
    Joined:
    Nov 20, 2020
    Likes Received:
    69
    Dislikes Received:
    0
    Trophy Points:
    13
    Location:
    Wisconsin, USA
    @Suuup
    Might be a bit nit-picky but can you add labels and units to the axes? We all know they are FR dB as a function of frequency (Hz) but it'd be good to have this on the graphs for the sake of completeness.

    Thanks.
     
    Philimon likes this.
  8. Bill-P

    Bill-P Level 42 Mad Wizard

    Friend
    Joined:
    Sep 25, 2015
    Likes Received:
    4,056
    Dislikes Received:
    15
    Trophy Points:
    113
    Location:
    San Jose, CA
    This is actually the problem. I've looked at RAW results from 2 different EARS rigs, and they differ quite a lot. The end result is very different even if you use the same headphone and have carefully placed it on the rig. MiniDSP didn't really do a good job at addressing channel imbalance and evenness among the two sides. Heck, even on the same rig, I can measure the exact same driver (it's symmetrical, by the way) on either side, and I still get different results. :bow:

    So unless you have that exact same rig, there is no way for you to really get the exact same result. What you'll get is at most something that closely approximates the original result. Basically, you're fighting: headphone variation, rig variation, human error...

    ...all of that just to obtain "perfect-looking" frequency response graphs. I don't think that's worth it.

    On top of that, everybody will most likely end up with differing compensation curves because we each have a sound signature that we prefer, so in the end, an "optimum" compensation curve may not be desirable after all. I think what may make sense is a way for you to take RAW data, then apply a custom compensation curve file to it.

    I already have that capability on my website, but I haven't really pushed it up yet. Going to make a visual compensation curve tool so that people can tweak, tinker, and make their own compensation curves... so that they can tell us what "looks right" to them. And I think that may be the way to move forward.
     
  9. Philimon

    Philimon Friend

    Friend
    Joined:
    May 6, 2016
    Likes Received:
    408
    Dislikes Received:
    10
    Trophy Points:
    63
    Location:
    Portland
    The pretend ears on my EARS are actually not symmetrical. They are each uniquely physically shaped. What was minidsp’s reasoning for this? Perhaps they chose extra variation ahead of channel balance. I think thats reasoning for better averaging.
     
  10. Vtory

    Vtory Illogical Spock

    Friend MZR
    Joined:
    Sep 12, 2016
    Likes Received:
    6,384
    Dislikes Received:
    39
    Trophy Points:
    113
    Location:
    Greater Philadelphia Area, PA
    As many already pointed out, I'd also reiterate that EARS needs a healthy amount of effort in fine tuning with a bunch of representative headphones and one's own ears. My experience confirms both channel to channel and unit to unit variance (not to mention human variance). Default calibration files help but let's not trust them entirely.

    However I won't bitch much because "solid out of the box" ones (B&K, Gras, etc) are crazily expensive.. and my purpose is mostly for the fun. And I somehow managed to tune mine to my liking.

    That said I am a little concerned with the idea to collect random EARS measurements without strict validations. Could be meaningful if you can collect super many (n>30 for each model) because then we can play with math and stat (yeah!). Otherwise all sort of variability make upper treble results quite questionable.

    By the way, the OP reads to me that @Suuup own neither his EARS nor multiple headphones for scientific sacrifices (lol). Better get them for yourself as you have to understand all possible BS by yourself.

    Trust me. There was really, really a huge chasm between what I thought measurements (and their process) would be and what they actually were.
     
    Philimon and cameng318 like this.
  11. SickMind

    SickMind Rando

    Joined:
    Dec 3, 2020
    Likes Received:
    4
    Dislikes Received:
    0
    Trophy Points:
    3
    Location:
    Saint-Petersburg, Russia
    Just tried to upload measurements, got "Internal Server Error" message.
     
    Suuup likes this.
  12. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    Huh, that is odd. What does the file you tried to upload look like?
     
  13. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    I am not trying to create measurements myself. I am trying to provide a platform for people to share measurements. I know that EARS have many flaws, but nevertheless, people use them.

    Wrt to the mentions of channel imbalance, currently I've only looked at one channel (left). I am mostly concerned with the validity of my way of removing the SBAF compensation. I think it's fine personally, just seeking others opinions.
    The way I see it, by gathering multiple samples of multiple headphones, I can get rid of placement variation (multiple samples from a single headphone should cancel this out) and headphone variation (by having multiple different headphones). Of course, there will still be some error, but this should approach zero when you increase the amount of headphones/samples. It should essentially mimic the reference EARS that I have data for, and should thus produce results of almost same quality.

    [​IMG]
    Example here, comparing HD600 from Purr1n after I've removed the SBAF compensation, and a measurement of HD6XX by user elQuixxote.

    For the overall goal of the website, I agree that there will be variations between EARS rigs, so even the same headphone with the same placement on different EARS rigs will produce different results. In the ideal case, enough people would upload their measurements and a mean could be taken of all these measurements. There's still the problem that the EARS rig is not capable of actually producing completely valid measurements, and I'll skimp on the details here to not spread any kind of misinformation, but it's not "advanced enough", like the proper setups (B&K, Gras, etc)

    Because of this, I have partnered with Oratory1990, who has access to a proper setup, and can provide high quality measurements. These are also available on the site.
    I further plan to make a database specifically for high quality measurements, so we can collaborate with other partners who can produce these measurements.
     
  14. SickMind

    SickMind Rando

    Joined:
    Dec 3, 2020
    Likes Received:
    4
    Dislikes Received:
    0
    Trophy Points:
    3
    Location:
    Saint-Petersburg, Russia
    Suuup likes this.
  15. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    Do you have multiple channels in this file? The system doesn't currently support multiple channels. Can you try with only a single channel?
     
  16. SickMind

    SickMind Rando

    Joined:
    Dec 3, 2020
    Likes Received:
    4
    Dislikes Received:
    0
    Trophy Points:
    3
    Location:
    Saint-Petersburg, Russia
    Yes, 2 channels, left and right.
     
  17. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    This would be the reason. I've put in some error handling so it doesn't crash but instead informs the user to upload only 1 channel.
     
    SickMind likes this.
  18. Vtory

    Vtory Illogical Spock

    Friend MZR
    Joined:
    Sep 12, 2016
    Likes Received:
    6,384
    Dislikes Received:
    39
    Trophy Points:
    113
    Location:
    Greater Philadelphia Area, PA
    I don't disagree that much, although still being a little skeptical (as I surmise variability occurring not purely at random but endogenously via some covariates). But it's your site, totally up to you, heck fine to go for what you believe.

    Just a few minor opinions apart from the philosophical aspects
    • The site doesn't show any raw data downloading features. I don't think it fair to take raw (= rich) information and give back graphical outputs only (kinda summaries in some sense). Particularly, the site as is has no way if users want to compensate differently from harman/hpn/others (e.g., I never use any of them for myself). Personalization could be one way to solve, but eventually allowing users to download and work on their ends could be the best.
    • "Oratory-graphing" sounds just weird.. I believe categorizing by measurement rig types (e.g., XXX's DIY, EARS, GRAS, etc) could look less daunting.
    • Clicking some items and seeing "The community hasn't added any frequency responses" feel somewhat irritating. Better show only available items atm.
     
    Suuup likes this.
  19. Suuup

    Suuup Rando

    Joined:
    Jun 18, 2016
    Likes Received:
    45
    Dislikes Received:
    10
    Trophy Points:
    13
    All fair points.
    Wrt point 1: I plan to allow users to download their data as well. As for downloading other people's data, I have to think about that. One idea I'm toying with is creating essentially something like REW that allows you to play around with graphs in much more detail. Creating this wouldn't be a small task though.
    Point 2: I plan to categorize into different types of rigs, just like I've done for MiniDSP EARS, however there aren't many different kinds of rigs available. Oratory-grapher is specifically for measurements from Oratory, as he has a large collection of high quality measurements and is willing to share them with the community. He is also a partner in creating the site.
    Point 3: A few years ago I created https://docs.google.com/spreadsheets/d/10Rrh9IOZBKNOQjEbb3b9BB_l0vurApr2cWF6o3zo-vw/edit#gid=0 with help from the community. I plan to incorporate the information from this into the database as a starting point, and continue to expand this information database. My hope is that over time, we can get measurements for a majority of these headphones as well, but we have not officially "released" the site, it is still under development.
     
    Lyander and SickMind like this.
  20. SickMind

    SickMind Rando

    Joined:
    Dec 3, 2020
    Likes Received:
    4
    Dislikes Received:
    0
    Trophy Points:
    3
    Location:
    Saint-Petersburg, Russia
    Yep, works with single chanel well.
    Uploaded Utopia measurements.
     
    Suuup likes this.

Share This Page