REL Subwoofer Measurements using Design of Experiments

Discussion in 'Portable and Other Gear Measurements' started by Puma Cat, Oct 18, 2017.

  1. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    Hi Guys,
    Back in 2010, I started to look at subwoofer integration with a pair of "mains" speakers using a statistical approach called Design of Experiments to optimize integration of subwoofers for a given room with a set of "mains" speakers.

    To start with here is an introduction to Design of Experiments.

    Description
    Design of experiments (DOE) is a powerful tool that can be used in a variety of experimental situations. DOE allows for multiple input factors to be manipulated determining their effect on a desired output (response). By manipulating multiple inputs at the same time, DOE can identify important interactions that may be missed when experimenting with one factor at a time. All possible combinations can be investigated (full factorial) or only a portion of the possible combinations (fractional factorial). Fractional factorials will not be discussed here.

    When to Use DOE
    Use DOE when more than one input factor is suspected of influencing an output. For example, it may be desirable to understand the effect of temperature and pressure on the strength of a glue bond.

    DOE can also be used to confirm suspected input/output relationships and to develop a predictive equation suitable for performing what-if analysis.

    DOE Procedure
    Acquire a full understanding of the inputs and outputs being investigated. A process flow diagram or process map can be helpful. Utilize subject matter experts as necessary.

    Determine the appropriate measure for the output. A variable measure is preferable. Attribute measures (pass/fail) should be avoided. Ensure the measurement system is stable and repeatable.

    Create a design matrix for the factors being investigated. The design matrix will show all possible combinations of high and low levels for each input factor. These high and low levels can be generically coded as +1 and -1. For example, a 2 factor experiment will require 4 experimental runs:

    [​IMG]

    Note: The required number of experimental runs can be calculated using the formula 2n where n is the number of factors.

    For each input, determine the extreme but realistic high and low levels you wish to investigate. In some cases the extreme levels may be beyond what is currently in use. The extreme levels selected should be realistic, not absurd. For example:

    Enter the factors and levels for the experiment into the design matrix. Perform each experiment and record the results. For example:

    [​IMG]

    Calculate the effect of a factor by averaging the data collected at the low level and subtracting it from the average of the data collected at the high level. For example:

    Effect of Temperature on strength:
    (51 + 57)/2 - (21 + 42)/2 = 22.5 lbs

    Effect of Pressure on strength:
    (42 + 57)/2 - (21 + 51)/2 = 13.5 lbs

    The interaction between two factors can be calculated in the same fashion. First, the design matrix must be amended to show the high and low levels of the interaction. The levels are calculated by multiplying the coded levels for the input factors acting in the interaction. For example:

    [​IMG]

    Calculate the effect of the interaction as before.

    Effect of the interaction on strength:
    (21 + 57)/2 - (42 + 51)/2 = -7.5 lbs

    The experimental data can be plotted in a 3D Bar Chart.

    [​IMG]

    [​IMG]


    This simple-minded example above shows that there is an interaction between temperature and pressure in the strength of the glue bond. This is one feature of DOEs that is particularly useful when looking at the effect of a number of factors and their effect on the critical functional response.

    Now that that is out of the way as intro, let's look at the specific experiment I had in mind in the next post.

    The experiment I was trying to reproduce was the one based on the graph from research from Harnan showing that listeners perceived a room as having flat room reponse when the measured frequency actually looked more like this. This is because our hearing is not linear at lower frequencies:

    [​IMG]

    Given this, I set out to to perform a set of experiment to see if I could determine the optimal settings of speakers, REL sub settings and other factors, like grilles on or off, speaker toe-in, port plugs, etc. that would optimize the in-room response.


    With that in mind, I set out to see what I could do with my set up to emulate that via DOE.

    System used at the time was: Conrad-Johnson Premier 17LS Preamp, Conrad-Johnson LP70S, Dynaudio Contour S3.4 speakers, REL R-305, M-Audio MobilePre USB mike preamp, Dayton calibrated measurement microphone with calibration file, and Room EQ Wizard Software. The Room EQ Wizard software generates and simultaneously measures the frequency sweep from 10 Hz to 20,000 Hz.

    The desired responses were to maximize 20 and 50 Hz repsonse in dB, and minimize the 150 and 500 Hz responses. These 150 and 500 Hz responses were nodes that I wanted to minimize, if possible.

    An example trace as my starting point for reference is shown. The purple trace is where I started with the sub in, the brown trace with the sub out, before optimizing things using the DOE approach. The following is the real-time, IN-ROOM measured response for the right speaker:

    [​IMG]

    The goal here was to maximize the 20-50Hz node to range, while smothing the 70, and minimizing the 105, 155 and 500 Hz node peaks, so as to emulate the JBL graph.

    The factors I used for the DOE were sub gain (as clicks up from zero), sub crossover, likewise clicks up from zero, plug or no plug in the speaker reflex port. So, four responses being measured as the result of 3 factors at two different levels (low, high as in the examples above shown).

    Setting up a full-factorial DOE in JMP (a stastical package), here are the experimental matrix I ran per JMP's output for the experimental design and the ACTUAL measurements for each frequency (as measured in dB by room Eq Wizard).

    [​IMG]

    The next post will be the analysis of the data...
     
  2. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    Here is the result of analyzing the 20 Hz response of the DOE (warning: large image)

    [​IMG]


    We can see from the actual by predicted plot that the R^2 and r^2 adjusted are strongly correlated at 0.97 and 0.94, respectively, indicating that our results fit our model pretty well. Also the leverage plots (the factors affecting the response (20Hz output), show that Sub gain has a large effect, and quite possibly that the interaction of sub gain and sub crossover may have an effect, as the p-value is just barely above p>-.05, which tends to suggest there may be a significant effect if we gather more data or relax our confidence level from 95% to 90%. In, fact, if we relax our alpha from 5% (the chance we are willing to accept that we are wrong) to 10%, then the Sub crossover point becomes significant. In addition, our analysis of variance with probability of F >0.0016, suggests that that this result occurred from our null hypothesis, that the model does not predict sub 20 Hz performance is very low.

    The interaction profile plots also suggest that there is a likely interaction between sub XO setting and sub gain, and the plot lines are not perfectly parallel, but appear to intersect.

    Looking at the Prediction Profiler from JMP, comparing Sub gain with Sub XO, we can see that the maximum 20 Hz response is obtained by the Sub gain at 12 clicks up, and sub set at zero clicks, and we could expect a level of 75.1875 dB.

    [​IMG]



    JMP is also cool because it will show you a 3-D image of two factors at one time as they affect the desired response in a 3-D cube, in this case, the 20 Hz response. As you can see, the upper left corner of the rectangle shows, which is max 20 Hz response is at a sub crossover of zero, and a gain setting of 12.

    [​IMG]


    These analyses show how you can utitlize DOE to predict with good confidence the response from setting the factors (crossover point and gain) to give the desired response at 20 Hz.

    I used this approach to determine where to set the subwoofer controls & speaker port plug for all the frequencies I wanted to optimize, then measured the BEFORE and AFTER responses, in this graph, BEFORE is BLUE and AFTER is GREEN. Note the similarity of the overall frequency response for the green trace to the ideal trace above from Harman labs.

    [​IMG]

    I then confirmed these settings by actual listening tests, and was very happy with the results. I am still using these settings today.
     
    Last edited: Oct 18, 2017
  3. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,775
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Nope. Not really. Your curve looks totally different. You need to align X and Y axis.
    FR.jpg

    Sorry, your ears and calcuations are better than Harmon's consumer research. Looks like you prefer accurate bass rather than Harmon MEGABASS.
     
    Last edited: Nov 20, 2017
  4. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    Hi Marvey. Thanks for input about aligning the X and Y axis; you are absolutely correct. I did confirm my DOE model by listening, which is critical when testing a model. After all, the DOE is only a model. As the statistiician George Box said, "All models are imperfect; some are useful". This is why they must be confirmed in the real world. And I do prefer accurate bass to MEGABASS.

    The nice thing about this approach is I could use the DOE to point me in the right direction with the subwoofer's settings and speaker's bass port plug, and then confirm the settings with listening tests. Trying to do this with OFAT (one factor at a time) approach would have been potentially confusing and very time-consuming.

    Cheers and thanks for the revised graph and input.
     
    Last edited: Nov 22, 2017
  5. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,775
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    As I like to say to say: all models are imperfect, Harmon's or Sean Olive's, especially so.

    A lot of old school tools, or tools based on old school principles are tried and true. I have found them to be extremely useful in predicting behavior, giving us a head start so we can get 90% there with a minimum of fuss.

    The Harmon targets seem to be marketing research, tuned to test subjects who took a two hour online crash course in ear training or people with untrained ears. The fact that many millennial redditors and people with facist proclivities see Harmon's "research" as the one true way, mainly because they are just out of school and the only thing they know is from books, not from doing, is disconcerting.

    Again, I see the Harmon models and Sean Olive's research as at worst, harmful to the hobby and at best, causing confusion and annoyance.

    All this shit on how stuff is supposed to sound, how neutral sounds like, target FRs for studios, movie theaters, speakers, has been figured out decades and decades ago. Ironically, JBL of yore was really good at it. Also, anyone who has diligently put together a home theater, two channel system, or built a speaker using any sort measurements probably has a better idea what sounds right more than Harmon research.
     
    Last edited: Nov 22, 2017
  6. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    Not familiar with Sean Olive. The Harman model was used as a "jumping off" point. The DOE then helped me to quickly and effectively end up where I wanted to be, in particular how to cancel the modes and nulls I originally had in the in-room frequency response. It was also fun to do; I was able to use this approach for a hobby rather than using it to teach scientists how to optimize a PCR reaction mix.
     
  7. murphythecat

    murphythecat GRU-powered uniformed trumpkin

    Pyrate Banned
    Joined:
    Apr 4, 2016
    Likes Received:
    1,201
    Trophy Points:
    93
    couldnt agree more with MArvey. the target curve of boosting the sub bass to my ears is plain wrong. I like flat sub sound.
    then harman take on acoustic is in many ways another nonsense.
     
  8. Hekeli

    Hekeli Facebook Friend

    Contributor
    Joined:
    Sep 28, 2015
    Likes Received:
    149
    Trophy Points:
    43
    Location:
    Finland
    I have no idea if Harman takes in account bass decay time etc, but surely things will sound a little different for someone with a shitty echo chamber vs super trapped dead room. Mine is the latter type and without even having a specific target in mind, my curve actually looks pretty much like Harman. Little help with a 80hz high shelf around -3-4dB, even if my small room already gives a natural 5dB boost around 20-50hz vs flat. There certainly isn't any bloat going on, super clean fast dry bass thanks to my 30hz tuned 15" "midwoofers"..

    Flat response would be like listening to a stock hd800, something I've also done in the past a lot. But there's zero tactility when required, especially so if you are a "quiet" listener. Everything is a matter of taste yes.
     
    Last edited: Nov 22, 2017
  9. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,775
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Bass doesn't work like that in terms of decay time. Wavelengths on the scale of tens of feet. Bass traps, unless they are the size of refrigerators don't work. Live vs. dead rooms affect higher up frequencies. Below 100Hz, bass "echo" is a becomes a matter of room modes. Certain narrow frequencies will be excited and others nulled from standing waves. Doubtful you will get a nice 5db boost across the board between 20-50Hz in your room. More likely than not, you will have two narrow peaks between 20-50Hz, and maybe even a dip, especially if you are only running one sub.

    No, HD800 is slightly light on sub and low bass. Should be 4-7db more at 40Hz.
     
    Last edited: Nov 23, 2017
  10. Hekeli

    Hekeli Facebook Friend

    Contributor
    Joined:
    Sep 28, 2015
    Likes Received:
    149
    Trophy Points:
    43
    Location:
    Finland
    Of course there are peaks and valleys, I was just smoothing out my description. But as I see, waterfall/RT60 can be different without frequency response itself changing much. Boomy long decaying bass vs dry quick bass, in the latter case "megabass" doesn't sound bad to me. I have 1000 litres of helmholz resonators and 20 feet of insulation tubes in my concrete basement. They make very little general frequency response changes, but RT60 <100hz drops from >2s to 0.5s. Day and night difference in bass quality.

    (My point is basically that frequency response doesn't tell exacly how the bass will sound, ignore all the itsy bitsy details. How this all relates to harman and whether it assumes something about listening environment, I do not know.)
     
    Last edited: Nov 23, 2017
  11. Poleepkwa

    Poleepkwa Friend

    Pyrate BWC
    Joined:
    Oct 1, 2015
    Likes Received:
    1,557
    Trophy Points:
    93
    Location:
    Finland
    @Puma Cat . That Harman Kardon curve for speakers you compared to looks quite a bit different than the one on Tyll's website.
    Mind pointing me to your source?

    [​IMG]

    This has much less bass than the one you have use for comparison. It looks lot more like the B&K curve, which Marvey uses.
     
  12. anetode

    anetode Friend

    Staff Member Pyrate
    Joined:
    Sep 25, 2015
    Likes Received:
    479
    Trophy Points:
    93
    Location:
    How did I get here
    Home Page:
    Gotta admit, I'm 97% to a hard-on looking at this.
    That's kind of a cynical way of looking at Olive's work, especially since you're familiar with the foundational 70s NRC research by Toole/Barton/etc.. There's clearly more to their approach than marketing research.

    Mind that I don't really agree with Olive's target curve, unless I'm trying to fall asleep, and PSB Speakers are a good value only to a point.

    The know-it-all-ism problem is the same short-sightedness that's been around since the advent of nerds. Best way to cure it is more data, not less.
     
  13. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    Hi Poleepkwa, you're correct. I went back and looked at my records and the "target" response curve I used as a basis for the DOE model was the JBL curve, not the Harman-Kardon one. I did this back in 2012, and it took me a while to go back and find the origin of the curve. Apologies for any confusion this may have caused. It did not impact the finding of my DOE, though. And I agree with Marvey, that I ended up with setting that put me more in the "more accurate/natural" sounding bass response.
     
  14. Puma Cat

    Puma Cat Friend

    Pyrate Contributor
    Joined:
    Sep 7, 2017
    Likes Received:
    869
    Trophy Points:
    93
    Location:
    East Bay, CA
    As a scientist and Design for Six Sigma guy, I completely agree. Plus, doing DOEs are fun! They provide "actual" insight rather than "hypothetical' insight that one might obtain running traditional hypothesis tests. Plus the ability to interrogate interactions of the factors impacting the response, and determine what settings to use to obtain optimal response is very powerful. Hope you liked the DOE! :D
     

Share This Page