1.     Don't be a dotard!
    Read these "rules" AND introduce
    yourself     before your first post
  2. A healthy approach to understanding measurements
    Being true to what the artists intended
  3. After being asked for the 308th time, Merv lists top 5 headphones
    (opinion / entertainment piece)
    Dismiss Notice
  4. $1000 to the first to design a headamp using discrete components with better measurements than Magni 3+
    Dismiss Notice
IMPORTANT: New Members Read This Don't Be a Measurement Fascist Merv's Top 5 Headphones - 2023 Edition $1000 Challenge

Some simulated results from (single/mult)ibit and (non/over)sampling DACs

Discussion in 'Digital: DACs, USB converters, decrapifiers' started by Rynsin, Jan 17, 2018.

Page 1 of 2
  1. Rynsin

    Rynsin New

    Contributor
    Joined:
    May 8, 2016
    Likes Received:
    41
    Trophy Points:
    13
    Home Page:
    I've always wanted to know what sort of distortions, noise, etc. would be produced by various DAC configurations (e.g., multi-bit, filter type, oversampling, etc). Since it isn't practical for me to go buy hundreds of DACs (plus, I doubt I could find a one-bit non-oversampling DAC, for example), I've started to write some software to investigate the properties and behaviors of some theoretical DACs. While I'm not even close to done, I thought maybe some of y'all might be interested in my early results or be able to offer feedback.

    Questions
    • What is the effect of oversampling, or not oversampling, on the output of a DAC?
    • What is the effect of the bit count, either "pure" binary delta-sigma or multibit, on the general output of a DAC?

    DAC Design
    For those interested in the technical details, here's a description of the DAC I've implemented in software.

    The theoretical DAC that I've designed is very simple. It takes in input as 16-bit, 44.1Khz PCM. If the oversampling factor is greater than 1, the input signal is oversampled via a simple zero-order hold. Next, the signal is delta-sigma modulated: first, triangular dither is added, and then, for a given bit-count b, the potentially oversampled wave is quantized to the closest digital boundary (e.g., with a 1-bit DAC, values over (2^16)/2 are sent to 1, and values under (2^16)/2 are sent to zero). Then, the quantization error is passed through a simple 2nd order IIR noise filter that performs a standard rolling average ("slow rolloff"). The output of this filter is added to the next sample. Once the signal is produced, it is passed through a 5th order Bessel filter with a critical frequency of 28Khz.

    Here's a graphical representation of the DAC design:
    [​IMG]

    The simplest, 1-bit NOS DAC

    For illustrative purposes, consider a 1-bit DAC with no oversampling. Essentially, our DAC simply takes in an input signal and maps the value to a 1 if the input value is above the midpoint, and to a zero if the value is below the midpoint. Afterwards, the quantization error (the difference between the zero or one we emitted and the actual value) is added to the next input value. Finally, the signal is sent through a low-pass filter.

    As you'd expect, our 1-bit DAC doesn't perform very well. Here's an image showing how it reconstructs a simple 5Khz signal:
    [​IMG]

    The topmost plot shows the original image. The middle plot shows that 0s and 1s outputted by the quantizer, and the bottommost plot shows the resulting signal after a low-pass filter has been applied. Clearly, the reconstruction wasn't very good. To see just how not-good it was, we can take the FFT of the (filtered) output:
    [​IMG]

    This plot shows us that the 5Khz signal is indeed reproduced, but a significant amount of noise is also added (at a strength of almost 0.025 times the original signal). There's also some harmonic spikes (in theory, their spacing should follow a sinc pattern) that reach up to 0.075 times the original signal! The red line on the right represents the far limits of human hearing (28Khz). Clearly, a 1-bit NOS DAC performs very poorly. So, let's add some upsampling and see if we do any better.

    A 1-bit, 4x oversampling DAC
    Again, we'll feed the DAC a simple 5Khz signal and see how it reconstructs it:

    [​IMG]

    Wow! With just 4x oversampling, our DAC is already doing a way better job at reconstructing the input signal. There is clearly still a lot of distortion, but the quality is greatly improved from doing no oversampling at all. Here's the frequency response (this isn't the kind of frequency response you might be used to seeing -- this is showing the response from a single 5Khz input. The "correct" output would have only a single spike a 5Khz and everything else would be zero):
    [​IMG]

    The "noise floor" (using the term loosely here, as this isn't normally what that means) is much lower -- reduced by almost half -- as compared to doing only 1x oversampling. There is still a pesky harmonic spike around 15Khz, but overall, a major improvement. In the real world, you probably still won't find a 1-bit 4xOS DAC... Before going into multi-bit territory, let's look at a more reasonable design.

    A 1-bit, 32x oversampling DAC (similar to Modi2 USB)
    Stepping up to 32x oversampling gives us a DAC that starts to look a lot like the delta/sigma DACs used in something you'd actually buy, the like Modi2's AK4490 (which does 16x oversampling combined with AKM's "OSRD" technology gives an effective 32x oversampling). Let's look at the reconstructed signal...

    [​IMG]

    Now things are starting to look a bit better! While we can clearly see the group delay from the Bessel filter (the first cycle of the reconstructed wave is delayed, because it's phase has been shifted by some constant), our reconstructed wave is finally starting to look like a wave!

    [​IMG]

    In the frequency response, the "noise floor" has been substantially lowered again, and the size of our harmonic artifacts have also significantly decreased.

    A 1-bit, 256x oversampling DAC (Philips SAA7320 / CD-635)
    For better or for worse, you won't find a lot of 256x oversampling 1-bit DACs anymore, but they used to dominate the high-end back in the late 80s / early 90s (at least according Willenswaard &Atkinson). As you'd expect, a 256x oversampling DAC has even cleaner responses:

    [​IMG]
    [​IMG]

    The "noise floor" is now almost completely eliminated, but sadly our sinc artifacts are here to stay. Actual 1-bit D/S chips deal with these distortions with additional bandpass filters, but the design of these filters seems to (still!) be kept under proprietary control. Many academic papers detail how to design theoretically optimal filters to deal with these distortions, and the theoretical limit for their elimination suggests that they can be brought down to at most 0.0001 times the input signal -- that's pretty small! Who knows how well the actual implementations perform, though.

    So what if we stopped using just 0s and 1s, and gave ourselves a little bit more resolution in the quantization?

    An 8-bit, NOS DAC
    Now instead of representing the signal with 1-bit outputs, we'll represent the signal with 8-bit outputs. Now instead of having two different possible values, we have 256 different values.

    [​IMG]
    [​IMG]

    Wow! Even the unfiltered reconstruction looks pretty good, and the frequency response is amazing (I zoomed in by 10x on the Y-axis from the previous plots). There's still some noise, but it's been reduced to less than 0.0025 times the input signal! That's pretty amazing.

    Other DACs and conclusions

    If you'd like, I've created a PDF with frequency responses for a lot of DACs here: https://ryns.in/multibit_lpf.pdf As you can see, there does seem to be some benefit from oversampling with an 8-bit DAC. The benefits of oversampling seem to tail off pretty quickly once you get into 16-bit territory, though.

    Obviously, as I've mentioned before, most production DACs that you would buy off the shelf will have a more complex design than mine. Especially 1-bit DS DACs, which will almost always have some kind of sinc filter at the end of the chain to remove those pesky harmonic artifacts.

    However, I do think it's still pretty interesting to look at the multibit technologies in their barest form -- from my experiments, it really looks like adding more bits to the quantizer should make a difference.
     
    Rynsin, Jan 17, 2018
    #1
  2. mrflibble

    mrflibble Friend

    Pyrate
    Joined:
    Oct 4, 2016
    Likes Received:
    381
    Trophy Points:
    63
    Location:
    United Kingdom
    Very interesting! Wish I was smart enough to fully understand it. One small point, I think your frequency response graphs are missing their data - those in the PDF are fine however.
     
    mrflibble, Jan 17, 2018
    #2
  3. Rynsin

    Rynsin New

    Contributor
    Joined:
    May 8, 2016
    Likes Received:
    41
    Trophy Points:
    13
    Home Page:
    Oh no! I put the images in SVG format, maybe they aren't working in your browser? I just tried in Firefox and Chrome on my phone and they worked. But I'll replace them with regular ol PNGs sometime tomorrow. Sorry about that.
     
    Rynsin, Jan 17, 2018
    #3
  4. mrflibble

    mrflibble Friend

    Pyrate
    Joined:
    Oct 4, 2016
    Likes Received:
    381
    Trophy Points:
    63
    Location:
    United Kingdom
    The images of the frequency graphs are visible, but there is no data shown (no blue lines). Only the red line is visible. I have tried Chrome and Opera on my phone.
     
    mrflibble, Jan 17, 2018
    #4
  5. AllanMarcus

    AllanMarcus Friend

    Pyrate
    Joined:
    Oct 23, 2015
    Likes Received:
    2,982
    Trophy Points:
    113
    Location:
    Los Alamos, NM
    Home Page:
    I see all the graphs correctly using Safari on a Mac.

    I wonder is that sinc distortion at 15k (or similar distortions) represents the "digital glare" some people hear with a DS DACs.
     
    AllanMarcus, Jan 29, 2018
    #5
  6. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    91,556
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    On the 1-bit sampling rates, the delta-sigma DACs would actually internally sample at frequencies in the MHz region. For example, the "1-bit, 32x oversampling DAC (similar to Modi2 USB)" scenario, the 32x oversampling refers to the PCM oversampling, not the 1-bit processing. 32x 44.1kHz is hardly enough for 1-bit.

    The DAC converts the PCM to 1-bit (or in reality these days, a hybrid, but for the sake of conversation, let's simplify and assume 1-bit). Then this 1-bit signal is processed as a MUCH higher rate than PCM (multi-bit), usually in MHz because we are now processing in the 1-bit domain. Whether we work in PCM or 1-bit, bandwidth must not decrease.

    In other words, the simulations above for delta-sigma DACs do not reflect actual implementations.
     
    Last edited: Jan 29, 2018
    purr1n, Jan 29, 2018
    #6
  7. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    I'll put together something later maybe. I was expecting more out of band noise in your sim. And yes, usually higher OSR is better.

    EDIT: Actually, more out of band noise and the characteristics of the noise shaper may become more apparent if using a noise like input signal instead of a tone.
     
    Last edited: Jan 29, 2018
    ultrabike, Jan 29, 2018
    #7
  8. Rynsin

    Rynsin New

    Contributor
    Joined:
    May 8, 2016
    Likes Received:
    41
    Trophy Points:
    13
    Home Page:
    Ah, I see -- so there's an initial oversampler, which is performing an n-bit PCM to m-bit PCM, with m > 1, and then the oversampled sequence is passed into a DS modulator that performs additional "processing" -- by which I assume you mean some kind of noise shaping / filtering.

    I shouldn't have asserted that the DAC I simulated was similar to the Modi2 USB, since the chip itself is so opaque (not to mention whatever else Schitt packs in there). There's no such post-processing on the SAA7320, though -- that really is a 256x oversampling DAC, at least according to the folks I've talked to.

    Do you have any idea where I could find more detailed information on the design / basic specs of these DACs? Like you said, most of them (claim that they) are not just basic 1-bit oscillators either. We've got some high-end programmable oscillators in the EE lab which are documented end-to-end. I'll take a look at some of those to start.

    I can see how playing a single tone might make the results a little deceptive. Once I understand the inner-workings of these things a little bit better, I'll run the simulation again.
     
    Rynsin, Jan 29, 2018
    #8
  9. Gleb Lev

    Gleb Lev New

    Joined:
    Sep 16, 2017
    Likes Received:
    14
    Trophy Points:
    8
    Location:
    Minsk
    I guess for 1 bit its 2.8224 MHz (dsd format based on this).
    Also I agree with post above about using noise instead of the sine. Pink or white noise should do the job.
    There is a block diagram in datasheet wich could help in understanding structure of ak4490 a little.
    P.S. If you take sine with higher frequency you won't get such good thd on binary weighted dacs with nos, if I recall it correctly.
     
    Gleb Lev, Jan 29, 2018
    #9
  10. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    @Rynsin, there could be an initial oversampler for your model. But one could do a S/H.

    Anyhow, the output of a delta sigma does not have to be 1-bit. But I'll make it so below. OSR is 64.
    Either way, here is a classic first order:

    Code:
    % 1st order delta sigma
diff = zeros(2,length(x));
acc = zeros(2,length(x));
for i=2:length(x),

    % first stage
    diff(1,i) = x(i) - ds(1,i-1);
    acc(1,i) = acc(1,i-1) + diff(1,i) + dither(i);
 
    % output
    ds(1,i) = gain*(2*(acc(1,i)>0)-1);
end
    The input is x, and the output is ds(1,i).

    Here is a classic second order one:

    Code:
    % 2st order delta sigma
diff = zeros(2,length(x));
acc = zeros(2,length(x));
for i=2:length(x),

    % first stage
    diff(1,i) = x(i) - ds(2,i-1);
    acc(1,i) = acc(1,i-1) + diff(1,i);
 
    % second stage
    diff(2,i) = acc(1,i) - ds(2,i-1);
    acc(2,i) = acc(2,i-1) + diff(2,i) + dither(i);
 
    % output
    ds(2,i) = gain*(2*(acc(2,i)>0)-1);
end
    The input is x, and the output is ds(2,i).

    Here is a filter after the fact:

    Code:
    % analog output filter
[B, A] = butter(5,1/OSR);
out1 = filter(B,A,ds(1,:));
out2 = filter(B,A,ds(2,:));
    The parameter gain is set to 1. But the input signal needs to be bounded.

    Here is an input signal that can work with the examples above:

    Code:
    [B, A] = butter(7,0.5/OSR);
x=filter(B,A,randn(1,NT));
x = sgain*x/std(x);
    where sgain is set to 0.25.

    If you know Matlab the above should hopefully make sense. If not we can go through it.

    There are more interesting architectures than can be explored of course. The above yield classic and simple noise transfer functions (NTFs). But more interesting ones can be obtained.

    The advantage of the second order (and higher order) are lower signal pass band noise, at the expense of higher out of band noise.
     
    Last edited: Jan 30, 2018
    ultrabike, Jan 30, 2018
    #10
  11. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    Here are some plots of the results while running the above:

    Signal Output (after filtering):

    DS_Signal_Out.png

    Delta-sigma spectrum (before filtering) - For the noise one I'm proly not aligning things exactly right as there is still signal there, but too tired right now:

    DS_Spectrum.png

    Noise transfer functions (blue are expected, check the slopes):

    DS_Noise_NTF.png

    These are very rough and simple.
     
    Last edited: Jan 30, 2018
    ultrabike, Jan 30, 2018
    #11
  12. skem

    skem Friend

    Pyrate
    Joined:
    Nov 1, 2017
    Likes Received:
    1,913
    Trophy Points:
    93
    Location:
    Charles River
    What is the x-axis? Sample #?

    On PSD plots, i presume we don’t care about things above ~40,000 Hz. It would be nice to be able to see low frequency stuff better. Unless this is actually inverse sample time not Hz?
     
    skem, Jan 30, 2018
    #12
  13. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    In the first plot x-axis is sample number. It is a time domain plot.

    In the second and third plots the x-axis is frequency. All of them are PSD plots. I think I used pwelsh. May get better resolution with other windows.

    We do care about 40 kHz because we want to see the NTF. It shows that we implemented things right. Low frequency should look almost like the original.

    I may try another window and plot the PSDs of the input, and the two outputs on top of eachother. The time domain plots are there though. It's the first plot. There are much more samples of course, but I zoomed in so it's obvious that things are working. The unfiltered delta sigma output in time domain is not very interesting. It's just 1's and -1's.
     
    ultrabike, Jan 30, 2018
    #13
  14. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    I should also stress that it makes little sense to apply delta sigma to non-oversampling situations. I would go as far as saying that even w/o delta sigma, w/o oversampling one pays the price of complexity and distortion with the analog reconstruction filter. NOS has it's sound and it may not be bad. But it's hard to get a flat distortionless passband with it.
     
    ultrabike, Jan 30, 2018
    #14
  15. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    BTW, the last plots shows low frequency better.

    0 to about 20 kHz is the passband for both 1st and 2nd order delta sigma. And it's a flat white noise like signal, very close to the original.

    Outside that range one can see the modulated quantization noise rising in agreement with the 1st and 2nd order expected behavior.

    With the 2nd order, the modulated quantization noise is below around -60 dBr at 20 kHz and decreasing toward DC. But it raises much quicker and eventually surpasses the 1st order quantization noise out of band around 200 kHz. The 1st order seems at -40 dBr at 20 kHz and decreasing toward DC, but slower.

    This is all in the last plot.

    Again, I can use a different window and add more labels to make it easier to visualize.

    Edit: Here you go. A quick and dirty labeling of the plot:

    DS_Noise_NTF.png

    It is not as clear as I would want, but it says -60 dBr for the second order knee there.
     
    Last edited: Jan 30, 2018
    ultrabike, Jan 30, 2018
    #15
  16. Rynsin

    Rynsin New

    Contributor
    Joined:
    May 8, 2016
    Likes Received:
    41
    Trophy Points:
    13
    Home Page:
    I see, this makes sense. What's the relationship between the 1st/2nd/nth order modulation (which, correct me if I'm wrong, is essentially stacking two D/S modulators on top of each other -- you measure the error and add it to the next value to quantize, and then you do it again), and extending the number of taps on noise filter (e.g., looking farther back at the error values)?

    For example, instead of adding my last quantization error to my current target value, I could add 0.9 times my last error value plus 0.1 times the error value before that. In the time domain, this makes things converge slower, but it's less "jumpy". Is this extended noise filter strictly different than the multi-order D/S modulation, or are they related?

    (On an unrelated note, both my and your example noise filter are IIR -- the wiki page for noise shaping mentioned FIR filters as well, but I'm not sure what the practical purpose of that would be?)

    Also, I sort of assumed the Butterworth filter wouldn't see a lot of use in audio applications since it doesn't have linear phase, like a Bessel filter. Or is it close enough that it doesn't really matter?

    I'm a Python person myself, but I've got enough matlab experience that I understand what's going on. :) (Just have to remind myself that its 1-indexed). Thank you, this is very helpful!

    I see, this does make things more clear. Tricky stuff! Kinda sucks we are just in the dark in terms of guessing what we're actually buying with these things. This is helping me understand the manual for the equipment we have in the EE lab, though.

    After our next paper deadline I'll come back to this and make sure I can reproduce your results for my own education.
     
    Rynsin, Jan 30, 2018
    #16
  17. skem

    skem Friend

    Pyrate
    Joined:
    Nov 1, 2017
    Likes Received:
    1,913
    Trophy Points:
    93
    Location:
    Charles River
    If the PSD plots are in Hz (1/seconds), then I presume you had to have assumed some sample rate (e.g., 44.1 kHz). What did you assume? I'm guessing OSR * 44.1kHz = 2.8 MHz.

    [ I presume you assumed some sample rate because if they are in frequency (1/sample) then all the second-order noise is pushed into the audio band since the crossing point of roughly 5E5/samples * (44,100 samples/sec) = 11 Hz. Also, the passband cut-off would be in the wrong place. ]

    A second question is how are these filters implemented in real life. Many times they seem to be on-board the chips. Are they analog components on the chip die?

    I guess take away for most people is that the noise is pushed way out of the audio band... Then a followup question is whether this energy has the potential to excite lower harmonics in drivers, and should it be dealt with in a well-designed system?
     
    Last edited: Jan 30, 2018
    skem, Jan 30, 2018
    #17
  18. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    The second order is not exactly stacking two delta sigmas. Note the two stages are not decoupled. To see what's going on you could work with the linearized model which assumes quatization noise is a second input.

    I will give you the STF and NTF if you use a first order with an IIR on the quantized feedback. I do not think you will get the same thing as the second order above. There are many classes of DS, and the one you are proposing has been generalized as well. Like I said, gotta go to lunch, but will draw something for you later so you get a better feel for what you are doing. All that said, I would have to simulate your approach as well to get a feel of it's behavior. But again, I think it still pretty standard.

    One could play around with the filters. It is possible to get better performance close to the transition band of the NTF and STF at the expense of low frequency performance. Hell, one could make the STF and NTF behave like Butterworth or something.

    Butterworth filters have much better stop band rejection and shorter transition band than Bessel. To get the stop band performance of a Butter, you need a much higher order Bessel. High order analog filters can blow up due to component tolerances. They can even blow up numerically due to precision.

    Kool! :)

    You presume correctly. This is why some of the plots above go to 1.4 MHz (2.8 MHz sampling rate).

    Yup, I assumed a sample rate. Possible I set the corner frequency of the input signal to 12 kHz or so. I would have to check. But as far as the second order noise, it's pretty much outside the audio range. Again, right at 20 kHz is well below -60 dBr (relative to the signal passband). Possible I'm not following you.

    There are many implementations. Full digital delta sigmas are a bunch of transistor logic. But you could have full analog ones or hybrids. All of these would be on a chip die. Or could be programmed on an FPGA. Hell, they could be SW into a DSP and out of a GPIO pin or something.

    That's why you add an analog filter at the output of the delta sigma. You are not going to pass all those 1's and -1's directly into the driver. You pass them through a filter and things should look nice (out of band noise be gone - or as much as possible depending on the analog filter). See the first plot. That's time domain. That's the 1's and -1's after going through another Butterworth filter. The outputs look pretty close to the original bandlimited signal.

    Note multi-bit DS will exhibit lower out of band noise as well. Trade-offs...
     
    Last edited: Jan 30, 2018
    ultrabike, Jan 30, 2018
    #18
  19. skem

    skem Friend

    Pyrate
    Joined:
    Nov 1, 2017
    Likes Received:
    1,913
    Trophy Points:
    93
    Location:
    Charles River
    Thanks @ultrabike. I won’t pester you with more Qs; this is a great education.

    Some random thoughts: I find it remarkable that DACs still sound as different as they do given how good this reconstruction is.

    I looked at the ESS Sabre ES9038Pro, which is only 8x oversampling, but which nonetheless claims (with THD filters) THD at -120dB across the audioband.
    [​IMG]

    While power supplies and general good design remain important to noise, I presume these are solvable in any half-decent design. My ill-informed guess would be that all remaining art would be in the implementation of the reconstruction filters (if even that, given the on-chip ones may be nicely optimized) and any amplifier/buffer circuit needed to get to line level—which begs the question, is audio DAC design rapidly coming to a resting place where the reconstruction part is as good as humanly perceptible?
     
    skem, Jan 30, 2018
    #19
  20. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,976
    Trophy Points:
    113
    Location:
    Irvine CA
    I think there will always be a push for better. Also things can sound different for a variety of reasons. Implementation is one. Operating range when used with different devices. Many things can go well or wrong.
     
    ultrabike, Jan 30, 2018
    #20
Page 1 of 2

Share This Page