HQPlayer vs SOX

Discussion in 'Computer Audiophile: Software, Configs, Tools' started by Woland, Aug 7, 2021.

  1. fastfwd

    fastfwd Friend

    Pyrate
    Joined:
    Aug 29, 2019
    Likes Received:
    988
    Trophy Points:
    93
    Location:
    Silicon Valley
    Ok, I looked at the code (in github, though, so not very closely).

    The table of filters has versions for 256x, 128x, and 64x 44100 frequencies. Your frequency must be greater than or equal to one of those, or it gives the misleading error message you saw.

    See here for the three clans-8 filter configurations:

    256x: https://github.com/mansr/sox/blob/master/src/sdm.c#L236C1-L257
    128x: https://github.com/mansr/sox/blob/master/src/sdm.c#L424-L445
    64x: https://github.com/mansr/sox/blob/master/src/sdm.c#L612-L633

    See https://github.com/mansr/sox/blob/master/src/sdm.c#L664 for the frequency check that follows a successful name match.
     
  2. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    I'm grateful but... maybe it's the scotch... I'm still a bit confused. What do you mean by

    I can't follow the code closely enough to understand what frequency means in this case (I assume sampling rate?) and what relationship it has to those numbers.
     
  3. fastfwd

    fastfwd Friend

    Pyrate
    Joined:
    Aug 29, 2019
    Likes Received:
    988
    Trophy Points:
    93
    Location:
    Silicon Valley
    Yeah, I was worried you'd say that.

    I assume that "frequency" is DSD sample rate, but I don't know where it's set. I'll look at the code some more after I finish the work I've been procrastinating on.
     
  4. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    A friend of mine did some digging and found that it's the input file frequency that needs to be greater than or equal to 44100 to use the filters. Except... the input file's frequency is equal to 44100 Hz. I even tried again with a 96kHz file with no luck.

    I really don't want to have to make an ASR account just to contact the guy who wrote this thing, but he is active over there.
     
  5. fastfwd

    fastfwd Friend

    Pyrate
    Joined:
    Aug 29, 2019
    Likes Received:
    988
    Trophy Points:
    93
    Location:
    Silicon Valley
    No, the test is for much higher frequencies than 44100. They're testing against DSD sample rates:

    64*44100 = 2822400 Hz
    128*44100 = 5644800 Hz
    256*44100 = 11289600 Hz
     
    Last edited: Oct 20, 2023
  6. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    That's what my friend and I eventually arrived at, too. So the input needs to be a DSD file already to use those filters. and without specifying the filters, I end up with a 1bit 352.8kHz file.

    Damn. I really wanted an offline DSD upsampler but it seems this thing is broken... or I'm doing something very wrong. But the commands I'm using came straight off of forum posts by the guy who made the fork.

    I'll bite the bullet and sign up on ASR to send the author a message.
     
  7. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    Well, the author of this sox fork doesn't appear interested in getting back to me, so I guess this little project is dead. I really wanted some DSD upsamples of my favorite albums to carry on my DAP, but alas, I cannot read C. I guess I'll have to settle for 24bit/352.8kHz with a stupidly high-tap sinc filter.

    I'm also going to experiment with HQPlayer OS later tonight with my USBridge Sig and Bifrost 2/64.

    EDIT: And, just as I give up, I come across a command that actually works:

    Code:
    sox input.flac output.dsf rate -v 2822400 sdm
    This produces a proper DSD64 file, as far as I can tell (I'm a bit confused about the way it's displayed on my DAP, as in the top right corner it says DSD64 but in the "now playing" window it lists: DSF 352.8kHz/1bit/5644Kbps).

    In either case, I'll convert a few albums and do some listening tonight.
     
    Last edited: Oct 31, 2023
  8. fastfwd

    fastfwd Friend

    Pyrate
    Joined:
    Aug 29, 2019
    Likes Received:
    988
    Trophy Points:
    93
    Location:
    Silicon Valley
    You can choose which noise-shaping filter to use by appending, e.g., "-f clans-8" to that command.
     
  9. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    I think something is still wrong here--while my DAP does recognize the file as DSD64 in format, it also displays a sample rate of 352.8kHz. I just compared and my "real" DSD files show 2.822MHz in the same place.
     
  10. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    Well, my DAP continues to insist that the sampling rate is 352.8kHz but I'm plowing ahead anyway, since other devices recognize them as DSD64 and they do sound quite good.

    Listening on Shanling M8 > Jotunheim 2 > ZMF Atticus.Bass definition and texture, instrument separation especially during chaotic passages. and resolve are all improved. I can't comment on treble or air since I'm using a new SPC cable (4.4 > XLR) that needs some burn in time, and I'm getting some stiffness/brittleness from that.

    Below is a simple bash script that goes through all flac files in the present working directory and processes them into DSD64 files. It took about two hours with ChatGPT to get this, because ChatGPT is awful at bash (but I'm worse). If any of the tracks clip, the volume is decreased by -0.1dB and starts over, so that every track on the album has the same volume decrease. This can take a while depending on the clipping, but the results are well worth it imo:

    Code:
    #!/bin/bash
    
    # Function to process a file with a specific volume level
    process_file() {
        local filename="$1"
        local volume="$2"
        local output_file="/home/atom/Dropbox/FLAX/upsampled/$filename.dsf"
        local input_file="$filename.flac"
        local clipping_warning="sox WARN rate: rate clipped"
    
        # Process the file with the given sox command
        sox_output=$(sox "$input_file" "$output_file" vol "$volume"dB rate 2822400 sdm -f clans-8 2>&1)
     
        # Check for clipping in sox_output
        if [[ $sox_output == *"$clipping_warning"* ]]; then
            volume=$(echo "$volume - 0.1" | bc)
            echo "Clipping detected in $input_file. Decreasing volume to ${volume}dB and starting over..."
            process_files "$volume"  # Start over with the decreased volume level
        else
            echo "Processed: $input_file"
        fi
    }
    
    # Main processing logic
    function process_files() {
        local volume="$1"
    
        local files=(*.flac)  # Get all .flac files in the current directory
        local total_files=${#files[@]}
    
        for ((i=0; i<$total_files; i++)); do
            local filename=$(basename "${files[$i]}" .flac)
            process_file "$filename" "$volume"
        done
    }
    
    # Start processing with initial volume level 0dB
    process_files "0"
    Just edit the output file path, go to the folder containing the flac files, and run the script.

    EDIT: In the meantime, a friend of mine who can read C will hopefully look into why the sampling rate appears to be 8x lower than it should for a proper DSD file. Hopefully he can sort it out and fix it, because if something is wrong and the sound quality gets even better, that's a win!

    EDIT2: My friend has been unsuccessful. @fastfwd is this sampling rate discrepancy something you think you can sort out? I suspect there's just a missing *8 somewhere. I sent another message to mansr on ASR but I didn't hear back from him after my first message so I doubt the second one will get a response.
     
    Last edited: Nov 6, 2023
  11. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    As it turns out, I was doing something wrong after all: as per the instructions in the man page, the file needs to be upsampled to a DSD sampling rate first. That still doesn't make a difference to my DAP, which still reads the files as 1bit 357.8kHz.

    Anyway, using a slight modification of @Garns 's command to upsample to a 2.8224 MHz file:

    Code:
    sox -V3 input.flac -b 24 -r 2822400 output.wav upsample 64 sinc 22050 -n 16777216 -L -b 0 vol 64
    yields a shitton of clipping. Really, any volume value above 1 yields a shitton of clipping. A volume of 1 yields an unlistenably quiet file.

    What am I doing wrong?

    edit: I was passing "sinc 22050", not "sinc -22050" which is correct. This was the cause of the clipping. It works now, but the files are truncated. Probably a problem with the way I'm parallelizing the processing. I'll figure it out.
     
    Last edited: Nov 13, 2023
  12. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    Well, f**k me. I am really out of my depths now: SoX is prodcuing truncated .wav files no matter what I do. For example, a song that is 9m16s long ends up with a .wav file that is 48s long. Another file that should be 11m28s ends up as 2m07s.

    When I run the upsampling in parallel, I get premature EOF warnings... but I get none of these when I run them in series, and the files are still truncated.

    When I use the -S flag to show current progress, it does show the correct timing on the file, it shows that it is processing all 9:16 of the song. But the resulting .wav file is still 0:48.

    This happens even when I don't specify the filter, e.g.

    Code:
    sox -S -V6 input.flac -b 24 -r 2822400 output.wav upsample 64 vol 64
    I've uploaded my fork to https://github.com/ohshitgorillas/sox where I've applied @Garns' modifications to the DSD-capable fork.

    Any troubleshooting help or advice would be greatly appreciated. I can at least confirm that this isn't a disk space issue.

    EDIT: mystery solved. don't know how I missed this one:

    Code:
    /home/atom/sox-dsd/sox/src/.libs/sox WARN wav: File too large, writing truncated values in header
    Cool, now I just need to figure out how to increase the maximum file size in sox. I do not understand how this goddamn DSD converter was supposed to work in the first place if the files need to be pre-upsampled to 2822.4kHz and sox won't make .wav files that large.

    EDIT2: Another mystery solved: WAV doesn't support file sizes above 4GB. WAV64 does, though, and that seems to work so far.
     
    Last edited: Nov 14, 2023
  13. Baten

    Baten Friend

    Pyrate
    Joined:
    Mar 18, 2018
    Likes Received:
    1,128
    Trophy Points:
    93
    Location:
    EU
    Yeah, many formats have 4GB (FAT32) limits.. :)
     
  14. ohshitgorillas

    ohshitgorillas Friend

    Pyrate
    Joined:
    Nov 27, 2015
    Likes Received:
    680
    Trophy Points:
    93
    Location:
    Sacramento, CA
    Well, as it turns out, I thought I was doing something wrong because my DAP, Shanling M8, displayed the wrong sampling rates when playing the file back. That led me to the intermediate WAV64 conversion phase, however, even still my DAP displays the wrong sampling rates. One album (Sunbather by Deafheaven) shows the correct sampling rate, another shows 1411.2 kHz, but most still show 357.8 kHz.

    I have come to the conclusion that my DAP software is wrong, especially given that both mediainfo and soxi disagree with my DAP:

    Code:
    FLAX/upsampled/Heilung - Lifa                                                                                                             
    ▶ soxi 01.\ Opening\ Ceremony.dsf
    
    Input File     : '01. Opening Ceremony.dsf'
    Channels       : 2
    Sample Rate    : 2.8224e+06
    Precision      : 1-bit
    Duration       : 00:00:56.87 = 160515648 samples ~ 4265.4 CDDA sectors
    File Size      : 40.1M
    Bit Rate       : 5.65M
    Sample Encoding: 1-bit Direct Stream Digital
    
    
    FLAX/upsampled/Heilung - Lifa                                                                                                             
    ▶ mediainfo 01.\ Opening\ Ceremony.dsf
    General
    Complete name                            : 01. Opening Ceremony.dsf
    Format                                   : DSF
    Format/Info                              : Direct Stream Digital Stream File
    Format version                           : Version 1
    File size                                : 38.3 MiB
    Duration                                 : 56 s 872 ms
    Overall bit rate                         : 5 645 kb/s
    
    Audio
    Format                                   : DSD
    Format/Info                              : Direct Stream Digital
    Commercial name                          : DSD64
    Format settings                          : Little
    Duration                                 : 56 s 872 ms
    Bit rate                                 : 5 645 kb/s
    Channel(s)                               : 2 channels
    Channel layout                           : L R
    Sampling rate                            : 2 822 kHz
    Compression mode                         : Lossless
    Stream size                              : 38.3 MiB (100%)
    
    This likely means that the intermediate phase of converting to WAV64 is unnecessary, and reviewing old forum posts by the author of the SoX fork confirms that FLAC can be converted directly to DSF. This doesn't explain the misleading message in the SDM add-on man page that says the file "must be previously upsampled to a high rate for DSD", but whatever.

    I guess it wasn't a complete waste of time since I learned something, but otherwise... yeah, a waste of time.
     
  15. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,331
    Trophy Points:
    113
    This is stupid. Just use a converter with an anti-alias filter with adequate band rejection at 44.1 and 48 kHz. There are many.
     

Share This Page