I was curious about the new rate conversion for audio in SDL2, specifically in SDL_ResampleAudioSimple. How does this compare to a linear resampler?
Thanks!
I was curious about the new rate conversion for audio in SDL2, specifically in SDL_ResampleAudioSimple. How does this compare to a linear resampler?
Here’s more information than you probably wanted. 
The resampler it replaced was not good, but it was very fast. The original SDL2 resampler was (no kidding) the Bresenham line drawing algorithm: use the same audio sample for a run-length of the “line”, move to the next when the “line” would bump up a row. Fast but low quality, as one would expect.
The new one is faster than the old one when downsampling, but slower when upsampling (because we read from the same piece of memory multiple times; probably fixable). On systems without floating point, the new one is probably taking a larger hit, too.
Unrelated to the resampler, we also made SDL’s internal audio pipeline use float32 samples; this simplified things immensely and guarantees us that the data will be in a format we can use SIMD instructions on. Previously, we had a perl script generate code for every possible format, and this turned out to be 15% of libSDL’s binary size, so dropping that was a win for several reasons. We’ve taken effort to put SSE2 code in where possible, to regain some speed we lost when breaking conversion back down into multiple steps. The old perl script would do all the steps in one place, with a separate function for each possible conversion, which meant a bloated library but you could do all the math without leaving the CPU registers in many cases. I hope to get NEON in there for ARM processors at some point, to match the SSE2 code.
We put a special case fast-path in there for resampling stereo Sint16 audio, though, which will probably cover the most common uses…so Raspberry Pi users, etc, don’t usually have to pay a floating point penalty here, we hope.
As for the quality: the new resampler is “good enough” for common use cases (moving from 44.1KHz to 48KHz, etc), but if you stretch it too far (go to like…1000Hz or 196KHz) you will definitely hear artifacts, getting progressively worse as you go. The old resampler would often fail (buffer overflows, etc) with sample rate conversions that weren’t powers of two.
If you build SDL with libsamplerate support (also new!) you get much better quality, but the resampler is more expensive–12x more expensive!–but often times the higher cost isn’t severe anyhow. libsamplerate is only usable for audio streaming (so SDL_AudioCVT can’t use it, since the API doesn’t keep state between blocks of data and would introduce “skips” in between blocks…I’d like to introduce a new API for streaming conversions soon), and the end-user needs to have libsamplerate installed for SDL to dynamically load the library (license issues!). Without libsamplerate, we always fall back to the “simple” new resampler.
It’s also worth noting that in terms of feeding an audio device, SDL would let the target API handle resampling if possible, so results were platform dependent but generally pretty good, and certainly better than what we offered. WASAPI, though, doesn’t offer resampling (before Windows 10), so that was a large part of the motivation to do all this audio subsystem redesign.
The new resampler isn’t very high-quality, but it’s “good enough” in most cases. Still, exploring the algorithm from https://ccrma.stanford.edu/~jos/resample/ is a todo list item that might replace the “simple” resampler at some point.
If you build SDL with libsamplerate support (also new!) you get much
better quality, but the resampler is more expensive–12x more
expensive!–but often times the higher cost isn’t severe anyhow.
libsamplerate is only usable for audio streaming (so SDL_AudioCVT can’t
use it, since the API doesn’t keep state between blocks of data and
would introduce “skips” in between blocks…I’d like to introduce a new
API for streaming conversions soon), and the end-user needs to have
libsamplerate installed for SDL to dynamically load the library (license
issues!). Without libsamplerate, we always fall back to the “simple” new
resampler.
By the way, libsamplerate has been released under 2-clause BSD license
last year.
That’s still not as liberal as SDL’s zlib license, but definitely makes
it more feasible to use or statically link than GPL (used before).
Maybe making it easy to statically link (or otherwise “compile into
SDL”) libsamplerate would be nice for most platforms (e.g. mobile,
Windows, OSX etc where the user can’t just do “apt-get install
libsamplerate0”)?
Cheers,
Daniel
Maybe making it easy to statically link
My hope is that it’s a stopgap until the next iteration of our resampler.
Thanks for all the info! I was mostly wondering about the quality of the resampler, and I think you answered that Where does the algorithm come from? I have been looking at other resamplers lately, and SDL is the only library with an averaging resampler (instead of point/linear/etc. resamplers).
Where does the algorithm come from?
I just made it up, which is why it’s not exactly professional quality.