SDL_MixAudioFormat distorting waveform

Hi, I’m trying for the first time to write my own waveform (a simple square with some ramping) and send it to my audio device, but running into some confounding problems at the very last stage of the process.

Context: I am on a little endian system, using 16 bit unsigned samples, which is what my device seems to want. So the zero amplitude of each sample should be 65535 / 2.

In my callback function, the samples I’m copying from to write to the stream are fine. I’ve printed and graphed them to confirm this. However, something goes wrong when I write them to the stream – or, depending on method, the problem occurs after I do the copy to the stream, but before my device plays them.

I’ve tried two methods of writing my samples to the stream: SDL_MixAudioFormat, and a simple memcpy. While the distortion is only visible in the stream waveform after calling SDL_MixAudioFormat, and presents no visual problem when using memcpy, the same distortions are still audible in the memcpy’d audio. I tested this by reducing the amplitude of the input signal to almost nothing, which had no effect on the amplitude (volume) of the output signal, even though the values in stream still show a small waveform.

This is all quite hard to articulate, so here’s an illustration:

The blue line is a fragment of my input waveform, which looks exactly as I expect. The orange line is the waveform produced by printing unsigned 16bit values directly from the stream after copying over my waveform with SDL_MixAudioFormat. I’ve confirmed that the format value I’m passing to that function specifies 16-bit, unsigned, little-endian samples.

Here’s my whole callback function, where you can see exactly how I’m printing the values that generate these waveforms:

int callback_i = 0; // for graphing

void audio_callback(void* userdata, Uint8* stream, int streamLength)
{
  AudioData* audio = (AudioData*)userdata;

	if(audio->length == 0) {
		return;
	}

	Uint32 length = (Uint32) streamLength;
	length = (length > audio->length ? audio->length : length);

  Uint16 zero_amp = 65535 / 2;

  for (int i=0; i<streamLength-1; i+=2) {
    memcpy(&(stream[i]), &zero_amp, 2);
  }
  
  printf("\nFormat Details => isSigned: %d, bitSize: %d, isBigEndian: %d", (Uint8) SDL_AUDIO_ISSIGNED(audio->format), (Uint8) SDL_AUDIO_BITSIZE(audio->format), (Uint8) SDL_AUDIO_ISBIGENDIAN(audio->format));

  /* Method 1: memcpy. Printed waveform from stream looks fine, but the audio still sounds clearly incorrect. */
  /* memcpy(stream, audio->pos, length); */

  /* Method 2: SDL_MixAudioFormat. Problems are visible in the printed waveform. */
  SDL_MixAudioFormat(stream, audio->pos, audio->format, length, 128);

  /* Graphing data */
  if (callback_i < 2000) {
    FILE* f1;
    FILE* f2;
    f2 = fopen("callback_origin.txt", "w");
    f1 = fopen("callback_waveform.txt", "w");
    for (int i=0; i<length; i+=2) {
      Uint16 sample_origin = 0;
      Uint16 sample = 0;
      memcpy(&sample, &(stream[i]), 2);
      memcpy(&sample_origin, &((audio->pos)[i]), 2);
      fprintf(f1, "%d\n", sample);
      fprintf(f2, "%d\n", sample_origin);
    }
    callback_i += 1;
  }
	audio->pos += length;
	audio->length -= length;
}

What am I misunderstanding or doing wrong? Any help would be tremendously appreciated!

EDIT:
Passing AUDIO_U16LSB explicitly to SDL_MixAudioFormat actually fixes the appearance of the orange waveform. However, the audio I’m hearing is very loud (and also, I think, distorted in a few other ways) regardless of how small I make the input signal amplitude. This is the same problem I have with memcpy. I had thought earlier on that this was due to some automatic normalization that SDL does behind the scenes, but it seems more likely that the waveform is getting distorted based on some format misunderstanding or disagreement somewhere.

Did you inspect the “obtained” SDL_AudioSpec after calling SDL_OpenAudioDevice or SDL_OpenAudio to make sure you got the format that you requested?

Unsigned audio formats have always confused me. Have you tried using AUDIO_S16LSB instead?

Hey Peter, thanks for your reply. Yes; I have this in the function that calls SDL_PauseAudioDevice:

  SDL_AudioSpec* wantSpec = malloc(sizeof(SDL_AudioSpec));
  SDL_AudioSpec* audioSpec = malloc(sizeof(SDL_AudioSpec));

  SDL_memset(wantSpec, 0, sizeof(*wantSpec));
  SDL_memset(audioSpec, 0, sizeof(*audioSpec));


  wantSpec->freq = 44100;
  wantSpec->channels = 1;
  wantSpec->samples = 4096;
  wantSpec->callback = audio_callback;
  Uint8* wavStart;

  SDL_AudioDeviceID device = SDL_OpenAudioDevice(NULL, 0, wantSpec, audioSpec, SDL_AUDIO_ALLOW_ANY_CHANGE);
  if (device == 0) {
    fprintf(stderr, "Error: failed to open device.");
  }

  SDL_CloseAudioDevice(device);
  free(wantSpec);
  Uint8 isBigEndian = SDL_AUDIO_ISBIGENDIAN(audioSpec->format);
  Uint8 isFloat = SDL_AUDIO_ISFLOAT(audioSpec->format);
  Uint8 isSigned = SDL_AUDIO_ISSIGNED(audioSpec->format);
  Uint16 sampleSize = SDL_AUDIO_BITSIZE(audioSpec->format);
  Uint32 sampleRate = audioSpec->freq;

  printf("\nisBE: %d, isFloat: %d, isSigned: %d, sampleSize: %d, sampleRate: %d, samples: %d, silence: %d", isBigEndian, isFloat, isSigned, sampleSize, sampleRate, audioSpec->samples, audioSpec->silence);

Which prints:
isBE: 0, isFloat: 0, isSigned: 0, sampleSize: 16, sampleRate: 44100, samples: 4096, silence: 0

Notably, if the isSigned value is correct, than the silence value is wrong. But there’s no documentation for the silence member, so maybe it can’t be trusted.

I haven’t tried forcing AUDIO_S16LSB, because my code is slapdash and unportable, and it’ll take some work :slightly_smiling_face: but that does seem like a reasonable next step.

Does it really want U16? We’re removing this from SDL3, because nothing uses it.

In the history of sound cards and video games, when they want to use a 16-bit integer format, it’s always been signed.

(and yes, the silence value for U16 is wrong, because silence would be 0x8000, but we can’t specify that in a single byte for memset()'ing a buffer, so we had to settle for 0x80, which will memset to 0x8080, which is almost silent. It’s a mess. Don’t use U16.)

No, it didn’t. I was being dumb:
Uint8 isSigned = SDL_AUDIO_ISSIGNED(audioSpec->format);
I ignored the significant bit by using Uint8 here instead of Uint16 :man_facepalming:

At least I learned a lot along the way! Thanks for your reply, I appreciate it.

1 Like