Feeding the audio buffer

Craig_Edwards · October 26, 2004, 5:21am

I am almost at the point where I can integrate my XBOX audio port into
SDL, however, I have a question about how audio buffers are normally fed
to the audio chip.

The way my code currently works is to trigger an interrupt when the audio
chip has finished processing each buffer. This invokes a callback
function which should give some more data to the chip… however, if you
only provide one buffer’s worth of data before starting to play,
everything will be slightly askew (ie. there is a gap between when the
chip is finished playing and when the callback gives it more data).
However, if you feed it, say, 3 buffers before playing, then the chip
should be playing the 2nd while the callback is setting up the 4th
buffer… yes?

How does SDL (or indeed, any audio library) handle this sort of scenario?
Any advice would be much appreciated. Thanks.–
Craig Edwards

Greg_Lee · October 26, 2004, 12:59pm

Craig Edwards wrote:

I am almost at the point where I can integrate my XBOX audio port into
SDL, however, I have a question about how audio buffers are normally
fed to the audio chip.

The way my code currently works is to trigger an interrupt when the
audio chip has finished processing each buffer. This invokes a
callback function which should give some more data to the chip…
however, if you only provide one buffer’s worth of data before starting
to play, everything will be slightly askew (ie. there is a gap between
when the chip is finished playing and when the callback gives it more
data). However, if you feed it, say, 3 buffers before playing, then
the chip should be playing the 2nd while the callback is setting up the
4th buffer… yes?

How does SDL (or indeed, any audio library) handle this sort of
scenario? Any advice would be much appreciated. Thanks.

For Linux/Alsa and Linux/OSS, SDL doesn’t need to handle it, since the
work is done by the kernel driver. When the sound system is opened
for output, you notify the driver about the “period” (Alsa) or
"fragment" (OSS) size, which is the amount of data you will send it
at one time, when it’s ready. A period/fragment is a portion of the
buffer that is sent to the sound chip when it generates an interrupt.
So it isn’t necessary to provide a callback – just send the kernel
driver a period/fragment’s worth of data whenever it’s ready.

Greg

Craig_Edwards · October 27, 2004, 11:19pm

Thanks for the reply. As I understand it, when the interrupt is
generated, the driver (ALSA/OSS/my XBOX code) is responsible for giving
the next buffer to the chip. This has two separate (but related)
challenges:

Where does the driver get the data from? You said “just send the
kernel driver a period/fragment’s worth of data whenever it is ready”… I
presume the “sender” in this case is the application code? If I
understand you correctly, you are saying the driver code is interrupt
based, but the application code is not. i.e. The application code just
gives buffers to the driver as they become available, and the driver gives
them to the chip upon interrupt. Is that right? How does this cope with
the fact that the application code might not be cooking the buffers fast
enough to keep up with the interrupts? My audio driver (and SDL, as I
understand it) has the capability to register a application code callback
to get more data if the driver has none left when an interrupt is
triggered.
How do we cope with possible lag between interrupt firing and setting
up of new data? This scenario might happen if the application code has
not provided enough data to the driver. For example, it has sent only one
buffer to the driver which then passed it on to the chip. The chip
processes that buffer, realises there are no more descriptors set up, so
fires an interrupt. The driver catches that interrupt and calls the
application callback for more data. However, there is a time-gap between
the interrupt firing and the application code providing more data, which
means the audio chip will either play silence (or the last sample). This
is what I am trying to avoid. My current thinking is that before playing
the very first sample, my driver could call the application callback a
couple of times to make sure it has a couple of buffers that can be
playing while waiting for the callback to provide the next buffer (an
audio form of double-buffering, I guess).

My problem is that this is not my area of expertise, so I am not sure if
what I am proposing is typical, not recommended, or just plain foolish.
Looking through the SDL code, I see that it appears (I think) to have the
notion of double buffering through the user of the fake_stream element.
However, I am just not sure…On Tue, 26 Oct 2004 02:59:53 -1000, Greg Lee wrote:

A period/fragment is a portion of the buffer that is sent to the sound
chip when it generates an interrupt.
So it isn’t necessary to provide a callback – just send the kernel
driver a period/fragment’s worth of data whenever it’s ready.

–
Craig Edwards

Craig_Edwards · October 27, 2004, 11:30pm

Actually further to my little monologue, I believe that the SDL stuff
isn’t actually interrupt-based, but rather, relies on a separate thread
that uses a timer to invoke the application callback. Compare this to my
code which invokes the callback based on the interrupt. I suppose I can
just not set the callback in my driver, because the SDL timer thread
should be invoking the callback on a regular basis, and thus ensuring the
driver has a constant input of data buffers.

Or at least that is my understanding… if I am incorrect, please feel
free to correct. Thanks.On Thu, 28 Oct 2004 09:19:51 +1000, Craig Edwards <@Craig_Edwards> wrote:

My problem is that this is not my area of expertise, so I am not sure if
what I am proposing is typical, not recommended, or just plain foolish.
Looking through the SDL code, I see that it appears (I think) to have
the notion of double buffering through the user of the fake_stream
element. However, I am just not sure…

–
Craig Edwards

Greg_Lee · October 28, 2004, 11:50am

Craig Edwards wrote:

A period/fragment is a portion of the buffer that is sent to the
sound chip when it generates an interrupt.
So it isn’t necessary to provide a callback – just send the kernel
driver a period/fragment’s worth of data whenever it’s ready.

Thanks for the reply. As I understand it, when the interrupt is
generated, the driver (ALSA/OSS/my XBOX code) is responsible for giving
the next buffer to the chip.

Yes, but let’s call it a period or fragment, instead of “buffer”,
because it’s only a portion of the output buffer.

This has two separate (but related)
challenges:

Where does the driver get the data from?

From the SDL output driver, which got it from an SDL output routine,
which got it from an application program, which calculated it somehow
and called the SDL output routine to play it.

You said “just send the
kernel driver a period/fragment’s worth of data whenever it is ready”…
I presume the “sender” in this case is the application code? If I
understand you correctly, you are saying the driver code is interrupt
based, but the application code is not.

It might be interrupt based, but it needn’t be. The Alsa library
routines let you set up callbacks to service the buffer, but you
don’t need to use callbacks. You (the application) can just
deposit a fragment into the buffer whenever there’s room for it.

i.e. The application code
just gives buffers to the driver as they become available, and the
driver gives them to the chip upon interrupt. Is that right?

Yes.

How does
this cope with the fact that the application code might not be cooking
the buffers fast enough to keep up with the interrupts?

Then the kernel driver tells you there was an underflow, and you just
have to deal with that.

I think the SDL library code handles this situation on behalf of an
application by putting silence into the kernel driver’s buffer,
so it’s happy and never knows about the underflow.

My audio
driver (and SDL, as I understand it) has the capability to register a
application code callback to get more data if the driver has none left
when an interrupt is triggered.

That sounds right, but I’ve never used either SDL’s or Alsa’s callback
facilities, and I’m not familiar with them.

How do we cope with possible lag between interrupt firing and
setting up of new data?

Well, that’s the problem, isn’t it? We cope the best we can. If
there’s a lot of calculation to do, it may just not be possible
for the application to feed the driver fast enough.

This scenario might happen if the application
code has not provided enough data to the driver. For example, it has
sent only one buffer to the driver which then passed it on to the
chip. The chip processes that buffer, realises there are no more
descriptors set up, so fires an interrupt. The driver catches that
interrupt and calls the application callback for more data. However,
there is a time-gap between the interrupt firing and the application
code providing more data, which means the audio chip will either play
silence (or the last sample). This is what I am trying to avoid. My
current thinking is that before playing the very first sample, my
driver could call the application callback a couple of times to make
sure it has a couple of buffers that can be playing while waiting for
the callback to provide the next buffer (an audio form of
double-buffering, I guess).

Alsa allows you to specify whether you want it to wait until the buffer
is full of fragments, on startup, before initiating the playback.
That’s a good option to choose. It helps prevent an initial stutter.

My problem is that this is not my area of expertise, so I am not sure
if what I am proposing is typical, not recommended, or just plain
foolish. Looking through the SDL code, I see that it appears (I think)
to have the notion of double buffering through the user of the
fake_stream element. However, I am just not sure…

I think the fake stream business concerns concealing underflows
by inserting silence into the output buffer, but I don’t
understand the details. I don’t think it’s necessary to
do it that way. You can just let the underflow happen, and
it you service the error you get from the kernel driver
appropriately, you will just get silence during the
dropout.

Greg> On Tue, 26 Oct 2004 02:59:53 -1000, Greg Lee <@Greg_Lee> wrote:

Craig_Edwards · October 28, 2004, 9:17pm

Thanks very much for the detailed reply. It is slowly starting to sink in

If I can indulge your patience one more time… I am curious as to how it
would work if the application is cooking fragments faster than the chip
can process. I know that AC97 has 32 available descriptors for
fragments… I wonder what happens if the driver gets 33 fragments from
the application code before the chip has finished playing the first one?
Would the driver buffer it internally? Or would is just overwrite the
next descriptor (which happens to be the currently playing one)? It
probably will never happen… I am just curious. Thanks again for your
patience and explanations.On Thu, 28 Oct 2004 01:50:51 -1000, Greg Lee wrote:

a lot of good stuff

–
Craig Edwards

Greg_Lee · October 28, 2004, 10:50pm

Craig Edwards wrote:

a lot of good stuff

Thanks very much for the detailed reply. It is slowly starting to sink
in

If I can indulge your patience one more time… I am curious as to how
it would work if the application is cooking fragments faster than the
chip can process.

Then it has to wait, giving the time up to the system for other tasks,
or find something else useful to do.

I know that AC97 has 32 available descriptors for
fragments… I wonder what happens if the driver gets 33 fragments from
the application code before the chip has finished playing the first
one?

The situation can’t arise (speaking only about Alsa, where the
term is “period”, not “fragment”). When you open, you choose
a buffer size, the number of periods, and a period size, and the
driver tells you what the size limits are. Once you’ve chosen,
protocol is to ask what you were actually given, in case it’s
not what you asked for. The periods have to fit inside the buffer
(I think). If you try writing a period when there’s not enough
free room in the buffer to hold it, the driver just says “No”
(-EAGAIN). It’s the driver that maintains the buffer. (You could
implement your own private buffer, of course, in addition to the
driver’s buffer.)

Would the driver buffer it internally? Or would is just
overwrite the next descriptor (which happens to be the currently
playing one)? It probably will never happen… I am just curious.
Thanks again for your patience and explanations.

Greg> On Thu, 28 Oct 2004 01:50:51 -1000, Greg Lee <@Greg_Lee> wrote:

Patrice_Mandin · October 29, 2004, 2:29pm

Le Thu, 28 Oct 2004 09:30:41 +1000
"Craig Edwards" a ?crit:

Actually further to my little monologue, I believe that the SDL stuff
isn’t actually interrupt-based, but rather, relies on a separate thread
that uses a timer to invoke the application callback. Compare this to
my code which invokes the callback based on the interrupt. I suppose I
can just not set the callback in my driver, because the SDL timer
thread should be invoking the callback on a regular basis, and thus
ensuring the driver has a constant input of data buffers.

Using threads or interrupts to feed the audio buffer is done by the
driver. For example, the MacOS classic audio driver (macrom subdir) and
the Atari ones (mint subdir), use an interrupt because threads are not
available on the OS they run.

The OpenAudio() function of these drivers return 1 instead of 0 for a
threaded audio feeder. Check also around line 400 in SDL_audio.c for some
IFDEFS that prevent creating a mutex when using interrupts.
Semaphores/mutexes/conditions are thread-related stuff, so you don’t want
it to mess your interrupt-based audio driver.

How do we cope with possible lag between interrupt firing and
setting up of new data? This scenario might happen if the
application code has not provided enough data to the driver. For
example, it has sent only one buffer to the driver which then passed
it on to the chip.

I have the same problem on Atari, but is caused by a different source,
which is that the CPU is too slow in some cases to feed the buffer before
the next interrupt comes. The only solution for me is to skip the
audio-feeding part of the interrupt, so I have time to do the next. SDL
does not provide any mechanism to the application to tell it that some
audio has been skipped.

The loopwave example program shows it: it just feed the data when the
callback is called. If the system is not fast enough, the replay will take
longer, as silent pauses (or replay of previous buffers) are introduced.
And I don’t speak about the conversions done in the audio callback because
the sample is not in the format the harware waits, and consume CPU time.

It could be nice for SDL to tell the application audio callback that
audio buffers have been skipped, so the application can resynchronize its
audio output. Something like FPS-independent video output, but for audio,
because the SDL_GetTicks() and SDL timers are not precise enough to do
this.–
Patrice Mandin
WWW: http://membres.lycos.fr/pmandin/
Programmeur Linux, Atari
Sp?cialit?: D?veloppement, jeux

Craig_Edwards · November 2, 2004, 6:54am

Using threads or interrupts to feed the audio buffer is done by the
driver. For example, the MacOS classic audio driver (macrom subdir) and
the Atari ones (mint subdir), use an interrupt because threads are not
available on the OS they run.

On the XBOX, I have both threads and interrupts available to me. The
challenge is figureing out which is the best way to integrate my audio
library into the SDL audio. To me, interrupt based seems simpler, but
then again, I freely admit I know diddley-squat!

The OpenAudio() function of these drivers return 1 instead of 0 for a
threaded audio feeder. Check also around line 400 in SDL_audio.c for some
IFDEFS that prevent creating a mutex when using interrupts.
Semaphores/mutexes/conditions are thread-related stuff, so you don’t want
it to mess your interrupt-based audio driver.

I saw those #ifdefs, but there seems to be some overlap between the MINT,
macintosh and ENABLE_AHI #defines. I wasn’t sure if they were
specifically related, or if they just happened to be needing to doing the
same general behaviour. I found that the windib audio used threads and
semaphores, so I have tried to follow that model, but have had no luck.
As you are no doubt aware, it is very difficult to debug two threads with
a couple of semaphores and mutexes, and interrupts firing a zillion times
a second

Anyway, thanks for your feedback. Much appreciated.On Fri, 29 Oct 2004 16:29:35 +0200, Patrice Mandin <mandin.patrice at wanadoo.fr> wrote:

–
Craig Edwards

slouken · November 12, 2004, 11:38pm

I am almost at the point where I can integrate my XBOX audio port into
SDL, however, I have a question about how audio buffers are normally fed
to the audio chip.

The way my code currently works is to trigger an interrupt when the audio
chip has finished processing each buffer. This invokes a callback
function which should give some more data to the chip… however, if you
only provide one buffer’s worth of data before starting to play,
everything will be slightly askew (ie. there is a gap between when the
chip is finished playing and when the callback gives it more data).
However, if you feed it, say, 3 buffers before playing, then the chip
should be playing the 2nd while the callback is setting up the 4th
buffer… yes?

How does SDL (or indeed, any audio library) handle this sort of scenario?

Typically an SDL audio driver requests 2 buffers at whatever sample size
is requested by the application and then pre-fills them with silence and
then double-buffers filling one while the audio hardware is reading the
other.

e.g.
Buffers A and B are both 4K large (1024 samples of 16-bit stereo data)

A and B are allocated and filled with silence
Audio hardware is triggered to play back A
Application gets a callback to fill B
Audio driver waits for A to finish playing and triggers playing B
Application gets a callback to fill A
…

This scheme assumes that the application is able to fill the buffers in
time, which is usually true with current CPU and OS combinations.

See ya,
-Sam Lantinga, Software Engineer, Blizzard Entertainment