Audio callback function is not called in time sometimes

Hi All,
I’m developing a media player using SDL, and now I met the problem that
the audio callback function is sometimes not called in time, and cause the
audio a little fitful.
I did some investigation about this problem. I use such piece of code to
open the audio device:
wanted_spec.xxx = xxx;
wanted_spec.callback = audio_callback; //audio_callback is my
audio callback function
SDL_OpenAudio(&wanted_spec, &spec);
I put some log in audio_callback function, and find that when the
phenomenon occurs the audio_callback function is not called in time, which
cause a split of the audio. And continuously not in time called
audio_callback causes the fitful sound.

My OS is Windows XP, and generally this phenomenon occurs when there're

task switches, like focus on other windows, minimum/maximum a window or
something like that. And if there’s no such task switch, the sound is
smooth.
Do you know anything about that?–
Mine

Anyone know about that?
Here’s some data about the delay of audio_callback function:

  1. when it’s playing normally, the interval between two audio_callback
    function is about 25~31 ticks in SDL (get from SDL_GetTicks );
  2. when the sound is fitful, the interval between two audio_callback is
    totally variable from 40 to 80, or larger sometimes.

My question is, how does this problem come? And how to resolve it if
possible?On Nov 30, 2007 9:33 PM, Mine wrote:

Hi All,
I’m developing a media player using SDL, and now I met the problem
that the audio callback function is sometimes not called in time, and cause
the audio a little fitful.
I did some investigation about this problem. I use such piece of code
to open the audio device:
wanted_spec.xxx = xxx;
wanted_spec.callback = audio_callback; //audio_callback is my
audio callback function
SDL_OpenAudio(&wanted_spec, &spec);
I put some log in audio_callback function, and find that when the
phenomenon occurs the audio_callback function is not called in time, which
cause a split of the audio. And continuously not in time called
audio_callback causes the fitful sound.

My OS is Windows XP, and generally this phenomenon occurs when

there’re task switches, like focus on other windows, minimum/maximum a
window or something like that. And if there’s no such task switch, the sound
is smooth.
Do you know anything about that?

Mine


!!!

Just guessing here, and assuming you have a busy main loop running?

Do you have any SDL_Delay calls in your main loop? If not, and your app is using up all the CPU time it can get, Windows might tag your program as a resource hog and give it less CPU time especially when other processes are busy. Like minimizing/maximizing etc.

Hoping it?s of any help,–

Jukka-Pekka Manninen


From: sdl-bounces@lists.libsdl.org [mailto:sdl-bounces at lists.libsdl.org] On Behalf Of Mine
Sent: 4. joulukuuta 2007 16:00
To: A list for developers using the SDL library. (includes SDL-announce)
Subject: Re: [SDL] Audio callback function is not called in time sometimes

Anyone know about that?
Here’s some data about the delay of audio_callback function:

  1. when it’s playing normally, the interval between two audio_callback function is about 25?31 ticks in SDL (get from SDL_GetTicks );
  2. when the sound is fitful, the interval between two audio_callback is totally variable from 40 to 80, or larger sometimes.

My question is, how does this problem come? And how to resolve it if possible?

On Nov 30, 2007 9:33 PM, Mine wrote:

Hi All,
I’m developing a media player using SDL, and now I met the problem that the audio callback function is sometimes not called in time, and cause the audio a little fitful.
I did some investigation about this problem. I use such piece of code to open the audio device:
wanted_spec.xxx = xxx;
wanted_spec.callback = audio_callback; //audio_callback is my audio callback function
SDL_OpenAudio(&wanted_spec, &spec);
I put some log in audio_callback function, and find that when the phenomenon occurs the audio_callback function is not called in time, which cause a split of the audio. And continuously not in time called audio_callback causes the fitful sound.

My OS is Windows XP, and generally this phenomenon occurs when there're task switches, like focus on other windows, minimum/maximum a window or something like that. And if there's no such task switch, the sound is smooth. 
Do you know anything about that?


Mine


!!!

All sorts of things can create latencies that will result in audio drop outs.
As you mentionned in your first post: task swiches, windows minimisation, etc…

To try and avoid this problem:
Disable the power saving functions.
If you can, kill all processes that are necessary.
If you want to have task switches, increase the sound buffer length.

Christophe PallierOn 12/4/07, Mine wrote:

Anyone know about that?
Here’s some data about the delay of audio_callback function:

  1. when it’s playing normally, the interval between two audio_callback
    function is about 25~31 ticks in SDL (get from SDL_GetTicks );
  2. when the sound is fitful, the interval between two audio_callback is
    totally variable from 40 to 80, or larger sometimes.

My question is, how does this problem come? And how to resolve it if
possible?

On Nov 30, 2007 9:33 PM, Mine wrote:

Hi All,
I’m developing a media player using SDL, and now I met the problem
that the audio callback function is sometimes not called in time, and cause
the audio a little fitful.
I did some investigation about this problem. I use such piece of code
to open the audio device:
wanted_spec.xxx = xxx;
wanted_spec.callback = audio_callback; //audio_callback is my
audio callback function
SDL_OpenAudio(&wanted_spec, &spec);
I put some log in audio_callback function, and find that when the
phenomenon occurs the audio_callback function is not called in time, which
cause a split of the audio. And continuously not in time called
audio_callback causes the fitful sound.

My OS is Windows XP, and generally this phenomenon occurs when

there’re task switches, like focus on other windows, minimum/maximum a
window or something like that. And if there’s no such task switch, the sound
is smooth.

Do you know anything about that?


Mine


!!!


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


Christophe Pallier (http://www.pallier.org)

Thanks guys.
I sure have SDL_Delay in my main loop, or the CPU usage will be full.
I did further investigation, it looks like SDL_Cond_Wait/Signal takes the
most extra time, and SDL_Lock/Unlock_Mutex takes less extra time when the
problem occurs. I now use a while loop & SDL_Lock/Unlock_Mutext instead of
SDL_Cond_Wait/Signal, then it works better, there’s less audio drop. But
sometimes there still is.
Is it possible that SDL_Lock/Unlock_Mutex will take about 30~40 ticks?
(I’m sure that my threads won’t Lock a mutex for so long time.)On Dec 4, 2007 10:11 PM, Christophe Pallier wrote:

All sorts of things can create latencies that will result in audio drop
outs.
As you mentionned in your first post: task swiches, windows minimisation,
etc…

To try and avoid this problem:
Disable the power saving functions.
If you can, kill all processes that are necessary.
If you want to have task switches, increase the sound buffer length.

Christophe Pallier

On 12/4/07, Mine wrote:

Anyone know about that?
Here’s some data about the delay of audio_callback function:

  1. when it’s playing normally, the interval between two audio_callback
    function is about 25~31 ticks in SDL (get from SDL_GetTicks );
  2. when the sound is fitful, the interval between two audio_callback is
    totally variable from 40 to 80, or larger sometimes.

My question is, how does this problem come? And how to resolve it if
possible?

On Nov 30, 2007 9:33 PM, Mine wrote:

Hi All,
I’m developing a media player using SDL, and now I met the problem
that the audio callback function is sometimes not called in time, and
cause
the audio a little fitful.

I did some investigation about this problem. I use such piece of

code

to open the audio device:

    wanted_spec.xxx = xxx;
    wanted_spec.callback = audio_callback;    //audio_callback is

my

audio callback function

    SDL_OpenAudio(&wanted_spec, &spec);
I put some log in audio_callback function, and find that when the

phenomenon occurs the audio_callback function is not called in time,
which
cause a split of the audio. And continuously not in time called
audio_callback causes the fitful sound.

My OS is Windows XP, and generally this phenomenon occurs when

there’re task switches, like focus on other windows, minimum/maximum a
window or something like that. And if there’s no such task switch, the
sound
is smooth.

Do you know anything about that?


Mine


!!!


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


Christophe Pallier (http://www.pallier.org)


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


!!!

This should happen on the thread level rather than the process
level, and I think even Windows gets that pretty much right. :wink:

(What I’m saying is, as a non-vsync main loop burns all CPU it can
get, the OS will consider it a CPU hog and lower it’s priority, to
allow somewhat fair system wide time sharing. Meanwhile, the audio
thread blocks once per buffer, and usually doesn’t burn more than a
few % of the CPU time, so it keeps it’s “nominal” priority, or gets a
higher dynamic priority, depending on how the OS scheduler works.)

I think the problem is simply that this particular system is
misbehaving slightly. For Windows, not being able to go below 50 ms
audio latency is pretty common, and I wouldn’t expect it to work
reliably on your average system. Don’t go that low, at least not if
hardwiring it into the application…

If you really want to go that low (without resorting to pro
audio/studio system requirements), you should talk directly to the
shared output buffer, doing “mix-ahead” as new sounds are started,
while using more buffering (say, 200 ms) for “old” sounds. This gives
you close to zero latency for sound triggering with some risk of
glitches, but reliable, glitch free playback for anything that’s been
playing for more than those 200 ms or whatever.

Bad news (apart from this being a bit tricky to get right) is that
you’ll need to do an SDL deep hack, or bypass SDL audio entirely to
do this. The good news is that most SDL backends already use shared
memory I/O, so you might be able to piggy-back on that. (I’ve been
meaning to demonstrate how to do this some time, but well… Five
million projects…)

//David Olofson - Programmer, Composer, Open Source Advocate

.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
’-- http://www.reologica.se - Rheology instrumentation --'On Tuesday 04 December 2007, Jukka-Pekka Manninen wrote:

Just guessing here, and assuming you have a busy main loop running?

Do you have any SDL_Delay calls in your main loop? If not, and your
app is using up all the CPU time it can get, Windows might tag your
program as a resource hog and give it less CPU time especially when
other processes are busy. Like minimizing/maximizing etc.

[…]

I did further investigation, it looks like SDL_Cond_Wait/Signal
takes the most extra time, and SDL_Lock/Unlock_Mutex takes less
extra time when the problem occurs. I now use a while loop &
SDL_Lock/Unlock_Mutext instead of SDL_Cond_Wait/Signal, then it
works better, there’s less audio drop.

What are you using these for? You might want to try some lock-free
solution for passing “commands” to the audio thread, if that’s where
the problem is…

Is it possible that SDL_Lock/Unlock_Mutex will take about 30~40
ticks?

Well, not really, but they certainly increase the risk of you being
blocked, and whenever that happens, there’s a quite substantial risk
of various other threads running for a while before you get the CPU
back. You’re not on a real time OS, after all.

Also note that due to this dynamic priority deal that any normal OS
will have, using locks or similar to protect data actually gives you
a sort of priority inversion problem. It doesn’t help that the audio
thread is behaving and given high priority, if it has to wait on a
lock held by your CPU hog main thread…!

(As opposed to an RTOS, a normal OS doesn’t implement priority
inheritance and similar solutions to this problem, as priorities are
just considered a way of tweaking the timesharing a little,
basically. So, you have to work around it one way or another.)

//David Olofson - Programmer, Composer, Open Source Advocate

.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
’-- http://www.reologica.se - Rheology instrumentation --'On Tuesday 04 December 2007, Mine wrote:

[…]

I did further investigation, it looks like SDL_Cond_Wait/Signal
takes the most extra time, and SDL_Lock/Unlock_Mutex takes less
extra time when the problem occurs. I now use a while loop &
SDL_Lock/Unlock_Mutext instead of SDL_Cond_Wait/Signal, then it
works better, there’s less audio drop.

What are you using these for? You might want to try some lock-free
solution for passing “commands” to the audio thread, if that’s where
the problem is…

I use these stuff to synchronize the buffer read & write. I have two thread,
one for decoding and write the audio data to my buffer, the other for
reading the audio data from the buffer, so I use mutex/cond to protect the
buffer.
I would like to send the buffer to audio device with nearly 0 latency, which
means I need my audio callback function works as fast as possible, but I
find it sometimes takes more than 40 ticks, stuck in the SDL_Lock/Unlock.
Maybe I should not use any protection in my read thread so it would work
pretty fast?On Dec 5, 2007 4:29 AM, David Olofson wrote:

On Tuesday 04 December 2007, Mine wrote:

Is it possible that SDL_Lock/Unlock_Mutex will take about 30~40
ticks?

Well, not really, but they certainly increase the risk of you being
blocked, and whenever that happens, there’s a quite substantial risk
of various other threads running for a while before you get the CPU
back. You’re not on a real time OS, after all.

Also note that due to this dynamic priority deal that any normal OS
will have, using locks or similar to protect data actually gives you
a sort of priority inversion problem. It doesn’t help that the audio
thread is behaving and given high priority, if it has to wait on a
lock held by your CPU hog main thread…!

(As opposed to an RTOS, a normal OS doesn’t implement priority
inheritance and similar solutions to this problem, as priorities are
just considered a way of tweaking the timesharing a little,
basically. So, you have to work around it one way or another.)

//David Olofson - Programmer, Composer, Open Source Advocate

.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
’-- http://www.reologica.se - Rheology instrumentation --’


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


!!!

[…]

I use these stuff to synchronize the buffer read & write. I have two
thread, one for decoding and write the audio data to my buffer, the
other for reading the audio data from the buffer, so I use
mutex/cond to protect the buffer.

Using multiple threads and locking means you’re letting go of the CPU
every now and then - and when you do, you’re not always getting it
back ASAP. General purpose operating systems don’t behave very well
in this regard, and there is little we can do about that, short of
switching to a different OS.

If you need to do things like this, you need to use lock-free
solutions. You can never turn a non RTOS into an RTOS, but at least,
you can reduce the risk of drop-outs by not asking for more trouble
than you have to.

However, before even considering this (lock-free solutions are hairy
business), you should make sure your solution is theoretically
correct in the first place…!

I would like to send the buffer to audio device with nearly 0
latency, which means I need my audio callback function works as fast
as possible, but I find it sometimes takes more than 40 ticks, stuck
in the SDL_Lock/Unlock.

You can’t affect latency in that way. You can be en time, or you can
be late. If you’re on time everything works fine, whereas if you’re
late, you get a drop-out. That’s all there is to it, really.

The audio callback is called as soon as the driver has room for
another buffer of audio, and from that point, you have practically
the full playback time of that audio buffer (ie samples_per_buffer /
output_sample_rate) until you actually have to return. Returning
earlier has no effect whatsoever on latency.

That said, there is a reason to keep the audio callback light:
Finishing quickly means you have more margin for OS induced “coffee
breaks”. The callback may (or rather, will) sometimes fire a bit
late, and then, obviously, you have a better chance of still
finishing in time if you have less work to do. However, you shouldn’t
really care about this unless you’re using substantial amounts of CPU
power for audio processing. (Your average games sound engine would
use a few % of the CPU time available. That’s not an issue worth
considering in this regard.)

You cannot avoid the problem by moving the work elsewhere. All that
can ever do is make things worse. If you can’t deliver the data in
time, there is no way you can make the buffer deadline, no matter
what you do.

Maybe I should not use any protection in my read thread so it would
work pretty fast?

That would solve only one thing: It would avoid blocking the audio
callback. However, that would just mean your audio callback runs with
no input every now and then, so you still get your output ruined.
This is only of use when you have other streams playing at the same
time, or when you’re streaming from disk or similar. In the latter
case, you’d add some intermediate buffering between the
streaming/decoder thread and the audio callback, to allow for more
timing variations without causing drop-outs in the stream.

If you really need low latency all the way from input to output (which
is unusual in the context of codecs), the only real solution is to
just do all your sound processing in the context of the audio
callback.

//David Olofson - Programmer, Composer, Open Source Advocate

.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
’-- http://www.reologica.se - Rheology instrumentation --'On Thursday 06 December 2007, Mine wrote: