Audio Engine question

Hi Everyone,

My name is Marc, I am new to the list as I just started to use SDL recently.
I am currently developping a tracker (yes, they still exist :slight_smile: and do a
combined version for GP32 (a korean portable game console), Windows, &
possibly others in the future as the code base is pretty much independent.
At first, the windows version was just a testing & debugging platform (the
main objective was to do a portable tracker) but recently some friends of
mine started to use the windows version so I thought I’d be nice and do a
better job for it.

I had no real trouble for the graphic part, after a day, I had a decent SDL
based version running in 320x240 full screen for that old school feel :slight_smile:

No comes the part of the audio. My current windows version uses winmm and,
although it works, it needs a huge pre buffering (hence lag) and I got no
real way (at least didn’t find one) to know what is currently playing which
makes the sync of audio and midi very troublesome.

To do a better job, I have two solutions: use SDLaudio or DirectX. I’ve been
looking at SDL_audio but it gave me the impression it was using winmm which
means I’ll end up with the same trouble with respect to lag & knowing where
I am in the audio stream. I’ve read in some post that there was a directX
based version but I didn’t really find it anywhere.

What would be the advice of the experts ? Is SDL up the part ? Since I am
doing all processing & mixing myself, all I need it the ability to queue
buffers (preferably of variable sizes) and know which buffer is currently
played…

Thanks
Marc
http://discodirt.10pm.org/

Hi M-._n,

I can’t help you with your question, but I thought you’d like to know
that there’s already an SDL port for the GP32.

cheers,
Florian

M-.-n schrieb:>Hi Everyone,

My name is Marc, I am new to the list as I just started to use SDL recently.
I am currently developping a tracker (yes, they still exist :slight_smile: and do a
combined version for GP32 (a korean portable game console), Windows, &
possibly others in the future as the code base is pretty much independent.
At first, the windows version was just a testing & debugging platform (the
main objective was to do a portable tracker) but recently some friends of
mine started to use the windows version so I thought I’d be nice and do a
better job for it.

I had no real trouble for the graphic part, after a day, I had a decent SDL
based version running in 320x240 full screen for that old school feel :slight_smile:

No comes the part of the audio. My current windows version uses winmm and,
although it works, it needs a huge pre buffering (hence lag) and I got no
real way (at least didn’t find one) to know what is currently playing which
makes the sync of audio and midi very troublesome.

To do a better job, I have two solutions: use SDLaudio or DirectX. I’ve been
looking at SDL_audio but it gave me the impression it was using winmm which
means I’ll end up with the same trouble with respect to lag & knowing where
I am in the audio stream. I’ve read in some post that there was a directX
based version but I didn’t really find it anywhere.

What would be the advice of the experts ? Is SDL up the part ? Since I am
doing all processing & mixing myself, all I need it the ability to queue
buffers (preferably of variable sizes) and know which buffer is currently
played…

Thanks
Marc
http://discodirt.10pm.org/


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

Thanks Florian;

My GP32 version does not really need SDL at the moment as I’ve got most
of what’s needed so far… I want it more to simplify the windows port &
prepare the GP2x one :slight_smile:

But does not seems too many poeple here care about audio :frowning:
Cheers,
Marc

Florian Hufsky wrote:> Hi M-._n,

I can’t help you with your question, but I thought you’d like to know
that there’s already an SDL port for the GP32.

cheers,
Florian

M-.-n schrieb:

Hi Everyone,

My name is Marc, I am new to the list as I just started to use SDL
recently.
I am currently developping a tracker (yes, they still exist :slight_smile: and do a
combined version for GP32 (a korean portable game console), Windows, &
possibly others in the future as the code base is pretty much
independent.
At first, the windows version was just a testing & debugging platform
(the
main objective was to do a portable tracker) but recently some
friends of
mine started to use the windows version so I thought I’d be nice and
do a
better job for it.

I had no real trouble for the graphic part, after a day, I had a
decent SDL
based version running in 320x240 full screen for that old school feel :slight_smile:

No comes the part of the audio. My current windows version uses winmm
and,
although it works, it needs a huge pre buffering (hence lag) and I
got no
real way (at least didn’t find one) to know what is currently playing
which
makes the sync of audio and midi very troublesome.

To do a better job, I have two solutions: use SDLaudio or DirectX.
I’ve been
looking at SDL_audio but it gave me the impression it was using winmm
which
means I’ll end up with the same trouble with respect to lag & knowing
where
I am in the audio stream. I’ve read in some post that there was a
directX
based version but I didn’t really find it anywhere.

What would be the advice of the experts ? Is SDL up the part ? Since
I am
doing all processing & mixing myself, all I need it the ability to queue
buffers (preferably of variable sizes) and know which buffer is
currently
played…

Thanks
Marc
http://discodirt.10pm.org/


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

–
M-.-n
http://discodirt.10pm.org

What would be the advice of the experts ? Is SDL up the part ? Since I am
doing all processing & mixing myself, all I need it the ability to queue
buffers (preferably of variable sizes) and know which buffer is currently
played…

You can’t know in the SDL API, but you can estimate fairly well, based
on how often the callback is triggered and how much you are feeding it
per callback iteration.

SDL should be using DirectSound by default, though (and waveout as a
fallback).

–ryan.

Ryan C. Gordon <icculus icculus.org> writes:

What would be the advice of the experts ? Is SDL up the part ? Since I am
doing all processing & mixing myself, all I need it the ability to queue
buffers (preferably of variable sizes) and know which buffer is currently
played…

You can’t know in the SDL API, but you can estimate fairly well, based
on how often the callback is triggered and how much you are feeding it
per callback iteration.

SDL should be using DirectSound by default, though (and waveout as a
fallback).

–ryan.

I got pretty much the same problem as I wanted to do
MIDI playback (which works) and in another thread I
tried to do a graphic equilyzer, that is updated every 20 ms
or so.

I also use the raw audio playback function, but unfortunately on
some computers it requests about 16000 bytes per call @ 16 kHz,
on other systems it only requests 4000 samples or even only 2048
(so the buffersize that you can set at initialisation does not
work on all computers with all cards).

I wonder if the number of audio blocks that is send to the audio
driver via the callback function before you hear anything in the
lsp is also dependend on the hardware/driver below SDL, does
anybody know that?

/BR

(Denis)-----

Indeed… I would not mind relying on the currently queued buffer if only I
could queue variable size and be able to have a musical accuracy in the
buffer’s boundaries… otherwise it is just going to be too screwy. Any
chances of having the len parameter being updatable to some length of data
to be queued (providing it is not bigger than the buffer size specified when
opening the audio) ? I’m not familiar with the development process so
forgive me if this is not the right place to ask;

Good to know the default behaviour is DS tho…

Cheers,
Marc> ----- Original Message -----

From: sdl-bounces+nostromo=arkaos.net@libsdl.org
[mailto:sdl-bounces+nostromo=arkaos.net at libsdl.org]On Behalf Of Ryan C.
Gordon
Sent: lundi 2 janvier 2006 12:43
To: A list for developers using the SDL library. (includes SDL-announce)
Subject: Re: [SDL] Audio Engine question

You can’t know in the SDL API, but you can estimate fairly well, based
on how often the callback is triggered and how much you are feeding it
per callback iteration.

SDL should be using DirectSound by default, though (and waveout as a
fallback).

–ryan.


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

M-.-n wrote:

Indeed… I would not mind relying on the currently queued buffer if only I
could queue variable size and be able to have a musical accuracy in the
buffer’s boundaries… otherwise it is just going to be too screwy. Any
chances of having the len parameter being updatable to some length of data
to be queued (providing it is not bigger than the buffer size specified when
opening the audio) ? I’m not familiar with the development process so
forgive me if this is not the right place to ask;

No, this won’t change; you have to feed the audio device at a constant
rate, or it would result in clicks, silence, etc.

What you CAN do is build something on top of that that you feed variable
chunks, and when the audio callback runs, it decides what needs to go to
the device at that moment (or silence, if there’s nothing to put to the
card).

SDL’s audio callback is necessarily low level and basic, though.

–ryan.

Indeed… I would not mind relying on the currently queued buffer if
only I could queue variable size and be able to have a musical
accuracy in the buffer’s boundaries…

That’s not how it’s normally done, regardless of API. The main reason
is that most hardware and some APIs don’t support anything but
power-of-two buffer sizes (64, 128, 256, 512, …).

Even if you can select “arbitrary” buffer sizes, there will usually be
a lower limit - usually because there are only two buffers,
regardless of buffer size - and this minimum buffer size may be far
too large for any usable musical timing accuracy.

So, you’ll have to support several “musical ticks” per buffer no
matter what.

otherwise it is just going to be too screwy. Any
chances of having the len parameter being updatable to some length
of data to be queued (providing it is not bigger than the buffer
size specified when opening the audio) ?

Well, there is always the option of implementing some sort of OSS-like
write() API over the SDL_audio API. However, that will add latency
due to intermediate buffering, and it may also increase the risk of
drop-outs, as a result of uneven CPU load across actual audio
buffers. I would strongly recommend against any such approach.

It’s best if you can have your sound engine generate exactly the
number of samples needed to fill one buffer. This will minimize CPU
load variations, and lets you avoid intermediate buffering.

Musical/tick timing can be implemented in a number of ways, but I
prefer to handle musical/control timing by means of timestamped
events, completely decoupling it from the buffer size.

Below is a stripped down version of the voice mixer (a_voice.c) from
Audiality. It handles sample looping and control events with sample
accurate timing. I added some comments regarding the handling of
buffer boundaries, event timing etc.

If you were to use tracker/mod style “step” timing, you’d replace the
event handling loop here with the mod “step” function (check if it’s
time to handle a new pattern line, update FX etc), and have the
following audio processing loop process all voices in one go.

(Audiality calls voice_process_mix() for one voice at a time, to avoid
a massive stream of events for one voice impacting all voices.)On Monday 02 January 2006 22:21, M-.-n wrote:


void voice_process_mix(A_voice v, unsigned frames)
{
unsigned s, frag_s;
…
/
Loop until buffer is full, or the voice is “dead”. /
s = 0;
while(frames)
{
/

* The while() loop below handles timestamped control
* events. When there are no more events for the
* current sample, frag_frames will be set to the
* number samples until the next event.
*/
unsigned frag_frames;
while( !(frag_frames = aev_next(&v->queue, s + aev_timer)) )
{
AEV_event *ev = aev_read(&v->queue);
switch(ev->type)
{
case VE_START:
…
case VE_START_NOMIXER:
…
case VE_STOP:
…
case VE_SET:
…
case VE_IRAMP:
…
}
aev_free(ev);
}

	/*
	 * Process until the next event, or the end of the
	 * output buffer, whichever comes first.
	 */
	if(frag_frames > frames)
		frag_frames = frames;

	/* Handle fragmentation, end-of-waveform and looping */
	frag_s = (VS_PLAYING == v->state) ? 0 : frag_frames;
	while(frag_s < frag_frames)
	{
		/* Start of this fragment in the buffer. */
		unsigned offs = (s + frag_s) << 1;

		/* Samples until next loop point */
		unsigned do_frames = endframes(v, frag_frames - frag_s);

		/* Generate a bunch of output samples! */
		if(do_frames)
		{
			..
			fragment_single(v, v->bus1->buffer + offs,
					do_frames);
			frag_s += do_frames;
			...
		}
		...
	}
	s += frag_frames;
	frames -= frag_frames;
}

}

Wait… I was working on an improved version of the simplemixer SDL
example a good while ago. It has a simple drum pattern sequencer that
runs inside the audio callback - and more interestingly, it runs the
sequencer every N samples, where N is any integer value, completely
independent of the SDL_audio buffer size.

I got sidetracked by some other ideas (graphical pattern editor and
stuff), but the thing compiles and runs, so I guess I could just
strip it down, wrap it up and release it.

The old version is found here, but I don’t think it contains anything
of interest to you:
http://olofson.net/examples.html

//David Olofson - Programmer, Composer, Open Source Advocate

.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,… |
`-----------------------------------> http://audiality.org -’
— http://olofson.net — http://www.reologica.se —

Hi, and thanks for the suggestion. The buffer size limitation is in
itself not a huge issue for the audio part. My root problem is that I
need synchronisation of two output streams that are independent and
have different latencies: Sound output & MIDI. If I want them to be
thight they need to be driven by a common timebase and I thought using
the audio stream would be best but it seems it is not.

What would be your alternative ?–
M-.-n
http://discodirt.10pm.org

It’s best if you can have your sound engine generate exactly the
number of samples needed to fill one buffer. This will minimize CPU
load variations, and lets you avoid intermediate buffering.

Musical/tick timing can be implemented in a number of ways, but I
prefer to handle musical/control timing by means of timestamped
events, completely decoupling it from the buffer size.

Hi, and thanks for the suggestion. The buffer size limitation is in
itself not a huge issue for the audio part. My root problem is that
I
need synchronisation of two output streams that are independent and
have different latencies: Sound output & MIDI. If I want them to be
thight they need to be driven by a common timebase and I thought
using the audio stream would be best but it seems it is not.

Using the audio stream for MIDI timing can be a good solution, but
only on an operating system that can handle very low latency audio
reliably. (You’d want around 1000 audio buffers/s, so you can deal
with MIDI events with roughly ms accuracy.) Vanilla Linux or Windows
won’t cut it, unless you’re fine with MIDI timing granularity and
jitter in the tens of ms range.

What would be your alternative ?

Well, there are basically two ways of doing it;
1) using an MIDI sequencer API with timestamped events,
or…
2) rolling your own sequencer that runs off some high
resolution timer.

Both Win32 and Linux supports a few variations of both methods.

Win32 has a “new” MIDI API with timestamping, and both OSS and ALSA
have sequencer APIs. (Though I don’t remember if the OSS sequencer
supports recording…) ALSAs sequencer API is pretty advanced, and
can be used for a lot more than plain MIDI I/O. The basic idea with
either of these is that input events are timestamped as they are
received (so you know when they were received even if you don’t read
them instantly), and output events are timestamped and enqueued,
leaving it to the the OS, driver and/or hardware to deliver them on
time. Pretty much like buffered audio I/O, that is, except MIDI is
structured variable rate data rather than raw streams of samples.

As to rolling your own, that’s obviously the most powerful method, as
it puts your own code right in the middle of the real time action,
which is perfect for real time MIDI effects and MIDI thru with
advanced routing and filters. However, it can be hard to get right,
and it doesn’t really buy you anything if all you need is plain
record and playback. Anyway, to drive your sequencer, you’d probably
use “mmtimers” on Win32, and the RTC on Linux. (Or high resolution
POSIX timers, if you’re on a Linux kernel that has those.)

Either way, the audio/MIDI sync problem still remains unsolved.
Buffered audio is no problem. Buffered/timestamped MIDI shouldn’t be
a problem, these days. But if they have no common time base, you
cannot translate back and forth, and thus, you cannot compensate for
latency or drift. To synchronize the two, you need information that
some APIs (including SDL_audio) do not provide in any reliable way.

You may get away with the infering tricks that old audio/MIDI
sequencers played (timing audio buffer delivery and stuff, trying to
calculate the actual latency), but those may not work reliably on all
systems. Your application may have to consult the user to tell what
works and what doesn’t.

Maybe you can ask the user to tap along with an audio metronome and
just measure where the resulting events land in relation to the audio
buffers…? Not exactly plug’n’play, but it would probably work just
about anywhere.

//David Olofson - Programmer, Composer, Open Source Advocate

.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,… |
`-----------------------------------> http://audiality.org -’
— http://olofson.net — http://www.reologica.se —On Tuesday 03 January 2006 10:28, M-.-n wrote:

Thanks for the excellent post. I guess I’ll try to revert off the idea
of audio-driven midi sync; and maybe work with external hi-)res timers
or something. Note that my applicaton has very crude midi resolution
(everything is hugely quantised) and I am totally ok with it. It might
make my life easier compared to full fledged support like the one you
are refering to.

Thanks
Marc

David Olofson wrote:>On Tuesday 03 January 2006 10:28, M-.-n wrote:

Hi, and thanks for the suggestion. The buffer size limitation is in
itself not a huge issue for the audio part. My root problem is that
I
need synchronisation of two output streams that are independent and
have different latencies: Sound output & MIDI. If I want them to be
thight they need to be driven by a common timebase and I thought
using the audio stream would be best but it seems it is not.

Using the audio stream for MIDI timing can be a good solution, but
only on an operating system that can handle very low latency audio
reliably. (You’d want around 1000 audio buffers/s, so you can deal
with MIDI events with roughly ms accuracy.) Vanilla Linux or Windows
won’t cut it, unless you’re fine with MIDI timing granularity and
jitter in the tens of ms range.

What would be your alternative ?

Well, there are basically two ways of doing it;

  1. using an MIDI sequencer API with timestamped events,
    or…
  2. rolling your own sequencer that runs off some high
    resolution timer.

Both Win32 and Linux supports a few variations of both methods.

Win32 has a “new” MIDI API with timestamping, and both OSS and ALSA
have sequencer APIs. (Though I don’t remember if the OSS sequencer
supports recording…) ALSAs sequencer API is pretty advanced, and
can be used for a lot more than plain MIDI I/O. The basic idea with
either of these is that input events are timestamped as they are
received (so you know when they were received even if you don’t read
them instantly), and output events are timestamped and enqueued,
leaving it to the the OS, driver and/or hardware to deliver them on
time. Pretty much like buffered audio I/O, that is, except MIDI is
structured variable rate data rather than raw streams of samples.

As to rolling your own, that’s obviously the most powerful method, as
it puts your own code right in the middle of the real time action,
which is perfect for real time MIDI effects and MIDI thru with
advanced routing and filters. However, it can be hard to get right,
and it doesn’t really buy you anything if all you need is plain
record and playback. Anyway, to drive your sequencer, you’d probably
use “mmtimers” on Win32, and the RTC on Linux. (Or high resolution
POSIX timers, if you’re on a Linux kernel that has those.)

Either way, the audio/MIDI sync problem still remains unsolved.
Buffered audio is no problem. Buffered/timestamped MIDI shouldn’t be
a problem, these days. But if they have no common time base, you
cannot translate back and forth, and thus, you cannot compensate for
latency or drift. To synchronize the two, you need information that
some APIs (including SDL_audio) do not provide in any reliable way.

You may get away with the infering tricks that old audio/MIDI
sequencers played (timing audio buffer delivery and stuff, trying to
calculate the actual latency), but those may not work reliably on all
systems. Your application may have to consult the user to tell what
works and what doesn’t.

Maybe you can ask the user to tap along with an audio metronome and
just measure where the resulting events land in relation to the audio
buffers…? Not exactly plug’n’play, but it would probably work just
about anywhere.

//David Olofson - Programmer, Composer, Open Source Advocate

.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,… |
`-----------------------------------> http://audiality.org -’
— http://olofson.net — http://www.reologica.se —


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

–
M-.-n
http://discodirt.10pm.org

David Olofson wrote:

  1. rolling your own sequencer that runs off some high
    resolution timer.

I routine I wrote to play midi files uses the audio sample count as a timer. Midi is parsed a bit at a time in a call from the synthesizer, every time a timing code is found the parsing code returning with a count of how many samples until the next midi event.

Planing to release code when I get it cleaned and debugged a bit more.