Patch for SDL audio recording

SDL Development
1

Attached is a patch for audio recording in SDL. The patch is for SDL
1.2.3 but should apply against CVS too (last I checked, anyway). It
retains source compatibility with the SDL 1.2 audio playback API, so all
your old programs should still compile against and work with this
patched SDL (let me know if they don’t). It most definitely does not
retain binary compatibility, so I don’t expect this patch to make it
into SDL 1.2. I’m hoping that some sort of recording ability will make
it into 1.3.

Because I tried to retain source compatibility with the current API, I
could not make the most elegant modifications possible, and indeed the
patch is rather hackish and currently only works for DSP audio.

Here’s how you use it in your programs. With standard SDL audio playing,
you setup a callback in your audio spec that is called whenever SDL
needs a chunk of audio to fill its playback buffer. What I did was add a
second callback (callback_record) that SDL calls whenever it has a chunk
of recorded data (such as from a microphone) available for you. If you
don’t set the recording callback to anything, then SDL will behave
exactly as the unpatched SDL with playback-only capabilities. But if
there is a callback_record function, then /dev/dsp will be opened for
read-write and recording should work. Note that if you don’t want to do
any playback at all, you can choose not to set an audio play callback,
but still set a recording callback.

Feedback is definitely welcome. I’m particularly interested in people
just trying out the patch and reporting whether it works. I’m currently
working on adding wav file writing support so that you can actually do
something with all of the nice recorded audio data you get from this
patched SDL. I’ll post new versions here and possibly put up a web page
for the patch if anyone’s interested.

For what it’s worth, this patch is Copyright © 2002 Jim Henson’s
Creature Shop, and it is of course LGPL.–
Dan Helfman
Jim Henson’s Creature Shop
-------------- next part --------------
A non-text attachment was scrubbed…
Name: SDL-1.2-record.diff.gz
Type: application/x-gzip
Size: 5257 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20020409/f3faa0b9/attachment.bin

2

Thanks Dan. I’m sure many people will appreciate it. :slight_smile:

See ya,
-Sam Lantinga, Software Engineer, Blizzard Entertainment

3

[…]

Here’s how you use it in your programs. With standard SDL audio
playing, you setup a callback in your audio spec that is called
whenever SDL needs a chunk of audio to fill its playback buffer. What I
did was add a second callback (callback_record) that SDL calls whenever
it has a chunk of recorded data (such as from a microphone) available
for you. If you don’t set the recording callback to anything, then SDL
will behave exactly as the unpatched SDL with playback-only
capabilities. But if there is a callback_record function, then /dev/dsp
will be opened for read-write and recording should work. Note that if
you don’t want to do any playback at all, you can choose not to set an
audio play callback, but still set a recording callback.

Are you planning on supporting different input and output rates and
formats?

If not, I think it would be better to use only one callback, but add a
flag to request input. That could even be binary compatible. (If you’re
running on an older SDL version, opening for input will just fail.)

Unfortunately, there’s no real ‘flags’ field anywhere in the current API,
so you’d have to abuse the ‘format’ field or something. Looking at the
code, it seems that SDL just gives up if you give it a “bogus” sample
format - such as something with new flags ORed in.

Would it be safe to use bits 8…11, something like this?

#define	AUDIO_INPUT	0x0100
#define	AUDIO_OUTPUT	0x0200

Passing any of these flags to an older SDL version will just give you an
error return - and here’s the ugly part: If you use this new feature
you,ll have to be aware of the fact that SDL could potentially not even
bother to try to understand what you want. Specifying AUDIO_OUTPUT will
fail, even if it’s technically “supported”.

ORing AUDIO_INPUT to the sample format would give your callback input
audio data in the buffer, but would disable output. (SDL knows that
you’re using the new API as soon as you set one of these flags!)

Specify AUDIO_OUTPUT, or none of these bits, and you get the traditional
behavior.

Specify both AUDIO_INPUT and AUDIO_OUTPUT, and you get full duplex
operation; your callback will receive input data in the buffer, and is
expected to write back output data into it. (Do nothing, and you have
nice, “free” software monitoring of the input…! :wink:

How about that?

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Wednesday 10 April 2002 03:09, Dan Helfman wrote:

4

How about that?

I don’t think there’s a need to wedge it in. It’s probably best to look at
it for 1.3 so that we don’t have to force an inelegant solution. In the
meantime, the patch is there if needed right away.

(I haven’t looked at it yet, but I think it’s cool that someone sat down
to do this, btw.)

–ryan.

5

How about that?

I don’t think there’s a need to wedge it in. It’s probably best to look at
it for 1.3 so that we don’t have to force an inelegant solution. In the
meantime, the patch is there if needed right away.

I agree.

(I haven’t looked at it yet, but I think it’s cool that someone sat down
to do this, btw.)

Hear, hear. :slight_smile:

See ya,
-Sam Lantinga, Software Engineer, Blizzard Entertainment

6

Are you planning on supporting different input and output rates and
formats?

Right now I have the playback and recording sharing the same audio spec,
and so they have to share the same rate/format. (But of course SDL will
do transparent conversions if this shared rate/format is different than
what the hardware supports.) If anyone has a need for separate audio
specs for playback and recording, I can probably do that. And in fact,
doing so might even make some of my code cleaner… (I wouldn’t have to
make as many new “_record” variables.)

[…]

How about that?

This might work, but I’m with Sam and Ryan on this… Let’s wait for 1.3
to do it right. What I have now is just sort of to tide people over
until then and maybe get the ball rolling in terms of SDL audio
recording. But if you have any sort of other clever ideas to improve the
existing patch, or even for a potential SDL 1.3 audio redesign, I’d like
to discuss it.On Tue, 2002-04-09 at 20:35, David Olofson wrote:

//David Olofson — Programmer, Reologica Instruments AB

–
Dan Helfman
Jim Henson’s Creature Shop

7

At 07:18 PM 4/9/02 -0700, you wrote:

Thanks Dan. I’m sure many people will appreciate it. :slight_smile:

Anyone offers himself to port that patch to Win32 and other target
different from *x? :slight_smile:

Bye,
Gabry (gabrielegreco at tin.it)

8

It turns out that I misspoke about the full source compatibility of
this patch. Apparently you have to set the callback_record to NULL in
any program that doesn’t use the new recording callback. I didn’t notice
this earlier because I was using SDL_LoadWav to create the spec in my
test program, which conveniently zeros out the appropriate memory,
leaving the recording callback NULL. I don’t think this is a huge
problem since those people who apply the patch are likely to be using
the new recording functionality. But if I figure out a way around this,
I’ll definitely fix it.–
Dan Helfman
Jim Henson’s Creature Shop

9

It turns out that I misspoke about the full source compatibility of
this patch. Apparently you have to set the callback_record to NULL in
any program that doesn’t use the new recording callback. I didn’t
notice this earlier because I was using SDL_LoadWav to create the spec
in my test program, which conveniently zeros out the appropriate
memory, leaving the recording callback NULL.

Oops… Almost the same problem as with the glSDL “wrapper hack” - I’m
using the “unused1” field, which is always there, but isn’t cleared by
SDL. heh

I don’t think this is a
huge problem since those people who apply the patch are likely to be
using the new recording functionality.

Then again, if you apply this patch, you’ll break all SDL code that isn’t
aware of the change - even if you recompile it… Not a major problem on
Win32, but on Linux, you tend to have all libraries "in the system"
rather than in the same dir as the game, so you can’t easilly work around
this.

But if I figure out a way around
this, I’ll definitely fix it.

Well, my solution with flags would solve this problem, as well as the
binary compatibility problem with the SDL library… :wink:

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Wednesday 10 April 2002 23:21, Dan Helfman wrote:

10

How about that?

I don’t think there’s a need to wedge it in. It’s probably best to look
at it for 1.3 so that we don’t have to force an inelegant solution.

Right; my sugestion was just an attempt to avoid major headaches for
those that actually apply the patch.

In
the meantime, the patch is there if needed right away.

…but it forces you to patch and recompile all code that uses SDL on
your system, unless you give the patched SDL lib a different name, or
something.

(I haven’t looked at it yet, but I think it’s cool that someone sat
down to do this, btw.)

Agreed!

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Wednesday 10 April 2002 06:31, Ryan C. Gordon wrote:

11

Are you planning on supporting different input and output rates and
formats?

Right now I have the playback and recording sharing the same audio
spec, and so they have to share the same rate/format. (But of course
SDL will do transparent conversions if this shared rate/format is
different than what the hardware supports.) If anyone has a need for
separate audio specs for playback and recording, I can probably do
that. And in fact, doing so might even make some of my code cleaner…
(I wouldn’t have to make as many new “_record” variables.)

Ok. This should probably be done for the 1.3 version - but also keep in
mind that input and output that isn’t in sync is pretty hard to use, if
there’s any form of connection between input and output, not to mention
actual sample/sample synchronization.

Most professional audio APIs assume that you’ll use the same format
throughout the system, and there are several reasons for that; most
importantly avoiding sync problems and conversion. (Doing sample rate
conversions right and fast on a general purpose CPU is very hard.)

[…]

How about that?

This might work, but I’m with Sam and Ryan on this… Let’s wait for 1.3
to do it right. What I have now is just sort of to tide people over
until then and maybe get the ball rolling in terms of SDL audio
recording. But if you have any sort of other clever ideas to improve
the existing patch, or even for a potential SDL 1.3 audio redesign, I’d
like to discuss it.

I have plenty of ideas… :wink: (I think I have figured out a relatively
nice way of supporting shared memory “zero latency” mixing on the API
level, if nothing else.)

Either way, I still think breaking both binary and source code
compatibility is a bad idea, even for an unofficial patch - especially if
it can be avoided without too much work.

Anyway, are you seeing any problems with my approach, or should I go
ahead and implement it?

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Wednesday 10 April 2002 11:09, Dan Helfman wrote:

On Tue, 2002-04-09 at 20:35, David Olofson wrote:

12

I could have ported from DirectX, but I don’t have any serious native
development tools for Win32, so I’d have to cross compile… heh

I could probably hack a useless but fun demo application, though. :wink:

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Wednesday 10 April 2002 15:00, Gabriele Greco wrote:

At 07:18 PM 4/9/02 -0700, you wrote:

Thanks Dan. I’m sure many people will appreciate it. :slight_smile:

Anyone offers himself to port that patch to Win32 and other target
different from *x? :slight_smile:

13

Then again, if you apply this patch, you’ll break all SDL code that isn’t
aware of the change - even if you recompile it… Not a major problem on
Win32, but on Linux, you tend to have all libraries "in the system"
rather than in the same dir as the game, so you can’t easilly work around
this.

But if I figure out a way around this, I’ll definitely fix it.

Well, my solution with flags would solve this problem, as well as the
binary compatibility problem with the SDL library… :wink:

Okay, okay. I’ll try your approach. :slight_smile: My main concern though is that a
program using recording might not want to always be playing back audio
constantly. But with the single callback, any incoming audio data will
also get played back… unless the callback remembers to zero out the
recorded data after it is copied.On Wed, 2002-04-10 at 15:22, David Olofson wrote:

//David Olofson — Programmer, Reologica Instruments AB

–
Dan Helfman
Jim Henson’s Creature Shop

14

Right - but that’s not too different from most other APIs: if you request
audio output, you have to “drive” it as well, or Bad Things will happen.
The biggest difference between my suggested approach, and say, ASIO, is
that the latter would give you two different buffers, whereas we reuse
the same buffer, for API compatibility reasons. (We’d need a new callback
prototype otherwise.)

BTW, obviously, if you don’t want output at all, just don’t set the
AUDIO_OUTPUT flag. (I suggested that using AUDIO_INPUT only should
disable output.)

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Thursday 11 April 2002 01:05, Dan Helfman wrote:

On Wed, 2002-04-10 at 15:22, David Olofson wrote:

Then again, if you apply this patch, you’ll break all SDL code that
isn’t aware of the change - even if you recompile it… Not a major
problem on Win32, but on Linux, you tend to have all libraries “in
the system” rather than in the same dir as the game, so you can’t
easilly work around this.

But if I figure out a way around this, I’ll definitely fix it.

Well, my solution with flags would solve this problem, as well as the
binary compatibility problem with the SDL library… :wink:

Okay, okay. I’ll try your approach. :slight_smile: My main concern though is that a
program using recording might not want to always be playing back audio
constantly. But with the single callback, any incoming audio data will
also get played back… unless the callback remembers to zero out the
recorded data after it is copied.

15

Oh, and BTW, you should generally keep the audio device running all the
time anyway, as anything else is likely to produce clicks, unknown
latencies and other problems.

In fact, I don’t even think it’s “legal” to return from the current SDL
audio callback without doing something with the buffer. (AFAIK, that
would result in “looping” on most targets.) I can’t seem to find any
comments on that in the docs. Did I just miss it, or is this considered
too obivous to comment on? :wink:

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Thursday 11 April 2002 01:05, Dan Helfman wrote:

On Wed, 2002-04-10 at 15:22, David Olofson wrote:

Then again, if you apply this patch, you’ll break all SDL code that
isn’t aware of the change - even if you recompile it… Not a major
problem on Win32, but on Linux, you tend to have all libraries “in
the system” rather than in the same dir as the game, so you can’t
easilly work around this.

But if I figure out a way around this, I’ll definitely fix it.

Well, my solution with flags would solve this problem, as well as the
binary compatibility problem with the SDL library… :wink:

Okay, okay. I’ll try your approach. :slight_smile: My main concern though is that a
program using recording might not want to always be playing back audio
constantly. But with the single callback, any incoming audio data will
also get played back… unless the callback remembers to zero out the
recorded data after it is copied.