Creating audio samples via code

SDL Development
1

Hi guys,

We’re making a game where we need to time-stretch some audio samples. My
idea is to load the samples normally and then create the stretched
samples on the fly.

The thing is, I can’t find anything in SDL_mixer.h to create and
manipulate samples, except for the Mix_Chunk itself.

So two questions. First, if I just allocate a Mix_Chunk and its abuf
field and put stuff there, assuming the stuff is in the correct format,
should it work?

Second, would you be interested in a patch to add some functions to do
that? Say, Mix_CreateChunk(), Mix_GetSample() and Mix_PutSample()?

BTW, if someone has SDL_Mixer code to do time-stretching, you’ll save me
a lot of work :wink:

Thanks,

    --Gabriel________________________________________________________________________

Gabriel Gambetta
Mystery Studio - http://www.mysterystudio.com
Gabriel’s Stuff - http://www.mysterystudio.com/gabriel

2

Stretching the sound wave will change its pitch, and will sound differently.
You might want to make the sound longer in time, but probably the
same, then you’ll need high-tech algorythms to reconstruct the right
pitch.

This type of algorythm is used to strech the audio part of a movie to
help synchronise the movie with the sound without loosing any frames.
(And in fact i heard there was a patent on it, for movie playback that
is!).

If you wish to learn about this type of calculation, give a visit to
http://csounds.com/ a nice place to learn about audio theories.

Good luck,
SimonOn 5/21/07, Gabriel Gambetta wrote:

Hi guys,

We’re making a game where we need to time-stretch some audio samples. My
idea is to load the samples normally and then create the stretched
samples on the fly.

The thing is, I can’t find anything in SDL_mixer.h to create and
manipulate samples, except for the Mix_Chunk itself.

So two questions. First, if I just allocate a Mix_Chunk and its abuf
field and put stuff there, assuming the stuff is in the correct format,
should it work?

Second, would you be interested in a patch to add some functions to do
that? Say, Mix_CreateChunk(), Mix_GetSample() and Mix_PutSample()?

BTW, if someone has SDL_Mixer code to do time-stretching, you’ll save me
a lot of work :wink:

Thanks,

    --Gabriel

Gabriel Gambetta
Mystery Studio - http://www.mysterystudio.com
Gabriel’s Stuff - http://www.mysterystudio.com/gabriel

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

3

El lun, 21-05-2007 a las 14:28 -0500, Simon escribi?:

Stretching the sound wave will change its pitch, and will sound differently.
You might want to make the sound longer in time, but probably the
same, then you’ll need high-tech algorythms to reconstruct the right
pitch.

I know. That’s what I was referring to as “time-stretching” (as opposed
to resampling). Thanks for the link!

    --Gabriel________________________________________________________________________

Gabriel Gambetta
Mystery Studio - http://www.mysterystudio.com
Gabriel’s Stuff - http://www.mysterystudio.com/gabriel

4

Hi guys,

We’re making a game where we need to time-stretch some audio
samples. My idea is to load the samples normally and then create the
stretched samples on the fly.

Have you considered the alternative of (re)synthesizing the sounds in
real time? For example, use one or more looping waveforms along with
and a bunch of “transient” samples that are actually synchronized
with their respective visual events. If applicable, this method might
be simpler, less CPU intensive and/or better sounding than true
time-stretching.

The thing is, I can’t find anything in SDL_mixer.h to create and
manipulate samples, except for the Mix_Chunk itself.

Mix_QuickLoad_RAW()…? However, if you want to do this on-the-fly, in
real time, I don’t think this is the API for you. You could probably
abuse the Mix_Chunk as an intermediate buffer, allowing your
time-stretch code to run with a bit more relaxed timing (ie not
hard-sync’ed to the SDL audio buffer size), but you’d still need to
stay in sync with the channel playback position.

Is there some reason why your algorithm cannot generate N samples at a
time, on demand? I’m thinking about abusing the effect API. (See
Mix_RegisterEffect etc.) Keep the channel alive using a dummy looping
sample, or hack SDL_mixer to support channels starting with some sort
of constantly playing generator “effect”. Have your effect/generator
ignore the normal input and produce output based on it’s
own “private” data.

Second, would you be interested in a patch to add some functions to
do that? Say, Mix_CreateChunk(), Mix_GetSample() and
Mix_PutSample()?

Not sure if this is the correct type of interface for this - but then
I’m not sure I understand what you actually want to do. :slight_smile:

BTW, if someone has SDL_Mixer code to do time-stretching, you’ll
save me a lot of work :wink:

Sorry, haven’t played with time-stretching since the Amiga days, and
no code at all for SDL_mixer…

//David Olofson - Programmer, Composer, Open Source Advocate

.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
’-- http://www.reologica.se - Rheology instrumentation --'On Monday 21 May 2007, Gabriel Gambetta wrote:

5

Hi David,

Not sure if this is the correct type of interface for this - but then
I’m not sure I understand what you actually want to do. :slight_smile:

Thanks for your reply. I think I may have been somewhat unclear. I’m not
trying to synchronize the audio with anything else. I have an .ogg file
with a 2 or 3 second segment of Beethoven’s 5th, and I need to play it
faster or slower without altering the pitch. So yes, the
Mix_QuickLoad_RAW() solution may work.

We could precompute the time-stretched samples with sox or a similar
program, but we want to do it at runtime (not necessarily realtime) to
save disk and memory space.

Thanks,

    --Gabriel________________________________________________________________________

Gabriel Gambetta
Mystery Studio - http://www.mysterystudio.com
Gabriel’s Stuff - http://www.mysterystudio.com/gabriel

6

Is there some reason why your algorithm cannot generate N samples at a
time, on demand? I’m thinking about abusing the effect API. (See
Mix_RegisterEffect etc.) Keep the channel alive using a dummy looping
sample

I was going to recommend something similar, actually, but didn’t want to
be the first to offer it because it’s a Big Nasty Hack…but it’ll work.

David’s other solution would be way better, but would require further
development work in SDL_mixer itself: if there was going to be an
extension at all, it would be to have SDL_mixer fire a callback where
you generate the audio for the channel…much like what SDL’s audio
callback does, but you can have several independent ones running at once
that SDL_mixer mixes into a final output buffer. This way you could mix
and match premade audio with algorithmically generated noise…that’s
basically what was described with the Effects API, but it just cleans it
up into something formal and removes the need to have a buffer of
looping silence with an effect.

If you’re generating all your own audio, just use SDL directly and not
SDL_mixer…obviously, SDL_mixer is most useful if you want to fire off
some .wav files or whatnot and not worry about it too much. If you just
want to decode some .oggs, maybe SDL_sound will do, too.

–ryan.