Transferring Decoded Audio

Hello, (last one sent from the wrong address…)

I crossposted this on the ffmpeg mailing lists, but upon thinking
today I realized that since my problem might involve threads, or
likely has something to do with getting audio to SDL (and since you
guys respond infinitely better to requests than anyone on the ffmpeg
lists), I’d give it a shot here, too. Sorry if anyone received this
already.

I’m using the ffplay source to try to help me with decoding and
playing back audio using SDL and the libavcodec and its associated
libraries.

I’m using C++, and I have a class which maintains the ffmpeg
interface-type code (AVCodecContext, etc…) and runs a thread to
grab the next frame whilst the current one draws. The video fetching
code works fine, but I’ll post it anyway for completeness’ sake.
Here’s the threaded function:

int VE_Media::thread_func() {
mMutex = SDL_CreateMutex();

while( mThreadLives ) {
	if( needNextFrame ) {
		int frameFinished;
		SDL_mutexP(mMutex);	//lock
		av_read_frame(pFormatCtx, &packet);// >= 0) {
			if(packet.stream_index == videoStream ) {
				// Decode video frame
				avcodec_decode_video(pCodecCtx, pFrameYUV, &frameFinished,
										packet.data, packet.size);
				
				if(frameFinished) {
					sws_scale( pSwsCtx, pFrameYUV->data, pFrameYUV->linesize,
								0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize );
				}
			} else if( packet.stream_index == audioStream ) {
				mAudioBuffer = (int16_t *)av_fast_realloc(mAudioBuffer,  

&mAudioBufferSize, VEMAX(packet.size, AVCODEC_MAX_AUDIO_FRAME_SIZE));
mAudioDataSize = mAudioBufferSize;
int result = avcodec_decode_audio2( mAudioCodecCtx,
mAudioBuffer, &mAudioDataSize,
packet.data, packet.size );
if( result == -1 )
printf( “error decoding audio\n” );
//else
// printf( “decoded audio packet just fine!\n” );
}
av_free_packet( &packet );
SDL_mutexV(mMutex); //unlock
}
SDL_Delay(1);
}

printf( "VE_Media:  thread dying\n" );
SDL_DestroyMutex( mMutex );

return 0;

}

…the code I call to grab the audio buffer is as follows:

int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
if( audStreamExists ) {
audioBuffer = mAudioBuffer;
return mAudioDataSize;
} else
return NULL;
}

…In my sound class, I have created the callback function for SDL’s
audio, basing it largely on the code within ffplay.c:

void VE_Sound::audio_callback( void *userdata, Uint8 *stream, int
len ) {
int dLen;

//grab audio from all media objects and mix together
if( mMediaList.size() > 0 ) {
	while( len > 0 ) {
		//check if we need to grab more audio...
		if( mAudioBufferIndex >= mAudioBufferSize ) {
			mAudioBufferSize = mMediaList[0]->getCurrentAudioFrame 

( mAudioBuffer );
if( mAudioBufferSize < 0 ) {
mAudioBufferSize = 1024;
memset(mAudioBuffer, 0, 1024);
} else {
//synchronize for audio to transfer…
//…look at ffplay.c for help
}
mAudioBufferIndex = 0;
}
dLen = mAudioBufferSize - mAudioBufferIndex;
if( dLen > len )
dLen = len; //is this even necessary?

		memcpy(stream, (uint8_t *)mAudioBuffer + mAudioBufferIndex, dLen);
		len -= dLen;
		mAudioBufferIndex += dLen;
		printf( "length of audio buffer: %i\nnew index: %i\n",  

mAudioBufferSize, mAudioBufferIndex );
}
}
}

…What’s happening is my program invariably crashed on the “memcpy
()” function call, stating EXC_BAD_ACCESS, which I think means that
I’m trying to write/read memory from an array which is not assigned
to that array.

If someone could offer me some help on this, I would greatly
appreciate it. I’m sure it’s something I’ve missed with either
threading or memory/data shuffling that I’ve screwed up.

– Scott Harper

As an update, I managed to get SOME sort of audio, though I’m not
sure it’s quite what I expected. I slowed down the rate of decoding
frames (was not actually waiting for me to need the net one at the
time) so that it depends on decoding the audio first, and the sound I
get is very annoying, almost like an amplified electric buzz (like
the kind heard under power lines sometimes, only MUCH louder).

The code I changed to get SOME kind of audio (rather than an outright
crash) was:

int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
if( audStreamExists ) {
audioBuffer = mAudioBuffer;
return mAudioDataSize;
} else
return NULL;
}

replaced with

int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
if( audStreamExists ) {
int16_t *copy = (int16_t *)malloc( mAudioDataSize );
memcpy( copy, mAudioBuffer, mAudioDataSize );
audioBuffer = copy;
needNextFrame = true;
return mAudioDataSize;
} else
return NULL;
}

I tried throwing in a free() before calling this (inside
VE_Sound::audio_callback(…) to avoid a memory leak, but I get an
error saying I called free() too many time on one variable, so I’ve
probably created a horrific memory leak, and the sound I’m getting is
basically garbage, so I’m sure I’m still doing something VERY wrong.

Anything would help.
Thanks,
– Scott

(PS: Sorry for the top-post; I couldn’t think of a better way to
post an update, though.)On Feb 2, 2007, at 5:00 PM, Scott Harper wrote:

Hello, (last one sent from the wrong address…)

I crossposted this on the ffmpeg mailing lists, but upon thinking
today I realized that since my problem might involve threads, or
likely has something to do with getting audio to SDL (and since you
guys respond infinitely better to requests than anyone on the ffmpeg
lists), I’d give it a shot here, too. Sorry if anyone received this
already.

I’m using the ffplay source to try to help me with decoding and
playing back audio using SDL and the libavcodec and its associated
libraries.

I’m using C++, and I have a class which maintains the ffmpeg
interface-type code (AVCodecContext, etc…) and runs a thread to
grab the next frame whilst the current one draws. The video fetching
code works fine, but I’ll post it anyway for completeness’ sake.
Here’s the threaded function:

int VE_Media::thread_func() {
mMutex = SDL_CreateMutex();

while( mThreadLives ) {
if( needNextFrame ) {
int frameFinished;
SDL_mutexP(mMutex); //lock
av_read_frame(pFormatCtx, &packet);// >= 0) {
if(packet.stream_index == videoStream ) {
// Decode video frame
avcodec_decode_video(pCodecCtx, pFrameYUV, &frameFinished,
packet.data, packet.size);

  			if(frameFinished) {
  				sws_scale( pSwsCtx, pFrameYUV->data, pFrameYUV->linesize,
  							0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize );
  			}
  		} else if( packet.stream_index == audioStream ) {
  			mAudioBuffer = (int16_t *)av_fast_realloc(mAudioBuffer,

&mAudioBufferSize, VEMAX(packet.size, AVCODEC_MAX_AUDIO_FRAME_SIZE));
mAudioDataSize = mAudioBufferSize;
int result = avcodec_decode_audio2( mAudioCodecCtx,
mAudioBuffer, &mAudioDataSize,
packet.data, packet.size );
if( result == -1 )
printf( “error decoding audio\n” );
//else
// printf( “decoded audio packet just fine!\n” );
}
av_free_packet( &packet );
SDL_mutexV(mMutex); //unlock
}
SDL_Delay(1);
}

printf( “VE_Media: thread dying\n” );
SDL_DestroyMutex( mMutex );

return 0;
}

…the code I call to grab the audio buffer is as follows:

int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
if( audStreamExists ) {
audioBuffer = mAudioBuffer;
return mAudioDataSize;
} else
return NULL;
}

…In my sound class, I have created the callback function for SDL’s
audio, basing it largely on the code within ffplay.c:

void VE_Sound::audio_callback( void *userdata, Uint8 *stream, int
len ) {
int dLen;

//grab audio from all media objects and mix together
if( mMediaList.size() > 0 ) {
while( len > 0 ) {
//check if we need to grab more audio…
if( mAudioBufferIndex >= mAudioBufferSize ) {
mAudioBufferSize = mMediaList[0]->getCurrentAudioFrame
( mAudioBuffer );
if( mAudioBufferSize < 0 ) {
mAudioBufferSize = 1024;
memset(mAudioBuffer, 0, 1024);
} else {
//synchronize for audio to transfer…
//…look at ffplay.c for help
}
mAudioBufferIndex = 0;
}
dLen = mAudioBufferSize - mAudioBufferIndex;
if( dLen > len )
dLen = len; //is this even necessary?

  	memcpy(stream, (uint8_t *)mAudioBuffer + mAudioBufferIndex, dLen);
  	len -= dLen;
  	mAudioBufferIndex += dLen;
  	printf( "length of audio buffer: %i\nnew index: %i\n",

mAudioBufferSize, mAudioBufferIndex );
}
}
}

…What’s happening is my program invariably crashed on the “memcpy
()” function call, stating EXC_BAD_ACCESS, which I think means that
I’m trying to write/read memory from an array which is not assigned
to that array.

If someone could offer me some help on this, I would greatly
appreciate it. I’m sure it’s something I’ve missed with either
threading or memory/data shuffling that I’ve screwed up.

– Scott Harper


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org