[SDL_OpenAudioDevice] : can we open several recordable devices and select /change the used one?

In my software, I would like to mux several video sources with several audio sources.

For example several webcams (including audio capabilities) or one video playing, or rtsp flow.

Currently I can see everything:

SDL2 Window created 
SDL_VERSION_ATLEAST(2,0,9) 1
SDL_VERSION_ATLEAST(2,0,8) 1
SDL_VERSION_ATLEAST(2,0,7) 1
SDL_HAS_CAPTURE_AND_GLOBAL_MOUSE    SDL_VERSION_ATLEAST(2,0,4) = 1
SDL_HAS_WINDOW_ALPHA                SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_ALWAYS_ON_TOP               SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_USABLE_DISPLAY_BOUNDS       SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_PER_MONITOR_DPI             SDL_VERSION_ATLEAST(2,0,4) = 1
SDL_HAS_VULKAN                      SDL_VERSION_ATLEAST(2,0,6) = 1
SDL_HAS_MOUSE_FOCUS_CLICKTHROUGH    SDL_VERSION_ATLEAST(2,0,5) = 1
We compiled against SDL version 2.0.9 ...
And we are linking against SDL version 2.0.9.
OpenGL version: 4.6 (Core Profile) Mesa 20.1.0-devel (git-c38946e62d)
GLSL version: 4.60
Vendor: Intel Open Source Technology Center
Renderer: Mesa DRI Intel(R) UHD Graphics 620 (WHL GT2)
Audio device (no recording capability)   0: HDA Intel PCH, ALC293 Analog
Audio device (with recording capability) 0: HDA Intel PCH, ALC293 Analog
Audio device (no recording capability)   1: HDA Intel PCH, HDMI 0
Audio device (with recording capability) 1: HD Pro Webcam C920, USB Audio
Audio device (no recording capability)   2: HDA Intel PCH, HDMI 1
Audio device (with recording capability) 2: BRIO 4K Stream Edition, USB Audio
Audio device (no recording capability)   3: HDA Intel PCH, HDMI 2
Audio device (with recording capability) 3: (null)
Audio device (no recording capability)   4: HDA Intel PCH, HDMI 3
Audio device (with recording capability) 4: (null)
Audio device (no recording capability)   5: HDA Intel PCH, HDMI 4
Audio device (with recording capability) 5: (null)
Audio device (no recording capability)   6: (null)
Audio device (with recording capability) 6: (null)
Audio device (no recording capability)   7: (null)
Audio device (with recording capability) 7: (null)
Found Audio device (with recording capability) device number : 0 name : HDA Intel PCH, ALC293 Analog
Audio device (with recording capability) device number : 1 name : HD Pro Webcam C920, USB Audio
Audio device (with recording capability) device number : 2 name : BRIO 4K Stream Edition, USB Audio
3 recordable audio devices.
Number of Audio devices (with recording capability) 3 
Audio driver name :  alsa 
io =  0x560602973b78
style =  0x5606029751b8
Viewport enabled 

My question is about select one recording device, and be able to select another on in runtime.

Currently, I’m able to select one, and change as shown in the log below:

Requested device : 0
Previous device index was : 0 with name : HDA Intel PCH, ALC293 Analog
Current status for previous device is : stopped
New device requested : 0
status of the requested device  : stopped
currentRecordableAudioDevice is now : HDA Intel PCH, ALC293 Analog
Current status for new device is : stopped



Requested device : 1
Previous device index was : 0 with name : HDA Intel PCH, ALC293 Analog
Current status for previous device is : stopped
New device requested : 1
status of the requested device  : stopped
currentRecordableAudioDevice is now : HD Pro Webcam C920, USB Audio
Current status for new device is : stopped



Requested device : 1
Previous device index was : 0 with name : HDA Intel PCH, ALC293 Analog
Current status for previous device is : stopped
New device requested : 1
status of the requested device  : stopped
currentRecordableAudioDevice is now : HD Pro Webcam C920, USB Audio
Current status for new device is : stopped

I think I start to see how make it work, but I prefer ask, to be sure :slight_smile:

My questions:

  1. does SDL2 allow to open several record-able Audio devices (already identified) e.g. at launch time (during initialization), open and set every of them as “paused” ? (current status is stopped)

  2. on demand; set any of them as paused / recording , to be able to record and mix any audio source with any video source, to record them in runtime ? More precisely : record ONLY one at the same time

Thanks in advance for any help, or advice :slight_smile:

Nobody can confirm it is possible … or not ?

Since nobody answered, I’ll try to add more information. Don’t forget I’m a newbie with SDL audio, so please be gentle :slight_smile::

The current work in progress is:

  • implement a selector to choose a device able to record audio
  • create a callback, and “visualize” its behavior (maybe will change for a ring buffer soon)
  • read the buffer and playback or send the content to a stream, to mux audio with another image source (using ffmpeg)

N.B. : the muxer will be the next big task

All the code is there:

Interface:

Audiodevice class : https://framagit.org/ericb/miniDart/-/blob/master/Sources/inc/audiodevice.hpp

Audiomanager class : https://framagit.org/ericb/miniDart/-/blob/master/Sources/inc/audiomanager.hpp

Implementation:

audiodevice.cpp, audiomanager.cpp : https://framagit.org/ericb/miniDart/-/tree/master/Sources/src/Audio

miniDart.cpp (line around 2008) : https://framagit.org/ericb/miniDart/-/blob/master/Sources/src/Application/miniDart.cpp

The log says:

dpi                   : 166.444
defaultDpi            : 150
windowDpiScaledWidth  : 1553
windowDpiScaledHeight : 865
SDL2 Window created 
SDL_VERSION_ATLEAST(2,0,9) 1
SDL_HAS_CAPTURE_AND_GLOBAL_MOUSE    SDL_VERSION_ATLEAST(2,0,4) = 1
SDL_HAS_WINDOW_ALPHA                SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_ALWAYS_ON_TOP               SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_USABLE_DISPLAY_BOUNDS       SDL_VERSION_ATLEAST(2,0,5) = 1
SDL_HAS_PER_MONITOR_DPI             SDL_VERSION_ATLEAST(2,0,4) = 1
SDL_HAS_VULKAN                      SDL_VERSION_ATLEAST(2,0,6) = 1
SDL_HAS_MOUSE_FOCUS_CLICKTHROUGH    SDL_VERSION_ATLEAST(2,0,5) = 1
We compiled against SDL version 2.0.12 ...
And we are linking against SDL version 2.0.12.
OpenGL version: 4.6 (Core Profile) Mesa 20.2.0-devel (git-b8df1c43d2)
GLSL version: 4.60
Vendor: Intel Open Source Technology Center
Renderer: Mesa DRI Intel(R) UHD Graphics 620 (WHL GT2)
Audio device (no recording capability)   0: HDA Intel PCH, ALC293 Analog
Audio device (with recording capability) 0: HDA Intel PCH, ALC293 Analog
Audio device (no recording capability)   1: HDA Intel PCH, HDMI 0
Audio device (with recording capability) 1: HD Pro Webcam C920, USB Audio
Audio device (no recording capability)   2: HDA Intel PCH, HDMI 1
Audio device (with recording capability) 2: BRIO 4K Stream Edition, USB Audio
Audio device (no recording capability)   3: HDA Intel PCH, HDMI 2
Audio device (with recording capability) 3: (null)
Audio device (no recording capability)   4: HDA Intel PCH, HDMI 3
Audio device (with recording capability) 4: (null)
Audio device (no recording capability)   5: HDA Intel PCH, HDMI 4
Audio device (with recording capability) 5: (null)
Audio device (no recording capability)   6: (null)
Audio device (with recording capability) 6: (null)
Audio device (no recording capability)   7: (null)
Audio device (with recording capability) 7: (null)
Found : 
Audio device (with recording capability) device number : 0 name : HDA Intel PCH, ALC293 Analog
Audio device (with recording capability) device number : 1 name : HD Pro Webcam C920, USB Audio
Audio device (with recording capability) device number : 2 name : BRIO 4K Stream Edition, USB Audio
3 recordable audio devices.
Number of Audio devices (with recording capability) 3 
Audio driver name :  alsa 
Audio subsystem initialized; driver = alsa.
io =  0x55b5eb494e88
style =  0x55b5eb4964c8
Viewport enabled 

And I can select one between 3 “able to record” devices.
select_recording_devices
Step 1 : select
Step 2 : record

Just one example to understand:

isFullVideo vaut 1 
Gtk-Message: 23:40:28.214: GtkDialog mapped without a transient parent. This is discouraged.
Success!
(some path)/a_song.mp4
video_init done
calling main2(filename), filename in video_init contains : (some path)/a_song.mp4
 av_sync_type =  0
b_video_running = 1 (currently in video_init) 
Initialization : origin         =  0
                 max_position   =  0
                 video_duration =  0
Video Open : done 
is->ic->duration : 83754367,000000 1000000 
video_duration = 83,754367 (currently in read_thread) 
Requested device : 2
Current status for previous device is : stopped
New device requested : 2
status of the requested device  : playing
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : playing

>>>>  Opening : 
aDevice         = 2
currentAudioDev = 3
device 16384 recording will b triggered
Device 2 recording ....  
... done 
>>>>  Opening : 
aDevice         = 2
currentAudioDev = 0
[audio]could not open audio, error:ALSA: Couldn't open audio device: Périphérique ou ressource occupé
device -1 recording will b triggered
Device 2 recording ....  
... done 
Requested device : 2
Current status for previous device is : paused
New device requested : 2
status of the requested device  : paused
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : paused

>>>>  Opening : 
aDevice         = 2
currentAudioDev = 0
[audio]could not open audio, error:ALSA: Couldn't open audio device: Périphérique ou ressource occupé
device -1 recording will b triggered
Device 2 recording ....  
... done 
Requested device : 0
Current status for previous device is : paused
New device requested : 0
status of the requested device  : stopped
currentRecordableAudioDevice is now : HDA Intel PCH, ALC293 Analog
Current status for new device is : stopped

Requested device : 2
Current status for previous device is : stopped
New device requested : 2
status of the requested device  : paused
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : paused

>>>>  Opening : 
aDevice         = 2
currentAudioDev = 0
[audio]could not open audio, error:ALSA: Couldn't open audio device: Périphérique ou ressource occupé
device -1 recording will b triggered
Device 2 recording ....  
... done 
Requested device : 0
Current status for previous device is : paused
New device requested : 0
status of the requested device  : stopped
currentRecordableAudioDevice is now : HDA Intel PCH, ALC293 Analog
Current status for new device is : stopped

>>>>  Opening : 
aDevice         = 0
currentAudioDev = 4
device 16384 recording will b triggered
Device 0 recording ....  
... done 
Requested device : 2
Current status for previous device is : stopped
New device requested : 2
status of the requested device  : paused
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : paused

>>>>  Opening : 
aDevice         = 2
currentAudioDev = 0
[audio]could not open audio, error:ALSA: Couldn't open audio device: Périphérique ou ressource occupé
device -1 recording will b triggered
Device 2 recording ....  
... done 
>>>>  Opening : 
aDevice         = 2
currentAudioDev = 0
[audio]could not open audio, error:ALSA: Couldn't open audio device: Périphérique ou ressource occupé
device -1 recording will b triggered
Device 2 recording ....  
... done 
 entering in do_exit(is) 
 is->tead_tid : 0x55d90d152000
currentAudioDevicesList vector size : 2
currentAudioDevicesList vector size : 1
currentAudioDevicesList vector size : 0
currentAudioDevicesList vector size : 2
currentAudioDevicesList vector size : 1
currentAudioDevicesList vector size : 0

(miniDart has quit)

FYI, aDevice is the right device number, while currentAudioDev is plain wrong, and comes from:

currentAudioDev = SDL_OpenAudioDevice(SDL_GetAudioDeviceName(aDevice, SDL_TRUE), SDL_TRUE, &wanted, &have, SDL_AUDIO_ALLOW_FREQUENCY_CHANGE | SDL_AUDIO_ALLOW_CHANNELS_CHANGE);

My questions are :slight_smile:

  1. is SDL2 able to select ONE recording audio device, while another is doing playback ?

  2. when closing a given recording device, WHY does the one doing playback STOP too ?

  3. why does only one callback work. e.g. when I’m connecting 2 webcams able to record audio + the internal microphone (means I got 3 recording devices connected), only 1 triggered the correct callback behavior (see below):

Requested device : 2
Current status for previous device is : stopped
New device requested : 2
status of the requested device  : stopped
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : stopped

>>>>  Opening : 
aDevice         = 2
currentAudioDev = 2
device 16384 recording will b triggered
Device 2 recording ....  
Using callback at 204362
md::AudioDevice::audioRecordingBuffer = 0 
Using callback at 204405
md::AudioDevice::audioRecordingBuffer = 16384 
Using callback at 204447
md::AudioDevice::audioRecordingBuffer = 32768 
Using callback at 204490
md::AudioDevice::audioRecordingBuffer = 49152 
Using callback at 204533
md::AudioDevice::audioRecordingBuffer = 65536 
Using callback at 204577
md::AudioDevice::audioRecordingBuffer = 81920 
Using callback at 204619
md::AudioDevice::audioRecordingBuffer = 98304 
Using callback at 204662
md::AudioDevice::audioRecordingBuffer = 114688 
Using callback at 204705
md::AudioDevice::audioRecordingBuffer = 131072 
Using callback at 204748
md::AudioDevice::audioRecordingBuffer = 147456 
Using callback at 204790
md::AudioDevice::audioRecordingBuffer = 163840 
Using callback at 204833
md::AudioDevice::audioRecordingBuffer = 180224 
Using callback at 204874
md::AudioDevice::audioRecordingBuffer = 196608 
Using callback at 204917
md::AudioDevice::audioRecordingBuffer = 212992 
Using callback at 204960
md::AudioDevice::audioRecordingBuffer = 229376 
Using callback at 205002
md::AudioDevice::audioRecordingBuffer = 245760 
Using callback at 205044
md::AudioDevice::audioRecordingBuffer = 262144 
Using callback at 205087
md::AudioDevice::audioRecordingBuffer = 278528 
Using callback at 205130
md::AudioDevice::audioRecordingBuffer = 294912 
Using callback at 205172
md::AudioDevice::audioRecordingBuffer = 311296 
Using callback at 205215
md::AudioDevice::audioRecordingBuffer = 327680 
Using callback at 205258
md::AudioDevice::audioRecordingBuffer = 344064 
Using callback at 205301
md::AudioDevice::audioRecordingBuffer = 360448 
Using callback at 205343
md::AudioDevice::audioRecordingBuffer = 376832 
Using callback at 205386
md::AudioDevice::audioRecordingBuffer = 393216 
Using callback at 205429
md::AudioDevice::audioRecordingBuffer = 409600 
Using callback at 205472
md::AudioDevice::audioRecordingBuffer = 425984 
Using callback at 205514
md::AudioDevice::audioRecordingBuffer = 442368 
Using callback at 205557
md::AudioDevice::audioRecordingBuffer = 458752 
Using callback at 205601
md::AudioDevice::audioRecordingBuffer = 475136 
Using callback at 205644
md::AudioDevice::audioRecordingBuffer = 491520 
Using callback at 205687
md::AudioDevice::audioRecordingBuffer = 507904 
Using callback at 205727
md::AudioDevice::audioRecordingBuffer = 524288 
Using callback at 205770
md::AudioDevice::audioRecordingBuffer = 540672 
Using callback at 205813
md::AudioDevice::audioRecordingBuffer = 557056 
Using callback at 205856
md::AudioDevice::audioRecordingBuffer = 573440 
Using callback at 205899
md::AudioDevice::audioRecordingBuffer = 589824 
Using callback at 205942
md::AudioDevice::audioRecordingBuffer = 606208 
Using callback at 205984
md::AudioDevice::audioRecordingBuffer = 622592 
Using callback at 206026
md::AudioDevice::audioRecordingBuffer = 638976 
Using callback at 206068
md::AudioDevice::audioRecordingBuffer = 655360 
Using callback at 206111
md::AudioDevice::audioRecordingBuffer = 671744 
Using callback at 206154
md::AudioDevice::audioRecordingBuffer = 688128 
Using callback at 206196
md::AudioDevice::audioRecordingBuffer = 704512 
Using callback at 206239
md::AudioDevice::audioRecordingBuffer = 720896 
Using callback at 206281
md::AudioDevice::audioRecordingBuffer = 737280 
... done 
  1. When audio device (0,0) doe playback, recording stops it (no more sound). How to retrigger it ?

THANKS in advance for any advice, suggestion or help !!

[Edit : fixed presentation details, typos …]

1 Like

Update : I fixed several dumb issues, and made some progress, like it seems to no longer crash (nor leak), but it still does not work.

The problem seems to be I’m trying to (or I can’t) open the wrong device. Got either

SDL_GetError() result = Invalid audio device ID 

or

SDL_GetError() result = ALSA: Couldn’t open audio device: Device or resource busy

Last but not least, it seems to work only for the last device found. e.g. when I connect 2 webcams, only the device seen as the third one (number 2) can record (able to record devices 0 and 1 do not). Maybe I’m not using SDLOpenAudioDevice() correctly ?

The current log :



**=> See SDL_GetError()**


Opening :
aDevice = 0
currentAudioDev = 2
SDL_GetError() result = Invalid audio device ID
Device 0 recording will b triggered … with size : 16384
Device 0 recording …
… done
Requested device : 1
Current status for previous device is : stopped
New device requested : 1
status of the requested device : stopped
currentRecordableAudioDevice is now : BRIO 4K Stream Edition, USB Audio
Current status for new device is : stopped

Opening :
aDevice = 1
currentAudioDev = 0
SDL_GetError() result = ALSA: Couldn’t open audio device: Device or resource busy
Device 1 recording will b triggered … with size : 16384
Device 1 recording …
… done
Requested device : 2
Current status for previous device is : stopped
New device requested : 2
status of the requested device : paused
currentRecordableAudioDevice is now : C922 Pro Stream Webcam, USB Audio
Current status for new device is : paused

Opening :
aDevice = 2
currentAudioDev = 0
SDL_GetError() result = Couldn’t set hardware audio parameters: Operation not permitted
Device 2 recording will b triggered … with size : 16384
Device 2 recording …
Using callback at 273704
md::AudioDevice::audioRecordingBuffer = 0
Using callback at 273750
md::AudioDevice::audioRecordingBuffer = 16384
Using callback at 273797
md::AudioDevice::audioRecordingBuffer = 32768
Using callback at 273844
md::AudioDevice::audioRecordingBuffer = 49152
Using callback at 273890
md::AudioDevice::audioRecordingBuffer = 65536
Using callback at 273937
md::AudioDevice::audioRecordingBuffer = 81920
Using callback at 273983
md::AudioDevice::audioRecordingBuffer = 98304
Using callback at 274029
md::AudioDevice::audioRecordingBuffer = 114688
Using callback at 274077
md::AudioDevice::audioRecordingBuffer = 131072
Using callback at 274123
md::AudioDevice::audioRecordingBuffer = 147456
Using callback at 274170
md::AudioDevice::audioRecordingBuffer = 163840
Using callback at 274216
md::AudioDevice::audioRecordingBuffer = 180224
Using callback at 274263
md::AudioDevice::audioRecordingBuffer = 196608
Using callback at 274310
md::AudioDevice::audioRecordingBuffer = 212992
Using callback at 274356
md::AudioDevice::audioRecordingBuffer = 229376
Using callback at 274403
md::AudioDevice::audioRecordingBuffer = 245760
Using callback at 274449
md::AudioDevice::audioRecordingBuffer = 262144
Using callback at 274496
md::AudioDevice::audioRecordingBuffer = 278528
Using callback at 274542
md::AudioDevice::audioRecordingBuffer = 294912
Using callback at 274589
md::AudioDevice::audioRecordingBuffer = 311296
Using callback at 274636
md::AudioDevice::audioRecordingBuffer = 327680
Using callback at 274682
md::AudioDevice::audioRecordingBuffer = 344064
Using callback at 274729
md::AudioDevice::audioRecordingBuffer = 360448
Using callback at 274776
md::AudioDevice::audioRecordingBuffer = 376832
Using callback at 274822
md::AudioDevice::audioRecordingBuffer = 393216
Using callback at 274869
md::AudioDevice::audioRecordingBuffer = 409600
Using callback at 274915
md::AudioDevice::audioRecordingBuffer = 425984
Using callback at 274962
md::AudioDevice::audioRecordingBuffer = 442368
Using callback at 275003
md::AudioDevice::audioRecordingBuffer = 458752
Using callback at 275050
md::AudioDevice::audioRecordingBuffer = 475136
Using callback at 275097
md::AudioDevice::audioRecordingBuffer = 491520
Using callback at 275144
md::AudioDevice::audioRecordingBuffer = 507904
Using callback at 275190
md::AudioDevice::audioRecordingBuffer = 524288
Using callback at 275237
md::AudioDevice::audioRecordingBuffer = 540672
Using callback at 275283
md::AudioDevice::audioRecordingBuffer = 557056
Using callback at 275330
md::AudioDevice::audioRecordingBuffer = 573440
Using callback at 275377
md::AudioDevice::audioRecordingBuffer = 589824
Using callback at 275424
md::AudioDevice::audioRecordingBuffer = 606208
Using callback at 275470
md::AudioDevice::audioRecordingBuffer = 622592
Using callback at 275516
md::AudioDevice::audioRecordingBuffer = 638976
Using callback at 275563
md::AudioDevice::audioRecordingBuffer = 655360
Using callback at 275609
md::AudioDevice::audioRecordingBuffer = 671744
… done

Hello,

After a looong time completely busy, I finaly found the time to investigate, and start to see some light :slight_smile:

After the recent changes, I finally can create a recording callback, and it seems to worrk as expected. If I understand correctly, the mistake was caused by a bad deviceID, and the trick was to only trust (and use) the value returned by
SDL_GetAudioDeviceName(aName.c_str(), SDL_TRUE, &wanted, &have, SDL_AUDIO_ALLOW_FREQUENCY_CHANGE | … ). Use the array number does not work, since it never match with the ID.

For the one interested, I implemented 3 classes:

  • audiodevices : objects with audio devices properties, including callbakcs, and their own buffers
  • audiomanager : create the list of audio devices as objects, including their own callback
  • engine : SDL / OpenGL 3.x

In runtime : (debug is included, during the development phase), things work this way:

engine initializes the audiomanager. The audiomanager creates the list of all recordable audio devices.
A checkbox allows to diaplay the list, and once they are selected, every device is initialised once only (currently only recording the callback is created). Once this is done, one can select and start / stop recording.
Well, the next step is to check what contains the callback (stream of data if I’m not toowrong, and how to copy all of them in a file ( e.g. aac). Current track is to use ffmpeg (I got some working code, investigating).

Other current tracks : using a circular buffer (to help Windows protability).

The code is here :

What helped me ?

NEXT STEP : implement both recording and playback callbacks, and use them.

Feel free to suggest improvements, and thanks a lot to SDL2 : this API is black magic !