Question regarding SDL2 + libVLC and threading

Hi all

I’m building a simple media player around libVLC and lately I’ve been having a look at SDL.

There is one thing I can’t find a clear answer to however; I’ve seen several questions posted in different places on the Internet regarding making Renderer calls from different threads. The most common method/answer seems to be that you generally have one renderer per window and calls based on this renderer should only be made in the same (main) thread it was created in.

However, the libVLC/SDL examples break this rule straightaway because when when a player is created and started in libVLC it creates it’s own thread and from there calls back to the lock/unlock/display callbacks you supply. In the example (SDL2), the texture is locked/unlocked in the lock/unlock functions and then rendered to the screen in the display callback which is clearly in a different thread. It works though, maybe because the main thread isn’t doing any rendering?

Also, I don’t see the point of the mutex on the context in that example - vlc doesn’t call lock, unlock and display at the same time, only sequentially, so what is the mutex intending to protect?

Anyway, I have tried to look at this clean as maybe the libVLC/SDL example is misleading; in my scenario I am rendering a user interface overlay, and I’ve structured it as follows:

  1. main thread creates a “video texture” and makes the call to start vlc player, which then locks, draws to, and unlocks this texture in it’s own thread. I don’t use the display callback and I don’t make any rendering calls in this libVLC thread. Just locking the texture in the lock() callback, vlc then writes to it, then I unlock it again in the unlock() callback.

  2. main thread/loop deals with events and user input, and drawing to a UI texture.

  3. at the bottom of the loop the video texture is rendered first, then the UI texture ontop with appropriate alpha, and then this is presented to the screen

  4. my app has some networking stuff going on too, which takes place in a separate thread so that the main thread stays as fast as possible.

I am not using any mutexes at all and it seems to be working just fine. This is on Windows using the default Direct3D renderer. Am I likely to run into other problems on other platforms though or is this approach ok? Thing is, according to the SDL help the functions that operate on textures should also only be called from the main thread, but I’m doing it and I’ve not had any problems at all and with video streaming at 30fps and the render loop compositing the video with the UI at 60fps I would have thought if there was an issue I would have seen it?

Presumably when the texture is locked by the libvlc thread, the call to render it to the screen deals with this somehow, presumably it’s only reading it so the texture being locked for writing doesn’t cause any problems?

Cheers

Oliver

https://wiki.videolan.org/LibVLC_SampleCode_SDL/

Well I can probably answer my own question now having spent some more time experimenting and trying things out.

Since I am developing my app on a Windows Parallels VM this appeared to hide some nasty realities from me which came out of the woodwork when I tried the app on a regular Windows machine.

I didn’t find any solution that was stable when making SDL2 function calls relating to rendering and textures from a thread other than the main thread.

My final solution was quite simple in the end, and is as follows:

  • I malloc a YUV buffer (3 planes) in the callback from libVLC. libVLC then decodes the video to this buffer.
  • in the libVLC display callback, which is meant to be the place where you render to the screen I basically lock a mutex and wait on a condition variable.
  • in my main thread/event loop in my render section I lock the mutex and then I call UpdateYUVTexture passing in my malloc’d YUV planes. I then unlock the mutex and signal the condition variable which basically tells libVLC it can carry on.
  • I then render the YUV texture to the screen, followed by the UI overlay.

A variation on this I was using for a while was instead of having the display callback wait on the condition variable I had it copy the YUV buffer to another YUV buffer (this buffer can’t be a texture - as this would involve calling UpdateYUVTexture from the libVLC thread which proved very unstable), and then the render section of the main loop would grab this copy.

Although it worked and I couldn’t detect any difference in performance using my eyes, I didn’t like the additional buffer and copy - I think having libVLC wait a tiny amount of time for the main loop to grab the buffer into the texture and signal the thread is more elegant although I have a slight nagging worry about whether it’s a good idea to hold up libVLC like that.

Anyway, maybe this will help someone someday…