UI frameworks and SDL

SDL Development
1

Hello!

I’m developing a video player which is using SDL to render video and to play audio. My application is going to be cross-platform and i’m using gcc (and mingw on windows) to compile it. Everything is going fine. But i want to have some buttons (such as “Play”) and other widgets in my application. So i’ve started search on the internet how can i do it. First of all, i can use a lot of SDL-based ui frameworks. But all of them was developed to make UI in games, not in desktop applications. So these frameworks will always look ugly on desktop application and also there are no platform-depended widgets such as MainMenu. Next step i made was looking for solution which will be able to make working SDL with other UI frameworks such as wxWidgets, QT and GTK. So if to sum up the task it should look like this: draw UI widgets on the SDL_Surface. And here is my results:

wxWidgets:
In the wxWidgets there is an ugly hack here http://code.technoplaza.net/wx-sdl/part1/ , which doesn’t help me because: 1) it draws RGB images from SDL_Surface on the panel (all sdl’s power is useless here) 2) It doesn’t support SDL_Overlay which i’m using to render YUV images. 3) It doesn’t support YUV at all.
Also i found nice one library SDL_wx (http://sourceforge.jp/projects/azn-wind/releases/), which is doing all things i need, but last update was in 2007. I’ve built it, but test application is crashing on my Mac in sdlwxtest (i’m using wxWidgets 2.9.1, OS X 10.6). I tried to contact with author by e-mail, but his address is no longer valid :frowning: I’m sure that problem should be solved easy here, but all my tries was unsuccessful to do it myself.

QT, GTK
No “union” libraries at all. Only hacks with window id, but they don’t work in mac afaik (http://forums.libsdl.org/viewtopic.php?t=4798&sid=068976e0f56f02ff4bed6ca96174eab8).

So is it really impossible to develop desktop applications with SDL? I’m sure that someone should faced with this problem. Is there any advice on how can i solve this problem?

Thank you, for attention!

2

Honestly, if you just need buttons, it is easier to just make your own than
it is to load up some huge UI lib for the sake of three or four buttons.
That’s just my opinion, though.On Tue, Mar 15, 2011 at 7:20 AM, Gordiychuck Oleg wrote:

Hello!

I’m developing a video player which is using SDL to render video and to
play audio. My application is going to be cross-platform and i’m using gcc
(and mingw on windows) to compile it. Everything is going fine. But i want
to have some buttons (such as “Play”) and other widgets in my application.
So i’ve started search on the internet how can i do it. First of all, i can
use a lot of SDL-based ui frameworks. But all of them was developed to make
UI in games, not in desktop applications. So these frameworks will always
look ugly on desktop application and also there are no platform-depended
widgets such as MainMenu. Next step i made was looking for solution which
will be able to make working SDL with other UI frameworks such as wxWidgets,
QT and GTK. So if to sum up the task it should look like this: draw UI
widgets on the SDL_Surface. And here is my results:

wxWidgets:
In the wxWidgets there is an ugly hack here
http://code.technoplaza.net/wx-sdl/part1/ , which doesn’t help me
because: 1) it draws RGB images from SDL_Surface on the panel (all sdl’s
power is useless here) 2) It doesn’t support SDL_Overlay which i’m using to
render YUV images. 3) It doesn’t support YUV at all.
Also i found nice one library SDL_wx (
http://sourceforge.jp/projects/azn-wind/releases/), which is doing all
things i need, but last update was in 2007. I’ve built it, but test
application is crashing on my Mac in sdlwxtest (i’m using wxWidgets 2.9.1,
OS X 10.6). I tried to contact with author by e-mail, but his address is no
longer valid :frowning: I’m sure that problem should be solved easy here, but all my
tries was unsuccessful to do it myself.

QT, GTK
No “union” libraries at all. Only hacks with window id, but they don’t work
in mac afaik (
http://forums.libsdl.org/viewtopic.php?t=4798&sid=068976e0f56f02ff4bed6ca96174eab8
).

So is it really impossible to develop desktop applications with SDL? I’m
sure that someone should faced with this problem. Is there any advice on how
can i solve this problem?

Thank you, for attention!

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

3

Hello!

I’m developing a video player which is using SDL to render video and to
play audio. My application is going to be cross-platform and i’m using gcc
(and mingw on windows) to compile it. Everything is going fine. But i want
to have some buttons (such as “Play”) and other widgets in my application.
So i’ve started search on the internet how can i do it. First of all, i can
use a lot of SDL-based ui frameworks. But all of them was developed to make
UI in games, not in desktop applications. So these frameworks will always
look ugly on desktop application and also there are no platform-depended
widgets such as MainMenu. Next step i made was looking for solution which
will be able to make working SDL with other UI frameworks such as wxWidgets,
QT and GTK. So if to sum up the task it should look like this: draw UI
widgets on the SDL_Surface. And here is my results:

wxWidgets:
In the wxWidgets there is an ugly hack here
http://code.technoplaza.net/wx-sdl/part1/ , which doesn’t help me
because: 1) it draws RGB images from SDL_Surface on the panel (all sdl’s
power is useless here) 2) It doesn’t support SDL_Overlay which i’m using to
render YUV images. 3) It doesn’t support YUV at all.
Also i found nice one library SDL_wx (
http://sourceforge.jp/projects/azn-wind/releases/), which is doing all
things i need, but last update was in 2007. I’ve built it, but test
application is crashing on my Mac in sdlwxtest (i’m using wxWidgets 2.9.1,
OS X 10.6). I tried to contact with author by e-mail, but his address is no
longer valid :frowning: I’m sure that problem should be solved easy here, but all my
tries was unsuccessful to do it myself.

QT, GTK
No “union” libraries at all. Only hacks with window id, but they don’t work
in mac afaik (
http://forums.libsdl.org/viewtopic.php?t=4798&sid=068976e0f56f02ff4bed6ca96174eab8
).

So is it really impossible to develop desktop applications with SDL? I’m
sure that someone should faced with this problem. Is there any advice on how
can i solve this problem?

Thank you, for attention!

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

I’m… kind of confused. Are you trying to draw to an SDL_Surface, or an
SLD_Overlay ?

I also concur with Alex – exactly how much widget stuff do you need ?
Buttons are simple and shouldn’t need a UI lib.

-WillOn Tue, Mar 15, 2011 at 6:20 AM, Gordiychuck Oleg wrote:

4

Thank you for reply!

I must to agree with you about “just buttons”! But, unfortunately, except buttons i need also such “objects” as OpenFileDialog - to open files which i want to play, Form - to make some “About” box, MainMenu - to place these items together. All these things are platform depended and must look “native” on each platform.

2 Will Langford: Thank you for reply, too. I’m using SDL_SetVideoMode to make a main screen and SDL_DisplayYUVOverlay to draw. My video player is based on dranger’s ffmpeg example:http://dranger.com/ffmpeg/

P.S.: sorry for duplicating message, my mistake.

5

Thank you for reply!

I must to agree with you about “just buttons”! But, unfortunately, except
buttons i need also such “objects” as OpenFileDialog - to open files which i
want to play, Form - to make some “About” box, MainMenu - to place these
items together. All these things are platform depended and must look
"native" on each platform.

Can’t particularly help you here, sorry. I’m sure others can. But - I can
make a mild suggestion concerning your second point:

2 Will Langford: Thank you for reply, too. I’m using SDL_SetVideoMode to
make a main screen and SDL_DisplayYUVOverlay to draw. My video player is
based on dranger’s ffmpeg example:http://dranger.com/ffmpeg/

Generally, nothing will draw to an SDL Overlay. There’s an experimental lib
listed libsdl.org for treating overlays the same as surfaces, but it’s not
really a solution. I’ve had some funky things happen when drawing an
sdl_surface on top of an sdl_overlay, so anything other than platform
specific calls might be a bit weird if it emulates the native platform.

In regards to the ffmpeg example / tutorial you mentioned, you can make a
slight tweak to it that will cost you a fair amount of performance (20% ish
?) but offers some more flexibility. Instead of rendering to a YUV overlay,
you can have the tutorial render to a SDL_Surface.

In that particular example, you’ll have to change the overlay variable to a
surface (and change corresponding SDL_* calls), and for the rendering:

SDL_LockSurface(vp->bmp);

dst_pix_fmt = PIX_FMT_RGB565; // FIXME

// point pict at the queue

pict.data[0] = vp->bmp->pixels;
pict.linesize[0] = vp->bmp->pitch;

Then call img_convert() or sws_scale() as you see fit.

Note that the FIXME should really detect what your current display mode is
and fill in the appropriate ffmpeg friendly define (possibly set after
setting the video mode). For a quick example, I just have it set to a 16bpp
surface.

In doing this, you’ll lose the speed of the YUV overlay, but you’ll be
drawing to a regular SDL_Surface with all of the simple advantages that
offers (including maybe easier working with UI libs).

-WillOn Tue, Mar 15, 2011 at 11:15 AM, Gordiychuck Oleg wrote:

6

15 ???. 2011, ? 18:39, Will Langford ???(?):

Generally, nothing will draw to an SDL Overlay. There’s an experimental lib listed libsdl.org for treating overlays the same as surfaces, but it’s not really a solution. I’ve had some funky things happen when drawing an sdl_surface on top of an sdl_overlay, so anything other than platform specific calls might be a bit weird if it emulates the native platform.

In regards to the ffmpeg example / tutorial you mentioned, you can make a slight tweak to it that will cost you a fair amount of performance (20% ish ?) but offers some more flexibility. Instead of rendering to a YUV overlay, you can have the tutorial render to a SDL_Surface.

In that particular example, you’ll have to change the overlay variable to a surface (and change corresponding SDL_* calls), and for the rendering:

  SDL_LockSurface(vp->bmp);

  dst_pix_fmt = PIX_FMT_RGB565; // FIXME

  // point pict at the queue 

  pict.data[0] = vp->bmp->pixels;
  pict.linesize[0] = vp->bmp->pitch;

Then call img_convert() or sws_scale() as you see fit.

Note that the FIXME should really detect what your current display mode is and fill in the appropriate ffmpeg friendly define (possibly set after setting the video mode). For a quick example, I just have it set to a 16bpp surface.

In doing this, you’ll lose the speed of the YUV overlay, but you’ll be drawing to a regular SDL_Surface with all of the simple advantages that offers (including maybe easier working with UI libs).

-Will

Hm, i want to try it, but i don’t know an analogue of SDL_DisplayYUVOverlay to call it for SDL_Surface. Can you help me with it? (I’ve googled but didn’t find yet)

Also when i tried to use sws_scale with RGB formats i’ve received warning from libswscale that the way i’m doing is slow. But i don’t know on how much is slow to compare. I want to be able watching HD videos, if it’s possible with RGB (i mean if performance will be good) than i can draw it on wxWIdgets panel!

Thank you for help!

7

Hm, if i understand right, there is no need to call analogue of SDL_DisplayYUVOverlay in SDL_Surface case at all (am i right?). Also i changed all code as you said to SDL_Surface analogues. Instead of SDL_CreateOverlay, i’m making vp->bmp = screen , where screen is created in main() with SDL_SetVideoMode. Also i used PIX_FMT_RGB565 as you.

But application is not rendering video (just empty screen). Is it because i’m using invalid pix_fmt?

8

Hm, if i understand right, there is no need to call analogue of
SDL_DisplayYUVOverlay in SDL_Surface case at all (am i right?). Also i
changed all code as you said to SDL_Surface analogues. Instead of
SDL_CreateOverlay, i’m making vp->bmp = screen , where screen is created in
main() with SDL_SetVideoMode. Also i used PIX_FMT_RGB565 as you.

But application is not rendering video (just empty screen). Is it because
i’m using invalid pix_fmt?

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

When I get a free moment at work, or later tonight i’ll post a more complete
description if it’s needed.

I chose RGB565 at random because it’s a common 16bpp color mode I’ve used in
the past. The PIX_FMT should match whatever your current display resolution
is.

I don’t think you should set vp->bmp to screen.

in alloc_picture(), you’d want to:

vp->bmp = SDL_CreateRGBSurface()

in video_display(), you’d want to:

SDL_BlitSurface (vp->bmp, NULL, screen, &rect);
SDL_Flip (screen);

lastly, you’d have to make sure to SDL_FreeSurface(vp->bmp) as appropriate.On Tue, Mar 15, 2011 at 12:46 PM, Gordiychuck Oleg wrote:

9

The buttons don’t need to appear over the video. Just resize the video to fit to the area with no buttons, and add buttons.

Making a non-text button over SDL is really simple:

Code:

struct SDL_Button
{
SDL_Surface* skin;
union
{
struct { int x; int y; int w; int h; };
SDL_Rect region;
}
void (onClick)(struct SDL_Button);
};

void SDL_Button_Draw(SDL_Button* b, SDL_Surface* screen)
{
SDL_BlitSurface(b->skin, NULL, screen, b->region);
};

[…]
while(true)
{
SDL_Event e;
SDL_PollEvent(&e);
if(e.type == SDL_MOUSEBUTTONDOWN)
{
// one for each button
if(e.button.x >= mybutton.x && e.button.x <= (mybutton.x+mybutton.w) && e.button.y >= mybutton.y && e.button.y <= (mybutton.y + mybutton.w))
{
mybutton.onClick(&mybutton);
}
// other mouse events
}
[…]
// at the appropriate time based on how you want it to look, but while rendering
mybutton.render(&mybutton, screen);
[…]
}------------------------
EM3 Nathaniel Fries, U.S. Navy

http://natefries.net/

10

The buttons don’t need to appear over the video. Just resize the video to
fit to the area with no buttons, and add buttons.

I was under the assumption this wasn’t the desire (resize video area), and I
also dunno of any ‘common dialog box’ style cross platform libs for SDL…

But, decent example :slight_smile:

-WillOn Tue, Mar 15, 2011 at 7:31 PM, Nathaniel J Fries wrote:

11

I’ve made my application working with SDL_Surface (thanks to Will Langford) and now i want to draw data to wxPanel. The main idea is to call Paint function in video_display() instead of SDL_BlitSurface (vp->bmp, NULL, screen, &rect);
SDL_Flip (screen); which looks like this:

void Paint(SDL_Surface *surface)
{
panel->setSurface(surface);
}

and OnPaint event of panel:

void SDLPanel::onPaint(wxPaintEvent &) {
// previous surface was already painted so must wait till next call of Paint(). surface is a private pointer of SDLPanel
if (surface==NULL)
return;

SDL_LockSurface(surface);

// create a bitmap from our pixel data
wxBitmap bmp(wxImage(surface->w, surface->h, 
                static_cast<unsigned char *>(surface->pixels), true));

SDL_UnlockSurface(surface);

// paint the screen
wxBufferedPaintDC dc(this, bmp);

surface = NULL; // this surface was already painted so it shouldn’t be painted next time OnPaint called until new surface is available
}

This code should work i guess. But i’ve faced to a new problem: SDL and wxWidgets have own event loops. And my first idea was to make SDL loop in separate thread. But i’m getting crash errors when i’m doing that. It’s due to Cocoa restrictions - you cannot poll events not in the main thread (windows have the same problem afaik). Any ideas how to solve this?

I tried next (but it doesn’t work yet): i made “own” events for FF_ALLOC_EVENT and FF_REFRESH_EVENT and run separate thread to manage them:

typedef struct DecodeEvent
{
int event_code;
void *data;
SDL_mutex *mutex;
}DecodeEvent;

static int EventLoop(void arg)
{
DecodeEvent decode_event = (DecodeEvent*)arg;

    for (;;)
    {
        SDL_LockMutex(decode_event->mutex);
        switch (decode_event->event_code)
        {
            case FF_ALLOC_EVENT:
                alloc_picture(decode_event->data);
                decode_event->event_code = -1;                    
            break;
            case FF_REFRESH_EVENT:
                video_refresh_timer(decode_event->data);
                decode_event->event_code = -1;
            break;
        }
        SDL_UnlockMutex(decode_event->mutex);
        SDL_Delay(100);
    }    
}

And here is how i’m calling it from dranger’s functions:

static Uint32 sdl_refresh_timer_cb(Uint32 interval, void opaque) {
SDL_LockMutex(decode_event->mutex);
decode_event->data = opaque;
decode_event->event_code = FF_REFRESH_EVENT;
SDL_UnlockMutex(decode_event->mutex);
return 0; / 0 means stop timer */
}

and the same code in queue_picture.

Any ideas what is wrong?

12

I’m looking to do something similar. I don’t need to allow both libraries to draw to the same window – I just want SDL to handle drawing, and wxWidgets to handle showing native dialog boxes.

Actually, all I need to do is have SDL process events when no dialog box is visible, then allow wxWidgets to “control” event handling when a dialog is shown, then when the dialog is dismissed, SDL gets control of event handling again. So unlike your app, in mine, they don’t both need to handle events at the same time. But I don’t think that even what I want to do is possible?

If you ever get events for both libraries working, let me know.

13

I’ve solved problem with event loop: you can place SDL events into onIdle event of any wxWidget. This is always main() thread so it will not crash on windows and os x and will work as you put it in main() function. To understand how it works you may read about wx event loop.

But i faced with another problem: video still doesn’t render. I only see the black screen. My “onPaint” event is calling a lot of times, but screen doesn’t change. Here is some code:

    screen = SDL_CreateRGBSurface(SDL_SWSURFACE, 640, 480,
                                  24, rmask, gmask, bmask, amask);

screen is creating in 24bit mode, because wx supports only this format. The same technique is for vp->bmp surface in alloc_picture():

    vp->bmp = SDL_CreateRGBSurface(SDL_SWSURFACE, 640, 480, 24, rmask, gmask, bmask, amask);

And in video_display function:

SDL_LockSurface(screen);
SDL_BlitSurface (vp->bmp, NULL, screen, &rect);
SDL_Flip (screen);
SDL_UnlockSurface(screen);

Now onPaint event:

void SDLPanel::onPaint(wxPaintEvent &ab)
{
SDL_LockSurface(screen);
wxBitmap bmp(wxImage(screen->w, screen->h,static_cast<unsigned char *>(screen->pixels), true));

SDL_UnlockSurface(screen);

wxBufferedPaintDC dc(this, bmp);
fprintf(stderr,“i was here\n”);
}

onPaint event is calling a lot of times so the problems lies in video_display, i think. Could someone help to solve this?

14

Solved this problem. Video is displaying if to remove SDL_LockSurface. But it is displaying monochrome with white stripes 0_0. What is that?

15

You only call SDL_LockSurface() when you want to directly access
surface->pixels. If you are calling SDL routines to blit or otherwise
manipulate the surface, it should be unlocked.On 21 March 2011 10:38, Gordiychuck Oleg wrote:

Solved this problem. Video is displaying if to remove SDL_LockSurface. But it is displaying monochrome with white stripes 0_0. What is that?

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

16

Solved problem with “monochrome and white stripes”, i forgot to switch sws_scale function from rgb32 to rgb24 mode. But now my onPaint event is very slow. I guess it happens because i’m trying to draw image too fast. So i decided to put mutex in video_display() and onIdle event that will make my painting only if “new” surface is avaliable. Here is my code for video_display():

SDL_BlitSurface (vp->bmp, NULL, screen, &rect);
SDL_Flip (screen);

SDL_LockMutex(can_draw_mut);
can_draw = 1;
SDL_UnlockMutex(can_draw_mut);

onIdle():

if (can_draw == 1)
Refresh(false); // Refresh calls onPaint event

onPaint():

void SDLPanel::onPaint(wxPaintEvent &ab)
{
wxBitmap bmp(wxImage(screen->w, screen->h,static_cast<unsigned char *>(screen->pixels), true));
wxBufferedPaintDC dc(this, bmp);
SDL_LockMutex(can_draw_mut);
can_draw = 0;
SDL_UnlockMutex(can_draw_mut);
}

But audio is still much faster than video. Can’t understand what is wrong