Multithreaded GUI done rigth

SDL Development
1

Hi
I’m not experienced with GUI applications and I’m not sure how to organize them, so I decided to have two separate threads for keyboard events handling and gui rendering: gui thread inits video subsystem and redraws a window and keyboard handler listens for keyboard events. It seemed to work fine until I tried to close my app and call SDL_Quit. It can either throw malloc_consolidate(): unaligned fastbin chunk detected or stuck on std::thread join function.

After I failed to use this approach I googled for SDL_CreateThread information and found this:

  • Don’t call SDL video/event functions from separate threads
  • Don’t use any library functions in separate threads

Due to this I have several questions:

  1. Isn’t it possible to draw and handle events in different threads? For example, I can limit drawing frequency with sleep(), but I wouldn’t like event handling thread to sleep at all.
  2. What does the second statement mean? How am I supposed to create a multithreaded app if I can use the library in one thread only?
  3. What’s the general approach of creating gui apps using sdl?

P.S. Extra question: I’m trying to get pressed keys with SDL_GetKeyboardState and I’m able to catch KEYDOWN event, but the array returned by SDL_GetKeyboardState is always filled with zeros. I’ using SDL_PumpEvents() to update the state and SDL scancodes as array indexes. What am I doing wrong?

Thanks in advance

2

First, get the pointer to the keyboard buffer once with SDL_GetKeyboardState, passing null as the argument. The received pointer can be used throughout the entire game session, because the address of the keyboard buffer will not change. To read the state of a specific key, check if the given byte of the array contains the value 0 or 1. The pointer retrieved using the function mentioned treat as a pointer to the uint8 array and the SDL key scancodes as indexes.

It may be better to respond to SDL_KeyboardEvent events, but it depends on your requirements.

3

Event handling and anything to do with drawing needs to be done on the main thread. Some of this is due to limitations of certain operating systems and certain graphics APIs. So your GUI and event handling will need to be on the main thread.

As to the second statement, it means a lot of SDL isn’t thread safe. Obviously, the stuff explicitly dealing with threads is fine. But a lot / most of the other stuff isn’t. You can do other parts of your program on different threads, but only make SDL calls (except the ones dealing with threads, atomic variables, etc.) on the main thread.

edit: When it comes to doing a UI in SDL for game tools or apps that aren’t games at all, I usually recommend Dear ImGui.

1 Like
4

This is exactly what I do, but as I mentioned, the array is always filled with 0. As far as I understand, SDL_KeyboardEvent represents the last active key only, but I want to handle simultaneously pressed keys. Thanks for the answer, anyway

5

Event handling and anything to do with drawing needs to be done on the main thread.

Well, this sound like a mess in a main thread. Can you advise, is it safe for the main thread to sleep then? Can I miss input events if I limit the frequency to 60 Hz or is it ok?

6

I don’t know why this is happening, but there must be a problem somewhere if you have all zeros. In support of my words, I can inform that I have already used this way of handling the keyboard in a real project — Fairtris — and I have not observed any problems, everything works just fine.

These generic objects can store the previous and current state (of anything, not only keys) and are used in this case so that anywhere in the game code I can check whether a given key has been pressed recently, whether it has been held at least two frames, whether it has been recently released, etc. A very simple and, at the same time, very useful thing.

I understand — nothing bizarre. It would be good if you could make a little test program so we know how you handle the keyboard buffer. A minimal program to reproduce the problem, in the technology you use.

Of course you can, and more than that, you should do it.

A well-written main loop should update game logic a fixed number of times per second (e.g. 60ups), render limited or no frames (depending on user settings), and sleep the rest of the time when the game has nothing to do. The game process should always use as little CPU power as possible — the rest of the time should be given to the system and other processes, which not only promotes responsiveness of the entire system, but also saves battery, which is very important nowadays.

No, you should never miss inputs. It doesn’t matter how many times the logic is updated per second and how many frames are rendered, all inputs should be processed. This is just my opinion of course, but I don’t see any practical use for missing inputs.

7

Use SDL_GetKeyboardState() if you want to know if a key is currently held down. This is usually good when you want to repeat something for as long as a key is held down (e.g. turn or accelerate) but it is possible that a key is pressed and released again quickly since last time you checked so if you want to react to a key press without the chance to “miss” it then you might want to use events instead.

8

Not really. Keep in mind that while you can use SDL for stuff that isn’t games, games are the main focus. The traditional game loop is something like

while(running) {
    while(eventsPending) {
        processEvent();
    }
    updateGameState();
    drawStuff();
}

If you’re trying to create a regular GUI app then it might be better to use something like Qt or WxWidgets. You’ll still run into the problem of bad things potentially happening if you do graphics or input handling off the main thread, depending on what platform your app is running on, but their interaction and event model is more like traditional UI toolkits. Like, you write callback functions that respond to buttons being clicked, keys being pressed, that sort of thing, instead of having to loop through events or use functions like SDL_GetKeyboardState().

It Depends

If the main thread goes for too long without processing events the operating system will think your program is hung. On macOS you get the beachball/pinwheel cursor, on Windows you get the “This application is not responding” warning or whatever, etc.

Can you miss events while the main thread is sleeping? Yes, depending on how you’re processing them and how long it sleeps. If you’re using functions like SDL_GetMouseState() or SDL_GetKeyboardState() then you will miss anything that happened while the main thread was sleeping; if the main thread slept too long and the user pressed and released the Escape key while it was sleeping, your input code won’t see it when it wakes up and calls SDL_GetKeyboardState(). If you’re just doing the standard while loop with SDL_PollEvent() then no, you won’t miss any events.

Another potential issue is if you’re trying to manually put the thread to sleep yourself. Implementations of sleep that aren’t just busy waits can only guarantee that the thread will sleep for at least the specified amount of time. It’s common that it’ll sleep longer (even several milliseconds), and if you’re trying to manually hit 60hz then you may not be able to consistently do so.

What you can do is use SDL_Renderer’s vsync support. Create the renderer with the flag SDL_RENDERER_PRESENTVSYNC. When you call SDL_RenderPresent() it will submit whatever drawing needs to be done and then block until the next display refresh after the drawing is complete. Doing a little test app with vsync on that draws a square every frame and nothing else, CPU usage sits around 1-3%.

If you’re worried about CPU usage when idle, this is an area where “traditional” UI toolkits have the upper hand if you aren’t making a game. If your program isn’t doing anything, and the user isn’t doing anything, your program’s main thread will be blocked and get no CPU time at all. The OS will wake up your program when there are new events to handle (user clicked a button, moved the mouse over your program’s window, a timer expired, etc) and once those are handled it goes back to sleep.

1 Like
9

You ahould be able to drain the event queue in another thread using SDL_PeepEvents while filling it in the main thread with SDL_PumpEvents, but it’s generally a bad idea, rendering always depends on user input processing results.

What you’re almost certainly going to wind up with is a situation where the rendering (main) thread is just priming the event queue and waiting on the event handling thread in a loop, and the event handling thread just waits for the main thread to prime the event queue in a loop. That’s just using two threads to make a program that has exactly zero actual concurrency.

UI doesn’t need to be quicker to respond to user input than you draw either, the user can’t be expected to click a button you haven’t shown them yet.

But do feel free to use threads for long-running tasks, or tasks unrelated to user input or rendering that need to loop, that’s normal and useful.

10

That would be a massive limitation, if taken literally. In practice there are many SDL functions that can safely be called from threads other than the main thread. Major ones are file handling (SDL_RWread(), SDL_RWclose() etc.), functions operating on surfaces (textures must be accessed from the main thread, but that limitation doesn’t apply to surfaces) and placing user events on the event queue (SDL_PushEvent()) which can be used for inter-thread communication.

1 Like
11

Thanks a lot for your help, but I’ve already figured out where was mistake in keyboard handling, I also was able to handle simultaneous inputs and even found some good examples of gui rendering

12

Thank you very much for your help, however, I’ve already found out how to handle inputs and draw gui properly. Thought I have a question regarding you example, where you have a nested loop inside the main loop. Wouldn’t it block gui drawing if we hold a button, for example? Or you just not a state which can be updated independently of input events? Also, could you advise where to look for information about os/graphics API limitations of multithreaded drawing/inputs handling?

13

No. When you press a key on the keyboard you get a KEYDOWN event for that key (pressing multiple keys at once will give a separate KEYDOWN events for each of them). You get a KEYUP event when the key is released. If a key is held down for longer than a certain amount of time you will get more KEYDOWN events for it, but they’ll be marked as repeat events so you can ignore them, and they’ll be spaced at least a few frames apart.

In any case, if the user is holding down a key or a mouse button, your app won’t get stuck in the while loop.

In fact, since multiple events can be in the event queue, you need to process them in a loop. A common beginner mistake on these forums comes from people not realizing this.

Even if you are using SDL_GetKeyboardState() and/or SDL_GetMouseState(), you still need to check for other events, like SDL_QUIT, window resizing, the render device being lost, etc.

I don’t have any links, but both iOS and macOS, for instance, deliver all events to the main thread. All your callbacks for button clicks and such will be called on the main thread. Anything that touches the UI must also be done from the main thread. Sometimes doing it from another thread will work, but a lot (most) of the time it doesn’t and you get weird results.

Apple’s Metal graphics API itself is multithreaded (can create command buffers on different threads and submit them to the same command queue) but all window handling, creating the Metal layer, etc., still has to be done from the main thread.

I’m assuming Windows and most Linux UI stuff is also single-threaded, though I don’t know if they’re restricted to only doing UI stuff on the main thread.

1 Like
14

For SDL_KEYDOWN events you actually do get “repeated events”.

If you look at the docs for SDL_KeyboardEvent you’ll see that it has a repeat member that you can check if you want to know if it’s a repeated event.

If you open a text editor and hold down a letter key you’ll get multiple letters. There is a longer delay between the first and the second letter compared to the rest. The repeated SDL_KEYDOWN events in SDL works the same way.

It still shouldn’t cause problems because there is plenty of time between each repeated SDL_KEYDOWN event.

I’m not aware of any other type of event that is repeated like this.

15

You’re right, I’d forgotten about key repeats. Fixed my post to reflect this.

But, like you said, there’s time between them, repeat events are marked, and it’s not like your app gets stuck in the while loop.

16

I’ve opened a bug to start updating the API documentation with details on thread safety:

1 Like