Event queue size/limits

I’m occasionally a bit incoherent at times, so I apologize if this doesn’t make sense. How big are the queues for events when using SDL_PumpEvents? I started using SDL_AddEventWatcher and I like that more than an event queue thread or something like that. After I set it up, I noticed that it doesn’t work unless at least SDL_PumpEvents is called occasionally, so that’s why I’m asking. Right now I have this: while(SDL_PollEvent(&event)); which calls SDL_PumpEvents and ensures that the queue doesn’t get out of control, but I’d like to know if I can get rid of that and just call SDL_PumpEvents(); instead, and not have to worry about memory getting out of hand. I tried looking through the code for SDL_PumpEvents, and the video initialization code, but it’s a bit beyond me. I can see the PumpEvents inside of the SDL_VideoDevice, and where the pointers are being changed, but then I get stuck at a dead end.

Thanks.

SDL 2 currently has an event queue limit of 65535. This is the value on line 39 in SDL_events.c.

Once the queue is full, it can’t add any more events and functions like SDL_PollEvent will never be able to see these new events. However, the event filters still get called, if the events are added with SDL_PushEvent. Not sure if there are any side effects if you leave the queue full besides it using up 3.67 MB of memory.

Edit: Many of the window events seem to come through SDL_SendWindowEvent which uses SDL_PushEvent, so that should still work.

Thanks for that. I’m glad it wasn’t a snake or it would have bit me. At first glance, it looks like there’s no harm in having a filter set to do all events with the video thread pumping the event queue occasionally. I altered my SDL to return immediately and I still see all events. My Add Event function is like this:

SDL_AddEvent(SDL_Event * event)
  {
    return 0;
  }

I might only be saving a few nano/microseconds, but in a game, that can really help out when your loop is constantly running. One SDL_PumpEvents() can also read all of the queued up events as well. I put a delay for one second and all events still seem to come through. More testing is needed, but so far, so good.

Something that has bitten me is that SDL_PumpEvents() is very slow on iOS (much slower than on other platforms, in my experience). It’s even commented in the source code (SDL_uikitevents.m): “Let the run loop run for a short amount of time: long enough for touch events to get processed”. Supposedly it’s set to take two microseconds, but whether or not that’s true my app - which relies heavily on the SDL event mechanism for inter-thread communication - runs much more slowly as a result.

I’ve now taken to calling SDL_PumpEvents relatively infrequently which makes a massive difference. My custom events - which result from a SDL_PushEvent - are not affected.

Thanks for the heads up, won’t be too long before I’m coding sections for Apple products and the sooner I find out about quirks the better. I’d attempt to help you find the reason why it’s so slow right now, but I don’t have any Apple products at this time. :expressionless:

It’s deliberate, and commented (to a degree), but precisely what the underlying issue is I don’t know (the comment goes on to talk about “important to get certain elements of Game Center’s GKLeaderboardViewController to respond to touch input” which sounds pretty obscure). But my workaround is effective and my intention is always to limit the rate at which I call SDL_PumpEvents() from now on.

It’s deliberate, and commented (to a degree), but precisely what the
underlying issue is I don’t know (the comment goes on to talk about
important to get certain elements of Game Center’s
GKLeaderboardViewController to respond to touch input
” which sounds pretty
obscure). But my workaround is effective and my intention is always to
limit the rate at which I call SDL_PumpEvents() from now on.

I haven’t noticed this myself, but my suspicion is that it caused by
the way SDL does its event loop. The problem is that SDL tries to take
control of the event loop so you can write a top-to-bottom procedural
poll-driven “int main()”. This worked on the old platforms (e.g.
Windows, traditional Unix, etc.), but has become problematic on the
newer platforms (e.g. OSX, iOS, Android, Emscripten).

In these latter platforms, the OS wants to control the event loop and
you are not supposed to ever block it. So the typical SDL “int main()”
pattern would block these loops and not work. To preserve SDL’s
paradigm, hoop jumping acrobatics are done. Unfortunately, all of
these workarounds have various problems.

I know from personal experience doing a lot of Cocoa work, that not
using Apple’s normal event loop can break certain Apple APIs and
frameworks, because their own code depends on that behavior.
GameCenter was at one time (and still may be) one of those frameworks.
We’ve also seen menu and window focus problems with SDL on Mac due to
not using the standard event loop. As for the lag, I don’t know the
reason, but I could speculate that trying to query for input events at
random times, not tied to Apple’s standard event model may be a
reason…consider maybe the events are designed to come through at a
certain time into the Cocoa framework which was coordinated with the
rest of the Cocoa event loop. With SDL, since we are not synchronized
to that event loop, our queries might be just a little too early, and
we have to wait another frame for them to appear.

On Android, the problem is worse because the only workaround was to
put SDL on a background thread. If you only stick to only SDL APIs and
OpenGL, you won’t notice anything, but the moment you need to interact
with anything on the UI thread (e.g. In App Billing modules, Facebook
plugin, etc.), you are in for a world of hurt. Also, the whole Android
life-cycle is extra painful with SDL because there are a lot of sync
point issues with getting SDL to do the right thing immediately on a
different thread than the main Android app.

On Emscripten, SDL had to give up because there is no mechanism to
control the event loop in the web brower, PERIOD. And the web doesn’t
allow threads (and I don’t think web workers are sufficient for SDL’s
case). So for Emscipten, SDL introduced a new extension API that calls
back on a new frame, and you are expected to run through (just once)
your main event loop through that.

My belief is that the new SDL API for Emscpiten is the better way to
deal with the event loop moving forward. Most new platforms are moving
to the same event-driven OS loop control, for various reasons…but
battery life has become a strong motivator for vendors for this in
recent years. Also, most new platforms provide high performance
callbacks on frame draws, which SDL could utilized. And most
importantly for cross-platform, I think it is always possible to turn
a procedural top-to-bottom event loop into something that resembles
event driven, but it is not always possible to (perfectly) convert a
native event-driven into a procedural top-to-bottom-loop (as we’ve
seen with Emscripten, Android, and Apple).

I talked about cross-platform event loop issues in my talk about the
cross-platform native GUI framework, IUP. And I walk through how to
convert the poll-driven traditional loops used by Unix and Windows in
IUP into an event-driven model, to re-unify with the new Mac, iOS,
Android, and Emscripten backends I’ve been developing which all are
natively event-driven.

A digression, one of my (very) long term goals is to make SDL work with IUP.

-Eric

I would have no objection to an alternative style of event-driven operation but my app relies critically on very-low-latency events because they are used as the means of performing inter-thread procedure calls. The latency associated with waking up a thread for every event (e.g. by means of a semaphore) is unacceptable - it kills the performance of my app - so I have to run a busy-wait loop (calling SDL_PeepEvents) for a short time after each user event to achieve the required low latency.

This all works beautifully with SDL2 as it is now - on all platforms - although it took a lot of ‘tuning’ to get it to the current state. I would worry that I could not reproduce the same performance in a fully-event-driven environment. So whatever changes might be upcoming, please don’t remove the option of it working as it does now.

So to be clear, my argument is NOT that event-driven systems are
better. My argument is merely that the native operating systems are
now designed as event-driven which is creating an impedance mismatch
for us, which is resulting in some bugs and possible performance loss.
The truth is that because these systems use their native loops and are
already event driven to process things for their own internal stuff,
they are already occurring in SDL to various degrees and we are
already paying for it, but we are not getting any of the benefits
(i.e. all the downsides and none of the upsides).

In the 4 cases I mentioned (macOS, iOS, Android, Emscripten), all are
designed around controlling the event loop themselves, and all
actually happen to be single-threaded designs where they expect their
UI stuff to happen on the main UI thread. In fact, in this last
release, Apple has become unabashedly proud that their UI frameworks
must be called on the main UI thread and added more compile and
runtime checks to make sure their UI APIs are only called on the main
UI thread. Whereas many programmers criticized Apple for this decision
for years, they now wear it as a badge of honor because they claim
that by doing this, they were able to remove a lot of unnecessary
locking and context switches which reduces latency. (I remember some
mailing list conversations with early Cocoa engineers talking about
their Cocoa/Java bridge and Mac Java implementation (way back in
2000-2002) and one of the things they learned was trying to make
everything thread safe killed performance.)

As for the SDL implementation on Mac/iOS, I suspect things won’t need
to change that much. Ultimately, I think the primary trick would be to
either use Apple’s new high precision NSTimer to ensure timely
callbacks, or on iOS, use the CADisplayLink (assuming it still calls
back on the main UI thread), which is iOS’s GPU timer callback that is
synchronized with the draw request, to implement the equivalent API of
what the SDL/Emscripten extension does. (I actually vaguely recall a
different SDL extension for iOS which used CADisplayLink already which
might be very close to this already.) Then the final trick that needs
to be figured out is where are Cocoa’s events copied into the SDL
event queue.

The ususal basic way is through the set of touchesBegin/End/Cancelled
callbacks. And I think this is how SDL is already implemented. So you
are already enduring this event callback anyway, and there are no
guarantees when this is being called, even with SDL’s current design.
The SDL pollevent is only an illusion that everything is procedural
and atomic. So the latency you are complaining about in your app
design may already be suffering from unnecessary context switches and
you don’t recognize it.

Though I have heard of some other techniques to get touches faster.
One is coalesced touches, which I believe gets you everything in one
callback, instead of spread out against multiple. This might be the
opportune time to copy to the SDL event queue.

https://developer.apple.com/documentation/uikit/touches_presses_and_gestures/handling_touches_in_your_view/getting_high_fidelity_input_with_coalesced_touches?language=objc

Predictive touches are even more aggressive to reduce latency.

https://developer.apple.com/documentation/uikit/touches_presses_and_gestures/handling_touches_in_your_view/minimizing_latency_with_predicted_touches?language=objc

The other question I don’t know off hand is if there is a way to poll
for these directly without going through the callbacks. If there is,
we would still need to measure if that is actually any faster. (There
is always a risk that it could be slower due to implementation
details.)

-Eric

I’m not complaining about latency in my app design, indeed I stated “This all works beautifully with SDL2 as it is now”. Yes it took a while to work out how to achieve it, but I’m happy that the latency is now mostly very low because of my busy-wait loop (which of course wastes CPU time, and is not battery-life friendly, but only runs when necessary).

In respect of SDL getting “all the downsides and none of the upsides”, its current configuration (with a conventional ‘main loop’) is absolutely an ‘upside’ for me! Naturally I don’t want my app to be sacrificed on the altar of making things work better for the ‘majority’.