How to poll for SDL events and socket IO?

Hi,

I’m trying to write a networked app with OCamlSDL. For the purpose of
the question lets make it something real simple.

./sdl-wget http://host/file

This connects to the host and sends a GET command to download the
file. While it downloads it should display a progress bar and have a
cancel button.

So the main loop needs to do 2 things:

  1. poll the socket for downloaded data
  2. poll SDL for events

But it seems, and that might be a shortcoming of the ocaml bindings,
that there is no way to poll both. I don’t want to set a 0.1s timeout
and switch between the two all the time.

So how would you do this in C? How do you poll a socket and SDL with a
single blocking call?

Or does that require using one thread for the SDL part and one for the
socket? And if so can the socket thread update the progress bar while
the SDL thread is blocked in poll?

MfG
Goswin

Doing the download in a thread might be the best way to handle that. The
download thread can post custom progress messages to the SDL event queue as
well, if the SDL thread is doing the UI.On Sun, Jul 7, 2013 at 5:26 PM, Goswin von Brederlow wrote:

Hi,

I’m trying to write a networked app with OCamlSDL. For the purpose of
the question lets make it something real simple.

./sdl-wget http://host/file

This connects to the host and sends a GET command to download the
file. While it downloads it should display a progress bar and have a
cancel button.

So the main loop needs to do 2 things:

  1. poll the socket for downloaded data
  2. poll SDL for events

But it seems, and that might be a shortcoming of the ocaml bindings,
that there is no way to poll both. I don’t want to set a 0.1s timeout
and switch between the two all the time.

So how would you do this in C? How do you poll a socket and SDL with a
single blocking call?

Or does that require using one thread for the SDL part and one for the
socket? And if so can the socket thread update the progress bar while
the SDL thread is blocked in poll?

MfG
Goswin


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

Doing the download in a thread might be the best way to handle that. The download thread can post custom progress messages to the SDL event queue as well, if the SDL thread is doing the UI.

[note Ocaml cannot handle concurrent threads in Ocaml code,
works fine in C callouts though]

There is nothing wrong with polling loops with delays.
You can do async I/O with that (in Ocaml AFAIK).

Another alternative is to switch to Felix. This has much of
Ocaml stuff in it (variants, gc, strong typing) but is C/C++
compatible and has SDL binding (work in progress).

Felix also provides automatic async I/O with event notifications
for most major OS (epoll, poll, select, kqueue, Win32 and Solaris)
[fully transparent]On 08/07/2013, at 10:29 AM, Sam Lantinga wrote:


john skaller
@john_skaller
http://felix-lang.org

So the main loop needs to do 2 things:

  1. poll the socket for downloaded data
  2. poll SDL for events

The problem is that, since SDL works with a lot of different APIs on a
lot of different platforms, we can’t reasonably select() on a file
handle (or whatever) when polling for events. Some of the targets don’t
offer any such thing at all, others offer incompatible ways so you can’t
just pile them all into one big call.

With that in mind, we can’t give you a file handle (or several file
handles) to select() on, nor accept a handle from you to add to our
select().

It has come up before, though, that this would be nice to offer in terms
of power consumption if nothing else, and I think we could get 90%
there (and maybe 100% there in uncomplicated use cases), but it would
take a lot of reworking of SDL’s internals…since most things that use
SDL don’t actually want to block until there’s new data (since they want
to redraw the screen 60 times a second, etc), the gains of reworking
this are questionable…or at least very low priority.

–ryan.

So the main loop needs to do 2 things:

  1. poll the socket for downloaded data
  2. poll SDL for events

The problem is that, since SDL works with a lot of different APIs on a lot of different platforms, we can’t reasonably select() on a file handle (or whatever) when polling for events. Some of the targets don’t offer any such thing at all, others offer incompatible ways so you can’t just pile them all into one big call.

What do iOS and Android do?

[I have code that does async I/O on Windows, Solaris, Linux, BSD, OSX, generic Unix, but not iOS or Android support. However … it is C++ :]On 09/07/2013, at 2:12 AM, Ryan C. Gordon wrote:


john skaller
@john_skaller
http://felix-lang.org

What do iOS and Android do?

I don’t know what Android does; iOS eventually uses Apple’s CFRunLoop,
which isn’t a select()able object (and maybe we can wrap select()able
things in CFWhatevers, but now it’s getting complicated).

–ryan.

What do iOS and Android do?

I don’t know what Android does; iOS eventually uses Apple’s CFRunLoop,
which isn’t a select()able object (and maybe we can wrap select()able
things in CFWhatevers, but now it’s getting complicated).

I think the best way to do this in a cross-platform way, using as low
"power" as possible, and compatibile with every SDL target is to spawn a
new thread and do your network I/O there with a select() call or if you
have a single socket with blocking send()/recv(), then you can forward to
the SDL loop your events using SDL_USEREVENT.

The only penality I see is that to avoid using semaphores or other locking
methods you’ll have to allocate a buffer for every packet you receive to be
passed in your SDL_UserEvent struct (the SDL main thread will free the
buffer once it processes it).–
Bye,
Gabry

What do iOS and Android do?

I don’t know what Android does; iOS eventually uses Apple’s CFRunLoop, which isn’t a select()able object (and maybe we can wrap select()able things in CFWhatevers, but now it’s getting complicated).

I think the best way to do this in a cross-platform way, using as low “power” as possible, and compatibile with every SDL target is to spawn a new thread and do your network I/O there with a select() call or if you have a single socket with blocking send()/recv(), then you can forward to the SDL loop your events using SDL_USEREVENT.

The only penality I see is that to avoid using semaphores or other locking methods you’ll have to allocate a buffer for every packet you receive to be passed in your SDL_UserEvent struct (the SDL main thread will free the buffer once it processes it).

Actually you don’t. It can work like this:

(a) client allocates a buffer and intends it be filled from some socket
(b) client posts request to I/O thread
(c) I/O thread monitors socket and fills buffer
(d) I/O threads posts completion notice to client

Step (d) can use SDL event queue which is already thread safe, so no (additional)
locking is required. Step (b) requires some protocol such as a thread safe
request queue to send the request. Although this can be done with locks
it isn’t necessary if you have CAS: queues can be done without locks.
This actually suggests SDL could use a thread safe queue for portability.

Of course select is a pretty bad way to monitor sockets.
Might be OK for a game client but I’d hate to use it for a server ;(On 09/07/2013, at 6:06 PM, Gabriele Greco wrote:


john skaller
@john_skaller
http://felix-lang.org

Doing the download in a thread might be the best way to handle that. The download thread can post custom progress messages to the SDL event queue as well, if the SDL thread is doing the UI.

[note Ocaml cannot handle concurrent threads in Ocaml code,
works fine in C callouts though]

There is nothing wrong with polling loops with delays.
You can do async I/O with that (in Ocaml AFAIK).

There is everything wrong with polling loops. They needlessly cause
cpu load cause added noise (fan speed increase), heat and battery
drain. Esspecially battery drain is noticable.

MfG
GoswinOn Tue, Jul 09, 2013 at 12:04:24AM +1000, john skaller wrote:

On 08/07/2013, at 10:29 AM, Sam Lantinga wrote:

What do iOS and Android do?

I don’t know what Android does; iOS eventually uses Apple’s CFRunLoop, which isn’t a select()able object (and maybe we can wrap select()able things in CFWhatevers, but now it’s getting complicated).

I think the best way to do this in a cross-platform way, using as low “power” as possible, and compatibile with every SDL target is to spawn a new thread and do your network I/O there with a select() call or if you have a single socket with blocking send()/recv(), then you can forward to the SDL loop your events using SDL_USEREVENT.

The only penality I see is that to avoid using semaphores or other locking methods you’ll have to allocate a buffer for every packet you receive to be passed in your SDL_UserEvent struct (the SDL main thread will free the buffer once it processes it).

Actually you don’t. It can work like this:

(a) client allocates a buffer and intends it be filled from some socket
(b) client posts request to I/O thread
© I/O thread monitors socket and fills buffer
(d) I/O threads posts completion notice to client

Step (d) can use SDL event queue which is already thread safe, so no (additional)
locking is required. Step (b) requires some protocol such as a thread safe
request queue to send the request. Although this can be done with locks
it isn’t necessary if you have CAS: queues can be done without locks.
This actually suggests SDL could use a thread safe queue for portability.

This is a nice idea but often not that usable. Specifically the server
might send data to the client at any time. Like “player foo has moved
unit bar to baz”. The client would have to blindly allocate buffers
and request IO in the assumption that something probably will happen.

I think a better design is for the I/O thread to allocate buffers as
needed, fill them and pass them to the main thread. The main thread
can then either free them when it is done or hand them back to the I/O
thread for reuse.

Of course select is a pretty bad way to monitor sockets.
Might be OK for a game client but I’d hate to use it for a server ;(

You would be using poll or epoll where available for scalability
reasons on a server.

But on a client where you probably only have one socket (and a pipe,
see below) select is perfectly fine.

I’ve implemented the solution with a GUI thread and an IO thread. But
that exposed another kind of problem. The socket can close and then
the IO thread sends a signal to the GUI thread via the sdl event loop.
But the GUI can close too (user hitting the close button). Then the
GUI thread terminates while the IO thread remains stuck in the select
loop.

Usualy the OS has some way to kill threads or send signals to abort
the select loop. But that seems to have protability problems. So I
came up with another simple solution: a pipe.

At the start I create a pipe, named kill_pipe. The reading side goes
to the IO thread while the writing side remains with the GUI thread.
The IO thread includes the pipe in the select/poll/epoll call waiting
for read to be possible. When quiting the GUI thread simply closes the
writing end of the pipe. This causes the reading end of the pipe to
become ready for read and wakes up the select/poll/epoll. And the IO
thread detects that the FD ready to read is the kill_pipe and quits
too.

The same pipe could also be used to send other things, like to return
buffers. Simply write the address of the buffer to it and let the IO
thread read them back. Only on EOF on the pipe the IO thread would
terminate. Instant threadsafe queue.

MfG
GoswinOn Tue, Jul 09, 2013 at 10:02:58PM +1000, john skaller wrote:

On 09/07/2013, at 6:06 PM, Gabriele Greco wrote:

I noticed that too when I straced my binary. Even the event wait
function simply polls 100 times a second in a busy loop. Which makes
my question a bit of a mood point.

So I’ve thrown my 2 thread solution with GUI thread and IO thread out
and implemented a simple bussy loop myself:

let timeout = 0.01

while true do
(match Sdlevent.poll () with
| None -> ()
| Some event -> …);

Net.wait_for_io timeout;
done

So now I check for some SDL event and then wait up to timeout seconds
for some IO to happen, again and again and again.

MfG
GoswinOn Mon, Jul 08, 2013 at 12:12:57PM -0400, Ryan C. Gordon wrote:

So the main loop needs to do 2 things:

  1. poll the socket for downloaded data
  2. poll SDL for events

The problem is that, since SDL works with a lot of different APIs on
a lot of different platforms, we can’t reasonably select() on a file
handle (or whatever) when polling for events. Some of the targets
don’t offer any such thing at all, others offer incompatible ways so
you can’t just pile them all into one big call.

With that in mind, we can’t give you a file handle (or several file
handles) to select() on, nor accept a handle from you to add to our
select().

It has come up before, though, that this would be nice to offer in
terms of power consumption if nothing else, and I think we could
get 90% there (and maybe 100% there in uncomplicated use cases), but
it would take a lot of reworking of SDL’s internals…since most
things that use SDL don’t actually want to block until there’s new
data (since they want to redraw the screen 60 times a second, etc),
the gains of reworking this are questionable…or at least very low
priority.

–ryan.