Patch: non-blocking TCP in SDL_net

Hi everyone,

Well, I finally found some time to fulfil my threat, and added
non-blocking TCP sockets to SDL_net (current CVS, 1.2.6). Enclosed are
two diffs for SDL_net.h and SDLnetTCP.c, and a zip file containing two
test programs (source only). Please take a look at them, I’d love to
hear other people’s opinions.

Basically, non-blocking sockets mean that a Send or Recv can now return
0 as the number of bytes sent or read, if a send or read would otherwise
block.

Sockets can be made non-blocking by calling a SetNonBlocking function. I
made sure not to change the existing API; programs not using any
non-blocking functionality should still compile or link with the
modified SDL_net library, and behave exactly the same. Even the
TCPsocket struct is unchanged, though I did overload the sflag member a
bit…

The test programs: compile echoserver, start it, and run
telnet localhost 2004 . You should notice that the program still
generates output once a second, even though it’s waiting for input from
the socket. Then compile and run echoclient (kill the telnet session
first). It floods the echoserver with data, while reading none. If
compiled with SetNonBlocking(…, 0), the client stops at one point,
unable to proceed, because the Send call blocks. If you compile it with
SetNonBlocking(…, 1), the client does not block, it just repeatedly
sends 0 bytes.

What I’d really like is for these changes to make it into the official
SDL_net, since then I wouldn’t have to maintain my own version :wink: However,

  • I only tested it under Linux. The Win32 code is also in place, but
    untested.
  • Someone would need to write the MacOS::OpenTransport versions.

So, any comments on the design, interface, functionality?

Ben
-------------- next part --------------
A non-text attachment was scrubbed…
Name: echotest.zip
Type: application/x-zip-compressed
Size: 2087 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040427/c06b0c13/attachment.bin
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: SDLnetTCP.c.diff
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040427/c06b0c13/attachment.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: SDL_net.h.diff
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040427/c06b0c13/attachment.asc

Please upload/update those URL links on the SDL_net page
http://www.libsdl.org/projects/SDL_net/

Well, I finally found some time to fulfil my threat, and added
non-blocking TCP sockets to SDL_net (current CVS, 1.2.6). Enclosed are
two diffs for SDL_net.h and SDLnetTCP.c, and a zip file containing two
test programs (source only). Please take a look at them, I’d love to
hear other people’s opinions.

I’m actually looking for an SDL_net maintainer. The trick is you need to
get the code tested on all supported platforms, and support MacOS Classic.

Interested?
-Sam Lantinga, Software Engineer, Blizzard Entertainment

Well, I finally found some time to fulfil my threat, and added
non-blocking TCP sockets to SDL_net (current CVS, 1.2.6). Enclosed are
two diffs for SDL_net.h and SDLnetTCP.c, and a zip file containing two
test programs (source only). Please take a look at them, I’d love to
hear other people’s opinions.

What’s the difference between non-blocking TCP sockets and non-blocking TCP
server? There’s that nice TCP multi Server demo on the SDL_net website that
works nicely with threads.

Erm, where was this guys sourcecode for his implementation? Was it attached to
his newsgroup posting?

Benjamin Deutsch <ben fictiongroup.de> writes:

Hi everyone,

Well, I finally found some time to fulfil my threat, and added
non-blocking TCP sockets to SDL_net (current CVS, 1.2.6). Enclosed are
two diffs for SDL_net.h and SDLnetTCP.c, and a zip file containing two
test programs (source only). Please take a look at them, I’d love to
hear other people’s opinions.

Hmm, I had to rename the attachment from .bin to echotest.zip.
Strange but it worked!

Erm, how do we use your diff text?

Hi,

What’s the difference between non-blocking TCP sockets and non-blocking TCP
server? There’s that nice TCP multi Server demo on the SDL_net website that
works nicely with threads.

Well, a threaded implementation is probably to be preferred, but
sometimes it’s too much trouble, or reduces code readability. So I added
non-blocking TCP sockets to SDL_net.

A non-blocking socket is used just like a regular socket, except that it
will never, ever block. This has several implications:

  • A Recv call may also return 0 bytes read. This means that no data was
    available.
  • A Send call may not send all data at once, or even nothing. It will
    return the number of bytes sent. 0 is also possible.
  • A return value of 0 no longer signifies a closed connection. Now -1
    does that.

Of course, in their default state, sockets are still non-blocking, you
have to set them that way. Existing code should work fine.

The reason for creating non-blocking sockets was that you can do
networking in a central place in your main loop, without having it cause
any timing delays. The downside is that you must now take care of
correct input and output buffering. As I said, a multi-threaded approach
might be preferable, but hey, it’s all about choice, right?

Erm, where was this guys sourcecode for his implementation? Was it attached to
his newsgroup posting?

It was, but in a slightly awkward form, it seems. I reposted it in plain
form, as another reply in this thread.

Bye,

Ben

Well, a threaded implementation is probably to be preferred, but
sometimes it’s too much trouble, or reduces code readability. So I added
non-blocking TCP sockets to SDL_net.

Nice one.

A non-blocking socket is used just like a regular socket, except that it
will never, ever block. This has several implications:

  • A Recv call may also return 0 bytes read. This means that no data was
    available.
  • A Send call may not send all data at once, or even nothing. It will
    return the number of bytes sent. 0 is also possible.
  • A return value of 0 no longer signifies a closed connection. Now -1
    does that.

Oh, you mean like non-blocking polling opposed to interrupts?

Of course, in their default state, sockets are still non-blocking, you
have to set them that way. Existing code should work fine.

The reason for creating non-blocking sockets was that you can do
networking in a central place in your main loop, without having it cause
any timing delays. The downside is that you must now take care of
correct input and output buffering. As I said, a multi-threaded approach
might be preferable, but hey, it’s all about choice, right?

Excellent. That sounds great! Benjamin Deutsch, I found your old posting with
your patch code in it:From: Benjamin Deutsch <ben fictiongroup.de>
Subject: Patch: non-blocking TCP in SDL_net
Newsgroups: gmane.comp.lib.sdl
Date: Wed, 28 Apr 2004 00:16:04 +0200

Apparently the attached files was echoserver.c and echoclient.c. Your patch
code was embedded in the newsgroup posting. I copied it to a text file and ran:

patch -p0 < <diff_file1>

Seems to work. OMG! I patched your code on my SDL_net 1.2.5. Damn. Where can I
get SDL_net 1.2.6? And plus, have you updated your cool non-blocking patch
code? If so I would like latest version please. I can test on Cygwin/Windows.
You do linux. and I get my MACOSX friend Leynos to test/do MACOS. SDL_net
maintainers sorted! :wink:

Hi everyone,

Hmm, I had to rename the attachment from .bin to echotest.zip.
Strange but it worked!

Erm, how do we use your diff text?

Oh. Well, I was a bit overzealous and diff’d my code against the CVS
head… but given the relative stability of the code, that probably
wasn’t necessary ?-)

Anyway, I’ve attached the two files, and the contents of the zip file,
in their entirety. Enjoy!

Ben
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: SDL_net.h
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040517/2aecbb4f/attachment.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: SDLnetTCP.c
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040517/2aecbb4f/attachment.asc
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: echoclient.c
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040517/2aecbb4f/attachment-0001.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: echoserver.c
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20040517/2aecbb4f/attachment-0001.asc

Hi,

A non-blocking socket is used just like a regular socket, except that it
will never, ever block. This has several implications:

  • A Recv call may also return 0 bytes read. This means that no data was
    available.
  • A Send call may not send all data at once, or even nothing. It will
    return the number of bytes sent. 0 is also possible.
  • A return value of 0 no longer signifies a closed connection. Now -1
    does that.

Oh, you mean like non-blocking polling opposed to interrupts?

In a way, yes. I assume by “interrupts” you mean “wake me when there’s
data available”, not the “drop whatever you’re doing and deal with this
instead” meaning of interrupts.

I found your old posting with your patch code in it:

Apparently the attached files was echoserver.c and echoclient.c. Your patch
code was embedded in the newsgroup posting. I copied it to a text file and ran:

patch -p0 < <diff_file1>

That seems about right. The “echo” stuff was just a usage example.

Seems to work. OMG! I patched your code on my SDL_net 1.2.5. Damn. Where can I
get SDL_net 1.2.6?

I actually pulled the latest code from the CVS repository, since I
didn’t want to mess around in an already patched version. Check the
SDL_net main page or the documentation for CVS access instructions.

And plus, have you updated your cool non-blocking patch
code? If so I would like latest version please. I can test on Cygwin/Windows.
You do linux. and I get my MACOSX friend Leynos to test/do MACOS. SDL_net
maintainers sorted! :wink:

I had fixed a small problem with return codes. Then I foolishly posted
the complete C files to the mailing list (the complete files because the
.diff format seemed to be a bit troublesome, and foolishly because I
seem to have crossed a 50 KiB attachment limit !-) That post is now in
limbo, it seems. If the code doesn’t show up in the next two or three
days, I’ll mail it to you, if you’d like.

Bye,

Ben

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1On Monday 17 May 2004 20:23, Benjamin Deutsch wrote:

A non-blocking socket is used just like a regular socket, except that it
will never, ever block. This has several implications:

  • A Recv call may also return 0 bytes read. This means that no data was
    available.
  • A Send call may not send all data at once, or even nothing. It will
    return the number of bytes sent. 0 is also possible.
  • A return value of 0 no longer signifies a closed connection. Now -1
    does that.

This is a majorly bad idea. 0 should always be closed connection, or it
should always be 0 bytes received. There’s a reason BSD sockets work that
way.

It may seem like a minor point at first, but think about it.
Imagine you’ve written code that works with blocking sockets. This code will
certainly use some subroutines that receive a handle of the socket.
Now you decide that, for whatever reason, you want to use non-blocking
sockets from now on. You’ll have to change the error-handling in all those
subroutines.

It gets worse in cases like a C++ wrapper library that throws exceptions on
error conditions. What would a Recv() method look like in such a library?
It would have to check the type of socket along with the return type. Yet
another cause of potential bugs - and besides, it’s just ugly.

cu,
Nicolai
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAqnjOsxPozBga0lwRAifJAJkB9AOiEOihVW3xaflX/6mtMOC73QCghm5K
4rJEacRsDoIOlsu8BZlvXHc=
=Thwt
-----END PGP SIGNATURE-----

  • A return value of 0 no longer signifies a closed connection. Now -1
    does that.

This is a majorly bad idea. 0 should always be closed connection, or it
should always be 0 bytes received. There’s a reason BSD sockets work that
way.

Well, not really majorly, but I see your point. It’s true that one
consistency is broken, but another is gained: the sockets always return
the number of bytes actually read or sent.

It may seem like a minor point at first, but think about it.
Imagine you’ve written code that works with blocking sockets. This code will
certainly use some subroutines that receive a handle of the socket.

Um, by “handle of a socket”, are you talking about file descriptors? No,
the application code only sees the SDL_net “sockets”. It’s an
abstraction layer, after all. I’m going to assume you mean the SDL_net
constructs, please correct me if this is wrong.

Now you decide that, for whatever reason, you want to use non-blocking
sockets from now on. You’ll have to change the error-handling in all those
subroutines.

Whoa, the error handling should be the least of your problems.
Converting to non-blocking sockets is a major change, and needs to be
thoroughly planned. Sending gets much more complicated, since you now
need to cope with partial sends. The use of blocking or non-blocking
sockets is a big design choice, either the rest of your code is written
for the one, or the other. There’s no real point in being able to switch
in between.

Besides, error handling in applications that use SDL_net typically looks
like this:
if (Recv(bla, bla) <= 0) {
maybe_report_error(“Recv went wrong”);
forget_about_connection();
if (not_much_point_otherwise) exit(1);
}

Now imagine if non-blocking sockets returned -1 for EAGAIN and 0 for
EOF. You’ll have to change the error-handling in all those subroutines:

ret_code = Recv(bla, bla);
if (ret_code <= 0 unless errno == EAGAIN) {
drop_connection_and_probably_exit(1);
} else if (ret_code < 0 and errno == EAGAIN) {
pretend_zero_bytes_were_returned_as_read();
}

Ok, that’s not fully ANSI C :wink: The thing is, the error checking becomes
much worse. In fact, the first thing I’d do if faced with such an API
would be to create a wrapper that sets ret_code = 0 iff errno = EAGAIN.

It gets worse in cases like a C++ wrapper library that throws exceptions on
error conditions. What would a Recv() method look like in such a library?
It would have to check the type of socket along with the return type. Yet
another cause of potential bugs - and besides, it’s just ugly.

True, but the wrapper library will probably use its own socket creation
methods, so the socket type is not a problem, if it offers non-blocking
sockets at all. Anyway, you wouldn’t want to raise an exception on
EAGAIN, would you? So you’d still have to check the type of socket along
with the return type. As you say, yet another cause of potential bugs,
and just as ugly, if not uglier (since I can imagine EOF raising an
exception too, or will you check the return value for this?).

Anyway, I see SDL_net as a purposefuly simple abstraction layer, with an
emphasis on ease-of-use and understandability. Try telling a fledgling
network programmer why a seeminlgy normal operation (reading nothing)
returns an “error”, while a closed connection (much more an error in
their eyes) is treated differently. Hm, on the other hand, that is how
the “real world” works… perhaps a big red warning box in the
documentation?

What do others think?

Bye,
Ben

This is a majorly bad idea. 0 should always be closed connection, or it
should always be 0 bytes received. There’s a reason BSD sockets work that
way.

It may seem like a minor point at first, but think about it.
Imagine you’ve written code that works with blocking sockets. This code will
certainly use some subroutines that receive a handle of the socket.
Now you decide that, for whatever reason, you want to use non-blocking
sockets from now on. You’ll have to change the error-handling in all those
subroutines.

If you change a fairly network-dependent project from blocking to
non-blocking sockets, you will have a lot more work to do than just
changing the error handling. Changing something like that means
redesigning a pretty fundamental part of your program, which means if you
can’t count on it staying the same, you should have it abstracted with
wrappers anyway.

It seems there have been a lot of posts saying non-blocking socket support
is a bad idea, and I don’t quite understand that. Blocking sockets are a
problem: you can’t always sit and wait on a socket. So we need some sort
of solution for that. Telling people to use threads seems like an answer
people only settled on because it was easier than making the sockets
"fully featured" and allowing them to be non-blocking, giving more control
over their use to the programmers. Threads are one way to solve the
blocking problem, but they really don’t seem like the best way, and
definitely not always the best way.

Choosing 0 for the return value of Recv() on a closed socket was a bad
idea. And every piece of code that was written to expect that return
value was the price we paid for that bad idea :wink: Why? Because it’s
perfectly possible to read 0 bytes (never mind the many ways we could
interpret it), but it is not possible to read -1 bytes, for example. In a
perfect world, a return value of 0 would never have been used, so just
because BSD sockets have worked that way for however many decades it’s
been now, doesn’t mean it’s the best way. It just means everyone is
afraid to change it. (And rightly so, because in BSD’s case, it’s not
worth it.)

But if I read the previous e-mail correctly, the default behavior of a
socket is still blocking, which does not affect any existing code, and you
have to deliberately set a socket to non-blocking. Since that
non-blocking support wasn’t available before, there is no existing code to
break.

If I understand correctly, and that’s how it’s meant to work, then I would
be very glad to see non-blocking support added. I think it’s much
overdue :slight_smile:

Ryan