SDL_Net Server Stuff

I’m writing a server for an RPG using SDL_Net and I’m running into some
problems
early on.

I’m wondering how I should handle the UDP sockets for incoming messages
from
clients.

The easiest thing to do as I see it would be to create a socket for each
player on different
ports. So, I guess the question is: What is the best way to find an
unused port? I could
call SDLNet_UDP_Open(0), but the client has to know what port it ended
up on.
SDLNet_UDP_GetPeerAddress doesn’t work for ports opened that way though.

My next idea was to create one socket and accept all packets on that
port, and tag each
packet with an ID corresponding to a player ID. I could then go through
a binary tree
of the players placed by ID and find the player who needs the packet.
Then that Player
object would handle the body of the packet.

Any suggestions, answers, comments?
What’s the best way to do this?

-Randall Leeds

Basically, using one port is the best solution. However, do not rely on a
player ID - that would just be asking for attacks.
Instead, use the from address that should be returned to you by SDL_net (I
haven’t done UDP through SDL_net yet, but if it doesn’t return an address
that would be a really serious flaw ;))

cu,
NicolaiAm Samstag, 16. Februar 2002 21:03 schrieb Randall Leeds:

I’m writing a server for an RPG using SDL_Net and I’m running into some
My next idea was to create one socket and accept all packets on that
port, and tag each
packet with an ID corresponding to a player ID. I could then go through
a binary tree
of the players placed by ID and find the player who needs the packet.
Then that Player
object would handle the body of the packet.

I’m writing a server for an RPG using SDL_Net and I’m running into some
My next idea was to create one socket and accept all packets on that
port, and tag each
packet with an ID corresponding to a player ID. I could then go through
a binary tree
of the players placed by ID and find the player who needs the packet.
Then that Player
object would handle the body of the packet.

Basically, using one port is the best solution. However, do not rely on a
player ID - that would just be asking for attacks.
Instead, use the from address that should be returned to you by SDL_net (I
haven’t done UDP through SDL_net yet, but if it doesn’t return an address
that would be a really serious flaw ;))

Big guru R.W.Stevens says that connected UDP sockets have significantly less
overhead (the kernel seems to connect implicitly for each sendto which can be avoided
by this). (But maybe that’s not a problem with winsock and maybe current Linux has
solved this, after all, the books have been written years ago? Someone knows more?)

He recommends having one socket per connection. So I would think:

One primary socket on a well-known port on the server.
Client sends a packet. Server reads and opens a new socket on a different port and
connects it to the client, sends answer from there.
Client connects to the new socket. So both sockets are connected.
That way select() multiplexing tells you who wants to talk to you. The maximum number of
open sockets can be queried by WSAstartup() on windows (somewhere in the library, is
128 on my '98 box).

Should this be called wasting of resources?

On top of that some kind of protocol is needed to acknowledge sendings and resend lost
packets.

I am actually planning to implement this, but not with SDL_net, just bare sockets (does
not work on Mac then, but is only 100-200 lines of C). At the moment I am thinking about
the protocol logic which would also be needed on top of SDL_net with UDP. Someone
has a ready-to-use recipe for this?

Regards,

Carsten

Big guru R.W.Stevens says that connected UDP sockets have significantly
less overhead (the kernel seems to connect implicitly for each sendto which
can be avoided by this). (But maybe that’s not a problem with winsock and
maybe current Linux has solved this, after all, the books have been written
years ago? Someone knows more?)

He recommends having one socket per connection. So I would think:

One primary socket on a well-known port on the server.
Client sends a packet. Server reads and opens a new socket on a different
port and connects it to the client, sends answer from there.
Client connects to the new socket. So both sockets are connected.
That way select() multiplexing tells you who wants to talk to you. The
maximum number of open sockets can be queried by WSAstartup() on windows
(somewhere in the library, is 128 on my '98 box).

Ugh. I don’t know who R.W.Stevens is, but logic tells me that he’s just plain
wrong.
First of all, if there is any overhead with unconnected UDP sockets, it seems
to be a problem in one particular kernel, and as you say yourself should be
eliminated by now. This problem may or may not exist (it’s likely to not
exist I should think) in other kernels. Since SDL is all about writing
portable code, you shouldn’t optimize for a single crappy system, especially
when this “optimization” results in a general decrease of efficiency.

Firstly, using multiple sockets implies an OS-defined limit on the number of
clients you can have.
Secondly, select() and poll() reportedly don’t scale well with a huge number
of sockets to wait on. Using a single socket and then looking up the client
by remote address using binary search is much more efficient.
Thridly, the logon process you lined out is just ugly and has potential proxy
/ NAT problems.

Depending on the implementation, there may be an advantage for multiple
sockets though. The OS might maintain seperate packet queues for each socket,
which could lead to a bigger overall packet queue with multiple sockets (so
you drop less packets under load).
Unfortunately, I have never run a MMO server where that could be an issue.
Maybe someone has more information on that topic?

On top of that some kind of protocol is needed to acknowledge sendings and
resend lost packets.

I am actually planning to implement this, but not with SDL_net, just bare
sockets (does not work on Mac then, but is only 100-200 lines of C). At the
moment I am thinking about the protocol logic which would also be needed on
top of SDL_net with UDP. Someone has a ready-to-use recipe for this?

This is what Quake and Half-Life use, and what I’ve implemented into my own
game (more or less). The protocol supports both reliable and unreliable data,
however it does not (for simplicity) allow data to arrive out of order, as
this would make the processing of commands unnecessarily complex anyway, and
the focus of the protocol is on unreliable data.

Every packet is prefixed with a short header, which includes a sequence (SEQ)
and an acknowledge (ACK) number. The highest two bits of these numbers are
reserved for other purposes.
Whenever a packet is sent, the SEQ is increased by one. The sent ACK is
always the last received sequence number. Unreliable data is just sent like
this.

Whenever there’s reliable data to send, a bit in the SEQ is set, indicating
reliable data. If the packet only contains part of a block of reliable data,
a second bit is also set.
After a packet with reliable data has been sent, the sender will not send
anymore packets with reliable data until the previous one is acknowledged.

On the recieving end, packets with illogical SEQ/ACK (this includes old SEQs)
are silently ignored. This will prevent most IP spoofing attacks.
If the packet doesn’t contain reliable data, its contents are simply
forwarded to the game logic.
If, on the other hand, it does contain reliable data, a number of things is
done:
First of all, the protocol keeps track of a “reliable sequence”. The reliable
sequence is a single bit which is flipped whenever a reliable packet is
recieved. This reliable sequence is sent together with the ACK of a packet.
If a party receives a flipped reliable sequence bit, it knows that the
previous reliable packet has been received.

Secondly, if there is more reliable data to wait for (second bit in ACK is
set), the data is stored for later. Otherwise, all the reliable data received
up to now is forwarded to the game logic.

Unreliable packets are never resent (obviously)
Reliable packets are resent when we recieve an ACK greater than the original
SEQ number of the reliable packets without a change of the reliable sequence
bit.
Additionally, the protocol forces an empty packet every 500ms if no other
data is being sent.

Whenever a reliable packet is sent, the code looks for unreliable data that
might be put into the packet as well. The receiving game logic doesn’t have
to differentiate between reliable and unreliable data.

This protocol works pretty well for a shooter situation: both sides keep
sending data; the server sends frame updates at a constant rates, and the
client regularly sends movement commands. All these events are unreliable
(note that a client’s movement command also contains the two previous
movement commands as backup).
The occasional reliable data will eventually reach the other side without
impacting overall performance.

However, the maximum reliable data bandwidth is rather low, because no new
reliable data is sent until the previous block is acknowledged. So if you
really need lots of reliable data, you need to come up with something else.

cu,
NicolaiAm Sonntag, 17. Februar 2002 10:36 schrieb Carsten Burstedde:

I think, since I am writing an RPG, there doesn’t need to be much
reliable data
sent. If someones updated coordinates are sent, and not received, the
other
players will just deal with them skipping ahead a few feet when the next
updated
coordinates are sent (hopefully received). I’ve implemented the binary
tree of the
clients’ addresses and I have one socket accepting all the data.

I think I will have each client send what is similar to the SEQ number
you speak of,
this way if a packet is received out of order, it will be ignored. Any
packet loss is
not important as I see it, can anyone think of a situation where I would
need to get
information there?

The ACK and SEQ system you speak of sounds really nice, but as you said,
you have
the server and clients communicating constantly. I think that’s fine
for a 16-person game
of Quake, but not for 1000 RPGers.

Does anyone think I should put in a reliable packet system?

-Randall LeedsOn Sunday, February 17, 2002, at 09:10 AM, Nicolai Haehnle wrote:

Am Sonntag, 17. Februar 2002 10:36 schrieb Carsten Burstedde:

Big guru R.W.Stevens says that connected UDP sockets have significantly
less overhead (the kernel seems to connect implicitly for each sendto
which
can be avoided by this). (But maybe that’s not a problem with winsock
and
maybe current Linux has solved this, after all, the books have been
written
years ago? Someone knows more?)

He recommends having one socket per connection. So I would think:

One primary socket on a well-known port on the server.
Client sends a packet. Server reads and opens a new socket on a
different
port and connects it to the client, sends answer from there.
Client connects to the new socket. So both sockets are connected.
That way select() multiplexing tells you who wants to talk to you. The
maximum number of open sockets can be queried by WSAstartup() on
windows
(somewhere in the library, is 128 on my '98 box).

Ugh. I don’t know who R.W.Stevens is, but logic tells me that he’s just
plain
wrong.
First of all, if there is any overhead with unconnected UDP sockets, it
seems
to be a problem in one particular kernel, and as you say yourself
should be
eliminated by now. This problem may or may not exist (it’s likely to not
exist I should think) in other kernels. Since SDL is all about writing
portable code, you shouldn’t optimize for a single crappy system,
especially
when this “optimization” results in a general decrease of efficiency.

Firstly, using multiple sockets implies an OS-defined limit on the
number of
clients you can have.
Secondly, select() and poll() reportedly don’t scale well with a huge
number
of sockets to wait on. Using a single socket and then looking up the
client
by remote address using binary search is much more efficient.
Thridly, the logon process you lined out is just ugly and has potential
proxy
/ NAT problems.

Depending on the implementation, there may be an advantage for multiple
sockets though. The OS might maintain seperate packet queues for each
socket,
which could lead to a bigger overall packet queue with multiple sockets
(so
you drop less packets under load).
Unfortunately, I have never run a MMO server where that could be an
issue.
Maybe someone has more information on that topic?

On top of that some kind of protocol is needed to acknowledge sendings
and
resend lost packets.

I am actually planning to implement this, but not with SDL_net, just
bare
sockets (does not work on Mac then, but is only 100-200 lines of C).
At the
moment I am thinking about the protocol logic which would also be
needed on
top of SDL_net with UDP. Someone has a ready-to-use recipe for this?

This is what Quake and Half-Life use, and what I’ve implemented into my
own
game (more or less). The protocol supports both reliable and unreliable
data,
however it does not (for simplicity) allow data to arrive out of order,
as
this would make the processing of commands unnecessarily complex
anyway, and
the focus of the protocol is on unreliable data.

Every packet is prefixed with a short header, which includes a sequence
(SEQ)
and an acknowledge (ACK) number. The highest two bits of these numbers
are
reserved for other purposes.
Whenever a packet is sent, the SEQ is increased by one. The sent ACK is
always the last received sequence number. Unreliable data is just sent
like
this.

Whenever there’s reliable data to send, a bit in the SEQ is set,
indicating
reliable data. If the packet only contains part of a block of reliable
data,
a second bit is also set.
After a packet with reliable data has been sent, the sender will not
send
anymore packets with reliable data until the previous one is
acknowledged.

On the recieving end, packets with illogical SEQ/ACK (this includes old
SEQs)
are silently ignored. This will prevent most IP spoofing attacks.
If the packet doesn’t contain reliable data, its contents are simply
forwarded to the game logic.
If, on the other hand, it does contain reliable data, a number of
things is
done:
First of all, the protocol keeps track of a “reliable sequence”. The
reliable
sequence is a single bit which is flipped whenever a reliable packet is
recieved. This reliable sequence is sent together with the ACK of a
packet.
If a party receives a flipped reliable sequence bit, it knows that the
previous reliable packet has been received.

Secondly, if there is more reliable data to wait for (second bit in ACK
is
set), the data is stored for later. Otherwise, all the reliable data
received
up to now is forwarded to the game logic.

Unreliable packets are never resent (obviously)
Reliable packets are resent when we recieve an ACK greater than the
original
SEQ number of the reliable packets without a change of the reliable
sequence
bit.
Additionally, the protocol forces an empty packet every 500ms if no
other
data is being sent.

Whenever a reliable packet is sent, the code looks for unreliable data
that
might be put into the packet as well. The receiving game logic doesn’t
have
to differentiate between reliable and unreliable data.

This protocol works pretty well for a shooter situation: both sides keep
sending data; the server sends frame updates at a constant rates, and
the
client regularly sends movement commands. All these events are
unreliable
(note that a client’s movement command also contains the two previous
movement commands as backup).
The occasional reliable data will eventually reach the other side
without
impacting overall performance.

However, the maximum reliable data bandwidth is rather low, because no
new
reliable data is sent until the previous block is acknowledged. So if
you
really need lots of reliable data, you need to come up with something
else.

cu,
Nicolai


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

I just got an idea.
What if I keep the initial TCP connection that I use for login open, and
send reliable
information on that. I’ll send all my unreliable information on UDP,
but whenever I
need the occasional must-be-recieved packet to be sent, I’ll send it
over the TCP
connection.

Does this sound like a good or a bad idea?

-Randall LeedsOn Sunday, February 17, 2002, at 09:10 AM, Nicolai Haehnle wrote:

Am Sonntag, 17. Februar 2002 10:36 schrieb Carsten Burstedde:

Big guru R.W.Stevens says that connected UDP sockets have significantly
less overhead (the kernel seems to connect implicitly for each sendto
which
can be avoided by this). (But maybe that’s not a problem with winsock
and
maybe current Linux has solved this, after all, the books have been
written
years ago? Someone knows more?)

He recommends having one socket per connection. So I would think:

One primary socket on a well-known port on the server.
Client sends a packet. Server reads and opens a new socket on a
different
port and connects it to the client, sends answer from there.
Client connects to the new socket. So both sockets are connected.
That way select() multiplexing tells you who wants to talk to you. The
maximum number of open sockets can be queried by WSAstartup() on
windows
(somewhere in the library, is 128 on my '98 box).

Ugh. I don’t know who R.W.Stevens is, but logic tells me that he’s just
plain
wrong.
First of all, if there is any overhead with unconnected UDP sockets, it
seems
to be a problem in one particular kernel, and as you say yourself
should be
eliminated by now. This problem may or may not exist (it’s likely to not
exist I should think) in other kernels. Since SDL is all about writing
portable code, you shouldn’t optimize for a single crappy system,
especially
when this “optimization” results in a general decrease of efficiency.

Firstly, using multiple sockets implies an OS-defined limit on the
number of
clients you can have.
Secondly, select() and poll() reportedly don’t scale well with a huge
number
of sockets to wait on. Using a single socket and then looking up the
client
by remote address using binary search is much more efficient.
Thridly, the logon process you lined out is just ugly and has potential
proxy
/ NAT problems.

Depending on the implementation, there may be an advantage for multiple
sockets though. The OS might maintain seperate packet queues for each
socket,
which could lead to a bigger overall packet queue with multiple sockets
(so
you drop less packets under load).
Unfortunately, I have never run a MMO server where that could be an
issue.
Maybe someone has more information on that topic?

On top of that some kind of protocol is needed to acknowledge sendings
and
resend lost packets.

I am actually planning to implement this, but not with SDL_net, just
bare
sockets (does not work on Mac then, but is only 100-200 lines of C).
At the
moment I am thinking about the protocol logic which would also be
needed on
top of SDL_net with UDP. Someone has a ready-to-use recipe for this?

This is what Quake and Half-Life use, and what I’ve implemented into my
own
game (more or less). The protocol supports both reliable and unreliable
data,
however it does not (for simplicity) allow data to arrive out of order,
as
this would make the processing of commands unnecessarily complex
anyway, and
the focus of the protocol is on unreliable data.

Every packet is prefixed with a short header, which includes a sequence
(SEQ)
and an acknowledge (ACK) number. The highest two bits of these numbers
are
reserved for other purposes.
Whenever a packet is sent, the SEQ is increased by one. The sent ACK is
always the last received sequence number. Unreliable data is just sent
like
this.

Whenever there’s reliable data to send, a bit in the SEQ is set,
indicating
reliable data. If the packet only contains part of a block of reliable
data,
a second bit is also set.
After a packet with reliable data has been sent, the sender will not
send
anymore packets with reliable data until the previous one is
acknowledged.

On the recieving end, packets with illogical SEQ/ACK (this includes old
SEQs)
are silently ignored. This will prevent most IP spoofing attacks.
If the packet doesn’t contain reliable data, its contents are simply
forwarded to the game logic.
If, on the other hand, it does contain reliable data, a number of
things is
done:
First of all, the protocol keeps track of a “reliable sequence”. The
reliable
sequence is a single bit which is flipped whenever a reliable packet is
recieved. This reliable sequence is sent together with the ACK of a
packet.
If a party receives a flipped reliable sequence bit, it knows that the
previous reliable packet has been received.

Secondly, if there is more reliable data to wait for (second bit in ACK
is
set), the data is stored for later. Otherwise, all the reliable data
received
up to now is forwarded to the game logic.

Unreliable packets are never resent (obviously)
Reliable packets are resent when we recieve an ACK greater than the
original
SEQ number of the reliable packets without a change of the reliable
sequence
bit.
Additionally, the protocol forces an empty packet every 500ms if no
other
data is being sent.

Whenever a reliable packet is sent, the code looks for unreliable data
that
might be put into the packet as well. The receiving game logic doesn’t
have
to differentiate between reliable and unreliable data.

This protocol works pretty well for a shooter situation: both sides keep
sending data; the server sends frame updates at a constant rates, and
the
client regularly sends movement commands. All these events are
unreliable
(note that a client’s movement command also contains the two previous
movement commands as backup).
The occasional reliable data will eventually reach the other side
without
impacting overall performance.

However, the maximum reliable data bandwidth is rather low, because no
new
reliable data is sent until the previous block is acknowledged. So if
you
really need lots of reliable data, you need to come up with something
else.

cu,
Nicolai


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

I just got an idea.
What if I keep the initial TCP connection that I use for login open, and
send reliable
information on that. I’ll send all my unreliable information on UDP,
but whenever I
need the occasional must-be-recieved packet to be sent, I’ll send it
over the TCP
connection.

Does this sound like a good or a bad idea?

-Randall Leeds

That’s a good idea. In fact, many professional games do something like this.
Each client has one UDP and one TCP socket connected to the server, and the
server
has one UDP socket and TCP sockets connected to the clients.

Dirk Gerrits

i simply have a thread blocking on the known server port, then when the
join handshaking of a client takes place, I would spawn an individual
client thread and open a specific client socket, using 0 as you mentioned.
This is ok, but it does not exec a bind on the socket, which is not really
a problem in this design. The client thread simply blocks on the new
client socket waiting for info.On Sat, 16 Feb 2002, Randall Leeds wrote:

I’m writing a server for an RPG using SDL_Net and I’m running into some
problems
early on.

I’m wondering how I should handle the UDP sockets for incoming messages
from
clients.

The easiest thing to do as I see it would be to create a socket for each
player on different
ports. So, I guess the question is: What is the best way to find an
unused port? I could
call SDLNet_UDP_Open(0), but the client has to know what port it ended
up on.
SDLNet_UDP_GetPeerAddress doesn’t work for ports opened that way though.

My next idea was to create one socket and accept all packets on that
port, and tag each
packet with an ID corresponding to a player ID. I could then go through
a binary tree
of the players placed by ID and find the player who needs the packet.
Then that Player
object would handle the body of the packet.

Any suggestions, answers, comments?
What’s the best way to do this?

-Randall Leeds


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

Nicolai Haehnle <prefect_ at gmx.net> wrote:

Big guru R.W.Stevens says that connected UDP sockets have
significantly less overhead (the kernel seems to connect implicitly
for each sendto which can be avoided by this). (But maybe that’s not
a problem with winsock and maybe current Linux has solved this, after
all, the books have been written years ago? Someone knows more?)

He recommends having one socket per connection. So I would think:

[snip]

Ugh. I don’t know who R.W.Stevens is, but logic tells me that he’s
just plain wrong.
First of all, if there is any overhead with unconnected UDP sockets,
it seems to be a problem in one particular kernel, and as you say
yourself should be eliminated by now. This problem may or may not
exist (it’s likely to not exist I should think) in other kernels.
Since SDL is all about writing portable code, you shouldn’t optimize
for a single crappy system, especially when this "optimization"
results in a general decrease of efficiency.

He’s talking about W. Richard Stevens, the late author of numerous books
on network programming and advanced programming under Unix and winner of
the USENIX Lifetime Acheivement Award. In his lifetime he probably
forgot more about programming in general, and network programming
specifically, than you are ever going to know.

WRS Home Page: http://www.kohala.com/start/
WRS Obituary:
http://dan.yosemite.ca.us/6bone/w.richard.stevens.obituary.html

That said, I don’t think that Mr. Burstedde, the original poster,
accurately represents Mr. Steven’s position, which was hardly so
unequivocal.

S-

He’s talking about W. Richard Stevens, the late author of numerous books
on network programming and advanced programming under Unix and winner of
the USENIX Lifetime Acheivement Award. In his lifetime he probably
forgot more about programming in general, and network programming
specifically, than you are ever going to know.

Thanks for pointing that out. Important.

That said, I don’t think that Mr. Burstedde, the original poster,
accurately represents Mr. Steven’s position, which was hardly so
unequivocal.

Wait… some quotes from “Unix Networking Programming”, Vol. 1, second edition,
Prentice Hall:

Chapter 8 (Elementary UDP sockets), p. 226f.

“… Berkeley derived kernels temporarily connect the socket, send the datagram, and
then unconnect the socket … [Partridge and Pink 1993] note that the temporary
connecting of an unconnected UDP socket accounts for nearly ond-third of the cost of
each UDP transmission.”

I know that 1993 is prehistoric, and I mentioned that (though not in full explicity) in my
posting. I think that this is probably not an issue today, but wanted to ask anyway.

Chapter 20 (Advanced UDP sockets), p. 558 (yeah, it’s over 1000 pages, and vol. 2 is
equally big! Great stuff.)

“… but how does the server distinguish between subsequent datagrams from that client,
and new requests? The typical solution to this problem is for the server to create a new
socket for each client, bind an ephemeral port to that socket, and use that socket for all
its replies. This requires that the client look at the port number of the server’s first reply
and send subsequent datagrams for this request to that port.”

However, Nicolai is perfectly right - this would not work with strict NAT/firewall gateways.
(For quake 2 a special kernel module on the firewall for NAT was needed - similar
problems there??) So the approach which sounds more practicable is a single socket
(maybe on both UDP and TCP) and an internal client number header field (with some
startup uniqueness issues in case two clients go through the same NAT gateway).

Regards,

Carsten