CVS snapshot (networking)

slouken · November 18, 1998, 4:59am

The latest CVS snapshot, available from the download page, contains
cross-platform network code and GUI code, using a chat client/server
as the demo.

Grab both the SDL source and the example archive.

Check out the new stuff.
As an easter egg, if you compile the chat client, and run it as:
chat devolution.com
You’ll connect to a live chat server for SDL discussion.

I’ll be on as often as I can the next few days.

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

Matt_Slot · November 18, 1998, 5:07am

The latest CVS snapshot, available from the download page, contains
cross-platform network code and GUI code, using a chat client/server
as the demo.

hahah, thanks.

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */

Matt_Slot · November 18, 1998, 6:01am

You’re welcome. Do you have a system to compile it on?

I can send you win32 binaries if you want.

No need, I just wanted to scan the headers and sources. Looks like some
convenience wrappers around the BSD Sockets calls – pretty much what
you advertised.

I’ve attached the header file for my network abstraction layer – which
works on MacOS (MacTCP, OpenTransport, AppleTalk), sockets, and WinSock.
I still need to add Serial and IPX support, but the framework is in place.

Sorry, but the source isn’t available – we’re still trying to work out the
licensing details, but most likely it’ll be commercial instead of freebie.
I call it Light Sockets, more as a play on words than anything, as the API
doesn’t really relate to BSD Sockets.

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */-----

/* File “lsockets.h”, Light Sockets - Copyright © Matt Slot, 1996-7 /
/ Public file that has the relevant declarations to use Light Sockets. */

#ifndef LIGHT_SOCKETS_HEADER
#define LIGHT_SOCKETS_HEADER

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Preprocessor Includes */

#ifndef STANDARD_TYPES_HEADER
#include “stdtypes.h”
#endif /* STANDARD_TYPES_HEADER */

#ifndef STANDARD_ERROR_HEADER
#include “stderror.h”
#endif /* STANDARD_ERROR_HEADER */

#ifndef PLATFORM_HEADER
#include “platform.h”
#endif /* PLATFORM_HEADER */

#if defined(PLATFORM_MACINTOSH)
#include <Timer.h>
#include <OpenTransport.h>
#elif defined(PLATFORM_UNIX)
#include <sys/types.h>
#include <sys/socket.h>
#endif

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Functional Summary */

/*
Light Sockets provides a lightweight abstraction layer for networking routines.
It offers asynchronous throughput, interrupt-safe send or receive routines, and
callback, polled, or synchronous data access. Network endpoints, addresses, and
queues are managed using opaque data types to encourage platform independence.

This header is the primary public interface for Light Sockets, so it contains
alot of declarations. Please refer to the accompanying documentation for more
information, contextual overviews, and function descriptions.

This file is broken into 4 sections: fundamental types and enumerations,
address structures, data element structures, and public function prototypes.
*/

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Preprocessor Declarations */

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Structure/Class Declarations */

typedef UInt16 NetworkElemType;

enum {
eNetworkElemCustom = 0,
eNetworkElemResolver = 1,
eNetworkElemDatagram = 2,
eNetworkElemStream = 3
};

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef UInt16 NetworkStackType;
typedef UInt32 NetworkStackMaskType;

enum {
eNetworkStackTCPIP = 0,
eNetworkStackATalk = 1,

eNetworkStackTCPIPBit =     eNetworkStackTCPIP,
eNetworkStackATalkBit =     eNetworkStackATalk,

eNetworkStackTCPIPMask =    (1L << eNetworkStackTCPIPBit),
eNetworkStackATalkMask =    (1L << eNetworkStackATalkBit),
eNetworkStackAllMask =      eNetworkStackTCPIPMask | eNetworkStackATalkMask
};

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef UInt16 NetworkStatusType;

enum {
eNetworkStatusClosed = 0,
eNetworkStatusListening = 1,
eNetworkStatusAccepting = 2,
eNetworkStatusConnecting = 3,
eNetworkStatusTransfer = 4,
eNetworkStatusLocalClose = 5,
eNetworkStatusRemoteClose = 6
};

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef UInt16 NetworkEventType;
typedef UInt32 NetworkEventMaskType;

enum {
/* Network endpoint events */
eNetworkEventFirst = 0,
eNetworkEventIdle = 1,
eNetworkEventLast = 2,

/* Network operation events */
eNetworkEventResolveData =       8,
eNetworkEventResolveDone =       9,
eNetworkEventSendDone =         10,
eNetworkEventSendFailed =       11,
eNetworkEventRecvData =         12,
eNetworkEventRecvXData =        13,
eNetworkEventFlood =            14,
eNetworkEventServer =           15,
eNetworkEventTransfer =         16,
eNetworkEventClosing =          17,

/* Application defined events */
eNetworkEventCustomFirst =      24,
eNetworkEventCustomLast =       31,


/* Corresponding bitfield values */
eNetworkEventFirstBit =         eNetworkEventFirst,
eNetworkEventIdleBit =          eNetworkEventIdle,
eNetworkEventLastBit =          eNetworkEventLast,
eNetworkEventResolveDataBit =   eNetworkEventResolveData,
eNetworkEventResolveDoneBit =   eNetworkEventResolveDone,
eNetworkEventSendDoneBit =      eNetworkEventSendDone,
eNetworkEventSendFailedBit =    eNetworkEventSendFailed,
eNetworkEventRecvDataBit =      eNetworkEventRecvData,
eNetworkEventRecvXDataBit =     eNetworkEventRecvXData,
eNetworkEventFloodBit =         eNetworkEventFlood,
eNetworkEventServerBit =        eNetworkEventServer,
eNetworkEventTransferBit =      eNetworkEventTransfer,
eNetworkEventClosingBit =       eNetworkEventClosing,

/* Corresponding bitfield masks */
eNetworkEventFirstMask =        (1L << eNetworkEventFirstBit),
eNetworkEventIdleMask =         (1L << eNetworkEventIdleBit),
eNetworkEventLastMask =         (1L << eNetworkEventLastBit),
eNetworkEventResolveDataMask =  (1L << eNetworkEventResolveDataBit),
eNetworkEventResolveDoneMask =  (1L << eNetworkEventResolveDoneBit),
eNetworkEventSendDoneMask =     (1L << eNetworkEventSendDoneBit),
eNetworkEventSendFailedMask =   (1L << eNetworkEventSendFailedBit),
eNetworkEventRecvDataMask =     (1L << eNetworkEventRecvDataBit),
eNetworkEventRecvXDataMask =    (1L << eNetworkEventRecvXDataBit),
eNetworkEventFloodMask =        (1L << eNetworkEventFloodBit),
eNetworkEventServerMask =       (1L << eNetworkEventServerBit),
eNetworkEventTransferMask =     (1L << eNetworkEventTransferBit),
eNetworkEventClosingMask =      (1L << eNetworkEventClosingBit),
};

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef UInt32 NetworkControlMessageType;
typedef UInt32 NetworkControlFlagsType;

enum {
/* Network stack messages */
eNetworkControlStackFirst = 0x00000000,
eNetworkControlStackLast = 0x00001FFF,

    /* Suggest that our network stack make aggressive use of memory when
       allocating internal buffers (for the stack and its endpoints). The
       effects of this message will vary by platform, but the value passed
       in flags should indicate an approximate memory footprint (in bytes)
       or 0 for default. This message is supported on TCP/IP and AppleTalk
       network stacks. */
    eNetworkControlStackAggressive = 0x00000000,

    /* Enable packet delivery between endpoints on the localhost (Note:
       no way has been provided to disable the feature). This message
       is supported on AppleTalk network stack. */
    eNetworkControlStackSelfSend =  0x00020000,

    /* Return a list of zones on the AppleTalk network, where data is a 
       char ** (the address of a char *) and flags indicate which zones to
       return. The function allocates a packed struct of zero-terminated C
       strings, and returns the pointer to them in the data parameter. This
       message is supported on AppleTalk network stack. */
    eNetworkControlStackGetZones =  0x00020001,
    eNetworkControlFlagsGetMyZone =     0x00000000,
    eNetworkControlFlagsGetLocalZones = 0x00000001,
    eNetworkControlFlagsGetFullZones =  0x00000002,

/* Common endpoint messages */
eNetworkControlEndpointFirst =      0x00002000,
eNetworkControlEndpointLast =       0x00003FFF,

    /* Enable name binding for a data endpoint (via NBP), where flags is 1 
       to indicate enabling or 0 for disabling, and data points to a zero
       terminated string. To request specific service name, pass the object
       and type name string (eg, "xxx:yyy"); to use automatically use the
       workstation's "flagship" name, use the object wildcard (eg, "=:yyy").
       This message is supported on AppleTalk network stack. */
    eNetworkControlEndpointName =   0x00022000,

/* Resolver endpoint messages */
eNetworkControlResolveFirst =       0x00004000,
eNetworkControlResolveLast =        0x00005FFF,

    /* Enable multiple results for resolve and lookup calls where flags is
       the maximum number of results (0 = unlimited, 1 = default). This
       message is supported on TCP/IP and AppleTalk resolver endpoints. */
    eNetworkControlResolveMultiple =    0x00004000,

/* Datagram endpoint messages */
eNetworkControlDatagramFirst =      0x00006000,
eNetworkControlDatagramLast =       0x00007FFF,

    /* Enable reporting of failed datagram sends due to network failures.
       (Flags field should be 1 to enable, 0 to disable). This message
       is supported on TCP/IP and AppleTalk datagram endpoints. */
    eNetworkControlDatagramFailures =   0x00006000,

    /* Retreive the hostmask used for calculating broadcast addresses
       into the NetworkAddress structure referenced by data (an empty
       address for "no support"). This message is supported on *active*
       TCP/IP and AppleTalk datagram endpoints. */
    eNetworkControlDatagramHostMask =   0x00006001,

/* Stream endpoint messages */
eNetworkControlStreamFirst =        0x00008000,
eNetworkControlStreamLast =         0x00009FFF,

/* Application defined messages */
eNetworkControlCustomFirst =        0x0000C000,
eNetworkControlCustomLast =         0x0000FFFF
};

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

/* Opaque network address data structure – only length and raw bytes visible */
typedef struct NetworkAddress {
UInt32 length;
Byte8 bytes[256];
} NetworkAddress, *NetworkAddressPtr;

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef void *NetworkEndpointRef;
typedef void *NetworkResolverRef;
typedef void *NetworkDatagramRef;
typedef void *NetworkStreamRef;

typedef PASCAL void (*NetworkCallbackProc)(NetworkEndpointRef ref, void *refcon,
NetworkEventType event);

#define kNetworkCallbackSetSemaphore ((NetworkCallbackProc)(-1))

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

/* Public network data structures – application is free to modify fields */

typedef struct ResolverElem {
QStub qstub;
void * refcon;

NetworkAddress      address;
Char8               name[ARBITRARY_BUFFER_SIZE];
} ResolverElem, *ResolverElemPtr;

typedef struct DatagramElem {
QStub qstub;
void * refcon;

NetworkAddress      address;
UInt32              maxLen;
UInt32              actLen;
Byte8               data[ARBITRARY_BUFFER_SIZE];
} DatagramElem, *DatagramElemPtr;

typedef struct StreamElem {
QStub qstub;
void * refcon;

UInt32              maxLen;
UInt32              actLen;
Byte8               data[ARBITRARY_BUFFER_SIZE];
} StreamElem, *StreamElemPtr;

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

typedef PASCAL UInt32 (*NetworkBufferProc)(NetworkEndpointRef ref, void *refcon,
StreamElemPtr elem,Bool8 expedited);

#define kNetworkBufferBreakOnCRLF ((NetworkBufferProc)(-1))
#define kNetworkBufferBreakOnNull ((NetworkBufferProc)(-2))
#define kNetworkBufferFillBuffers ((NetworkBufferProc)(-3))
#define kNetworkBufferLengthPrefix ((NetworkBufferProc)(-4))

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Function Prototypes */

#ifdef __cplusplus
extern “C” {
#endif

/* High-level network routines */
EXTERN PASCAL Result32 NetworkStackLoad(NetworkStackMaskType stacks);
EXTERN PASCAL void NetworkStackUnload(void);
EXTERN PASCAL void NetworkStackTickle(void);
EXTERN PASCAL Result32 NetworkStackControl(NetworkStackType stack, void *data,
NetworkControlMessageType message,
NetworkControlFlagsType flags);

EXTERN PASCAL Bool8 NetworkStackGetAvail(NetworkStackType stack);
EXTERN PASCAL Bool8 NetworkStackGetActive(NetworkStackType stack);
EXTERN PASCAL Bool8 NetworkStackGetLoaded(NetworkStackType stack);
EXTERN PASCAL Result32 NetworkStackGetSelfAddress(NetworkStackType stack,
NetworkAddressPtr address);

EXTERN PASCAL void NetworkMakeCallback(NetworkCallbackProc callback,
void *refcon,
NetworkEndpointRef ref,
NetworkEventType event);
EXTERN PASCAL Result32 NetworkQueueFill(Queue *queue, NetworkElemType elemType,
UInt32 elemCount, UInt32 elemLen);
EXTERN PASCAL void NetworkQueueEmpty(Queue *queue);
EXTERN PASCAL QStubPtr NetworkQueuePoll(Queue *queue);
EXTERN PASCAL QStubPtr NetworkQueueWait(Queue *queue, UInt32 seconds);

EXTERN PASCAL NetworkElemType
EndpointGetElem(NetworkEndpointRef ref);
EXTERN PASCAL NetworkStackType
EndpointGetStack(NetworkEndpointRef ref);
EXTERN PASCAL NetworkStatusType
EndpointGetStatus(NetworkEndpointRef ref);

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Resolver-based network routines */

EXTERN PASCAL NetworkResolverRef
ResolverCreate (NetworkStackType stack, void *refcon,
NetworkCallbackProc callbackProc,
Queue *doneQueue, Queue *freeQueue);
EXTERN PASCAL void ResolverTickle (NetworkResolverRef ref);
EXTERN PASCAL void ResolverDispose (NetworkResolverRef ref);
EXTERN PASCAL Result32 ResolverControl (NetworkResolverRef ref, void *data,
NetworkControlMessageType message,
NetworkControlFlagsType flags);

EXTERN PASCAL Result32 ResolverOpen (NetworkResolverRef ref);
EXTERN PASCAL Result32 ResolveName (NetworkResolverRef ref,
ResolverElemPtr elem);
EXTERN PASCAL Result32 LookupName (NetworkResolverRef ref,
ResolverElemPtr elem);
EXTERN PASCAL void ResolverClose (NetworkResolverRef ref);

EXTERN PASCAL Result32 SimpleResolve (NetworkStackType stack,Char8 *hostname,
NetworkAddressPtr address);
EXTERN PASCAL Result32 SimpleLookup (NetworkAddressPtr address,
Char8 *hostname);

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Datagram-based network routines */

EXTERN PASCAL NetworkDatagramRef
DatagramCreate(NetworkStackType stack, void *refcon,
NetworkCallbackProc callbackProc,
Queue *doneQueue, Queue *freeQueue,
Queue *recvQueue,Queue *recvXQueue);
EXTERN PASCAL void DatagramTickle (NetworkDatagramRef ref);
EXTERN PASCAL void DatagramDispose (NetworkDatagramRef ref);
EXTERN PASCAL Result32 DatagramControl (NetworkDatagramRef ref, void *data,
NetworkControlMessageType message,
NetworkControlFlagsType flags);

EXTERN PASCAL Result32 DatagramOpen (NetworkDatagramRef ref,
NetworkAddressPtr localAddress);
EXTERN PASCAL Result32 DatagramSend (NetworkDatagramRef ref,
DatagramElemPtr elem);
EXTERN PASCAL void DatagramClose (NetworkDatagramRef ref, Bool8 orderly);

EXTERN PASCAL Result32 DatagramGetAddress(NetworkDatagramRef ref,
NetworkAddressPtr localAddress);
EXTERN PASCAL Bool8 DatagramGetFailedAddress(NetworkDatagramRef ref,
NetworkAddressPtr remoteAddress);

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Stream-based network routines */

EXTERN PASCAL NetworkStreamRef
StreamCreate(NetworkStackType stack, void *refcon,
NetworkCallbackProc callbackProc,
Queue *doneQueue, Queue *freeQueue,
Queue *recvQueue,Queue *recvXQueue);
EXTERN PASCAL NetworkStreamRef
StreamCreateBuffered(NetworkStackType stack,
void *refcon,
NetworkCallbackProc callbackProc,
NetworkBufferProc bufferProc,
Queue *doneQueue, Queue *freeQueue,
Queue *recvQueue,Queue *recvXQueue);
EXTERN PASCAL void StreamTickle (NetworkStreamRef ref);
EXTERN PASCAL void StreamDispose (NetworkStreamRef ref);
EXTERN PASCAL Result32 StreamControl (NetworkStreamRef ref, void *data,
NetworkControlMessageType message,
NetworkControlFlagsType flags);

EXTERN PASCAL Result32 StreamOpenServer(NetworkStreamRef ref,
NetworkAddressPtr localAddress);
EXTERN PASCAL Result32 StreamAcceptServer(NetworkStreamRef serverRef,
NetworkStreamRef acceptRef);
EXTERN PASCAL Result32 StreamOpenClient(NetworkStreamRef ref,
NetworkAddressPtr localAddress,
NetworkAddressPtr remoteAddress);
EXTERN PASCAL Result32 StreamSend (NetworkStreamRef ref,
StreamElemPtr elem);
EXTERN PASCAL void StreamClose (NetworkStreamRef ref, Bool8 orderly);

EXTERN PASCAL Result32 StreamGetAddress(NetworkStreamRef ref,
NetworkAddressPtr localAddress,
NetworkAddressPtr remoteAddress);

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ Network Address abstraction routines */

EXTERN PASCAL NetworkStackType
AddressToNetworkStack(NetworkAddressPtr addressPtr);
EXTERN PASCAL void AddressFromNetworkStack(NetworkAddressPtr addressPtr,
NetworkStackType stack);
EXTERN PASCAL void AddressDuplicate (NetworkAddressPtr dst,
NetworkAddressPtr src);
EXTERN PASCAL Bool8 AddressBroadcast (NetworkAddressPtr dst,
NetworkAddressPtr src,
NetworkAddressPtr hostmask);
EXTERN PASCAL void AddressIncrement (NetworkAddressPtr addressPtr);
EXTERN PASCAL Bool8 AddressCompare (NetworkAddressPtr addr1,
NetworkAddressPtr addr2);
EXTERN PASCAL Bool8 AddressIsEmpty (NetworkAddressPtr addressPtr);

EXTERN PASCAL void AddressComposeEmpty (NetworkAddressPtr addressPtr);
EXTERN PASCAL void AddressComposeTCPIP (NetworkAddressPtr addressPtr,
UInt16 port, UInt32 host);
EXTERN PASCAL void AddressComposeATalk (NetworkAddressPtr addressPtr,
UInt16 netID, UInt8 nodeID,
UInt8 sockID, UInt8 sockType);

EXTERN PASCAL void AddressDecomposeTCPIP(NetworkAddressPtr addressPtr,
UInt16 *port, UInt32 *host);
EXTERN PASCAL void AddressDecomposeATalk(NetworkAddressPtr addressPtr,
UInt16 *netID, UInt8 *nodeID,
UInt8 *sockID, UInt8 *sockType);

EXTERN PASCAL void AddressTranscribe(NetworkAddressPtr addressPtr,
Char8 *textPtr);

EXTERN PASCAL Result32 AddressPicker (Char8 *message,
NetworkStackMaskType stacks,
NetworkAddressPtr address,
Char8 *hostname, Char8 *hosttype);

#ifdef __cplusplus
}
#endif

/* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** /
/ **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */

#endif /* LIGHT_SOCKETS_HEADER */

Matt_Slot · November 18, 1998, 7:08am

Wow, complex. How much overhead is involved here?
I like the Datagram/Stream model. If I were going to do real transport-
independent network code, I would probably do something very similar.
How do you handle the various address types?

It’s pretty low overhead, but functionally very complete:

Endpoints are opaque objects, and when creating them, you specify
some interrupt-safe queues to hold free, busy, and incoming buffers.
Addresses are opaque objects, which you construct by resolving a
hostname, manually by host/port, or as part of a recv’d datagram.
There is also a call to display a platform-specific address “picker”,
where the user can enter an address for any available network stack.
Buffers are public structs that are allocated in batch, then queued.
The app fills in buffers to send or resolve, and incoming packets
are stored in buffers pulled from the free queue.
All data/requests are sent through buffers and queues, to preserve
the sequence of send/recvs while staying entirely interrupt-safe.
It also abstracts things well enough that the MacOS APIs are rather
moderate to implement.
All endpoints invoke an application for interesting events (recv data,
closing, etc). All endpoints proceed through well-defined states in
response to application and network events.
The network stacks provide robust controls, which are useful in PPP,
multihosted, or other “interesting” configurations.
Special control calls let you interact with endpoints in non-standard
ways (to take advantage of AppleTalk features such as NBP, and even
IP multicasting when I get around to it).

It’s actually pretty thin and fast, since endpoints are all async under
the hood. I use it under the Ambrosia Network_Tool, our game session lib,
which supports up to 32 players in a session, and boasts a synchronized
millisecond timer, high-resolution latency controls and stats, and even
arbitrary topologies (client/server, token ring, etc). The network stack
overhead is <1ms per packet.

I can answer specific questions a bit deeper, if you are interested.

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */

Stephane_Magnenat · November 18, 1998, 11:42am

At my opinion, there is two problem with componment that are

in external library :

They have a special tentation not to find what they want.
There is not the nice documentation that makes SDL a great part

of his power.

I think Netlib should be as good as the other part (grapohics) documented.

What do you think Sam ?

As I said in a previous mail the GUI exemple doesn’t want compile

Best regards,–
Stephane Magnenat
stephane.magnenat at urbanet.ch

P.S. Thanks foir your work Sam, it is coo.

Michael_Samuel · November 18, 1998, 1:07pm

I think Netlib should be as good as the other part (grapohics) documented.

Was that an offer? Documenting is a PITA, unless you have c2man or something.On Wed, Nov 18, 1998 at 11:42:06AM +0000, St?phane Magnenat wrote:

–
– Michael Samuel

slouken · November 18, 1998, 6:29pm

At my opinion, there is two problem with componment that are
in external library :

They have a special tentation not to find what they want.

What does this mean?

There is not the nice documentation that makes SDL a great part
of his power.

This is true. Documentation takes a fair bit of time to do properly,
and the examples are for the most part just pieces of code showing how
to do something cool with SDL. The two libraries which need documentation
are the screenlib and netlib libraries, and are not really high priority.

Someone volunteered to write documentation for screenlib, but I haven’t
heard back from him.

I will probably write documentation for netlib since it turned out to
have way more functions than I intended, and that should happen in a
few weeks. The CVS releases are intended as sneak previews, and should
never be looked at as a final product, though comments are always welcome.

I think Netlib should be as good as the other part (grapohics) documented.
What do you think Sam ?

I will not add netlib to the main SDL library, as it is neither as simple
nor as portable as I would like it to be. However, it is a very good
library for doing socket (TCP/UDP) networking on Win32 and UNIX.
As such, I will add some decent documentation.

As I said in a previous mail the GUI exemple doesn’t want compile

Try ‘make all CC=g++’

P.S. Thanks foir your work Sam, it is coo.

I’m glad you like it. I ended up writing it mainly for you.

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

slouken · November 18, 1998, 6:39pm

Endpoints are opaque objects, and when creating them, you specify
some interrupt-safe queues to hold free, busy, and incoming buffers.

Why do you have a separate queue for busy buffers?
Is the process:
Lock queues
dequeue buffer from free queue
enqueue buffer on busy queue
Unlock queues
Modify buffer (?)
Lock queues
dequeue buffer from busy queue
enqueue buffer on incoming queue
Unlock queues

That sounds like alot of overhead…

Addresses are opaque objects, which you construct by resolving a
hostname, manually by host/port, or as part of a recv’d datagram.
There is also a call to display a platform-specific address “picker”,
where the user can enter an address for any available network stack.

That’s cool.

All data/requests are sent through buffers and queues, to preserve
the sequence of send/recvs while staying entirely interrupt-safe.
It also abstracts things well enough that the MacOS APIs are rather
moderate to implement.

Do you do reliable sequencing of datagrams?
If so, how big do you make your hold queues, and what do you do on overflow?

All endpoints invoke an application for interesting events (recv data,
closing, etc). All endpoints proceed through well-defined states in
response to application and network events.

Do you handle OOB data?

The network stacks provide robust controls, which are useful in PPP,
multihosted, or other “interesting” configurations.

What do you mean by robust controls? You mean hooks for control panels?

It’s actually pretty thin and fast, since endpoints are all async under
the hood. I use it under the Ambrosia Network_Tool, our game session lib,
which supports up to 32 players in a session, and boasts a synchronized
millisecond timer, high-resolution latency controls and stats, and even
arbitrary topologies (client/server, token ring, etc). The network stack
overhead is <1ms per packet.

Very nice.
Let me do some quick math on the overhead…

Assume a game running at 30 FPS framerate, that’s 33 ms per frame.
Let’s say that a fully networked game has oh… 8 players, each sending
1 packet per frame (not necessarily a valid assumption, but hey) and
say the overhead is .6 ms per packet, then the overhead for network
packets alone is about 5 ms. A really well designed network game would
not send that much data and could handle largish latencies, but I could
see a mediocre game design having problems handling packets and AI and
drawing at a reasonable speed. How does it work out in practice?

I can answer specific questions a bit deeper, if you are interested.

Yup.

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

Matt_Slot · November 18, 1998, 8:25pm

Why do you have a separate queue for busy buffers?

Hmmm, “busy” is a bad word – maybe “pending” is better. I only maintain a
single outstanding send or DNS lookup, and queue the others. As soon as the
first operation completes (from the completion routine or SIGIO), I chain
the next call. Recv’s, of course, always operate one at a time.

Is the process:
Lock queues
dequeue buffer from free queue
enqueue buffer on busy queue
Unlock queues
Modify buffer (?)
Lock queues
dequeue buffer from busy queue
enqueue buffer on incoming queue
Unlock queues

That sounds like alot of overhead…

First, the “lock” and “unlock” calls are made as part of the actual enqueue
and dequeue operations – so that I can write my own fast and interrupt-safe
routines when possible.

App dequeues a free buffer
App fills with data
App passes buffer to StreamSend()
StreamSend() either 
    Sends data immediately
  - or -
    Places buffer in busy queue
When one send completes:
    The buffer is placed in the free queue
    The next busy buffer is dequeued and sent

I’ve found that almost anything you do in software is an order of magnitude
faster than anything that is placed on the wire. My queue routines are about
30 instructions on the Mac (and don’t disable interrupts), but on UNIX they
call the traditional signal functions.

The thing is, especially on the MacOS, if you want to do anything fast, you
have to do it async – and hence use queues. On UNIX, we tend to just block
and forget it – which causes everything in the thread to slowdown when flow
control kicks in. Both my MacOS and BSD Sockets code is async, so they avoid
alot of the slowdowns due to polling and flow control.

Do you do reliable sequencing of datagrams?
If so, how big do you make your hold queues, and what do you do on overflow?

Not at this layer, since this is just an abstraction layer.

The “session” layer I built over this, however, provides reliable datagram
service and tracks queue size on the fly. Basically, the app suggests how
many packets it expects to send/recv per player per second, and the library
preallocates that amount plus a bit more. I keep track of the “high water
mark”, so I can replenish it as necessary. On overflow, I just return an
error – it’s up to the application to wait a second or fail outright, but
he should have given me better info in the first place.

I actually track 2 different types of network data – unreliable/fast and
reliable/kinda-fast. The first are just datagrams but can be multicasted,
and out of order or dropped packets are not passed up. This is suitable for
"I am here" or “my score is 10200” state broadcasts. The second are also
datagrams, but organized to provide well-ordered, reliable, and kinda fast
transactions. These are suitable for negotiated events between 2 hosts (a
player and a game authority): “I capture resource X” or “collision between
object A and B”.

The library tracks latency in real-time, so that it can predict when a
reliable packet “should have arrived” given recent network conditions.
Say that the last 10 packets have been recv’d in the 200-250ms range, it
knows that at about 275ms the packet was likely dropped and should be
resent. This way I can also keep millisecond clocks in synch, because I
know when a “timer” packet arrives whether it was in the good or bad
30th percentile. Many of these ideas come right from the design of NTP,
although I took a little liberty in making them faster and more lightweight.

All endpoints invoke an application for interesting events (recv data,
closing, etc). All endpoints proceed through well-defined states in
response to application and network events.

Do you handle OOB data?

Actually, yes. You pass two recv queues to the create endpoint function, one
for normal and one for expedited/OOB data. If the app doesn’t care, it just
passes the same queue for both parameters. Similarly, the app can share a
single free queue among all it’s related endpoints, so that you are much less
likely to run out of buffers in one of them.

The network stacks provide robust controls, which are useful in PPP,
multihosted, or other “interesting” configurations.

What do you mean by robust controls? You mean hooks for control panels?

No, I just mean a way to check whether the network stack is active and loaded
or not, whether there are more than one host networks, etc. Things that you
have to consider on PC and Macs in dynamic enviroments, such as dialups.

Assume a game running at 30 FPS framerate, that’s 33 ms per frame.
Let’s say that a fully networked game has oh… 8 players, each sending
1 packet per frame (not necessarily a valid assumption, but hey) and
say the overhead is .6 ms per packet, then the overhead for network
packets alone is about 5 ms. A really well designed network game would
not send that much data and could handle largish latencies, but I could
see a mediocre game design having problems handling packets and AI and
drawing at a reasonable speed. How does it work out in practice?

heh, those are pretty large numbers. No game should send more than 10
packets per second per player, and then, most of those aren’t broadcasts.
Even in client/server, only one host is responsible for the gruntwork and
he should be prepared for a decent network load.

Since everything is handled during software interrupts, all the overhead is
smoothed out already. On the MacOS side, once data has been received, I’d
say that the abstraction layer takes about 150 instructions and the session
layer another 300 or so. Under UNIX, it’s probably a bit higher since I just
use the POSIX APIs and handle everything with kid gloves. If the application
was really concerned about overhead, it could do the session work itself.

In practice, we have yet to ship a game using the network libraries – we had
to yank support in the first one because the author was rather naive, and we
had to get the thing out the door. That said, part of the process is actually
educating the developer to design his game well – the same way someone can
use a 3D card really well or really poorly.

This type of abstraction and overhead wouldn’t have worked 5 or 10 years ago,
but just like the move to C++ from C, applications can take advantage of these
features without having to get their hands dirty at the cost of a few cycles.
The abstraction layer is relatively the same overhead that SDL offers in other
services (blitting, page flipping, mixing audio).

I can answer specific questions a bit deeper, if you are interested.

Yup.

Keep em coming.

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */

Stephane_Magnenat · November 18, 1998, 8:27pm

Sam Lantinga wrote:

At my opinion, there is two problem with componment that are
in external library :

They have a special tentation not to find what they want.

What does this mean?

Euh, sorry, it was a bad french=>english translation, I just mean
lirbaries and me are not friends sometimes

There is not the nice documentation that makes SDL a great part
of his power.

This is true. Documentation takes a fair bit of time to do properly,
and the examples are for the most part just pieces of code showing how
to do something cool with SDL. The two libraries which need documentation
are the screenlib and netlib libraries, and are not really high priority.

Someone volunteered to write documentation for screenlib, but I haven’t
heard back from him.

I will probably write documentation for netlib since it turned out to
have way more functions than I intended, and that should happen in a
few weeks. The CVS releases are intended as sneak previews, and should
never be looked at as a final product, though comments are always welcome.

I think Netlib should be as good as the other part (grapohics) documented.
What do you think Sam ?

I will not add netlib to the main SDL library, as it is neither as simple
nor as portable as I would like it to be. However, it is a very good
library for doing socket (TCP/UDP) networking on Win32 and UNIX.
As such, I will add some decent documentation.

As I said in a previous mail the GUI exemple doesn’t want compile

Try ‘make all CC=g++’

IT WORKS

I’m happy. Thanks a lot.
Why didn’t it worked ?
When I try to connect to devolution.com, it didn’t work, is it normal ?

P.S. Thanks foir your work Sam, it is coo.

I’m glad you like it. I ended up writing it mainly for you.

T H A N K S : - )

Bye,

Steph–
Stephane Magnenat
stephane.magnenat at urbanet.ch

Stephane_Magnenat · November 18, 1998, 8:28pm

michael at surfnetcity.com.au wrote:> On Wed, Nov 18, 1998 at 11:42:06AM +0000, St?phane Magnenat wrote:

I think Netlib should be as good as the other part (grapohics) documented.

Was that an offer? Documenting is a PITA, unless you have c2man or something.

–
– Michael Samuel

Why not, if I am enough good to understand all functions …

–
Stephane Magnenat
stephane.magnenat at urbanet.ch

slouken · November 19, 1998, 12:11am

Why didn’t it worked ?

The C compiler doesn’t know how to link C++ object files.

When I try to connect to devolution.com, it didn’t work, is it normal ?

Not if you are on a full internet connection. Are you behind a firewall?

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

Stephane_Magnenat · November 19, 1998, 6:59am

Sam Lantinga wrote:

Why didn’t it worked ?

The C compiler doesn’t know how to link C++ object files.

When I try to connect to devolution.com, it didn’t work, is it normal ?

Not if you are on a full internet connection. Are you behind a firewall?

My computer (linux) is a firewall for another win32 computer. Which port
does it uses ?
Can I connect using telnet just to see if it works. Normally, the linux
boxcan access everything.
When I launch the server in local, and then I do a ./chat localhost, it
doesn’t works too
Same problem with ./chat 127.0.0.1

Best regards,

Stephane–
Stephane Magnenat
stephane.magnenat at urbanet.ch

Matt_Slot · November 19, 1998, 8:32pm

I’ve tried to avoid SIGPOLL (and SIGURG) on library code because the
application might be using it for its own file descriptors. In your case
though, it makes a lot of sense.

My library is set up so I get all signals, handle the ones it likes, and then
call an application function. In addition to the networking ones SIGIO and
SIGURG (pretty much used for async i/o not block files), I have wrappers for
getitimer()/setitimer()/SIGALRM so that I can install multiple millisecond
clocks at once – much like the MacOS Time Manager.

Wow, it sounds like a really nice design.
How is the documentation coming along?

Well, the documentation for the Light Sockets layer is pretty far along,
although it hasn’t been touched for a while. You can peek at a working copy
at http://www.ambrosiasw.com/~fprefect/lsockets/ – but don’t pass that
URL out.

Since the API is pretty complex, I suggest grouping functions by common use
in the documentation. i.e. Normal functions used to get a network stack
up and running are grouped together, and all of the custom hooks and so forth
are put elsewhere. Just a thought. Your design sounds really nice, but
it’s hard for me to tell how it works just by looking at the header.

heh, yah, it’s a single header for a pretty complex library. Not to mention
that it uses another header that describes our platform abstraction library
(queues, timers, memory, signals, etc). The docs give a decent explanation
of what’s going on, but I’m more of a coder than a tech writer.

I’d really like to see the logic for a networked game.
When I added networking to Maelstrom, I couldn’t see a way to send fewer than
one packet per frame without keeping a rewindable stack of the entire game
state, or periodically sending the entire game state.
It’s a peer-to-peer model, so it’s a little harder than just getting periodic
state updates from a server.

Yah, to get a very good simulation, you need alot of packets. However, there
are ways to reduce the amount of traffic … at the cost of extra design. One
way is to assign each resource to an “authority”, typically the player who
owns it. The resource has predictable and well-defined behaviors so that all
can “extrapolate” it, and when the object needs to change, the authority
informs all those who care. Any collisions or interactions between two objects
are resolved by their authority or authorities.

One of these days, I’m going to sit down and write a series of short papers on
networked games – latency, authorities, time-synching. If I do, they will
likely be posted to the Web.

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */

Matt_Slot · November 19, 1998, 9:18pm

Wow, that’s really cool. How long has it taken for you (pl?) to write
your library? How long has it taken for you guys to write the other
cross-platform libraries? Do you expect cross-platform games soon?
How is Ambrosia Software supporting itself during the buildout?

Well, I’ve been working on the cross-platform collection since I’ve gotten
here 2 years ago, taking time off for other projects as necessary. I’m a
Mac coder who has written several UNIX network apps, and, well, I’m willing
to do a little Win32 when the need arises.

I had a pretty decent framework for the platform and network abstraction
layers, so it’s been more a matter of fleshing out the support for other
platforms and other internal libraries. We’re updating our displays and
sound libraries to the neutral APIs, and preparing them use. The network
session library has taken the longest… probably 4 man-months, but I
think it was well worth it.

We actually have several games in the pipes, most of which are Mac-only. It
looks like our first cross-platform game will be mine, but that’s still about
4-6 months off.

One of these days, I’m going to sit down and write a series of short papers
on networked games – latency, authorities, time-synching. If I do, they
willlikely be posted to the Web.

Will you send me a note when you do? I’d love to see it.

Absolutely.

Good chatting with you!

Matt

/* Matt Slot, Bitwise Operator * One box, two box, yellow box, blue box. *

* http://www.ambrosiasw.com/~fprefect/ */

CVS snapshot (networking)

See ya! -Sam Lantinga (slouken at devolution.com)– Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/

See ya! -Sam Lantinga (slouken at devolution.com)– Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/

See ya! -Sam Lantinga (slouken at devolution.com)– Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/

See ya! -Sam Lantinga (slouken at devolution.com)– Author of Simple DirectMedia Layer - http://www.devolution.com/~slouken/SDL/

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/

See ya!
-Sam Lantinga (slouken at devolution.com)–
Author of Simple DirectMedia Layer -
http://www.devolution.com/~slouken/SDL/