Endian Conversion?

(might be barking up the wrong tree here!)

Bit of background - I’ve created a pak like file to store all my game’s
data. The format is as follows,

filename - 56 bytes
file size - binary integer
file data - varaible length unsigned char data

It runs fine on x86 machines (Windows and Linux). However, I’d like to be
able to get the same file to run on other systems, like PPC. Basically the
only thing it trips up on is the reading of the binary integer,

fread(&packFSize, sizeof(int), 1, pak);

That’s the line used to read the integer. So all I’d like to know is, is
there anyway I can make use of the SDL_BYTEORDER to convert the binary
integer?

Cheers,
Steve

It runs fine on x86 machines (Windows and Linux). However, I’d like to be
able to get the same file to run on other systems, like PPC. Basically the
only thing it trips up on is the reading of the binary integer,

fread(&packFSize, sizeof(int), 1, pak);

That’s the line used to read the integer. So all I’d like to know is, is
there anyway I can make use of the SDL_BYTEORDER to convert the binary
integer?

No. You’re better off doing this:

int packFSize;

packFSize = fgetc(pak);
packFSize = ((int) fgetc(pak)) << 8;
packFSize = ((int) fgetc(pak)) << 16;
packFSize = ((int) fgetc(pak)) << 24;

(it’d be a good idea to check for end of file of course, but you can
figure that out)

No #ifdefs or anything funky required and it works on all platforms.

–>Neil-------------------------------------------------------------------------------
Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898

so that works if a file is being read on a little endian system that was
written on a big endian system?> ----- Original Message -----

From: nb@synthcom.com (Neil Bradley)
To:
Sent: Monday, January 06, 2003 10:57 AM
Subject: Re: [SDL] Endian Conversion?

It runs fine on x86 machines (Windows and Linux). However, I’d like to
be

able to get the same file to run on other systems, like PPC. Basically
the

only thing it trips up on is the reading of the binary integer,

fread(&packFSize, sizeof(int), 1, pak);

That’s the line used to read the integer. So all I’d like to know is, is
there anyway I can make use of the SDL_BYTEORDER to convert the binary
integer?

No. You’re better off doing this:

int packFSize;

packFSize = fgetc(pak);
packFSize = ((int) fgetc(pak)) << 8;
packFSize = ((int) fgetc(pak)) << 16;
packFSize = ((int) fgetc(pak)) << 24;

(it’d be a good idea to check for end of file of course, but you can
figure that out)

No #ifdefs or anything funky required and it works on all platforms.

–>Neil



Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

so that works if a file is being read on a little endian system that was
written on a big endian system?

Do the same “write every byte explicitly” approach that I recommended for
reading. Not like that’s a big problem. :wink:

–>Neil-------------------------------------------------------------------------------
Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898

no thats awesome (:> ----- Original Message -----

From: nb@synthcom.com (Neil Bradley)
To:
Sent: Monday, January 06, 2003 12:27 PM
Subject: Re: [SDL] Endian Conversion?

so that works if a file is being read on a little endian system that was
written on a big endian system?

Do the same “write every byte explicitly” approach that I recommended for
reading. Not like that’s a big problem. :wink:

–>Neil



Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

At 2003-01-06 20:37, you wrote:

so that works if a file is being read on a little endian system that was
written on a big endian system?

The mentioned conversion (with the correction that the ‘=’ should be
replaced by ‘|=’ or ‘+=’ in all but the first line) works if you write on a
little endian system with fwrite(value,sizeof(int),1,f) and want to read it
back on either a little or big endian.

If you want to write on either system and read on either system, you should
also adapt your write functions similarly, to something like:

unsigned char temp;
temp=value; /* value being the 32 bit integer */
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);

(This writes little endian format, you could also reverse it to write
big-endian, but as long as your read-code matches it shouldn’t matter)

The typecast to unsigned int should avoid shifting in sign-bits if your int
(value) is signed. Might want some extra typecasts to unsigned char to
avoid compiler warnings.

Since fwrite and fread are often (relatively) slow, you could optimize by
first putting your conversions into a 4-byte unsigned char array and
writing that, or (for reading) reading into a 4-byte unsigned char array
and converting from that. Useful if you read/write a large amount of data.

Additionally, you should make sure your ‘int’ is 32 bit on all platforms
you are going to support, otherwise the code will still fail.

Be sure to test the code since I might also have made a mistake somewhere.

I’m actually also looking into making sure my code is big/little endian
portable, and must say the original reply suggesting this ‘trick’ is quite
a nice alternative for #ifdefs.

Oops. Messed up my From: again. Need to fix that …On Mon, Jan 06, 2003 at 09:50:07PM +0100, M. Egmond wrote:

unsigned char temp;
temp=value; /* value being the 32 bit integer */
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);

You probably meant
fwrite(&temp, 1, 1, f);
(pointer to the char)

This should work:

value = SDL_SwapLE32(value);
fwrite(&value, sizeof(value), 1, f);

which converts the value to little-endian (if on a big endian machine)
and writes it; read it back with:

fread(&value, sizeof(value), 1, f);
value = SDL_SwapLE32(value);

Alternatively, use SDL_RWops:

SDL_WriteLE32(file, value);
value = SDL_ReadLE32(file);


Glenn Maynard

fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);
You probably meant
fwrite(&temp, 1, 1, f);
(pointer to the char)

Or better yet, how 'bout fputc/fgetc? Much better/faster than fwrite by a
LONG shot.

–>Neil-------------------------------------------------------------------------------
Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898

or of course if you know how much data there is (the entire file for
instance, or you know theres a block of 1k) you can fread into a buffer and
then decode it from there :P> ----- Original Message -----

From: nb@synthcom.com (Neil Bradley)
To:
Sent: Monday, January 06, 2003 1:20 PM
Subject: Re: [SDL] Endian Conversion?

fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);
You probably meant
fwrite(&temp, 1, 1, f);
(pointer to the char)

Or better yet, how 'bout fputc/fgetc? Much better/faster than fwrite by a
LONG shot.

–>Neil



Neil Bradley In the land of the blind, the one eyed man is not
Synthcom Systems, Inc. king - he’s a prisoner.
ICQ #29402898


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

Sweeney, Steven (FNB) wrote:

It runs fine on x86 machines (Windows and Linux). However, I’d like to be
able to get the same file to run on other systems, like PPC. Basically the
only thing it trips up on is the reading of the binary integer,

The “byte at a time” method will indeed work, but it’s slower than
reading the data 4 bytes at a time. Instead, read the integer into
memory and call ntohl() on it. When writing the file out, use htonl()
on it. Every modern *ix system has these functions, as does Windows if
you link to winsock.lib. Since every Windows system since Windows 95
comes with Winsock it’s safe to rely on this family of functions.
That’s not totally true: winsock.dll isn’t present on Windows when all
networking is disabled, including dialup networking. That’s pretty rare
these days.

I should point out that reading from a file 4 bytes at a time is slow,
too. Not as slow as reading 1 byte at a time, but… Point is, you
should be reading in 1K to 4K blocks at a time. Do some benchmarks –
betcha find that it’s a lot faster reading big chunks at once. I
realize this means you will usually have partial chunks of your
variable-sized data blocks in memory at one time; this requires more
clever coding to deal with, but it will net you a worthwhile speed
improvement if your file reading code currently takes more than about a
second to run. (That will depend on the data size and the speed of your
test machine.)

The “byte at a time” method will indeed work, but it’s slower than
reading the data 4 bytes at a time. Instead, read the integer into
memory and call ntohl() on it. When writing the file out, use htonl()
on it. Every modern *ix system has these functions, as does Windows if
you link to winsock.lib. Since every Windows system since Windows 95

Use SDL_SwapLE32; if you’re using SDL, it’s always there. Remember,
SDL works on more than just Windows and *nix, and although ntohl and
friends are probably available just about everywhere, you have to worry
about how to pull it in on each arch.

I should point out that reading from a file 4 bytes at a time is slow,
too. Not as slow as reading 1 byte at a time, but… Point is, you

For his application, it probably doesn’t matter; it seems he’ll be reading
and seeking in small units anyway (see original post; the metadata is
interleaved with the data, which is inherently inefficient).On Mon, Jan 06, 2003 at 05:02:25PM -0700, Warren Young wrote:


Glenn Maynard

It runs fine on x86 machines (Windows and Linux). However, I’d like
to be able to get the same file to run on other systems, like
PPC. Basically the only thing it trips up on is the reading of the
binary integer,

The “byte at a time” method will indeed work, but it’s slower than
reading the data 4 bytes at a time.

While this is generally true, please don’t forget that practically
every implementation of the stdio file functions (fgetc() and friends)
actually read (and often write) data in blocks (4K blocks or whatever
is generally efficient for the architecture). That was the main reason
for standardizing an extra layer of file I/O functions – so that each
program doesn’t have to do its own buffering in order to have
efficient file I/O.

Of course, doing your own buffering is still going to be faster,
because you can reduce the number of function calls, but the savings
is much less significant than that gained by buffering file I/O.

b

You probably meant
fwrite(&temp, 1, 1, f);
(pointer to the char)

This should work:

value = SDL_SwapLE32(value);
fwrite(&value, sizeof(value), 1, f);

which converts the value to little-endian (if on a big endian machine)
and writes it; read it back with:

fread(&value, sizeof(value), 1, f);
value = SDL_SwapLE32(value);

Alternatively, use SDL_RWops:

SDL_WriteLE32(file, value);
value = SDL_ReadLE32(file);On Mon, Jan 06, 2003 at 09:50:07PM +0100, M. Egmond wrote:

unsigned char temp;
temp=value; /* value being the 32 bit integer */
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);


Glenn Maynard

Thanks for the replies all :slight_smile: I’m going to try the method below… Think I
may have confused some people here,

I mainly code on and compile Linux (x86 AMD, if it matters) and I also
compile for Windows too (Intel x86). I’m actually able to take the Linux
packed up data file and use it with the Windows binary no problems… so I
know it works there.

Hopefully using the method below will also allow me to do the same for PPC,
etc versions… It’s basically to enable people who want to do ports to just
recompile the code without needing to pack up all the individual music,
graphics, data and sounds files again. But we’ll see how it goes :slight_smile:

Cheers,
Steve :))> -----Original Message-----

From: Glenn Maynard [SMTP:g_sm at zewt.org]
Sent: Monday, January 06, 2003 9:04 PM
To: sdl at libsdl.org
Subject: Re: [SDL] Endian Conversion?

On Mon, Jan 06, 2003 at 09:50:07PM +0100, M. Egmond wrote:

unsigned char temp;
temp=value; /* value being the 32 bit integer */
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>8;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>16;
fwrite(temp,1,1,f);
temp=((unsigned int)value)>>24;
fwrite(temp,1,1,f);

You probably meant
fwrite(&temp, 1, 1, f);
(pointer to the char)

This should work:

value = SDL_SwapLE32(value);
fwrite(&value, sizeof(value), 1, f);

which converts the value to little-endian (if on a big endian machine)
and writes it; read it back with:

fread(&value, sizeof(value), 1, f);
value = SDL_SwapLE32(value);

Alternatively, use SDL_RWops:

SDL_WriteLE32(file, value);
value = SDL_ReadLE32(file);


Glenn Maynard


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

.sophos.3.64.01.02.