Loading lots of PNG files

SDL Development
1

Hi,
I’m having pretty good progress with “AutoManga”,
but I’m still loading sprites as large arrays of
(separate) PNG files. I actually prefer this from
a resource-construction point of view, but as I
use SDL_Image to load each one individually, I’m
spending something like a whole minute loading
108 images of a character.

I have a feeling this will be noticed. :slight_smile:

There’s probably a way to speed this up, but I
don’t know enough about SDL_Image or what
exactly the problem is to know how best to
attack it.

Is there some way to make a single capsule
file so it can be loaded all at once (cheap
trick – I could tar the PNG images all into
one file – but how would I get them out?). Or
I could (somehow) build a single large PNG
file and load that, then dice it up to make
the sprite. The key point is that the designer
should be able to create and reference the
PNG files individually, but then have some
automated way to bundle them for loading.
This could happen at compile time or at
install time, but not at run time.

Or how about this – could I make an array of
SDL surfaces and then dump the result to disk?
The result would be machine-dependent, but
this could be part of my install process.

Will some of this problem go away if the
source files are on a CDROM? (I’ve noticed
some things will load faster off of a fast
CDROM than off of my hard disk – possibly
because the data is contiguous and the
CDROM has a faster data transfer rate,
offsetting the seek time).

I’m just looking for general rules of thumb –
how have people solved this before?–
Terry Hancock
@Terry_Hancock

2

Well, you could try libmng. MNG files (in a nutshell) are just sequential
png files.(Ignoring for the time being the JNG, delta image and sprite
support.)

You can find libmng at http://www.libmng.com/> ----- Original Message -----

From: owner-sdl@lokigames.com [mailto:owner-sdl at lokigames.com]On Behalf
Of Terry Hancock
Sent: May 14, 2001 8:05 PM
To: sdl at lokigames.com
Subject: [SDL] Loading lots of PNG files

Hi,
I’m having pretty good progress with “AutoManga”,
but I’m still loading sprites as large arrays of
(separate) PNG files. I actually prefer this from
a resource-construction point of view, but as I
use SDL_Image to load each one individually, I’m
spending something like a whole minute loading
108 images of a character.

I have a feeling this will be noticed. :slight_smile:

There’s probably a way to speed this up, but I
don’t know enough about SDL_Image or what
exactly the problem is to know how best to
attack it.

Is there some way to make a single capsule
file so it can be loaded all at once (cheap
trick – I could tar the PNG images all into
one file – but how would I get them out?). Or
I could (somehow) build a single large PNG
file and load that, then dice it up to make
the sprite. The key point is that the designer
should be able to create and reference the
PNG files individually, but then have some
automated way to bundle them for loading.
This could happen at compile time or at
install time, but not at run time.

Or how about this – could I make an array of
SDL surfaces and then dump the result to disk?
The result would be machine-dependent, but
this could be part of my install process.

Will some of this problem go away if the
source files are on a CDROM? (I’ve noticed
some things will load faster off of a fast
CDROM than off of my hard disk – possibly
because the data is contiguous and the
CDROM has a faster data transfer rate,
offsetting the seek time).

I’m just looking for general rules of thumb –
how have people solved this before?

–
Terry Hancock
hancock at earthlink.net

Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

3

I’m having pretty good progress with “AutoManga”,
but I’m still loading sprites as large arrays of
(separate) PNG files. I actually prefer this from
a resource-construction point of view, but as I
use SDL_Image to load each one individually, I’m
spending something like a whole minute loading
108 images of a character.

The sluggishness comes from 3 sources:

  1. PNG decoding in libpng
  2. SDL_image overhead
  3. File system overhead for loading many small files

The last is not likely to be the main problem if you only have 108 images,
but could become a problem when you are scaling up your game.
SDL_image supports reading from RWops objects, so you can define your own
sources - some kind of archive file, for instance. See the SDL_rwops.h
header file for more information

The first problem, PNG decoding, is hard to do anything about unless
switching formats is acceptable. TGA should be slightly faster to decode
but compresses less well (but should do OK for cartoonish graphics);
some formats don’t compress at all.

SDL_image is written for correctness first of all and performance second,
so it’s not surprising if there are inefficiencies here and there,
but I’m happy to look at it if you can point at something more
concrete

Is there some way to make a single capsule
file so it can be loaded all at once (cheap
trick – I could tar the PNG images all into
one file – but how would I get them out?).

This can be done with RWops. Use whatever format you prefer, like tar, ar,
zip, or something homemade. (No point in using a compressed archive format
if you are using PNG)

Or
I could (somehow) build a single large PNG
file and load that, then dice it up to make
the sprite. The key point is that the designer
should be able to create and reference the
PNG files individually, but then have some
automated way to bundle them for loading.

You can use the netpbm tools to group images together into a big image.
Netpbm can convert its output to most image formats, but SDL_image supports
its native formats (PPM etc) as well

Or how about this – could I make an array of
SDL surfaces and then dump the result to disk?
The result would be machine-dependent, but
this could be part of my install process.

This is equivalent to inventing your own image file format and you are free
to do that - if you are careful it won’t be machine-dependent at all

Will some of this problem go away if the
source files are on a CDROM?

definitely not

4

I’m having pretty good progress with “AutoManga”,
but I’m still loading sprites as large arrays of
(separate) PNG files. I actually prefer this from
a resource-construction point of view, but as I
use SDL_Image to load each one individually, I’m
spending something like a whole minute loading
108 images of a character.

The sluggishness comes from 3 sources:

  1. PNG decoding in libpng
  2. SDL_image overhead
  3. File system overhead for loading many small files

I don’t know about point 1., but the 2 and 3 should not be the bottleneck,
because my game loads more bmp’s (something like 120 or more) in less than 5
sec on Celeron416. So maybe you should profile you game and see which
functions exactly take so much time.

Kovacs
5

Hi,
I’m having pretty good progress with “AutoManga”,
but I’m still loading sprites as large arrays of
(separate) PNG files. I actually prefer this from
a resource-construction point of view, but as I
use SDL_Image to load each one individually, I’m
spending something like a whole minute loading
108 images of a character.

I have a feeling this will be noticed. :slight_smile:

108 images is kind of a bunch, but it shouldn’t really take a full minute to
load. You might try loading the whole image file into memory first, and then
decoding it. Some libraries just read one byte at a time from a file, and
some OSs aren’t very good at speeding that up.

Of you could use a more scientific approach and narrow down where your
bottleneck is. If it’s PNG decoding, then a faster to decode image format
might be better, etc.

There’s probably a way to speed this up, but I
don’t know enough about SDL_Image or what
exactly the problem is to know how best to
attack it.

Take a look into it then. You can tell it to load an image off of the disk,
or you can tell it to load an image stored in a block of memory. Reading
memory is of course faster than reading from disk. And if you can load one
big block off the disk at once, you can use DMA and stuff to speed that up.

Is there some way to make a single capsule
file so it can be loaded all at once (cheap
trick – I could tar the PNG images all into
one file – but how would I get them out?).

Doom used a WAD file format, which was just an archive of files. It didn’t
do any compression on the files. You might want to take a look at that.
It’s quite simple, but does exactly what you are looking for here I think.

Will some of this problem go away if the
source files are on a CDROM? (I’ve noticed
some things will load faster off of a fast
CDROM than off of my hard disk – possibly
because the data is contiguous and the
CDROM has a faster data transfer rate,
offsetting the seek time).

That’s pretty sad if your CDROM is faster than your harddrive. The first
thing I’d recommend you try is defragging your drive. That might speed
things up a lot for you.On Monday 14 May 2001 20:05, you wrote:

6

Doom used a WAD file format, which was just an archive of files. It
didn’t
do any compression on the files. You might want to take a look at
that.
It’s quite simple, but does exactly what you are looking for here I
think.

IIRC, WAD files never compressed very well, so I would assume they
were already compressed… no?–

Olivier A. Dagenais - Software Architect and Developer

7

IIRC, WAD files never compressed very well, so I would assume they
were already compressed… no?

no but the sprites were RLE-encoded in vertical strips. (I wrote a set of
perl scripts to extract WAD info once)

8

Nope. You basically just had a 16 byte header, then the individial file
data, followed finally by a directory structure for each of the files in it.
The individual files could be compressed, for for Doom they weren’t. I was
one of those people that wrote a map editor for it, so I became very familar
with the WAD format and the format of the individual files that were in it.On Tuesday 15 May 2001 20:01, you wrote:

Doom used a WAD file format, which was just an archive of files. It

didn’t

do any compression on the files. You might want to take a look at

that.

It’s quite simple, but does exactly what you are looking for here I

think.

IIRC, WAD files never compressed very well, so I would assume they
were already compressed… no?