SDL weakness for hi-res 2D applications

Hello everybody,

This thread has been already launched a few months ago but I think it
might be interested to re-start it now, as the lack is quite serious.

My project has just switched from a 320x240 resolution to a 640x480 one.
While the CPU usage in 320 was neglectable, doing such rendering in 640
really makes my machine suffer.

Here are the details of what I am doing:
I’m using a 16bpp color depth, on any of the targets I have made my
tests on. All surfaces (including the screen surface) are created with
the HWSURFACE flag. Surfaces intended for blitting are converted to the
display format. The screen surface is created with the DOUBLEBUF flag
too, because the entire screen is redrawn for every frame. So you guess
what happens: I make several blits for every frame (about 20-30 per
frame), then call SDL_Flip ().

The result, as you can guess, is that doing so results in something very
CPU intensive. I’d like to maintain a frame rate of 50fps, and it’s
quite hard, even though my machine should give good results (Athlon 850,
with a well-supported Voodoo3 card). The problem actually comes from
SDL’s targets: Using X11, surfaces are created in system memory and no
HW acceleration is used. Therefore, flipping the buffers takes a big
while. I’ve tried several accelerated targets: dga, segfault at startup.
fbcon, it runs, but badly-crash when I quit. Svgalib - no need to deal
with it :slight_smile: ggi uses the same targets as SDL, so it’s worthless here. I
guess that under Windows, with the DirectX target, things should be much
better and you can take full advantage of your hardware (even though I
can’t test it - but that’s at least what several people said here).

So here is the problem: using SDL, under GNU/Linux, you can’t compete
with the Windows target in terms of performance for hi-resolution 2D
applications. At least on my machine, which is quite a common one anyway
(someone who managed to take advantage of his hardware in this case,
please show up). And it’s a fact that there are no (or very few) such
gfx-intensive game using SDL that exists and run on Linux. This is quite
serious, because whenever you want to make a graphics-intensive, high
resolution 2D game using SDL, you end up with only the Windows target
usuable. It’s really sad that I can decently use only one target with a
cross-platform library… :confused: The fact is that ressources SDL can take
advantage of does exist under GNU/Linux, but doesn’t seems to be
exploited.

For example, a while ago David Olofson talked about an OpenGL target
hack he made for Kobo. Having tested it with Kobo, I can say it works
flawlessly - when the resolution raises, the frame rate becomes much
higher than the X11 software target. In short, it’s always fast when any
other target fall on the knees. But trying to use it on my project, it
unfortunately fails. Anyway, such a target is very interesting: why not
trying to tune it and include it into the main SDL tree? This would
solve all the above raised problems as OpenGL is the target that
supports the most acceleration cards under Linux: for most cards it
would rely on DRI, while for NVidia cards it would use the NVidia
drivers. It works directly under X11 (the most used target under Linux),
without any further configuration. It could even bring acceleration to
platforms others than Linux, as OpenGL itself is portable. In short,
it’s IMO The Right Thing To Do ©.

There are others exploitable ressources too, like DRI or DirectFB, but
they would be limited to certain cards, while every accelerated card
under Linux does support OpenGL.

SDL is mainly a 2D graphics platform, as it rely on OpenGL for 3D
rendering. Therefore, it should be unbeatable when it comes to 2D
graphics - and it can well be. I might be wrong, but it seems to me that
some others Linux graphics library (Clanlib for example) does provide
such a target. Is SDL late in that respect?

See you all and sorry for the long mail! :slight_smile:
Alex.

Note: Re-reading my mail, I realise it might sound a bit offensive and
critical, but don’t get me wrong: I just love SDL and am not attempting
to launch a troll or anything. I just feel like there is a weakness at
this precise point, and that it can easily be filled. Keep also in mind
that English isn’t my native language, and that I might not express
myself as well (and with as much courtesy) as I would like! :)–
http://www.gnurou.org

[…]

So here is the problem: using SDL, under GNU/Linux, you can’t compete
with the Windows target in terms of performance for hi-resolution 2D
applications. At least on my machine, which is quite a common one
anyway (someone who managed to take advantage of his hardware in this
case, please show up). And it’s a fact that there are no (or very few)
such gfx-intensive game using SDL that exists and run on Linux. This is
quite serious, because whenever you want to make a graphics-intensive,
high resolution 2D game using SDL, you end up with only the Windows
target usuable. It’s really sad that I can decently use only one target
with a cross-platform library… :confused: The fact is that ressources SDL can
take advantage of does exist under GNU/Linux, but doesn’t seems to be
exploited.

AFAIK, the real problem is that there are no useful features to exploit.

Sure, it would be nice if surfaces could be sent to the X server and
blitted by the X server (usually h/w accelerated) - but that would
probably only work in some special cases.

(Anyway, the latest stuff in XFree 4.1.x might change this, but I’m not
enough into it to tell.)

What we really need, the way the SDL API is designed, is asynchronous DMA
transfers from system RAM to VRAM - at least, that’s the easiest and most
flexible solution. It’s also a driver issue; nothing SDL can do much
about.

For example, a while ago David Olofson talked about an OpenGL target
hack he made for Kobo. Having tested it with Kobo, I can say it works
flawlessly - when the resolution raises, the frame rate becomes much
higher than the X11 software target. In short, it’s always fast when
any other target fall on the knees. But trying to use it on my project,
it unfortunately fails.

Why? It’s important to realize that glSDL is a source level wrapper. You
should not use any of the glSDL_* calls explicitly, but rather just
include glSDL.h instead of SDL.h and recompile. Exceptions:

* You need to | SDL_GLSDL to the flags to SDL_SetVideoMode(),
  or glSDL will just pass everything on to "standard SDL."

* glSDL uses an unused field (left in for binary compatibility)
  in the SDL_Surface struct. Since SDL doesn't initialize this
  field, you *MUST* use the GLSDL_FIX_SURFACE() macro on any
  surfaces you get from other libraries, such as SDL_image. If
  you don't, expect weird things to happen, as glSDL assumes
  that any other value than -1 in that field is a "TexInfo"
  handle. (In older versions, it was a pointer, which was even
  worse...)

The idea is that everything that works with SDL should work with glSDL -
although some operations will be slow, as a result of some insufficient
information being passed from the application. (For example, I’d really
like it if an SDL_Rect could be passed to SDL_LockSurface(), specifying
which are will be accessed. Would make occasional software rendering into
the “frame buffer” possible without killing the frame rate.)

As to the current state of glSDL, right now, you can’t:

* Lock the screen surface, for software rendering directly
  to the screen. (You *can* lock surfaces, though - and
  their textures will be updated as you unlock. That is,
  "software rendering into SDL_Surfaces" transparently
  becomes "procedural OpenGL textures".)

* Blit from the screen into surfaces.

* Blit from one rect to another in the screen surface.

* Use surfaces that cannot be tiled and squeezed into
  the area of a single texture. (You *can* use surfaces
  that are much wider or much taller, as long as the
  total *area* fits in an OpenGL texture.)

That’s pretty much it. Barring any bugs not known yet, everything else
should work - that is, basically everything that’s compatible with
"primarilly h/w accelerated rendering", including procedural textures.

Anyway, such a target is very interesting: why
not trying to tune it and include it into the main SDL tree?

That’s the idea. I’m just too busy hacking other stuff in Kobo Deluxe,
takeing care of my two sons (got another one 20020211) and other stuff,
and so far, I know of only one active glSDL user - although one who
does send bug reports and fixes every now and then. :slight_smile:

This would
solve all the above raised problems as OpenGL is the target that
supports the most acceleration cards under Linux: for most cards it
would rely on DRI, while for NVidia cards it would use the NVidia
drivers. It works directly under X11 (the most used target under
Linux), without any further configuration. It could even bring
acceleration to platforms others than Linux, as OpenGL itself is
portable. In short, it’s IMO The Right Thing To Do ©.

Well, you just about listed the major motivations behind glSDL. Indeed,
there are performance issues that are practically impossible to avoid
without changing the SDL API slightly - but most SDL applications should
get the full performance boost with few, if any changes. (The idea is
that th final version should be an SDL backend, so it better allow
applications to at least run without even recompiling…)

There are others exploitable ressources too, like DRI or DirectFB, but
they would be limited to certain cards, while every accelerated card
under Linux does support OpenGL.

Yeah, that’s pretty much it. Of accelerated targets, OpenGL is by far
the most powerful (alpha blending is just the beginning). As a result of
the 3D gaming industry, it probably also has the fastest drivers.

Besides, glSDL beats the crap out of SDL_OPENGLBLIT if you really don’t
want to code the 2D stuff in your 3D games in OpenGL. :wink:

SDL is mainly a 2D graphics platform, as it rely on OpenGL for 3D
rendering. Therefore, it should be unbeatable when it comes to 2D
graphics - and it can well be. I might be wrong, but it seems to me
that some others Linux graphics library (Clanlib for example) does
provide such a target. Is SDL late in that respect?

Dunno. I honestly haven’t even looked into what targets those libraries
support, or how they do it.

Anyway, the fact than SDL relies on OpenGL for 3D has very little to do
with the 2D part. You don’t even have to compile the OpenGL support in,
or have OpenGL to use the current 2D stuff! Thus, SDL cannot depend on it
internally. And there are still many systems out there with no OpenGL
support, or flaky OpenGL support that isn’t really usable for 2D
acceleration.

That is, it’s very hard to do anything for the users without decent
OpenGL support - but IMHO, that’s not a reason for SDL ignoring those who
do!

See you all and sorry for the long mail! :slight_smile:
Alex.

Note: Re-reading my mail, I realise it might sound a bit offensive and
critical, but don’t get me wrong: I just love SDL and am not attempting
to launch a troll or anything. I just feel like there is a weakness at
this precise point, and that it can easily be filled.
[…]

Well, if you look at the glSDL code, you’ll realize that maybe it’s not
that easy. :slight_smile:

But OTOH, I’ve already solved most of the problems that can be solved
without API changes, so… I’m planning to release Kobo Deluxe 0.4 soon,
and after that, I’ll look into cleaning up the source some more, and more
interestingly, splitting the “internal sub-projects” off, to live their
own lives.

The projects I’m talking about are:

* The Spitfire Engine
	* "Transparent" interpolated control system to
	  video frame rate conversion. (Always utilize
	  the full frame rate without tricky frame rate
	  dependent control code.)
	* Includes simple window oriented graphics
	  library with "smart refreshing", background
	  images and other handy features.
	* Graphics management system with plugin based
	  image processing system. Splits images up in
	  tiles or sprites, scales using various filters,
	  adjusts and cleans alpha channels and more...

* EEL - the Extensible Embeddable Language
	* Simple, readable C/Pascal/BASIC hybrid syntax.
	* Dynamic typing - variables take type *and*
	  value during assignment.
	* Expression evaluation that handles parentheses
	  and operator precedence.
	* Functions and procedures.
	* External files can be executed from within
	  scripts.
	* Very simple API for language extensions.
	* Language extensions can add *constructs*, not
	  just native code procedures and functions.
	* Language extensions can add new operators.
	* 100% ANSI C code! No external tools needed.
	Planned or in progress:
		* Types and "supported type lists" in
		  argument lists of function and
		  procedure declarations.
		* Support for "fixed type variables",
		  for faster execution and load or
		  compile time type checking.
		* "Source compressor" - strips and
		  tokenizes EEL source code, to save
		  space.
		* Semi-compiling, for faster execution.

* The Kobo Deluxe Audio Engine (no real name yet)
	* Software synth with dynamic voice allocation.
	* Scalable in performance/quality from "nearest
	  sample" through adaptive interpolation, using
	  cubic (4 point) interpolation or linear
	  interpolation w/ oversampling, depending on
	  in/out sample ratio.
	* Three voice allocation/control modes; "Mono",
	  "Poly" and "Detach", suitable for monophonic
	  instruments, polyphonic instruments and sound
	  effects, respectively.
	* Bus mixing architecture with insert plugin
	  support and fully configurable routing,
	  including sends from bus to bus, and sends
	  between each insert effect.
	* Simple "Waveform Construction API", that lets
	  you render sound effects using a few simple
	  calls.
	* Uses EEL to make the waveform contruction API
	  scriptable. This results in a way of creating
	  *very* compact sounds - a few hundred bytes
	  for rather nice instrumental sounds, as human
	  readable source with comments.
	* External MIDI input for real time playing.
	* Internal MIDI sequencer.
	* MIDI sequences can be played as waveforms!
	  (Within a few days, MIDI files will actually
	  be able to use other MIDI files as
	  "instruments", modulating their pitch,
	  velocity and so on.)
	* Wave files, raw audio files, MIDI files and
	  EEL (or rather AGW) scripts can be loaded
	  using the same call, and can be played and
	  modulated as if they were all the same thing.

* glSDL
	* Compile SDL programs to optionally run on
	  top of OpenGL.
	* Automatically tiles surfaces to fit in OpenGL
	  textures.
	* Uses OpenGL blending for alpha and colorkeying.
	* Supports using surfaces as procedural textures,
	  for efficient software rendering in combination
	  with h/w acceleration.
	Planned, or "probably broken right now":
		* Locking of the screen surface.
		* Blitting from and within the screen.
		* Minor optional API extensions to boost
		  performance of some operations. (Could
		  be of use to other targets as well.)
		* Transparent scaling from arbitrary
		  resolutions to the screen resolution.
		  (For emulation of unsupported
		  resolutions in fullscreen mode.)
		* Make it an SDL rendering backend!

There are some other smaller things that might be of general use as well,
such as the platform independent “filemapper” class, and the stack based
"state manager" that I’m using to keep track of the various screens and
states of the whole game. Probably others as well… I’ll filter them out
as I’m cleaning up, but only the 4 above will become stand-alone
libraries - the smaller ones are just headers and (sometimes) C files, so
I’ll release them like that, to be pasted into projects.

Oh, and it’s all LGPL, or at least will be - I’m still waiting for a
response regarding the remaining GPLed parts of the audio engine. (I
never actually deleted the original audio engine files, although most,
if not all, of that code is gone by now.)

(Note that the game is GPL just like the original XKobo - not LGPL.
Only the code that I’ve hacked from scratch, or borrowed from other
projects is LGPLed.)

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Sunday 17 February 2002 11:12, Alexandre Courbot wrote:

Why? It’s important to realize that glSDL is a source level wrapper. You
should not use any of the glSDL_* calls explicitly, but rather just
include glSDL.h instead of SDL.h and recompile. Exceptions:

Mmmm, actually I’ve just gave it another try with Kobo Deluxe and SDL
CVS source, I just can’t get anything to work now! Either it
hard-crashes, either I just get white squares on the screen. Feel free
to contact me privately if you want me to perform some tests.

Anyway, such a target is very interesting: why
not trying to tune it and include it into the main SDL tree?

That’s the idea. I’m just too busy hacking other stuff in Kobo Deluxe,
takeing care of my two sons (got another one 20020211) and other stuff,
and so far, I know of only one active glSDL user - although one who
does send bug reports and fixes every now and then. :slight_smile:

Oh, congrats! :wink: I’m sure such a target would be usefull (and
considering the latest discussions it looks like I’m not the only one,
hopefully! :)) There were especially an interesting thread with Dirk
Gerrits and Joseph Carter about the possible implementation, but nothing
you are not aware of, I guess.

Well, you just about listed the major motivations behind glSDL. Indeed,
there are performance issues that are practically impossible to avoid
without changing the SDL API slightly - but most SDL applications should
get the full performance boost with few, if any changes. (The idea is
that th final version should be an SDL backend, so it better allow
applications to at least run without even recompiling…)

Maybe it would be worth to take these elements into account when
designing SDL 1.3 API.

Anyway, the fact than SDL relies on OpenGL for 3D has very little to do
with the 2D part. You don’t even have to compile the OpenGL support in,
or have OpenGL to use the current 2D stuff! Thus, SDL cannot depend on it
internally. And there are still many systems out there with no OpenGL
support, or flaky OpenGL support that isn’t really usable for 2D
acceleration.

Right, of course. There is no reason to make SDL depend on OpenGL. But
anyway, none of the backends is mandatory when compiling SDL, so the
OpenGL one shouldn’t neither.

Well, if you look at the glSDL code, you’ll realize that maybe it’s not
that easy. :slight_smile:

Hehe :slight_smile: Haven’t looked yet, I admit. I lack time and am far from being
an OpenGL guru! :wink:

But OTOH, I’ve already solved most of the problems that can be solved
without API changes, so… I’m planning to release Kobo Deluxe 0.4 soon,
and after that, I’ll look into cleaning up the source some more, and more
interestingly, splitting the “internal sub-projects” off, to live their
own lives.

Excellent idea. There are some very interesting pieces of code in the
stuff you listed below.

Alex.–
http://www.gnurou.org

Why? It’s important to realize that glSDL is a source level wrapper.
You should not use any of the glSDL_* calls explicitly, but rather
just include glSDL.h instead of SDL.h and recompile. Exceptions:

Mmmm, actually I’ve just gave it another try with Kobo Deluxe and SDL
CVS source, I just can’t get anything to work now! Either it
hard-crashes, either I just get white squares on the screen. Feel free
to contact me privately if you want me to perform some tests.

White squares? Sounds like what happens on my G400 with the Win32 drivers
(which work great with Q3 and RTCW), but not on XFree 4.1.x on the same
card…

The bad news is that I have no idea whatsoever what’s going on. :-/

Anyway, such a target is very interesting: why
not trying to tune it and include it into the main SDL tree?

That’s the idea. I’m just too busy hacking other stuff in Kobo
Deluxe, takeing care of my two sons (got another one 20020211) and
other stuff, and so far, I know of only one active glSDL user -
although one who does send bug reports and fixes every now and
then. :slight_smile:

Oh, congrats! :wink:

Thanks!

I’m sure such a target would be usefull (and
considering the latest discussions it looks like I’m not the only one,
hopefully! :))

There were especially an interesting thread with Dirk
Gerrits and Joseph Carter about the possible implementation, but
nothing you are not aware of, I guess.

As far as I’ve seen, they’ve mostly discussed things that glSDL solves
already. glSDL just has to be cleaned up, transformed into a patch to
SDL, and the last few missing features have to be implemented. I don’t
think there are any really clever optimizations to make, except for
improving the “texture area management”. (Currently, a texture can
contain graphics from only one surface - no free space can be used for
other surfaces.)

Well, you just about listed the major motivations behind glSDL.
Indeed, there are performance issues that are practically
impossible to avoid without changing the SDL API slightly - but most
SDL applications should get the full performance boost with few, if
any changes. (The idea is that th final version should be an SDL
backend, so it better allow applications to at least run without
even recompiling…)

Maybe it would be worth to take these elements into account when
designing SDL 1.3 API.

Yes… It would be helpful just to know which features an application is
going to use on a per frame basis, so we can select the most suitable
backend. (This could be added to SDL 1.2 in the form of some new flags,
but that wouldn’t help applications that don’t care to use them…)

[…]

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Saturday 02 March 2002 13:15, Alexandre Courbot wrote:

Why? It’s important to realize that glSDL is a source level wrapper.
You should not use any of the glSDL_* calls explicitly, but rather
just include glSDL.h instead of SDL.h and recompile. Exceptions:

Mmmm, actually I’ve just gave it another try with Kobo Deluxe and SDL
CVS source, I just can’t get anything to work now! Either it
hard-crashes, either I just get white squares on the screen.
I doubt this is helpful… but white squares like you describe have
often been the result of a 32bpp Alpha channel surface being converted
to a non-alpha format.

Feel freeOn Mon, 2002-03-04 at 12:22, David Olofson wrote:

On Saturday 02 March 2002 13:15, Alexandre Courbot wrote:

to contact me privately if you want me to perform some tests.

White squares? Sounds like what happens on my G400 with the Win32 drivers
(which work great with Q3 and RTCW), but not on XFree 4.1.x on the same
card…

The bad news is that I have no idea whatsoever what’s going on. :-/

Anyway, such a target is very interesting: why
not trying to tune it and include it into the main SDL tree?

That’s the idea. I’m just too busy hacking other stuff in Kobo
Deluxe, takeing care of my two sons (got another one 20020211) and
other stuff, and so far, I know of only one active glSDL user -
although one who does send bug reports and fixes every now and
then. :slight_smile:

Oh, congrats! :wink:

Thanks!

I’m sure such a target would be usefull (and
considering the latest discussions it looks like I’m not the only one,
hopefully! :))

There were especially an interesting thread with Dirk
Gerrits and Joseph Carter about the possible implementation, but
nothing you are not aware of, I guess.

As far as I’ve seen, they’ve mostly discussed things that glSDL solves
already. glSDL just has to be cleaned up, transformed into a patch to
SDL, and the last few missing features have to be implemented. I don’t
think there are any really clever optimizations to make, except for
improving the “texture area management”. (Currently, a texture can
contain graphics from only one surface - no free space can be used for
other surfaces.)

Well, you just about listed the major motivations behind glSDL.
Indeed, there are performance issues that are practically
impossible to avoid without changing the SDL API slightly - but most
SDL applications should get the full performance boost with few, if
any changes. (The idea is that th final version should be an SDL
backend, so it better allow applications to at least run without
even recompiling…)

Maybe it would be worth to take these elements into account when
designing SDL 1.3 API.

Yes… It would be helpful just to know which features an application is
going to use on a per frame basis, so we can select the most suitable
backend. (This could be added to SDL 1.2 in the form of some new flags,
but that wouldn’t help applications that don’t care to use them…)

[…]

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -’


SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

Uh oh…

Under Linux, the G400 driver silently converts 32 bit RGBA into 16 bit
RGBA, while 24 bit BGB is used as is.

Is it possible that the Windows driver just won’t do this for some reason?

/David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |-------------------------------------> http://olofson.net -'On Monday 04 March 2002 18:31, Jimmy wrote:

On Mon, 2002-03-04 at 12:22, David Olofson wrote:

On Saturday 02 March 2002 13:15, Alexandre Courbot wrote:

Why? It’s important to realize that glSDL is a source level
wrapper. You should not use any of the glSDL_* calls
explicitly, but rather just include glSDL.h instead of SDL.h and
recompile. Exceptions:

Mmmm, actually I’ve just gave it another try with Kobo Deluxe and
SDL CVS source, I just can’t get anything to work now! Either it
hard-crashes, either I just get white squares on the screen.

I doubt this is helpful… but white squares like you describe have
often been the result of a 32bpp Alpha channel surface being converted
to a non-alpha format.