Using OpenGL for image transformations

SDL Development
1

I’ve seen several people ask about how to do image transformations (rotation and zooming, mostly) and the answer is usually “use SDL_gfx or do it in OpenGL.” Well, SDL_gfx is slow and I’d like to be able to do things in GL.

I’ve looked up a lot of stuff on OpenGL programming and I’ve got a handle on how texture mapping works and how you can map things at strange angles by messing with the vertices in order to rotate, scale or mirror a texture you’re drawing to the screen. Problem is, everything I find is about drawing to the screen. (Or to the GL buffer, which goes to the screen. Same difference in the end.) What I really want to do is take a surface and rotate, scale and mirror it, (or some combination of the three,) and possibly even run some shaders on it for advanced image processing, and then NOT draw it to the screen, but retrieve it back into an SDL_surface.

Does anyone know enough OpenGL to help me out? I need some sample code showing what to do with an SDL_surface I’ve loaded into a GL texture. Assume I already have a function written called GL_Mutilate() that does all the image processing, and throw that in at the appropriate place, then retrieve the result and return a SDL_surface pointer.

If someone could write up a sample routine illustrating the basic principles for doing this, I’d be very grateful.

Mason

2

Hi Mason,

I’ve seen several people ask about how to do image transformations
(rotation and zooming, mostly) and the answer is usually “use
SDL_gfx or do it in OpenGL.” Well, SDL_gfx is slow and I’d like to
be able to do things in GL.

I’ve looked up a lot of stuff on OpenGL programming and I’ve got a
handle on how texture mapping works and how you can map things at
strange angles by messing with the vertices in order to rotate,
scale or mirror a texture you’re drawing to the screen. Problem is,
everything I find is about drawing to the screen. (Or to the GL
buffer, which goes to the screen. Same difference in the end.) What
I really want to do is take a surface and rotate, scale and mirror
it, (or some combination of the three,) and possibly even run some
shaders on it for advanced image processing, and then NOT draw it to
the screen, but retrieve it back into an SDL_surface.

To read from an OpenGL context back to client memory (ie from GPU
memory to memory the CPU can access) there is no other method that I
know of than glReadPixels. I’m not sure how glReadPixels interacts
with the OpenGL framebuffer extension, but if that works you may find
using framebuffers useful as they could provide you someplace
offscreen to draw.

Does anyone know enough OpenGL to help me out? I need some sample
code showing what to do with an SDL_surface I’ve loaded into a GL
texture. Assume I already have a function written called
GL_Mutilate() that does all the image processing, and throw that in
at the appropriate place, then retrieve the result and return a
SDL_surface pointer.

I would first draw the GLTexture to the screen using OpenGL then (so
GL_Mutilate()), then assuming your SDL_Surface has an RGBA pixel
format with 8 bits per channel:

glReadPixels(x, y, surface.width, surface.height, GL_RGBA,
GL_UNSIGNED_BYTE, surface->pixels);

Where x,y are the OpenGL window coordinates where wish to start
copying to the surface.

Keep in mind that glReadPixels is slow (I guess because it transfers
between GPU memory and RAM), so my guess is this method would only be
warranted if the GL_Mutilate is sufficiently complex. I’m guessing if
you’re just rotating an image it might be slower, but I don’t know,
try it out.

  • HolmesOn Sep 16, 2008, at 7:51 PM, Mason Wheeler wrote:

If someone could write up a sample routine illustrating the basic
principles for doing this, I’d be very grateful.

Mason

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

3

Mason,

what are you trying to do with that resulting SDL_Surface? If you are going to
do any other thing than displaying it in any way on the screen, certainly
speed is not your worst problem. You should not use SDL nor OpenGL and
consider a general purpose library like libmagick (www.imagemagick.org)

If you are going to reuse it on the screen you are drawing, don’t take it back
into a SDL_Surface, nor use glReadPixels. Use it as a texture with the frame
buffer extension (FBO):

http://www.gamedev.net/reference/articles/article2331.asp

Alberto

El Mi?rcoles 17 Septiembre 2008ES 04:51:51 Mason Wheeler escribi?:> I’ve seen several people ask about how to do image transformations

(rotation and zooming, mostly) and the answer is usually “use SDL_gfx or do
it in OpenGL.” ?Well, SDL_gfx is slow and I’d like to be able to do things
in GL.

I’ve looked up a lot of stuff on OpenGL programming and I’ve got a handle
on how texture mapping works and how you can map things at strange angles
by messing with the vertices in order to rotate, scale or mirror a texture
you’re drawing to the screen. ?Problem is, everything I find is about
drawing to the screen. ?(Or to the GL buffer, which goes to the screen.
Same difference in the end.) ?What I really want to do is take a surface
and rotate, scale and mirror it, (or some combination of the three,) and
possibly even run some shaders on it for advanced image processing, and
then NOT draw it to the screen, but retrieve it back into an SDL_surface.

Does anyone know enough OpenGL to help me out? ?I need some sample code
showing what to do with an SDL_surface I’ve loaded into a GL texture.
?Assume I already have a function written called GL_Mutilate() that does
all the image processing, and throw that in at the appropriate place, then
retrieve the result and return a SDL_surface pointer.

If someone could write up a sample routine illustrating the basic
principles for doing this, I’d be very grateful.

Mason

4

(Sorry for top-posting.)

So, you’re trying to use OpenGL to render to an offscreen image of
your rotated sprites and then use those pixels with software 2D SDL?
I have to say I’m a little confused at why this becomes necessary. I
would suggest just going over to OpenGL for your drawing needs. It’s
really not that hard, and lets you ignore a lot of the more software-
oriented drawing ideas like dirty rectangles, etc. (You CAN use dirty
rects in OGL, however, but on most hardware it isn’t necessary for 2D
drawing.)

You mentioned speed as an issue with the software-based rotations and
scaling, which makes me think that you’re imagining transforming your
image in OpenGL and then drawing the resulting pixels to your SDL
context each frame, and I have to say this is probably (though your
situation may be an exception?) a tremendous waste of time and
effort. If you’re already using OpenGL, you may as well just use it
outright. As has been mentioned already, it’s pretty slow to read the
pixel data into a pixel array from the video hardware, so already the
speed you may have gotten with OpenGL is somewhat compromised.

Also, when they talk about using SDL_gfx, what they usually mean is
that you manipulate your image, and then save it into memory as a new
sprite or image, and then you can use the transformed sprites when
they’re needed, quickly, without having to load pre-transformed image
data from the disk, thereby lowering the amount of redundant data you
have to include with your program.

But if you’re going so far as to want to include fragment shader
programs in with your image manipulation, then you already have decent
OpenGL capabilities on your graphics card, and I’m pretty sure you’re
better off just using SDL to give you an OpenGL context, and rendering
into that…

– ScottOn Sep 16, 2008, at 8:51 PM, Mason Wheeler wrote:

I’ve seen several people ask about how to do image transformations
(rotation and zooming, mostly) and the answer is usually “use
SDL_gfx or do it in OpenGL.” Well, SDL_gfx is slow and I’d like to
be able to do things in GL.

I’ve looked up a lot of stuff on OpenGL programming and I’ve got a
handle on how texture mapping works and how you can map things at
strange angles by messing with the vertices in order to rotate,
scale or mirror a texture you’re drawing to the screen. Problem is,
everything I find is about drawing to the screen. (Or to the GL
buffer, which goes to the screen. Same difference in the end.) What
I really want to do is take a surface and rotate, scale and mirror
it, (or some combination of the three,) and possibly even run some
shaders on it for advanced image processing, and then NOT draw it to
the screen, but retrieve it back into an SDL_surface.

Does anyone know enough OpenGL to help me out? I need some sample
code showing what to do with an SDL_surface I’ve loaded into a GL
texture. Assume I already have a function written called
GL_Mutilate() that does all the image processing, and throw that in
at the appropriate place, then retrieve the result and return a
SDL_surface pointer.

If someone could write up a sample routine illustrating the basic
principles for doing this, I’d be very grateful.

Mason

5

So, you’re trying to use OpenGL to render to an offscreen image of
your rotated sprites and then use those pixels with software 2D SDL?
I have to say I’m a little confused at why this becomes necessary.
…
You mentioned speed as an issue with the software-based rotations and
scaling, which makes me think that you’re imagining transforming your
image in OpenGL and then drawing the resulting pixels to your SDL
context each frame,
…
Also, when they talk about using SDL_gfx, what they usually mean is
that you manipulate your image, and then save it into memory as a new
sprite or image, and then you can use the transformed sprites when
they’re needed, quickly, without having to load pre-transformed image
data from the disk, thereby lowering the amount of redundant data you
have to include with your program.

Sort of. What I’m trying to do involves tile-based sprites. I’ve got a bunch of tiles,
and I want to be able to assemble them into a sprite in real-time. Depending on
various circumstances, this might involve rotating or even mirroring some of the
tiles, and I haven’t seen any add-on SDL libraries that support mirroring yet.
(Why is that anyway? It seems to me it would be a lot easier to code than
rotation.) And then when the sprite is assembled, I might want to rotate or mirror
the finished product as well. (And yes, this is done to keep from having to load it
all from disc, because there is limited space in the sprite sheets.)

You can probably tell I’m a bit new to the whole OpenGL thing. I’m still thinking
in SDL, and the way I imagine doing this is about the way I’d do it in SDL_gfx, if
SDL_gfx could do mirroring: blit each tile that needed transformations onto a
temporary surface, perform transformations, assemble the tiles by blitting them
all onto a temporary surface, perform transformations on the final sprite, and then
load the result to a surface and have that be what I use. From various people’s
comments, I take it there’s an easier way in OpenGL?>----- Original Message ----

From: Scott Harper
Subject: Re: [SDL] Using OpenGL for image transformations

6

El Mi?rcoles 17 Septiembre 2008ES 14:08:17 Mason Wheeler escribi?:

?From various people’s
comments, I take it there’s an easier way in OpenGL?

Yes. Don’t think of blitting the pixels but moving the polygons. Suppose you
have a quad per tile, then you would do:

glRotate(final_rotation_of_all_the_map)
for(all_the_tiles)
glTranslate(x,y unrotated postions of this tile)
end

That would set a tile grid, then rotate it without secondary render buffers.
You can link any number of transformations without any blitting cost, except
for the last (the ony one).

Alberto

7

Also (and here’s something I haven’t done, actually) in OpenGL there’s
a way to take a frame buffer and turn that directly into a new
texture. In this case, you could (once) draw the newly configured
sprite as you describe into an offscreen frame buffer object, then
read that back in as a new texture, and finally draw the new texture
as a larger quad within the main view.

Basically, drawing sprites/tiles in OpenGL become drawing quads of the
appropriate size. You set up a, orthographic viewing area of the
dimensions you want to have (e.g. the dimensions of your screen or
window, assuming you want a pixel-per-pixel accurate rendering context
for your sprites and tiles) using something simple like gluOrtho(…),
and then instead of blitting rectangles, you draw quads of the same
size (keeping in mind that OpenGL uses a cartesion coordinate setup
rather than a screen coordinate setup ? origin is bottom left not top
left) wherever they should be in your field.

This is actually pretty handy because of how OpenGL’s stack works.
Say you have a scrolling, tile-based background that is larger than
your screen area, what you can do is before each draw call a
glPushMatrix(), then a glTraslate_() to move your “view” to wherever
is should be at the moment, and do all your drawing calls as normal,
drawing all of your tiles as you would just iterating through them.
Then just make sure to remember to call glPopMatrix() after drawing so
that you aren’t compounding your offset each frame.

If you have any more OpenGL specific questions, feel free to email me
off-list, though, since it kinds of strays from the whole SDL topic, I
think. ^_^;;

– ScottOn Sep 17, 2008, at 6:18 AM, Alberto Luaces wrote:

El Mi?rcoles 17 Septiembre 2008ES 14:08:17 Mason Wheeler escribi?:

From various people’s
comments, I take it there’s an easier way in OpenGL?

Yes. Don’t think of blitting the pixels but moving the polygons.
Suppose you
have a quad per tile, then you would do:

glRotate(final_rotation_of_all_the_map)
for(all_the_tiles)
glTranslate(x,y unrotated postions of this tile)
end

That would set a tile grid, then rotate it without secondary render
buffers.
You can link any number of transformations without any blitting
cost, except
for the last (the ony one).

Alberto

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

8

Hi,

El Mi?rcoles 17 Septiembre 2008ES 15:50:36 Scott Harper escribi?:

Also (and here’s something I haven’t done, actually) in OpenGL there’s ?
a way to take a frame buffer and turn that directly into a new ?
texture. ?In this case, you could (once) draw the newly configured ?
sprite as you describe into an offscreen frame buffer object, then ?
read that back in as a new texture, and finally draw the new texture ?
as a larger quad within the main view.

I submitted a link to a FBO tutorial on my first reply :slight_smile: However I think a
tile engine isn’t worth it, since on current cards the bottleneck is usually
on the fill rate and not the vertex processing. Why bother obfuscating the
code with several layers of intermediate surfaces when you can place a great
number of tiles almost for free? If you are after a tile engine, you can even
make a static grid that covers the whole screen and only change texture
coordinates from frame to frame. No translations or rotations of geometry
involved :wink:

For the general blitter case I suggest looking at SDL 1.3 source, since a
general GL blitter backend is being developed.

Alberto

9

Hey,

Sprig does have a mirroring ability (and I think SDL_gfx does too). Just use negative scaling factors when calling SPG_Scale or SPG_Transform, etc. It’s a little bit of a hidden feature since the documentation isn’t up to snuff…

For OpenGL, I have done exactly what you want… My code is on my home computer though. You can render directly to a texture, then save that as a bitmap file or whatever. I don’t remember the calls necessary, but if you look through the Red Book you may find it before I get back to you.

Jonny D> Date: Wed, 17 Sep 2008 05:08:17 -0700

From: masonwheeler at yahoo.com
To: sdl at lists.libsdl.org
Subject: Re: [SDL] Using OpenGL for image transformations

----- Original Message ----

From: Scott Harper
Subject: Re: [SDL] Using OpenGL for image transformations

So, you’re trying to use OpenGL to render to an offscreen image of
your rotated sprites and then use those pixels with software 2D SDL?
I have to say I’m a little confused at why this becomes necessary.
…
You mentioned speed as an issue with the software-based rotations and
scaling, which makes me think that you’re imagining transforming your
image in OpenGL and then drawing the resulting pixels to your SDL
context each frame,
…
Also, when they talk about using SDL_gfx, what they usually mean is
that you manipulate your image, and then save it into memory as a new
sprite or image, and then you can use the transformed sprites when
they’re needed, quickly, without having to load pre-transformed image
data from the disk, thereby lowering the amount of redundant data you
have to include with your program.

Sort of. What I’m trying to do involves tile-based sprites. I’ve got a bunch of tiles,
and I want to be able to assemble them into a sprite in real-time. Depending on
various circumstances, this might involve rotating or even mirroring some of the
tiles, and I haven’t seen any add-on SDL libraries that support mirroring yet.
(Why is that anyway? It seems to me it would be a lot easier to code than
rotation.) And then when the sprite is assembled, I might want to rotate or mirror
the finished product as well. (And yes, this is done to keep from having to load it
all from disc, because there is limited space in the sprite sheets.)

You can probably tell I’m a bit new to the whole OpenGL thing. I’m still thinking
in SDL, and the way I imagine doing this is about the way I’d do it in SDL_gfx, if
SDL_gfx could do mirroring: blit each tile that needed transformations onto a
temporary surface, perform transformations, assemble the tiles by blitting them
all onto a temporary surface, perform transformations on the final sprite, and then
load the result to a surface and have that be what I use. From various people’s
comments, I take it there’s an easier way in OpenGL?

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

Want to do more with Windows Live? Learn ?10 hidden secrets? from Jamie.
http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008

10

Okay, I found it. Pretty simple, but if you’re new to OpenGL, you might not have seen it.

glMatrixMode(GL_TEXTURE);

This sets the rendering destination to the currently bound texture.
I also (re)found this, which is neat:

/* Take snapshot of frame buffer, and store in texture object 1.

  • Take snapshot of center 400 by 400 pixels of frame buffer,
  • assuming center is at 600, 400.
    */
    glBindTexture( GL_TEXTURE_2D, 1 );
    glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, 400, 200, 512, 512, 0);
    glBindTexture( GL_TEXTURE_2D, 0 );

The comment is a little off because I changed the values, but you can figure it out. With this, you can copy the current screen to another texture. I used it for a little video screen in-game.

Jonny D_________________________________________________________________
See how Windows Mobile brings your life together?at home, work, or on the go.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/

11

glMatrixMode(GL_TEXTURE);

This sets the rendering destination to the currently bound texture.

Doesn’t this just allow you to perform texture transformation
routines? For instance, scaling, rotating, and transforming texture
coordinates rather than polygons or transforming view positions. I
was under the strong impression that glMatrixMode(…) doesn’t affect
where anything gets drawn to. That is always the current OpenGL
context, I believe.

/* Take snapshot of frame buffer, and store in texture object 1.

  • Take snapshot of center 400 by 400 pixels of frame buffer,
  • assuming center is at 600, 400.
    */
    glBindTexture( GL_TEXTURE_2D, 1 );
    glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, 400, 200, 512, 512, 0);
    glBindTexture( GL_TEXTURE_2D, 0 );

This, I believe, copies part of the viewport into the texture, so you
have to render into the viewport first. I found an NVidia
presentation on the matter as well as a NeHe tutorial on rendering to
texture:

http://developer.nvidia.com/object/gdc_oglrtt.html
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=36

It looks like you either have to render the new texture, copy it to a
texture, and then render the ACTUAL scene using the new texture (a
process you could shortcut by just prerendering all your textures in
this case, I think). Of course, I have no idea how to just do this
all offscreen without having a rendering context… The NVidia paper
mentions some ARB extensions (specific to Windows, it looks like,
though I know Mac has similar things you can do) that allow you to
render directly into textures, though.

– Scott

12

Hmm, maybe I grabbed the wrong stuff in haste? Or maybe it’s been too long since I touched that code. Either way, you can still get the desired effect with the stuff I gave. Use an orthogonal projection, render your texture to the buffer, make your other edits, then use glCopyTexImage2D to put the texture back. Clear the buffer and then do your thing. That may not be the best way around this, but it should work, right? Wrap it in a function for ease, I guess.

Jonny D----------------------------------------

From: orcein at gmail.com
To: sdl at lists.libsdl.org
Date: Fri, 19 Sep 2008 15:36:44 -0600
Subject: Re: [SDL] Using OpenGL for image transformations

glMatrixMode(GL_TEXTURE);

This sets the rendering destination to the currently bound texture.

Doesn’t this just allow you to perform texture transformation
routines? For instance, scaling, rotating, and transforming texture
coordinates rather than polygons or transforming view positions. I
was under the strong impression that glMatrixMode(…) doesn’t affect
where anything gets drawn to. That is always the current OpenGL
context, I believe.

/* Take snapshot of frame buffer, and store in texture object 1.

  • Take snapshot of center 400 by 400 pixels of frame buffer,
  • assuming center is at 600, 400.
    */
    glBindTexture( GL_TEXTURE_2D, 1 );
    glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, 400, 200, 512, 512, 0);
    glBindTexture( GL_TEXTURE_2D, 0 );

This, I believe, copies part of the viewport into the texture, so you
have to render into the viewport first. I found an NVidia
presentation on the matter as well as a NeHe tutorial on rendering to
texture:

http://developer.nvidia.com/object/gdc_oglrtt.html
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=36

It looks like you either have to render the new texture, copy it to a
texture, and then render the ACTUAL scene using the new texture (a
process you could shortcut by just prerendering all your textures in
this case, I think). Of course, I have no idea how to just do this
all offscreen without having a rendering context… The NVidia paper
mentions some ARB extensions (specific to Windows, it looks like,
though I know Mac has similar things you can do) that allow you to
render directly into textures, though.

– Scott

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

See how Windows Mobile brings your life together?at home, work, or on the go.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/