SetVideoMode Resize Crash (due to SDL_MouseRect)

SDL Development
1

Would you happen to be running a virtual desktop? (ie: viewable =
1024x768 but destkop is actually 1600x1200 and when the mouse hits the
border, the screen pans) Or maybe a multi-monitor set-up? I’m just
guessing, here…–

Olivier A. Dagenais - Software Architect and Developer

“Evil” wrote in message
news:OE34ZmcK8r6beUzmRf600000015 at hotmail.com…

Errr… Shouldn’t that be:

Uint32 * buffer = (Uint32 *) screen->pixels;
buffer[0] = 0xFF;

Yes, but it doesn’t really matter since a pointer fits into the
UINT32 (in
VC++). That was just an oversight I made while throwing together
the test
program - which didn’t effect the results, I did things correctly in
the
program where I initially detected the problem (which cast things to
a
Uint16*).

Also, check that the value returned by SetVideoMode is not NULL.

The problem is that SetVideoMode never returns! It stomps memory
within the
function.

I did a lot of debugging this afternoon and figured out exactly
where SDL is
stumbling, check it out: The failing function within the
SetVideoMode call
is (drumroll)… SDL_DrawCursorNoLock(). I discovered that the
’random’
factor affecting the crashes had to do with how far down the screen
I had my
cursor!

I marred the beautiful sdlcursor.c with some of my half-assed
debugging code
(since I couldn’t step through these functions in fullscreen mode)
and I
found that the memcpy in SDL_DrawCursorNoLock() was causing the
allocation
problem. Here are what the variable there (right before the memcpy)
look
like on a successful SetVideoMode():

w = 32, h = 16, screenbpp = 2, pitch = 1600

and here’s something typical of what I was seeing during the calls
that were
exploding:

w = 130638, h = 16, screenbpp = 2, pitch = 1600

I think we can all see which variable is suspicious here.

So, I figure that SDL_MouseRect() is where the true damage is done.
I put
some checks in there and here’s what I found:

successful: SDL_cursor->area.x = 593, .y =380, .w = 16, .h =16,

surface->w = 800, surface->h = 600

b4 failure:  SDL_cursor->area.x = 1017, .y =201, .w = 16, .h

=16,

surface->w = 800, surface->h = 600

Now, can someone look at the logic of SDL_MouseRect() and confirm
exactly
what the problem is? It looks as if the logic assumes your cursor
will
never be out of bounds of the new fullscreen resolution (not good),
eh?
I’ve been staring at this way too long to decide anything tonight or
how to
fix it - I’d rather defer that honor to someone more intimate with
the code
(since I’ve only been using SDL since last friday ).

Thanks in advance,
Jesse
www.eternaldusk.com

2

Would you happen to be running a virtual desktop? (ie: viewable =
1024x768 but destkop is actually 1600x1200 and when the mouse hits the
border, the screen pans) Or maybe a multi-monitor set-up? I’m just
guessing, here…

Olivier A. Dagenais - Software Architect and Developer

No, the bug occurs on normal desktop/single-monitor systems. It looks like
Ajay and I have the fix for it now, but thanks for taking the time to try to
help a fellow programmer!

-Jesse