Trouble with Windows dialogs in a fullscreen SDL app

I’m trying to prompt the user for a file, using GetOpenFileName, and I’ve run into a number of problems when my app is running in fullscreen mode.

I’d like the system to behave in one of the following ways:
Desired: The open file dialog displays on top of the fullscreen app.
Fallback: The fullscreen app minimizes itself and displays the dialog. When the dialog is closed, the app restores itself.

Existing behavior:
When the dialog is created, the app minimizes, but the dialog does not gain focus. Whichever explorer window was in focus before launching the app gets focus. This can cause the open file dialog to appear behind other windows obscuring it from the user. Also, when the dialog is closed, either by selecting a file to open or canceling, the app stays minimized and does not restore itself. If the dialog was parented, the app regains focus but does not restore, if the dialog was not parented the app doesn’t regain focus.

So there seem to be a number of problems.

  1. When the dialog window is created, it does not acquire focus.*
  2. When the dialog is closed, the app doesn’t restore.
  3. (optional) Even if the dialog is parented to the app, it minimizes the app.

*This also happens in windowed mode if I am launching/running the app in the Remote Debugger. If the app is launched by itself on the client machine it only appears to happen in fullscreen mode.

I’ve got possible fixes for 2 and 3, which I’ll post below so not to clutter this post, but I don’t know how to fix 1, which is the main problem. Any ideas how to fix this one?

I’m running on Windows XP SP3 (though Vista exhibits this behavior as well).
This is using SDL version 1.2.13, though from looking at the code from later versions (including 1.3) it seems 2 & 3 still exist there. I haven’t been able to evaluate 1 on any later versions.

For 3) Even if the dialog is parented to the app, it minimizes the app, the following fix works (:

In SDL-1.2.13/src/video/wincommon/SDL_sysevents.c
Function WinMessage
case WM_ACTIVATE

Code:

active = (LOWORD(wParam) != WA_INACTIVE) && !minimized;
if ( active ) {

} else {
/* Lose the following states */
appstate = SDL_APPINPUTFOCUS;
if ( minimized ) {
appstate |= SDL_APPACTIVE;
}
if ( this->input_grab != SDL_GRAB_OFF ) {
WIN_GrabInput(this, SDL_GRAB_OFF);
}
if ( SDL_GetAppState() & SDL_APPINPUTFOCUS ) {
if ( ! DDRAW_FULLSCREEN() ) {
DIB_SwapGamma(this);
}
if ( WINDIB_FULLSCREEN() ) {
// If the window being activated has this app as its parent, don’t restore desktop mode and minimize.
if( GetParent( (HWND)lParam ) != hwnd )
{
SDL_RestoreDesktopMode();

This seems to work, but I don’t know if it’s supported or undefined behavior. The problem is that it still has the focus bug on some versions of Windows so the dialog is sometimes created underneath the fullscreen app, which is obviously bad. Seems to work on XP SP3 but fails on Vista.

Fix for 2) When the dialog is closed, the app doesn’t restore:

In SDL-1.2.13/src/video/wincommon/SDL_sysevents.c
Function WinMessage
case WM_ACTIVATE

existing code:

Code:

SDL_VideoDevice *this = current_video;
BOOL active, minimized;
Uint8 appstate;

minimized = HIWORD(wParam);
active = (LOWORD(wParam) != WA_INACTIVE) && !minimized;

When the dialog is created, the app minimizes itself. Once the dialog is closed, the app receives a WM_ACTIVATE message while minimized. Since active is only set to true if minimized is false, the app doesn’t call SDL_RestoreGameMode(). This can be fixed by changing the line assigning active to the following:

Code:
active = (LOWORD(wParam) != WA_INACTIVE);

Does this seem reasonable? What is the reasoning for not restoring a minimized app on WM_ACTIVATE? Maybe I’m missing something.

Also, I don’t know if it helps but these are the Windows messages my app receives during the dialog creation/closing process, along with some of the SDL calls that handle it (with stock v1.2.13 code). Note especially the WinMessage WM_ACTIVATE re-entry, caused by the ShowWindow call in RestoreDesktopMode to minimize the app.

Ah… This is probably the issue. The first WM_ACTIVATE has the proper dialog hwnd in its lParam, but the ShowWindow(SDL_Window, SW_MINIMIZE) call to minimize the app causes another WM_ACTIVATE with a NULL hwnd lParam.

I still don’t know why the dialog’s focus would be lost if the app isn’t minimized, though. I’ve only seen that behavior on Vista, not XP, so it’s not as easy for me to test.

App hwnd = 0x580112
Dialog hwnd = 0x6600F6

Code:

Start App:

WM_GETMINMAXINFO – 0x0, 0x12D790
WM_NCCREATE – 0x0, 0x12D778
WM_NCCALCSIZE – 0x0, 0x12D7B0
WM_CREATE – 0x0, 0x12D764
WM_SETTEXT – 0x0, 0x12DF70
WM_DISPLAYCHANGE – 0x20, 0x3000400
WM_STYLECHANGING – 0xFFFFFFF0, 0x12DEE4
WM_STYLECHANGED – 0xFFFFFFF0, 0x12DEE4
WM_WINDOWPOSCHANGING – 0x0, 0x12DEFC
WM_NCCALCSIZE – 0x1, 0x12DED0
WM_WINDOWPOSCHANGING – 0x0, 0x12DEFC
WM_NCPAINT – 0x1, 0x0
WM_ERASEBKGND – 0xDF0105DC, 0x0
WM_WINDOWPOSCHANGED – 0x0, 0x12DEFC
WM_ACTIVATEAPP – 0x1, 0x34C
WM_NCACTIVATE – 0x1, 0x0
WM_ACTIVATE – 0x1, 0x0
WM_IME_SETCONTEXT – 0x1, 0xC000000F
WM_IME_NOTIFY – 0x2, 0x0
WM_SETFOCUS – 0x0, 0x0
WM_WINDOWPOSCHANGED – 0x0, 0x12DEFC
WM_MOVE – 0x0, 0x0
WM_SIZE – 0x0, 0x3000400
WM_NCCREATE – 0x0, 0x12D74C, 0x290264
WM_NCCALCSIZE – 0x0, 0x12D784, 0x290264
WM_CREATE – 0x0, 0x12D738, 0x290264
WM_SIZE – 0x0, 0xA000A, 0x290264
WM_MOVE – 0x0, 0x0, 0x290264
WM_GETICON – 0x2, 0x0
WM_GETICON – 0x0, 0x0
WM_GETICON – 0x1, 0x0
WM_NCHITTEST – 0x0, 0x1800200
WM_SETCURSOR – 0x580112, 0x2000001
WM_MOUSEMOVE – 0x0, 0x1800200
WM_PAINT – 0x0, 0x0
WM_DESTROY – 0x0, 0x0, 0x290264
WM_NCDESTROY – 0x0, 0x0, 0x290264
WM_NCHITTEST – 0x0, 0x1800200
WM_SETCURSOR – 0x580112, 0x2000001
WM_MOUSEMOVE – 0x0, 0x1800200
WM_NCHITTEST – 0x0, 0x1800200
WM_SETTEXT – 0x0, 0x12EC8C
WM_GETICON – 0x2, 0x0
WM_GETICON – 0x0, 0x0
WM_GETICON – 0x1, 0x0

Click Open File button

WM_LBUTTONDOWN – 0x1, 0x18001F9
WM_LBUTTONUP – 0x0, 0x18001F9
WM_CAPTURECHANGED – 0x0, 0x0
WM_NCHITTEST – 0x0, 0x18001F9
WM_NCHITTEST – 0x0, 0x18001F9
WM_SETCURSOR – 0x580112, 0x2000001
WM_MOUSEMOVE – 0x0, 0x18001F9

GetOpenFileName() called

WM_CANCELMODE – 0x0, 0x0
WM_KILLFOCUS – 0x0, 0x0
WM_IME_SETCONTEXT – 0x0, 0xC000000F
WM_IME_NOTIFY – 0x1, 0x0
WM_ENABLE – 0x0, 0x0
WM_NCACTIVATE – 0x0, 0x6600F6
WM_ACTIVATE – 0x0, 0x6600F6
RestoreDesktopMode()
ShowWindow(SDL_Window, SW_MINIMIZE)
(WinMessage re-entry)
WM_GETTEXT – 0x1FE, 0x12BDC8
(window minimizing animation, though desktop hasn’t repainted yet)
WM_WINDOWPOSCHANGING – 0x0, 0x12CFDC
WM_NCCALCSIZE – 0x1, 0x12CFB0
(desktop repaints)
WM_NCPAINT – 0x1, 0x0
WM_WINDOWPOSCHANGED – 0x0, 0x12CFDC
WM_MOVE – 0x0, 0x83008300
WM_SIZE – 0x1, 0x0
WM_NCACTIVATE – 0x0, 0x0
WM_GETTEXT – 0x1FE, 0x12B8D4
WM_ACTIVATE – 0x200000, 0x0
RestoreDesktopMode()
ShowWindow(SDL_Window, SW_MINIMIZE)
ChangeDisplaySettings()
(screen flash)
WM_DISPLAYCHANGE – 0x20, 0x4000500
SDL_PrivateAppActive()/SDL_ProcessEvents()
WM_ACTIVATEAPP – 0x0, 0x34C
(return from ShowWindow/re-entry)
ChangeDisplaySettings()
SDL_PrivateAppActive() state-no-change short-circuit
WM_WINDOWPOSCHANGING – 0x0, 0x12D304
WM_WINDOWPOSCHANGED – 0x0, 0x12D304
WM_ACTIVATEAPP – 0x1, 0x0
WM_MOUSELEAVE – 0x0, 0x0

…WM_ENTERIDLE – 0x0, 0x6600F6…

Select file and ‘Open’, or ‘Cancel’

WM_ENABLE – 0x1, 0x0
WM_WINDOWPOSCHANGING – 0x0, 0x12D300
WM_WINDOWPOSCHANGED – 0x0, 0x12D300
WM_NCACTIVATE – 0x200001, 0x6600F6
WM_GETTEXT – 0x1FE, 0x12BBF8
WM_ACTIVATE – 0x200001, 0x6600F6
(but active is set to 0 because app is minimized)
SDL_PrivateAppActive() state-no-change short-circuit

Click on taskbar icon to restore app

WM_NCACTIVATE – 0x1, 0x0
WM_GETTEXT – 0x1FE, 0x12E050
WM_WINDOWPOSCHANGING – 0x0, 0x12F758
WM_WINDOWPOSCHANGED – 0x0, 0x12F758
WM_QUERYOPEN – 0x0, 0x0
WM_GETTEXT – 0x1FE, 0x12E544
WM_WINDOWPOSCHANGING – 0x0, 0x12F758
WM_NCCALCSIZE – 0x1, 0x12F72C
WM_NCPAINT – 0x1, 0x0
WM_ERASEBKGND – 0xDF0105DC, 0x0
WM_WINDOWPOSCHANGED – 0x0, 0x12F758
WM_MOVE – 0x0, 0x0
WM_SIZE – 0x0, 0x3000400
WM_IME_SETCONTEXT – 0x1, 0xC000000F
WM_IME_NOTIFY – 0x2, 0x0
WM_SETFOCUS – 0x0, 0x0
WM_ACTIVATE – 0x1, 0x0
SDL_RestoreGameMode()
ShowWindow(SDL_Window, SW_RESTORE)
ChangeDisplaySettings()
WM_DISPLAYCHANGE – 0x20, 0x3000400
WM_SYNCPAINT – 0x4, 0x0
WM_NCPAINT – 0x1, 0x0
WM_ERASEBKGND – 0xDF0105DC, 0x0

Quit App:

WM_LBUTTONDOWN – 0x1, 0x2E103B3
WM_LBUTTONUP – 0x0, 0x2E103B3
WM_CAPTURECHANGED – 0x0, 0x0
WM_NCHITTEST – 0x0, 0x2E103B3
WM_NCHITTEST – 0x0, 0x2E103B3
WM_SETCURSOR – 0x580112, 0x2000001
WM_MOUSEMOVE – 0x0, 0x2E103B3
WM_WINDOWPOSCHANGING – 0x0, 0x12F948
WM_DISPLAYCHANGE – 0x20, 0x4000500
WM_SHOWWINDOW – 0x0, 0x0
WM_WINDOWPOSCHANGING – 0x0, 0x12F998
WM_WINDOWPOSCHANGED – 0x0, 0x12F998
WM_SIZE – 0x0, 0x3000400
WM_MOVE – 0x0, 0x0
WM_NCACTIVATE – 0x0, 0x0
WM_ACTIVATE – 0x0, 0x0
SDL_RestoreDesktopMode()
WM_KILLFOCUS – 0x0, 0x0
WM_IME_SETCONTEXT – 0x0, 0xC000000F
WM_IME_NOTIFY – 0x1, 0x0
WM_WINDOWPOSCHANGING – 0x0, 0x12F5C0
WM_NCCALCSIZE – 0x1, 0x12F594
WM_NCPAINT – 0x1, 0x0
WM_WINDOWPOSCHANGED – 0x0, 0x12F5C0
WM_MOVE – 0x0, 0x83008300
WM_SIZE – 0x1, 0x0
ChangeDisplaySettings()
SDL_PrivateAppActive/SDL_ProcessEvents
WM_ACTIVATEAPP – 0x0, 0xDC0
WM_WINDOWPOSCHANGING – 0x0, 0x12F944
WM_WINDOWPOSCHANGED – 0x0, 0x12F944
WM_DESTROY – 0x0, 0x0
WM_NCDESTROY – 0x0, 0x0

alk,

Did you ever get this to go as planned without any hitches? It seems like there is a little more troubleshooting that never took place in the thread…I’d love to hear more about it if you got it fixed it could be a really great learning tool.

Never got it to work the way I wanted, no. I had to ship something, so I just had the open file dialog minimize the fullscreen app, as ugly as that is. If I didn’t do that, then on other versions of Windows (Vista?) the dialog wouldn’t get focus and would pop up behind the fullscreen app, making it look like the app had hung, which was unacceptable.

I probably spent more time diagnosing this problem than it would have taken to write a custom, in-app file browser, instead of trying to use the Windows API. Oh well. If you want to investigate it any further, be my guest. Let me know if you figure anything out.

It seems like the fix for #2 should definitely be adopted into the SDL, as it is a legitimate bug. I’m not sure what the process is for that, though.

Could you take the time and file a bug at http://bugzilla.libsdl.org/
Please include as much detail as possible: SDL version, repro code,
target platform, compiler settings, screenshots, etc.
–Andreas> Never got it to work the way I wanted, no. I had to ship something, so

I just had the open file dialog minimize the fullscreen app, as ugly
as that is. If I didn’t do that, then on other versions of Windows
(Vista?) the dialog wouldn’t get focus and would pop up behind the
fullscreen app, making it look like the app had hung, which was
unacceptable.

I probably spent more time diagnosing this problem than it would have
taken to write a custom, in-app file browser, instead of trying to use
the Windows API. Oh well. If you want to investigate it any further,
be my guest. Let me know if you figure anything out.

It seems like the fix for #2 should definitely be adopted into the
SDL, as it is a legitimate bug. I’m not sure what the process is for
that, though.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org