WinRT support for SDL_WarpMouse* possible?

Hi,

I have successfully built SDL’s WinRT port :slight_smile: My only pitfall was that I
was using another drive (not C:) where the app will successfully compile but
never run :frowning:

Currently my app won’t work properly, because it expects a working “SDL_
WarpMouseInWindow”. The README states that it is currently not supported. Is
it possible to support it with the current Windows SDKs for Win10 at all, or
do I have to look for alternatives?

Regards,

Daniel

Looking through the UWP API docs on https://dev.windows.com, I don’t see anything.

Looking through the UWP header files (in the public, 10.0.10240.0, SDK), I see an undocumented API that seems like it may work: an ICoreWindow2 interface with a method to set the ‘PointerPosition’ property. Previously, and as documented, that property is read-only. I haven’t tried it, though.

So, maybe?

– David L.

Hi David,

maybe I was too fast thinking about this. Looks like my problem is more
generic. As soon as I run the app locally, my mouse disappears. It looks
like it is “frozen” at the last position, so I can click the last point, but
moving the mouse doesn’t do anything.

When I switch to a different program, the cursor reappears. Now when I
position it somewhere else, and then switch back to the app, this will be
the new position of the cursor - but the cursor itself is disabled again and
I can only click, not move.

Same code works when running as “desktop app” (cursor shown and can be
moved). What am I doing wrong?

Regards,

Daniel

---------- P?vodn? zpr?va ----------
Od: DLudwig
Komu: sdl at lists.libsdl.org
Datum: 4. 5. 2016 20:51:29
P?edm?t: Re: [SDL] WinRT support for SDL_WarpMouse* possible?

"

Looking through the UWP API docs on https://dev.windows.com,
(https://dev.windows.com,) I don’t see anything.

Looking through the UWP header files (in the public, 10.0.10240.0, SDK), I
see an undocumented API that seems like it may work: an ICoreWindow2
interface with a method to set the ‘PointerPosition’ property. Previously,
and as documented, that property is read-only. I haven’t tried it, though.

So, maybe?

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"

hardcoredaniel wrote:

maybe I was too fast thinking about this. Looks like my problem is more generic. As soon as I run the app locally, my mouse disappears. It looks like it is “frozen” at the last position, so I can click the last point, but moving the mouse doesn’t do anything.

When I switch to a different program, the cursor reappears. Now when I position it somewhere else, and then switch back to the app, this will be the new position of the cursor - but the cursor itself is disabled again and I can only click, not move.

Same code works when running as “desktop app” (cursor shown and can be moved). What am I doing wrong?

I’m not sure what’s up, off the top of my head. This sounds like it’s probably a bug in SDL. I have a few questions about the app you’re running:

  1. when you run it “locally”, are you running it on the local machine, but via the Windows Simulator?

  2. is the app configuring SDL to use non-default mouse state(s), such as a hidden cursor, relative mouse mode, and/or something else?

  3. is the app attempting to call SDL_WarpMouse* functions()?

  4. what version of WinRT are you building your app, and LibSDL, against? I.e. Windows 10/UWP? Windows 8.1? Windows 8.0? other?

  5. any chance you have sample code and/or project file(s) for this, that I could look at and try running? If not, I can try to make some on my end, using answer(s) to the above questions.

– David L.

"

                      hardcoredaniel wrote:	 

maybe I was too fast thinking about this. Looks like my problem is more
generic. As soon as I run the app locally, my mouse disappears. It looks
like it is “frozen” at the last position, so I can click the last point, but
moving the mouse doesn’t do anything.

When I switch to a different program, the cursor reappears. Now when I
position it somewhere else, and then switch back to the app, this will be
the new position of the cursor - but the cursor itself is disabled again and
I can only click, not move.

Same code works when running as “desktop app” (cursor shown and can be
moved). What am I doing wrong?

I’m not sure what’s up, off the top of my head. This sounds like it’s
probably a bug in SDL. I have a few questions about the app you’re running:

  1. when you run it “locally”, are you running it on the local machine, but
    via the Windows Simulator?

"

Running it on the local machine and the simulator shows the issue per
default. Once I change the UI from “mouse mode” to “basic touch mode” in the
simulator, I can operate my app.

?
"

  1. is the app configuring SDL to use non-default mouse state(s), such as a
    hidden cursor, relative mouse mode, and/or something else?

"

I disabled everything that looks suspicious, but no change.

?
"

  1. is the app attempting to call SDL_WarpMouse* functions()?

"

Yes, I disabled it as well, no difference.

?
"

  1. what version of WinRT are you building your app, and LibSDL, against? I.
    e. Windows 10/UWP? Windows 8.1? Windows 8.0? other?

"

I picked “Universal Windows” for everything.

?
"

  1. any chance you have sample code and/or project file(s) for this, that I
    could look at and try running? If not, I can try to make some on my end,
    using answer(s) to the above questions.

"

It’s a lot of code unfortunately. The main app is here:

with the UWP project in “vs2015_uwp\wmelite_sdl_mixer”. I am building
against recent SDL mercurial, and some dependencies from here:

You will also need a sample game image, please d/l from here:

https://bitbucket.org/hcdaniel/wmelite-rapaki-edition/downloads/data.dcp

and place it into the “Assets” folder. Currently it is not picked up by the
build (so please place it manually into the “AppX/Assets” folder), I still
need to figue that one out.

I hope to have committed all relevant files so you can easily build.

Many thanks for your help!

?
"

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"

I know what the problem is: SDL’s WinRT code for hiding a mouse is, unintentionally, causing Windows to stop emitting mouse motion events (that SDL uses to update its own state). Specifically, it sets the WinRT CoreWindow’s PointerCursor to nullptr. The workaround appears to be to use a Win32-style cursor-resource, one with blank content, then use that. Unfortunately, UWP APIs don’t allow the Win32 cursor-creation functions to be called, not if apps want to pass Microsoft’s Windows Store certification, and UWP doesn’t appear to provide analogues. Including those resource in the app does appear to be possible though.

I’ll see if I can write up a patch within the next week or so. It might require that apps directly link in a Win32 .res file though. :frowning:

– David L.

Hi David,

If I understand you correctly, the issue is related to the combination of
having an UWP app with mouse control, but hiding the Windows mouse cursor.
So if I decide to keep the Windows cursor shown, it would not appear? Can I
do that with the current SDL code base or do I need to modify sth? This is
just for understanding and experimenting, finally I’d like to use your
proposed solution anyway.

Another thing that I am wondering about: How could I “query” the platform
capabilities to find out whether I am on a pointer-only system, a touch-only
system, or whether I’d have both available? Some laptops might have touch
sensitive displays, and tablets can have a mouse connected. An app could
adapt its UI appropriately based on that info. I have to admit that I have
not checked the API, so forgive me if I asked a stupid question.

Thank you for your continuous effort for the WinRT/UWP port of SDL!

Regards,

Daniel

---------- P?vodn? zpr?va ----------
Od: DLudwig
Komu: sdl at lists.libsdl.org
Datum: 8. 5. 2016 20:49:23
P?edm?t: Re: [SDL] WinRT support for SDL_WarpMouse* possible?

"

I know what the problem is: SDL’s WinRT code for hiding a mouse is,
unintentionally, causing Windows to stop emitting mouse motion events (that
SDL uses to update its own state). Specifically, it sets the WinRT
CoreWindow’s PointerCursor to nullptr. The workaround appears to be to use a
Win32-style cursor-resource, one with blank content, then use that.
Unfortunately, UWP APIs don’t allow the Win32 cursor-creation functions to
be called, not if apps want to pass Microsoft’s Windows Store certification,
and UWP doesn’t appear to provide analogues. Including those resource in the
app does appear to be possible though.

I’ll see if I can write up a patch within the next week or so. It might
require that apps directly link in a Win32 .res file though. Sad

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"

hardcoredaniel wrote:

If I understand you correctly, the issue is related to the combination of having an UWP app with mouse control, but hiding the Windows mouse cursor. So if I decide to keep the Windows cursor shown, it would not appear? Can I do that with the current SDL code base or do I need to modify sth? This is just for understanding and experimenting, finally I’d like to use your proposed solution anyway.

I saw two, possible solutions to this bug:

  1. don’t hide the system’s cursor. Once I did this, the in-game cursor started showing up, as Windows started sending SDL mouse-motion events. (Oddly enough though, it’d still send mouse-down and mouse-up events.)

  2. work-around Windows’ bug/API-quirk within SDL. I’m planning on taking a stab at this, this upcoming weekend. I got something working on my own system, which required no changes to the game+engine’s code, minus the addition of a .res (Win32 data resource) file to the app/engine’s MSVC project file, along with some changes to SDL’s code.

hardcoredaniel wrote:

Another thing that I am wondering about: How could I “query” the platform capabilities to find out whether I am on a pointer-only system, a touch-only system, or whether I’d have both available? Some laptops might have touch sensitive displays, and tablets can have a mouse connected. An app could adapt its UI appropriately based on that info. I have to admit that I have not checked the API, so forgive me if I asked a stupid question.

There’s no API in SDL to do this, as of yet, but UWP does have APIs to list the available ‘pointer’ devices, with ‘pointer’ here being anything that a user could point to stuff with: mice, touch-screens, pen-input, and the like. Do note that lots of devices nowadays do feature multiple input types.

The UWP API to detect this is documented at https://msdn.microsoft.com/en-us/library/windows/apps/windows.devices.input.pointerdevice.aspx .

A quick C++ sample to get started with is below. A more extensive sample is also available, at https://code.msdn.microsoft.com/windowsapps/Input-device-capabilities-31b67745

Code:

// in a .cpp file compiled with MSVC’s /ZW flag, which enables WinRT-specific C++ extensions

#include <SDL_log.h>
#include <Windows.h>

extern “C” void LogPointerDevices()
{
using namespace Windows::Devices::Input;
auto allDevices = PointerDevice::GetPointerDevices();
SDL_Log(“num devices: %u”, allDevices->Size);
for (unsigned int i = 0; i < allDevices->Size; ++i) {
PointerDevice ^ dev = allDevices->GetAt(i);
const char * devTypeName = “unknown”;
switch (dev->PointerDeviceType) {
case PointerDeviceType::mouse: devTypeName = “Mouse”; break;
case PointerDeviceType::pen: devTypeName = “Pen”; break;
case PointerDeviceType::Touch: devTypeName = “Touch”; break;
}
SDL_Log(“device %u: %s, %s”, i, devTypeName);
}
}

hardcoredaniel wrote:

Thank you for your continuous effort for the WinRT/UWP port of SDL!

No problem! I hope this helps. :slight_smile:

– David L.

DLudwig wrote:

quick C++ sample to get started with is below.

Doh, my code sample had a bug in it, left from me trying some things out with the code, modifying it a bit, then not testing the change before posting it. Sorry about that.

Here’s a fixed, tested sample:

Code:

// in a .cpp file compiled with MSVC’s /ZW flag, which enables WinRT-specific C++ extensions

#include <SDL_log.h>
#include <Windows.h>

extern “C” void LogPointerDevices()
{
using namespace Windows::Devices::Input;
auto allDevices = PointerDevice::GetPointerDevices();
SDL_Log(“num devices: %u”, allDevices->Size);
for (unsigned int i = 0; i < allDevices->Size; ++i) {
PointerDevice ^ dev = allDevices->GetAt(i);
const char * devTypeName = “unknown”;
switch (dev->PointerDeviceType) {
case PointerDeviceType::mouse: devTypeName = “Mouse”; break;
case PointerDeviceType::pen: devTypeName = “Pen”; break;
case PointerDeviceType::Touch: devTypeName = “Touch”; break;
}
SDL_Log(“device %u: %s”, i, devTypeName);
}
}

Ok, my hack-fix/workaround for this bug, which seems like it might be a bug in Windows itself, is now in SDL, via https://hg.libsdl.org/SDL/rev/5b61e12c0a30 .

To get this working, you’ll need to:

  1. sync to the latest SDL sources
  2. include two extra files in your app’s MSVC project. These files have been added to SDL’s source distribution, in src/main/winrt/. Both begin with the name, “SDL-WinRTResource”.

As far as I can tell, this ‘fixes’ the bug you posted, and gets the app’s custom cursor to display.

– David L.

Thank you very much for your effort! I can confirm that these fix the issue,
both for mouse and touch input :slight_smile:

I also checked your example code for detecting pointers on WinRT. I was
surprized that on my standard Windows laptop (without touch screen) a
"touch" device was detected, but with the help of the MouseCapabilities and
TouchCapabilities classes I can query the “*Present” fields to find out
what’s really available.

Thanks also for this example code!

---------- P?vodn? zpr?va ----------
Od: DLudwig
Komu: sdl at lists.libsdl.org
Datum: 15. 5. 2016 5:34:11
P?edm?t: Re: [SDL] WinRT support for SDL_WarpMouse* possible?

"

Ok, my hack-fix/workaround for this bug, which seems like it might be a bug
in Windows itself, is now in SDL, via https://hg.libsdl.org/SDL/rev/5b61e12c
0a30(https://hg.libsdl.org/SDL/rev/5b61e12c0a30) .

To get this working, you’ll need to:

  1. sync to the latest SDL sources
  2. include two extra files in your app’s MSVC project. These files have been
    added to SDL’s source distribution, in src/main/winrt/. Both begin with the
    name, “SDL-WinRTResource”.

As far as I can tell, this ‘fixes’ the bug you posted, and gets the app’s
custom cursor to display.

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"

Hi David,

w.r.t. your trick to use a custom resource for the “invisible” cursor, I
found one occasion where it does not work as expected - but to me it looks
like this is caused by Windows itself. I would just like your opinion on
this.

When I run the app, I can use “Windows+G” to display a menu with some
options (screenshots and stuff), which will unconditionally display Windows’
mouse pointer. Unfortunately it won’t vanish when this menu disappears,
until I move the mouse up so that the window decoration shows, or I do a
task switch.

Is this something that can be prevented, or is this behaviour more or less
"given"?

Regards,

Daniel

---------- P?vodn? zpr?va ----------
Od: DLudwig
Komu: sdl at lists.libsdl.org
Datum: 15. 5. 2016 5:34:11
P?edm?t: Re: [SDL] WinRT support for SDL_WarpMouse* possible?

"

Ok, my hack-fix/workaround for this bug, which seems like it might be a bug
in Windows itself, is now in SDL, via https://hg.libsdl.org/SDL/rev/5b61e12c
0a30(https://hg.libsdl.org/SDL/rev/5b61e12c0a30) .

To get this working, you’ll need to:

  1. sync to the latest SDL sources
  2. include two extra files in your app’s MSVC project. These files have been
    added to SDL’s source distribution, in src/main/winrt/. Both begin with the
    name, “SDL-WinRTResource”.

As far as I can tell, this ‘fixes’ the bug you posted, and gets the app’s
custom cursor to display.

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"

hardcoredaniel wrote:

Hi David,

w.r.t. your trick to use a custom resource for the “invisible” cursor, I found one occasion where it does not work as expected - but to me it looks like this is caused by Windows itself. I would just like your opinion on this.

When I run the app, I can use “Windows+G” to display a menu with some options (screenshots and stuff), which will unconditionally display Windows’ mouse pointer. Unfortunately it won’t vanish when this menu disappears, until I move the mouse up so that the window decoration shows, or I do a task switch.

Is this something that can be prevented, or is this behaviour more or less “given”?

I just tried this on my development machine, and was able to reproduce it. I’d be inclined to argue that this is a bug in Windows itself, but that a work-around might be possible.

I’m able to re-hide the mouse via calls to SDL_SetCursor(SDL_GetCursor()). After a bit of digging, I found an API to detect when the “GameBar” redirects input (Windows::Gaming::UI::GameBar::IsInputRedirectedChanged). I wouldn’t be surprised if a work-around is possible, but don’t have time right now to confirm this. I’ll put it on my TODO list, but no guarantees as to when I’ll get to it.

– David L.

Thanks for mentioning a workaround! I do not see it as a very important
thing right now, and maybe it will even fix itself with a Windows update :slight_smile:
But in case it stays and becomes annoying, I know where to look.

Regards,

Daniel

---------- P?vodn? zpr?va ----------
Od: DLudwig
Komu: sdl at lists.libsdl.org
Datum: 13. 6. 2016 1:00:38
P?edm?t: Re: [SDL] WinRT support for SDL_WarpMouse* possible?

"

                      hardcoredaniel wrote:	 

Hi David,

w.r.t. your trick to use a custom resource for the “invisible” cursor, I
found one occasion where it does not work as expected - but to me it looks
like this is caused by Windows itself. I would just like your opinion on
this.

When I run the app, I can use “Windows+G” to display a menu with some
options (screenshots and stuff), which will unconditionally display Windows’
mouse pointer. Unfortunately it won’t vanish when this menu disappears,
until I move the mouse up so that the window decoration shows, or I do a
task switch.

Is this something that can be prevented, or is this behaviour more or less
"given"?

I just tried this on my development machine, and was able to reproduce it.
I’d be inclined to argue that this is a bug in Windows itself, but that a
work-around might be possible.

I’m able to re-hide the mouse via calls to SDL_SetCursor(SDL_GetCursor()).
After a bit of digging, I found an API to detect when the "GameBar"
redirects input (Windows::Gaming::UI::GameBar::IsInputRedirectedChanged). I
wouldn’t be surprised if a work-around is possible, but don’t have time
right now to confirm this. I’ll put it on my TODO list, but no guarantees as
to when I’ll get to it.

– David L._______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org"