SDL 1.3 keyboard input on X11

I have a little time right now and I’m trying to get X keyboard input
working on SDL 1.3. But, I am having a bit of a problem. The new SDL 1.3
way of doing keyboard input is to always pass an SDLK_* that represents
a real, physical, key on the keyboard. Personally I like that a lot
because it lets be be sure that if I get the SDLK_W key it is the same
physical key no matter what language is on the key caps.

The trouble is that X11 keyboard input is designed to hide the physical
keycodes. Oh, it gives them to you, but they are not documented, they
can be different on every implementation of X11 and even different for
different kinds of keyboards attached to the same implementation. The
idea is that an application running on X can be running on any kind of a
device and you shouldn’t have to worry about the kind of keyboard you
are using. X11 hides all that and just gives the application a "virtual"
keyboard that uses standard “keysyms”.

This was a great idea back in the early 1980s when everyone was building
their own unique keyboards. Now days, it kind of sucks… Not to
mention that it makes translating from X11 keycodes to SDLK_* a very
dicey proposition.

Now that I have explained the problem I am going to ask for some help. I
have been digging around in the xfree86 and X.org source code and it
looks like they both use the same keycodes on all versions of Linux, and
maybe on all versions of the server. I do not know for sure if this is
true. If it is, then we can handle a large number of systems very
simply.

So, if you know more than I do about what these servers are doing, then
please let me know and tell me where to look in the source and docs.

OTOH, if you are using X11 with a non US-English keyboard, please run
xmodmap -pk and send the out to me at @Bob_Pendleton

Thank You,

	Bob Pendleton-- 

±-------------------------------------+

Bob,

I have worked on X11 keyboard support some months ago. The problem about
the physical key codes is solved in my code (using X-server-specific
look-up tables), but I hit another road block later on, and since I
haven’t gotten a reply from Sam when I asked him for advice, that code
is still sitting on my hard drive, and I kinda forgot about it.

Rather than trying to recollect the problem from memory, here’s what I
wrote to Sam at the time (2007-10-03):

Status report: I’m somewhat stuck with the X11 keyboard
implementation. What I have now mostly works with the western
keyboard layouts I have tested, but most likely fails miserably as
soon as input methods are involved. I don’t consider this ready to be
committed. Specifically, GetLayoutKey() turned out to be less
straightforward to implement than I had hoped. What I need there, for
character keys, is basically a static mapping from X11 KeySyms to
one-character (unicode) key names (static because KeySyms correspond
1:1 to our concept of a “layout key code”). But I have been unable to
find an elegant way of doing this. Xutf8LookupString(), which I’m
using now, is inappropriate because it takes into account dead keys
and input methods (and probably messes up the XIC that I’m currently
borrowing from the first window). XLookupString() works, but only for
Latin-1 characters. XKeysymToString() produces ASCII strings, not
single characters. The only correct way I can imagine is to have a
giant look-up table containing all thousands of KeySyms. I’m hoping
some X11 expert (which I am not) has a better idea here… or perhaps
the API we designed is just unimplementable and needs to be revised.

Other unfinished or questionable points (might make more sense once
you see the code):

  • I haven’t implemented generation of text events because I don’t
    understand what’s going on in the commented-out SDL 1.2 code with
    saved_keysym.
  • I’ve resurrected the X11_KeyRepeat() function from SDL 1.2. Was
    there a particular reason why it was removed?
  • Due to what I wrote in
    http://article.gmane.org/gmane.comp.lib.sdl/34610, I can’t use
    SDL_iconv() for UTF-8/SDLKey conversions (it’s probably a bit
    heavy-weight anyway). Instead I’ve taken fragments of the integrated
    SDL_iconv() implementation and put them into the functions
    encodeUtf8() in SDL_keyboard.c and decodeUtf8() in SDL_x11keyboard.c.
    This should probably be centralized somewhere.

So, I’m a bit unsure about how to proceed. Should I send you a patch
for you personally to have a look at? Should I put it into bugzilla
and ask for help on the mailing list? Should we commit it nonetheless
and hope for someone to fix it sooner or later?

I’ll probably put a patch into Bugzilla just to get my code out there,
so you could have a look at it. I hope I’ll have time for that tonight.

-Christian

Bob,

I have worked on X11 keyboard support some months ago. The problem about
the physical key codes is solved in my code (using X-server-specific
look-up tables), but I hit another road block later on, and since I
haven’t gotten a reply from Sam when I asked him for advice, that code
is still sitting on my hard drive, and I kinda forgot about it.

Please send me those tables and your code. Please do not check it in.
That could mess me up and make it hard to get your code.

I haven’t looked at the higher level functions, but I have a pretty good
(table based) conversion from X11 keycodes to SDLK_*. The problem with
it is that it likely only works with X.org and Linux, and maybe only the
latest versions of both.

The following is more for everyone else than for you:

The latest Linux kernels convert keyboard scan codes to a fixed set of
keycodes. The result is that the physical location of the key is
preserved and nothing needs to care about the type of the keyboard.
Every application, including the X server, get the same kernel generated
keycodes. The X server then translates those keycodes to its own version
of keycodes and passes them along to X applications. There is not a 1:1
relationship between X server keycodes and kernel keycodes.

To confuse things even more X wants you to convert the keycodes to
"keysyms". Keysyms are X specific and date back to the 1980s before
anyone ever heard of Unicode. The key to generating SDLKey values from
keycodes is to ignore the X keysyms and use the keycodes. The trouble
with that is that I only know what the keycodes are for the 2.5 kernel
and for the X.org server shipped with Ubuntu 7.10.

The mod maps I asked for give me a real good idea of what X keycodes get
mapped to which keysyms so I can valid the table I am building. :slight_smile:

Personally, I believe the higher level SDL key functions should/must be
implemented strictly in terms of SDLKey values. How you do that, I have
no clue at this time. We may need to change the design.

	Bob PendletonOn Wed, 2007-12-19 at 07:27 +0100, Christian Walther wrote:

Rather than trying to recollect the problem from memory, here’s what I
wrote to Sam at the time (2007-10-03):

Status report: I’m somewhat stuck with the X11 keyboard
implementation. What I have now mostly works with the western
keyboard layouts I have tested, but most likely fails miserably as
soon as input methods are involved. I don’t consider this ready to be
committed. Specifically, GetLayoutKey() turned out to be less
straightforward to implement than I had hoped. What I need there, for
character keys, is basically a static mapping from X11 KeySyms to
one-character (unicode) key names (static because KeySyms correspond
1:1 to our concept of a “layout key code”). But I have been unable to
find an elegant way of doing this. Xutf8LookupString(), which I’m
using now, is inappropriate because it takes into account dead keys
and input methods (and probably messes up the XIC that I’m currently
borrowing from the first window). XLookupString() works, but only for
Latin-1 characters. XKeysymToString() produces ASCII strings, not
single characters. The only correct way I can imagine is to have a
giant look-up table containing all thousands of KeySyms. I’m hoping
some X11 expert (which I am not) has a better idea here… or perhaps
the API we designed is just unimplementable and needs to be revised.

Other unfinished or questionable points (might make more sense once
you see the code):

  • I haven’t implemented generation of text events because I don’t
    understand what’s going on in the commented-out SDL 1.2 code with
    saved_keysym.
  • I’ve resurrected the X11_KeyRepeat() function from SDL 1.2. Was
    there a particular reason why it was removed?
  • Due to what I wrote in
    http://article.gmane.org/gmane.comp.lib.sdl/34610, I can’t use
    SDL_iconv() for UTF-8/SDLKey conversions (it’s probably a bit
    heavy-weight anyway). Instead I’ve taken fragments of the integrated
    SDL_iconv() implementation and put them into the functions
    encodeUtf8() in SDL_keyboard.c and decodeUtf8() in SDL_x11keyboard.c.
    This should probably be centralized somewhere.

So, I’m a bit unsure about how to proceed. Should I send you a patch
for you personally to have a look at? Should I put it into bugzilla
and ask for help on the mailing list? Should we commit it nonetheless
and hope for someone to fix it sooner or later?

I’ll probably put a patch into Bugzilla just to get my code out there,
so you could have a look at it. I hope I’ll have time for that tonight.

-Christian


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


±-------------------------------------+

Personally, I believe the higher level SDL key functions should/must be
implemented strictly in terms of SDLKey values. How you do that, I have
no clue at this time. We may need to change the design.

Thank you guys for looking at this. It’s one of the big stumbling blocks
in 1.3, and I hadn’t had time to think it through really well.

See ya,
-Sam Lantinga, Lead Software Engineer, Blizzard Entertainment

Personally, I believe the higher level SDL key functions should/must be
implemented strictly in terms of SDLKey values. How you do that, I have
no clue at this time. We may need to change the design.

Thank you guys for looking at this. It’s one of the big stumbling blocks
in 1.3, and I hadn’t had time to think it through really well.

You are welcome. This is fun for me, I haven’t taken an in depth look at
this sort of thing since the '80s.

Bob PendletonOn Thu, 2007-12-20 at 11:39 -0800, Sam Lantinga wrote:

See ya,
-Sam Lantinga, Lead Software Engineer, Blizzard Entertainment


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


±-------------------------------------+

Bob Pendleton wrote:

Please send me those tables and your code.

It’s taken a day longer than I had hoped, but here it is:
http://bugzilla.libsdl.org/show_bug.cgi?id=523

I hope we can find a useful synthesis between this and your work.

-Christian