Problems with keymapping on AZERTY keyboards on Mac OS X

Hi folks,

some of our Mac OS X users are having problems with ScummVM <http://
www.scummvm.org> running on a machine with an AZERTY keyboard, a
layout that is standard in many french-speaking countries (France and
Belgium for example). The tricky thing about that keyboard layout is
that in it the “number row” differs considerably from that on a
QWERTY keyboard.In particular, the key which “normally” is labeled
"1" instead is labeled “&”, and to get a “1” you have to press “Shift-
&”, i.e. the opposite of a what happens in “normal” keyboard layouts.
To see what I mean, see here: <http://en.wikipedia.org/wiki/AZERTY.

Now, we offer our users certain hotkeys, among them are the Ctrl-
NUMBER and Alt-NUMBER hotkeys (for quick saving/loading). No issue,
you might say now – just use the keysym and be done with it, it
should return “1” for that key on AZERTY, too.

However, this is not the case on Mac OS X, due to some “clever” code
which tries to correct the default hard-coded keymap to accommodate
for non-US keyboards. The idea for adding that code back then (I was
involved in that) was to compensate for changed letter key positions
on non-QWERTY keyboards. For example I am using a QWERTZ keyboard
where y and z are swapped. That code in the Mac OS X variant of SDL
makes sure the keysyms for these two keys are swapped, too.

Why doing so? Because the purpose of keysyms is not completely well-
defined in SDL (and I think Sam is aware of that and maybe 1.3 will
improve in that regard). There are two different roles it tries to
support: One is to represent the symbol that is printed on the key
(which is what the OS X port currently tries to do). The other is to
represent the “position” of the key (so you don’t care whether the
key between “t” and “u” is labeled “y” or “z”, you you always want
SDLK_y, meaning “the key between t and u”).

Both functions have their merits, depending on what you are trying to
do, I guess… But at least for AZERTY keyboards, the fact that
pressing the “& resp. 1” key generates not a SDLK_1 keysym, but
rather SDLK_AMPERSAND. As a result, the above-mentioned quick save/
load hotkeys don’t work for those users. (see also ScummVM bug report
#1643848, “MACOSX: Unable to save/load using numbers on AZERTY
keyboard”, <https://sourceforge.net/tracker/index.php?
func=detail&aid=1643848&group_id=37116&atid=418820>)

My question now is: How should I resolve this “best” ? Several
possibilities come to mind. But I am not sure which to choose, and
maybe I am also missing some possible solutions. Finally, it would be
interesting to know how other SDL ports (e.g. on Windows on Linux)
behave in this regard, to make sure we get uniform behavior across
ports). My ideas so far:

  1. Do not remap the keysyms for the keys in the number row, like “1/
    &”. The attached patch does that. It’s simple, but is it “the right
    thing” to do? Are there maybe keyboard layouts where this produces
    bad results? Is there any code that will suffer from it

  2. Don’t perform any of the “remap to try to accommodate for
    international keyboard” voodoo, i.e. get rid of the “KCHRPtr” code in
    src/video/quartz/SDL_QuartzEvents.m

  3. Call SDL’s behavior “correct” and somehow fix the problem inside
    of ScummVM. Maybe by allowing the user to define a custom remapping,
    defining “quicksave to slot 1” as “Alt-&”

  4. Tell our users to use Ctrl/Alt+KEYPAD – although of course not
    all machines have a numerical keypad :confused:

  5. … ? Your idea here :slight_smile:

Cheers,
Max

-------------- next part --------------
A non-text attachment was scrubbed…
Name: macosx-azerty-fix.patch
Type: application/octet-stream
Size: 706 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20070627/ae79fda6/attachment.obj
-------------- next part --------------

Hello !

However, this is not the case on Mac OS X, due to some “clever” code
which tries to correct the default hard-coded keymap to accommodate
for non-US keyboards. The idea for adding that code back then (I was
involved in that) was to compensate for changed letter key positions
on non-QWERTY keyboards. For example I am using a QWERTZ keyboard
where y and z are swapped. That code in the Mac OS X variant of SDL
makes sure the keysyms for these two keys are swapped, too.

Why doing so? Because the purpose of keysyms is not completely well-
defined in SDL (and I think Sam is aware of that and maybe 1.3 will
improve in that regard). There are two different roles it tries to
support: One is to represent the symbol that is printed on the key
(which is what the OS X port currently tries to do). The other is to
represent the “position” of the key (so you don’t care whether the
key between “t” and “u” is labeled “y” or “z”, you you always want
SDLK_y, meaning “the key between t and u”).

Personally i always thought that the idea of SDLK_xyxy was, that
you get them, when pressing the key with the symbol xyxy
on them. As for most games or apps this is enough.
If you have to choose a wired comb. on your keyboard
to get that key, for example on a for small size designed
keyboards, like laptop ones, what the heck.

For more complex games coders can allow their users to
allow to define keyboard mappings themselves,
so that they can have the keys they want.

It would also be bad naming scheme if SDLK_xyxy is a key
that lies physically on line a and row b on the keyboard.
Then it should be named SDLK_a_b.

CU

Max Horn wrote:

Hi folks,

some of our Mac OS X users are having problems with ScummVM <http://
www.scummvm.org> running on a machine with an AZERTY keyboard, a
layout that is standard in many french-speaking countries (France and
Belgium for example). The tricky thing about that keyboard layout is
that in it the “number row” differs considerably from that on a
QWERTY keyboard.In particular, the key which “normally” is labeled
"1" instead is labeled “&”, and to get a “1” you have to press “Shift-
&”, i.e. the opposite of a what happens in “normal” keyboard layouts.
To see what I mean, see here: <http://en.wikipedia.org/wiki/AZERTY.

Now, we offer our users certain hotkeys, among them are the Ctrl-
NUMBER and Alt-NUMBER hotkeys (for quick saving/loading). No issue,
you might say now – just use the keysym and be done with it, it
should return “1” for that key on AZERTY, too.

However, this is not the case on Mac OS X, due to some “clever” code
which tries to correct the default hard-coded keymap to accommodate
for non-US keyboards. The idea for adding that code back then (I was
involved in that) was to compensate for changed letter key positions
on non-QWERTY keyboards. For example I am using a QWERTZ keyboard
where y and z are swapped. That code in the Mac OS X variant of SDL
makes sure the keysyms for these two keys are swapped, too.

Why doing so? Because the purpose of keysyms is not completely well-
defined in SDL (and I think Sam is aware of that and maybe 1.3 will
improve in that regard). There are two different roles it tries to
support: One is to represent the symbol that is printed on the key
(which is what the OS X port currently tries to do). The other is to
represent the “position” of the key (so you don’t care whether the
key between “t” and “u” is labeled “y” or “z”, you you always want
SDLK_y, meaning “the key between t and u”).

In my opinion, that would be ugly and would make it very hard for developers
using non Qwerty keyboards.

  1. Call SDL’s behavior “correct” and somehow fix the problem inside
    of ScummVM. Maybe by allowing the user to define a custom remapping,
    defining “quicksave to slot 1” as “Alt-&”

I’d say this is the correct solution. This is how UT 2004 handles it for the
quick chat menu (on windows at least, on Unix systems it fails miserably to
the point you HAVE to configure your keyboard layout in Qwerty to even have
a chance of using it correctly). Most commercial games allow you to
redefine the keyboard layout, and also chose the default values depending
on the installed language.

I’d say this is the correct solution. This is how UT 2004 handles it for the
quick chat menu (on windows at least, on Unix systems it fails miserably to
the point you HAVE to configure your keyboard layout in Qwerty to even have
a chance of using it correctly). Most commercial games allow you to
redefine the keyboard layout, and also chose the default values depending
on the installed language.

UT2004’s keyboard input is pretty broken in general. I wouldn’t show it
as an example of anything. I wrote it, but I’m not proud. :slight_smile:

SDL 1.3 is taking steps to split text input from key presses. Ultimately
the 1.2 API is going to always run into problems like this.

–ryan.

Max Horn wrote:

[…]

  1. Call SDL’s behavior “correct” and somehow fix the problem inside
    of ScummVM. Maybe by allowing the user to define a custom remapping,
    defining “quicksave to slot 1” as “Alt-&”

I’d say this is the correct solution. This is how UT 2004 handles
it for the
quick chat menu (on windows at least, on Unix systems it fails
miserably to
the point you HAVE to configure your keyboard layout in Qwerty to
even have
a chance of using it correctly). Most commercial games allow you to
redefine the keyboard layout, and also chose the default values
depending
on the installed language.

That’s not possible for us, there are hundreds of keyboard layout out
there, and SDL does not provide any way to detect the keyboard
layout – so we would have to add platform specific code for each
system we run on (several dozen), multiplied by the number of
keyboard layouts (which differ between systems – a Mac/French AZERTY
is different from PC/French AZERTY, and of course the Belgium AZERTY
differs from both agaiin).

I.e. we simply don’t have the manpower, nor the technical
possibilities, to do this. Hence the drawback of this solution (the
drawback I really really don’t like about it :/) is that we can’t run
"out of the box" for all our users but rather have to ask them to
reconfigure their keyboard layouts before playing (we still have to
add that “keyboard remapping” GUI code, too, but that’s planned
anyway, for other reasons).

Right now I am tempted to simply use my own hacked SDL version for
building OS X binaries, where I changed the number key to always
return the numeric keycodes. It might not be a solution fit for
everybody, but might be the best for our purposes anyway (in addition
to the keymapping), as the hotkeys will just work “as expected” ;).

Bye,
MaxAm 27.06.2007 um 20:28 schrieb Christophe Cavalaria:

SDL 1.3 formalizes the meaning of the keysyms: SDL_blah is the key with
the text “blah” printed on it based on the current keyboard layout.
Pressing that key may or may not give you a text input event with “blah”,
depending on modifiers, composition state, etc.

This allows you to say “Press the blah key to fire” and it’s actually
the blah key, not where the blah key would be on a US QWERTY keyboard. :slight_smile:

On the other hand, World of Warcraft makes a special exception for the
French keyboard and does not remap the number keys, and SDL will probably
have the same hack, basically to address your problem.

Most games do provide localized keyboard layout, or alternate layout
schemes to account for some uncommon layouts.

I’m open to suggestions, but I’ve been mulling this over for a few years,
and this is what I’ve come up with so far.

Darn users and their gaming expectations. :wink:

-Sam Lantinga, Lead Software Engineer, Blizzard Entertainment

Max Horn wrote:

… the purpose of keysyms is not completely well-defined in SDL …

I’m sure you have seen my numerous posts on this subject, but in case
you haven’t, here they are (some mentioned again below):

http://thread.gmane.org/gmane.comp.lib.sdl/20846/focus=20860
http://thread.gmane.org/gmane.comp.lib.sdl/24697/focus=24697
http://thread.gmane.org/gmane.comp.lib.sdl/25706/focus=25747
http://thread.gmane.org/gmane.comp.lib.sdl/26516/focus=26607
http://thread.gmane.org/gmane.comp.lib.sdl/27300/focus=27327
http://thread.gmane.org/gmane.comp.lib.sdl/29599/focus=29599

Finally, it would be interesting to know how other SDL ports (e.g. on
Windows on Linux) behave in this regard, to make sure we get uniform
behavior across ports)

Last time I checked
(http://thread.gmane.org/gmane.comp.lib.sdl/24697), Quartz, X11, and
windib implemented mapping-by-label, while windx5 implemented
mapping-by-position. Though that was a year and a half ago and things
may have changed in the meantime.

As far as I understand, Sam’s opinion is that all backends should
implement mapping-by-label, which would mean that DirectX is (or was)
wrong. My opinion is that mapping-by-position, done right, is more
useful, and I’ve proposed a prototype implementation for Quartz and X11
at http://thread.gmane.org/gmane.comp.lib.sdl/29599. I haven’t done
any more work on this since, in particular not integrating it into SDL
1.3 or providing a Windows implementation.

  1. Do not remap the keysyms for the keys in the number row, like
    "1/&". The attached patch does that. It’s simple, but is it “the
    right thing” to do?

Doesn’t feel right to me. Adding even more inconsistency than there is
already?

  1. Don’t perform any of the “remap to try to accommodate for
    international keyboard” voodoo, i.e. get rid of the “KCHRPtr” code in
    src/video/quartz/SDL_QuartzEvents.m

That would be my preferred solution (i.e. changing the system to
mapping-by-position), but as you can see I haven’t gotten very far with
this idea up to now… :slight_smile:

  1. Call SDL’s behavior “correct” and somehow fix the problem inside
    of ScummVM. Maybe by allowing the user to define a custom remapping,
    defining “quicksave to slot 1” as “Alt-&”

Sounds like the best immediate workaround to me.

-Christian

Christian Walther wrote:

Max Horn wrote:

  1. Don’t perform any of the “remap to try to accommodate for
    international keyboard” voodoo, i.e. get rid of the “KCHRPtr” code in
    src/video/quartz/SDL_QuartzEvents.m

That would be my preferred solution (i.e. changing the system to
mapping-by-position), but as you can see I haven’t gotten very far with
this idea up to now… :slight_smile:

Game: Press the M key
User presses the M key a few times, nothing happens
Game: Oh wait! Press the M key has it would be placed on a US QWERTY
keyboar, 1978 version
Users throws the computer out the window while cursing loudly

Additional funny fact, some keybords move around their keys. I’m sure US
QWERTY users would be rather interested by the very useful key one can find
near the left shift, on the other hand, the rest of the world is intrigued
by the small key one can find between the Enter key and the Backspace
key :slight_smile:

One last thing, on a lot of keyboard mappings, the right alt key has been
recycled to be AltGr allowing a few more symbols above the standard Not
shifted/shifted pattern. It might be a good idea if pressing that key
wouldn’t be considered as pressing the Alt modifier.

Christophe Cavalaria wrote:

Game: Press the M key
User presses the M key a few times, nothing happens
Game: Oh wait! Press the M key has it would be placed on a US QWERTY
keyboar, 1978 version
Users throws the computer out the window while cursing loudly

The other approach isn’t pretty either.

Game: Press the M key.
User presses the key that, in conjunction with AltGr, would produce the
’m’ character.
Game: No, not the key. The M key.
Users smashes the computer against the wall, cursing loudly.

I think the only real solution is to give the program access to the
keyboard layout. In particular:

  • The program needs to know how many keys there are, where they are,
    and how they are labeled.
  • Incoming key events should identify themselves by both label and
    position.
  • The program needs to be able to convert between key label codes and
    key position codes based on the current keyboard layout.
  • Ideally, each keyboard layout should have a unique and unambigiuous
    name that the program can use to make key mappimg decisions on a higher
    level.–
    Rainer Deyke - rainerd at eldwood.com

Christophe Cavalaria wrote:

Christian Walther wrote:

That would be my preferred solution (i.e. changing the system to
mapping-by-position), but as you can see I haven’t gotten very far with
this idea up to now… :slight_smile:

Game: Press the M key
User presses the M key a few times, nothing happens
Game: Oh wait! Press the M key has it would be placed on a US QWERTY
keyboar, 1978 version
Users throws the computer out the window while cursing loudly

That’s not at all what I’m advocating. Read
http://thread.gmane.org/gmane.comp.lib.sdl/29599, and do try the
example program attached to
http://bugzilla.libsdl.org/show_bug.cgi?id=319 to see what I mean.

-Christian

Rainer Deyke wrote:

Christophe Cavalaria wrote:

Game: Press the M key
User presses the M key a few times, nothing happens
Game: Oh wait! Press the M key has it would be placed on a US QWERTY
keyboar, 1978 version
Users throws the computer out the window while cursing loudly

The other approach isn’t pretty either.

Game: Press the M key.
User presses the key that, in conjunction with AltGr, would produce the
’m’ character.
Game: No, not the key. The M key.
Users smashes the computer against the wall, cursing loudly.

This reminds me of an old CPC game which would ask the user to press the M
key to pass the title screen to the main menu. It was a neat trick they
used to detect if the user was on a Qwerty keyboard or on an Azerty
keyboard :slight_smile: At the time, IIRC the CPC only had those two major layouts.

I think the only real solution is to give the program access to the
keyboard layout. In particular:

  • The program needs to know how many keys there are, where they are,
    and how they are labeled.

How many keys might not be possible/practical. Asking SDL if a specific key
exists could be interesting.

  • Incoming key events should identify themselves by both label and
    position.
  • The program needs to be able to convert between key label codes and
    key position codes based on the current keyboard layout.
  • Ideally, each keyboard layout should have a unique and unambigiuous
    name that the program can use to make key mappimg decisions on a higher
    level.

That one goes a little too far in my opinion. Are you suggesting we should
hardcode in SDL a list of all possible and existing keyboard layouts? I
don’t think such feat is possible in itself.

I don’t think it’s possible to work with a keyboard by “position” without
having some broken cases. Of course, most keys are in a standard position
but compare US Qwerty with English Qwerty for example. Same keys exactly
except for the physical location of the \ key. I’d bet there’s a good
chance the computer doesn’t even know about the different locations.

I’d say that the lesser evil in that whole mess would be :

  • Document the list of SDL keysyms that developers should expect to find on
    nearly every keyboard and those which can be missing. As it stands now,
    only SDLK_A-Z, space, enter etc… are available.
  • maybe rename the SDLK_… defines for keys that cannot be found reliably
    to SDLK_EX_…
  • On broken keyboards where you need Shift to type numbers, SDL will name
    those keys by the corresponding number anyway, like it is done in WoW.
    After all, most users will have difficulty finding the Ampersand key on a
    french keyboard whereas if you ask them for the 1 key, they’ll go right for
    it. This will help improve the list of SDL keysyms you can find nearly
    everywhere

And as a bonus, for those that make heavy use of SDL ports of Quake engines,
make it so that the console key has for keysym SDLK_CONSOLE. Opening the
console on ioQuake3 is rather difficult on a french keyboard :slight_smile:

Den Wed, 27 Jun 2007 16:15:42 -0700
skrev Sam Lantinga :

SDL 1.3 formalizes the meaning of the keysyms: SDL_blah is the key
with the text “blah” printed on it based on the current keyboard
layout. Pressing that key may or may not give you a text input event
with “blah”, depending on modifiers, composition state, etc.

Is there any way to get positional key information in SDL 1.3? This is
something I’ve been missing in SDL 1.2. Positional input can be a
nightmare without this, with either having to ship a gazillion
keymaps and having the user select one, or making people define
their own keymaps. (While the ability to redefine input in a game is
a good thing, making its use required is not so good, specially if
there’s many keys).

This allows you to say “Press the blah key to fire” and it’s actually
the blah key, not where the blah key would be on a US QWERTY
keyboard. :slight_smile:

Perhaps I’m misunderstanding here, but I think this is false (or, at
least, not always true). Take the SDL_EQUALS keysym, for example. On an
US qwerty keyboard, ‘=’ is available with one single keypress, but on eg
mine, I have to press shift+0 to get it. If a game tells me to press
’=’ to fire, what should I do? Press shift+0, or just 0? SDLK_0 is a
different keysym?

I think a better way to do this kind of thing would be to get the label
of a key through some function:

printf("Press the %s key to fire", SDL_GetKeyLabel(SDL_blah));

I can see arguments for both positional and labeled keysyms. In an
RPG, having the ‘I’ key bring up the inventory makes sense (at least
in english), but in an FPS, the labels of the qwerty WASD keys doesn’t
really matter, it’s the (relative) position of the keys that’s
important. Ideally there should be a way to do both of these.

Most games do provide localized keyboard layout, or alternate layout
schemes to account for some uncommon layouts.

This is difficult when you have to redefine most of the keyboard, eg to
get original native layout in an emulator, or in a game that uses
most of the keyboard (like most flight simulators seems to do).

  • Gerry