Hi,
Hmm, that’s a really interesting idea! I’ve never considered it. Accessibility on the web is a hot topic these days … perhaps due to the fact that HTML5 & friends present the capability to do so in a clean fashion.
I don’t know anything about a Windows implementation, but on OS X (at a glance), appear to have the necessary API hooks to do what you want in a clean fashion – https://developer.apple.com/library/mac/documentation/Accessibility/Conceptual/AccessibilityMacOSX/OSXAXModel/OSXAXmodel.html#//apple_ref/doc/uid/TP40001078-CH208-TPXREF101 (see The Accessibility Model, “An example of Accessibility” header sections) and https://developer.apple.com/library/mac/documentation/Accessibility/Conceptual/AccessibilityMacOSX/ImplementingAccessibilityforCustomControls/ImplementingAccessibilityforCustomControls.html#//apple_ref/doc/uid/TP40001078-CH256-SW1 (see “Controls Without Views”)
I’m not sure how well it could be implemented within SDL, but I think with some prototyping of the accessibility APIs with your use case could easily determine that. On OS X, this presumably means hooks for some portion of the “Informational Properties” and “Notifications” APIs. What about other platforms, such as Linux? (No idea there, what kind of accessibility standards exist?)
Maybe after I finish up my animation API, I could take a look at it… I’ve been wanting to learn Objective C, after all! 
Cheers,
Jeffrey Carpenter <@Jeffrey_Carpenter>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1572 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141211/7bce13d7/attachment.bin
2014-12-11 3:53 GMT-03:00, Marcus von Appen :
As I wrote, it used ATK (AT-SPI with CORBA) in the passed. The AT-SPI
bridges for accessibility tools were updated heavily (the implementation
originated from GNOME/GTK, KDE/QT however did not want to pull in Glib)
and I did not follow the latest changed since 2009.
Yeah, I was talking about the current status though. I recall reading
somewhere that they use dbus now (so confirming what you had guessed
before) but I don’t remember exactly where I saw this.
On, Wed Dec 10, 2014, Sik the hedgehog wrote:
I was trying to implement screen reader support in my game and reached
the conclusion that it’s a total mess, so I was wondering if it would
be possible to get this implemented as a feature in SDL itself
instead. Basically, just let the program set the string output to
accessibility tools by calling a function (think of how
SDL_SetWindowTitle works for the titlebar).
I was discussing with somebody and on Windows it seems you’d use
WM_GETOBJECT to tell Windows what the control is (you’d include this
text here). Not sure how you tell it when the text changed, but
anyway. No idea how to do it in other platforms, but I’d like to at
least start getting it implemented in some.
Can we get this feature inside SDL? (also what function name? maybe
SDL_SetAccessibilityText or something like that?)
I did that years ago for arbitrary application objects in Python for a
GUI toolkit based on pygame (SDL as back-end) on Win32 and Unix
platforms. Not sure about Win7 and Win8 these days, but I think, the
MSAA layer is still the same, which means that it should be pretty
straight forward for Windows and OS X. The accessibility status for
Unix/Linux may have changed, since there were several attempts to
modernize the whole stuff in the past (moving away from CORBA to DBUS).
You can find the MSAA implementation at
http://sourceforge.net/p/ocemp/svn/HEAD/tree/trunk/papi/src/msaa/
and the ATK-based version at
http://sourceforge.net/p/ocemp/svn/HEAD/tree/trunk/papi/src/atk/
I’ll gladly help anyone, who takes up that challenge, with extensive
explanations about both implementations.
Cheers
Marcus_______________________________________________
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
-------------- next part --------------
A non-text attachment was scrubbed…
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141211/f8796a93/attachment-0001.pgp
I think that on Linux the GUI libraries have some way to communicate
with screen readers and such, but I have no information about this
(but yeah, I think it’s something making use of dbus, and that’s about
as much as I know).
On, Thu Dec 11, 2014, Sik the hedgehog wrote:
I think that on Linux the GUI libraries have some way to communicate
with screen readers and such, but I have no information about this
(but yeah, I think it’s something making use of dbus, and that’s about
as much as I know).
As I wrote, it used ATK (AT-SPI with CORBA) in the passed. The AT-SPI
bridges for accessibility tools were updated heavily (the implementation
originated from GNOME/GTK, KDE/QT however did not want to pull in Glib)
and I did not follow the latest changed since 2009.
Cheers
Marcus
-------------- next part --------------
A non-text attachment was scrubbed…
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141211/673edf4d/attachment-0001.pgp
So, any new info on this?
I did not hear anything from anyone.
Cheers
Marcus> On 15.12.2014, at 00:00, Sik the hedgehog <sik.the.hedgehog at gmail.com> wrote:
So, any new info on this?
Sik the hedgehog,
Do you happen to have any prototype code for getting a screen reader functioning properly under Windows? If you are serious about the endeavor, it’s imaginably not so much a matter of if we could get this kind of feature implemented in SDL so much as it is a matter of somebody doing it.
It looks like it’ll be no less than ~2…4w before I feel comfortable putting what I’m working on now aside to take a serious look at prototyping a OS X screen reader using the official APIs available on the platform. It might be nice for me to have some reference code (that’s you) to go by to get a feel for things.
Cheers,
Jeffrey Carpenter <@Jeffrey_Carpenter>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1572 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141215/11756ba8/attachment-0001.bin
Hi again,
Mmm, I doubt that the code by itself would do me much good at this point. Maybe once I take a look at the Windows API for this stuff, I’ll understand things better and perhaps want to see the code of Sol.
The concept of a text buffer could indeed be a very useful application 
P.S. I casually browsed through Qt’s Accessibility API at http://qt-project.org/doc/qt-4.8/accessible.html and found it interesting that it appears – at a high level, at least – to be similar perhaps to the OS X API model. (I like looking at API documentation of other implementations to help get an idea at naming things, general structuring, etc.)
Cheers,
Jeffrey Carpenter <@Jeffrey_Carpenter>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1572 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141215/6664d065/attachment-0001.bin
2014-12-15 7:54 GMT-03:00, Jeffrey Carpenter :
Do you happen to have any prototype code for getting a screen reader
functioning properly under Windows? If you are serious about the endeavor,
it’s imaginably not so much a matter of if we could get this kind of
feature implemented in SDL so much as it is a matter of somebody doing it.
I have some code in Sol for screen reader support although it’s not
exactly any of the best methods I’ve discussed (one involves talking
to the speech synthesis engine, one involves using the clipboard, one
involves using the titlebar, one involves outputting to the standard
output). I could share them here if people want, but take that into
account, as none of them talk to the screen reader directly, which is
what I’m having problem with.
I suppose the whole concept of textbuffer (analogous to framebuffer)
used by it could be useful.
Looking around, on iOS apparently we have this:
That’s for iOS 7 onwards, though it seems there are answers there for
older versions too. I’ll let you decide what version is best to have
as minimum for this feature.
On Android we’ll have to use the speech API it seems.
2014-12-15 10:07 GMT-03:00, Jeffrey Carpenter :
The concept of a text buffer could indeed be a very useful application
The idea is simple: the same way a framebuffer holds the image to show
on screen, a textbuffer holds a string to show on the screen reader.
Basically:
- When the textbuffer is altered, the text is sent to the screen reader
- When the textbuffer becomes empty, the screen reader goes mute
- When the user presses some key to repeat the text, the text in the
textbuffer gets sent to the screen reader again
Does this make any sense? What I’m looking for here is to make a
function that writes to the textbuffer essentially (there would be one
for each window).
P.S. I casually browsed through Qt’s Accessibility API at
http://qt-project.org/doc/qt-4.8/accessible.html and found it interesting
that it appears – at a high level, at least – to be similar perhaps to the
OS X API model. (I like looking at API documentation of other
implementations to help get an idea at naming things, general structuring,
etc.)
Trying to get my head around it, though I have the feeling most of
that won’t be revelant ??? (we’re talking about enhancing just a
normal window after all, no more complex controls to deal with) What
is the name and the description of a control?
There’s also this if somebody wants to look into it:
http://hg.q-continuum.net/accessible_output2/
It’s a multiplatform library that handles screen readers. The code is
MIT licensed, so we probably shouldn’t worry much. The biggest problem
is that it’s written in Python so we can’t use it directly But to
get an idea of how things work this seems like it could help.
Hi,
Ah, the implementation would be beautifully elegant if we could manage to get it to work sensibly like so. It feels like an event queue sort of thing to me – that may because I’ve been dealing with animation queues, but anyhow – it definitely makes sense. Let me start thinking about that…
Screw the Qt Accessibility API, what you describe vaporizes all of that (so we would hope :-P).
Cheers,
Jeffrey Carpenter <@Jeffrey_Carpenter>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1572 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141216/b26a78d6/attachment-0001.bin
2014-12-16 7:42 GMT-03:00, Jeffrey Carpenter :
Screw the Qt Accessibility API, what you describe vaporizes all of that (so
we would hope :-P).
Well, it’s still useful to know what to look for to implement this
But yeah it’s like SAPI, it includes lots of functions and took me a
while to get my head around it… and in the end I only really needed
the Speak function. So much for complexity.
We could provide a SAPI backend on Windows later (especially since
it’s actually pretty easy even from pure C and I already have working
code), but on Windows I’d like to see if we could implement it using
the proper UI features so it works with screen readers directly.
2014-12-16 7:42 GMT-03:00, Jeffrey Carpenter :
Screw the Qt Accessibility API, what you describe vaporizes all of that (so
we would hope :-P).
Well, it’s still useful to know what to look for to implement this
But yeah it’s like SAPI, it includes lots of functions and took me a
while to get my head around it… and in the end I only really needed
the Speak function. So much for complexity.
We could provide a SAPI backend on Windows later (especially since
it’s actually pretty easy even from pure C and I already have working
code), but on Windows I’d like to see if we could implement it using
the proper UI features so it works with screen readers directly.
I sent some links around on that. For win32 it is pretty simple to write a generic wrapper, and somewhat complex for Unix.
Cheers
Marcus> On 16.12.2014, at 13:33, Sik the hedgehog <sik.the.hedgehog at gmail.com> wrote:
2014-12-16 9:51 GMT-03:00, Jeffrey Carpenter :
It’s too early to say for sure, but I’m getting the impression that, for the
textbuffer concept to work, one might essentially have to fake an UI element
and treat that as the buffer to make requests on behalf of, if that makes
sense … which shrug I’d be totally OK with if it is feasible without
much trouble … at least for an initial stab at it.
We shouldn’t need to fake an UI element, the window itself should be
enough (and if not, we could just make a custom control). The catch is
that SDL is the one who controls the window, so it’s SDL who needs to
handle this as well
(I’m aware that SDL can be made to use a window created elsewhere
which it can’t control… I think we should just resort to a different
backend in that case)
And here is Speech Dispatcher (used in Linux), in its own message for
the sake of organization. Again, same deal as with the SAPI example
code.
What we need:------------------------------------------------------------
#include <libspeechd.h>
static SPDConnection *speechd_connection = NULL;
To initialize:
speechd_connection = spd_open("",
NULL, NULL, SPD_MODE_THREADED);
if (speechd_connection == NULL) {
// Error…
}
spd_set_data_mode(speechd_connection, SPD_DATA_TEXT);
To say something:
spd_stop_all(speechd_connection);
spd_cancel_all(speechd_connection);
if (*str != ‘\0’) {
spd_say(speechd_connection, SPD_TEXT, str);
}
To deinitialize:
if (speechd_connection != NULL) {
spd_close(speechd_connection);
speechd_connection = NULL;
}
Oh, and if somebody gets confused: SAPI and Speech Dispatcher talk to
the speech synthesis engines directly, not to the screen readers, so
ideally they should be seen only as back-up backends and not as the
proper solution. Having them around is a good idea anyway.
We should probably also have a hint to select the backend much like
the case for the renderer API.
Sorry for multiple posting (for those in the forum). Here’s some quick
code for SAPI to get you the idea of how it would work (if we
implement that backend on Windows). Of course this would need to be
properly adapted for use in SDL and such (especially where sapi_voice
is stored).
Stuff we need:------------------------------------------------------------
#include <windows.h>
#include <sapi.h>
ISpVoice *sapi_voice = NULL;
(those headers are what’s needed on the most recent MinGW-w64 but I
think Visual Studio requires different headers, you may want to check)
To initialize:
if (FAILED(CoInitialize(NULL))) {
// Error…
}
if (FAILED(CoCreateInstance(&CLSID_SpVoice, NULL, CLSCTX_ALL,
&IID_ISpVoice, (void **) &sapi_voice))) {
// Error…
}
To say something (replace utf8_to_utf16 with whatever is relevant):
wchar_t *wstr = utf8_to_utf16((const char *) str);
sapi_voice->lpVtbl->Speak(sapi_voice, wstr,
SPF_PURGEBEFORESPEAK | SPF_IS_NOT_XML, NULL);
free(wstr);
To deinitialize:
if (sapi_voice != NULL) {
sapi_voice->lpVtbl->Speak(sapi_voice, L"",
SPF_IS_NOT_XML | SPF_PURGEBEFORESPEAK, NULL);
sapi_voice->lpVtbl->WaitUntilDone(sapi_voice, INFINITE);
sapi_voice->lpVtbl->Release(sapi_voice);
sapi_voice = NULL;
}
CoUninitialize();
Don’t know anything about SAPI 
Speech Application Programming Interface? Sounds neat. Reminds me of the say command in OS X – shell command for using speech vocals built-in with the environment (part of the accessibility stuff).
I do agree that it would be nice to interface with screen readers directly … I haven’t played with it yet, but I’ve found some sample code for implementing the OS X API for a “simple, accessible tic tac toe game” at https://developer.apple.com/wwdc/resources/sample-code/
It’s too early to say for sure, but I’m getting the impression that, for the textbuffer concept to work, one might essentially have to fake an UI element and treat that as the buffer to make requests on behalf of, if that makes sense … which shrug I’d be totally OK with if it is feasible without much trouble … at least for an initial stab at it.
Cheers,
Jeffrey Carpenter <@Jeffrey_Carpenter>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1572 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20141216/eecb8abe/attachment-0001.bin