What are threads allowed to do?

SDL Development
1

Looking here:
http://sdldoc.csn.ul.ie/thread.php

I find these bits of advice:

  1. Don’t use any library functions in separate threads

  2. Lock global variables which may be accessed by multiple threads

Cool. Taken literally, this means that a thread may
only burn CPU time. It can not access global variables,
because locking them would require a library function. The
thread can not exit; that would require a library function…

What’s the real story?

I had been planning to pre-fetch images and fonts in threads.
I guess I’ll just use fork(), and let the Windows users live
without decent performance as usual.

2

Looking here:
http://sdldoc.csn.ul.ie/thread.php

I find these bits of advice:

  1. Don’t use any library functions in separate threads

  2. Lock global variables which may be accessed by multiple threads

Cool. Taken literally, this means that a thread may
only burn CPU time. It can not access global variables,
because locking them would require a library function. The
thread can not exit; that would require a library function…

What’s the real story?

The thread related functions required for synchronization are of course
thread-safe. Those are guidelines, not strict rules.

-bobOn Jan 20, 2005, at 19:08, Albert Cahalan wrote:

3

I figured as much, but…

Mere guidelines alone don’t make for solid software. Am I to assume
that all other SDL functions are unsafe? The same for C library stuff
on the various supported platforms? If this is the case, then threads
are quite useless.

I tried using threads, hit memory corruption problems, and…
well, what am I to do? I can not even determine if my own code
is correct if thread-safety of various library functions is
undocumented. Really, it is completely unsafe to use threads
without documentation. I went with the optimistic approach, but
that obviously isn’t right. Pessimism suggests that all functions
are unsafe, and thus SDL threads are pointless. If reality is
somewhere in-between but undocumented, I’ll have to go with the
pessimistic assumption.On Thu, 2005-01-20 at 19:37, Bob Ippolito wrote:

On Jan 20, 2005, at 19:08, Albert Cahalan wrote:

Looking here:
http://sdldoc.csn.ul.ie/thread.php

I find these bits of advice:

  1. Don’t use any library functions in separate threads

  2. Lock global variables which may be accessed by multiple threads

Cool. Taken literally, this means that a thread may
only burn CPU time. It can not access global variables,
because locking them would require a library function. The
thread can not exit; that would require a library function…

What’s the real story?

The thread related functions required for synchronization are of course
thread-safe. Those are guidelines, not strict rules.

4

Mere guidelines alone don’t make for solid software. Am I to assume
that all other SDL functions are unsafe? The same for C library stuff
on the various supported platforms? If this is the case, then threads
are quite useless.

I suppose there has to be a better list of guidelines, then.

I’ve been doing cross-platform multithreading in SDL for quite some time
now, and about the only snag I hit is memory allocation across multiple
threads (and I’ve only experienced that problem in Windows). I do use
standard C library functions across threads, so that makes me a
rule-breaker. :wink:

I may be wrong, but I think the only reason for the “don’t call library
functions in separate threads” is because some of those library
functions change an implicit “state” in SDL’s own space, such that a
program may enter and exit that state only through the same thread. For
example, you shouldn’t call SDL_LoadWAV and SDL_FreeWAV in separate
threads because these functions implicitly do memory management
themselves, but theoretically you should be able to call both functions
on the same wave data array in the same thread. (I haven’t actually
tried breaking that rule, though – I ended up coding pessimistically
for this case and so I’ve never tested allocating memory using a thread
other than the main one.)

I tried using threads, hit memory corruption problems, and…
well, what am I to do? I can not even determine if my own code
is correct if thread-safety of various library functions is
undocumented. Really, it is completely unsafe to use threads
without documentation. I went with the optimistic approach, but

I don’t really know if documenting every single SDL function whether it
is thread-safe is actually feasible. As a corollary, the Java language
inherently supports multithreading, but it doesn’t have documentation
for every single function (and there are many of those in Java!) telling
us whether it is thread-safe either. But somehow that works well
enough. :slight_smile:

I would think that using the thread-locking mechanisms should already be
a safe bet, and instead there should be clear documentation whether a
library function behaves in a state-dependent manner ala
SDL_LoadWAV/SDL_FreeWAV.

Offtopic: I noticed that the Windows version of SDL uses the Win32
mutex objects. From what I’ve read, those are only useful for
inter-application synchronization, and critical sections should be used
instead to save some resources. I’ve recompiled SDL using critical
sections and things are indeed a bit snappier – I’m actually getting,
like, one more frame per second in my app. :slight_smile: I haven’t really
profiled my code yet, but it seems that each thread was able to do just
a little bit more in the same timeslice it gets from the processor,
(perhaps because it doesn’t have to compare any mutex “names”? Not that
it matters, SDL already passes NULL for the mutex name…). Is there an
argument for leaving the SDL implementation at mutexes? I don’t think
it would really break most applications except for those which had some
sort of internal behavior relying on an extremely “perfect” sync (which
would probably break on a different processor anyway). Oh, and I can
send in a quick patch if you do want the critical sections instead. :)–
Eric Vidal
Lecturer / Graduate Research Assistant, DISCS
Ateneo de Manila University
http://aegis.ateneo.net/evidal/

5

I may be wrong, but I think the only reason for the “don’t call library
functions in separate threads” is because some of those library
functions change an implicit “state” in SDL’s own space, such that a
program may enter and exit that state only through the same thread.

I think that there is another problem, too, which is a design problem,
in my opinion. I’ve faced it when I was working on the OS/2 port of SDL.

The problem does not exist when SDL is statically linked to the executable,
because in that case both SDL and the application uses the very same
C runtime library. In that case, it’s safe to use runtime library functions from
other threads.

However, once the application uses SDL dynamically linked (in a form
of a DLL), the following problem arises:

The SDL DLL uses its own C runtime library, let’s say, the RTL of OpenWatcom
(if it was compiled with OpenWatcom).
When an executable, which is compiled with a different compiler (e.g. GCC),
calls into the DLL to create a new thread (SDL_CreateThread), that new thread
will be created by the RTL of OpenWatcom, and the RTL of the executable
will not be initialized for that new thread.
So, it’s unpredictable what will happen if the application calls its own runtime
library functions from that thread (well, I had crashes).

This problem might not exist on other platforms, because
Windows and Linux usually uses common runtime libraries, so even though
SDL is in a separate DLL/shared object, the RTL calls will go to the same
runtime library at the very end.

DoodleOn Fri, 21 Jan 2005 16:50:19 +0800, Eric Vidal wrote:

6

Albert Cahalan wrote:

Looking here:
http://sdldoc.csn.ul.ie/thread.php

I find these bits of advice:

  1. Don’t use any library functions in separate threads

  2. Lock global variables which may be accessed by multiple threads

Cool. Taken literally, this means that a thread may
only burn CPU time. It can not access global variables,
because locking them would require a library function. The
thread can not exit; that would require a library function…

What’s the real story?

The thread related functions required for synchronization are of course
thread-safe. Those are guidelines, not strict rules.

I figured as much, but…

Mere guidelines alone don’t make for solid software. Am I to assume
that all other SDL functions are unsafe? The same for C library stuff
on the various supported platforms? If this is the case, then threads
are quite useless.

That’s the bottom line, if you don’t understand threads, they do more
harm than good.

I tried using threads, hit memory corruption problems, and…

Well, show us a small program demonstrating memory corruption with SDL
threads, and we’ll have a chance to find the reason for this problem.

Stephane>On Thu, 2005-01-20 at 19:37, Bob Ippolito wrote:

On Jan 20, 2005, at 19:08, Albert Cahalan wrote:

7

Albert Cahalan wrote:

Mere guidelines alone don’t make for solid software. Am I to assume
that all other SDL functions are unsafe? The same for C library stuff
on the various supported platforms? If this is the case, then threads
are quite useless.

That’s the bottom line, if you don’t understand threads, they do more
harm than good.

So I “don’t understand threads” if I haven’t carefully examined
all of the SDL source code for thread-safety problems, along with
careful examination of documentation for Windows, BeOS, MacOS,
Linux, FreeBSD, Solaris…?

By a more common definition, I think I understand threads quite
well. I will admit to understanding them more from the viewpoint
of a kernel hacker and /bin/ps author, but that shouldn’t be way
off from the user-centric viewpoint. I know spinlocks aren’t so
favored in userspace, and I know there is an appalling lack of
support for the memory barriers and atomic operations needed for
lockless code.

I tried using threads, hit memory corruption problems, and…

Well, show us a small program demonstrating memory corruption with SDL
threads, and we’ll have a chance to find the reason for this problem.

That’s a tough order to fill I think.

I saw the problem in Tux Paint, which is one 15469-line file.
(yes, yes, we know – it grew over time)

One thread was using libSDL_ttf. It scanned through about 200
font files. The thread would open a file, render some test text,
and close the file.

Meanwhile, the main app thread was mostly loading PNG images.
It would load a few hundred perhaps, along with a couple fonts
and perhaps dozens of *.wav files. The main thread has video
and sound going.On Sat, 2005-01-22 at 11:21, Stephane Marchesin wrote:

8

Albert Cahalan wrote:

I tried using threads, hit memory corruption problems, and…

Well, show us a small program demonstrating memory corruption with SDL
threads, and we’ll have a chance to find the reason for this problem.

That’s a tough order to fill I think.

I saw the problem in Tux Paint, which is one 15469-line file.
(yes, yes, we know – it grew over time)

We usually ask this because it works.
That said, if you happen to have a test case demonstrating a precise
problem, don’t hesitate.

One thread was using libSDL_ttf. It scanned through about 200
font files. The thread would open a file, render some test text,
and close the file.

Meanwhile, the main app thread was mostly loading PNG images.
It would load a few hundred perhaps, along with a couple fonts
and perhaps dozens of *.wav files. The main thread has video
and sound going.

I quickly looked at the file and I think you’ll really have a hard time
getting it to work, there are too many global variables, whereas I
couldn’t spot any mutex. Threaded programming must be thought about at
the beggining…

I use SDL threads for high performance duties (parallel visualization,
we’re talking about 10s of threads here) and I never had any problems
with these.
So I definitely think threads (be it SDL or posix theads) are neither
useless nor buggy.

Stephane

9

What, the tuxpaint.c file? The font thread was supposed to do
a very isolated task. The globals in use by the font thread
would not be touched by the main thread while the font thread
was running, except for reading one “volatile long” that the
font thread sets to indicate completion. The main thread would
do other stuff for a while, then display a progress bar while
waiting for font thread completion. When the font thread is done,
the main thread joins with it before using the global data.

That all worked pretty well, except for memory corruption.
Since the threads don’t race on shared globals, there should
not be any need for a mutex. (where would you put it?)

Well, there is a known memory corruption bug in libSDL_ttf
involving italic fonts. It normally doesn’t cause trouble,
but maybe the altered memory layout of the threaded solution
(different stack location, etc.) made the problem worse.
Perhaps Tux Paint even has a similar problem, though it
tolerates linking with Electric Fence (a malloc bounds checker)
just fine.

It’s looking now like I will use fork(), with a socket to
send back the list of font files. Windows is without fork(),
so the Windows users will just have to wait 20 seconds for
Tux Paint to start. (nothing unusual for them I think)On Sun, 2005-01-23 at 12:05, Stephane Marchesin wrote:

Albert Cahalan wrote:

I tried using threads, hit memory corruption problems, and…

Well, show us a small program demonstrating memory corruption with SDL
threads, and we’ll have a chance to find the reason for this problem.

That’s a tough order to fill I think.

I saw the problem in Tux Paint, which is one 15469-line file.
(yes, yes, we know – it grew over time)

We usually ask this because it works.
That said, if you happen to have a test case demonstrating a precise
problem, don’t hesitate.

One thread was using libSDL_ttf. It scanned through about 200
font files. The thread would open a file, render some test text,
and close the file.

Meanwhile, the main app thread was mostly loading PNG images.
It would load a few hundred perhaps, along with a couple fonts
and perhaps dozens of *.wav files. The main thread has video
and sound going.

I quickly looked at the file and I think you’ll really have a hard time
getting it to work, there are too many global variables, whereas I
couldn’t spot any mutex. Threaded programming must be thought about at
the beggining…

10

Albert Cahalan wrote:

Well, there is a known memory corruption bug in libSDL_ttf
involving italic fonts. It normally doesn’t cause trouble,
but maybe the altered memory layout of the threaded solution
(different stack location, etc.) made the problem worse.
Perhaps Tux Paint even has a similar problem, though it
tolerates linking with Electric Fence (a malloc bounds checker)
just fine.

have you ever tried with valgrind? has already safed my live once or
twice. :slight_smile:

clemens

11

Great idea! Highly recommended.

I’ve only used Valgrind for a year or so, and it’s already saved me
countless hours of boring, frustrating bug hunting. Most of the time
it leads you right at the bugs, and when it doesn’t, it usually says
something where you’d normally have a sleeping bug waiting to bite
you in the *ss way down the road.

That, all warnings enabled in the compiler, and a habit of writing
code in ways that tend to generate bugs that are easy to trap with
these tools, can increase productivity and lower frustration levels a
great deal. :slight_smile:

//David Olofson - Programmer, Composer, Open Source Advocate

.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,… |
`-----------------------------------> http://audiality.org -’
— http://olofson.net — http://www.reologica.se —On Sunday 23 January 2005 20.22, Clemens Kirchgatterer wrote:

Albert Cahalan wrote:

Well, there is a known memory corruption bug in libSDL_ttf
involving italic fonts. It normally doesn’t cause trouble,
but maybe the altered memory layout of the threaded solution
(different stack location, etc.) made the problem worse.
Perhaps Tux Paint even has a similar problem, though it
tolerates linking with Electric Fence (a malloc bounds checker)
just fine.

have you ever tried with valgrind? has already safed my live once
or twice. :slight_smile:

12

Albert Cahalan wrote:

I tried using threads, hit memory corruption problems, and…

Well, show us a small program demonstrating memory corruption with SDL
threads, and we’ll have a chance to find the reason for this problem.

That’s a tough order to fill I think.

I saw the problem in Tux Paint, which is one 15469-line file.
(yes, yes, we know – it grew over time)

We usually ask this because it works.
That said, if you happen to have a test case demonstrating a precise
problem, don’t hesitate.

One thread was using libSDL_ttf. It scanned through about 200
font files. The thread would open a file, render some test text,
and close the file.

Meanwhile, the main app thread was mostly loading PNG images.
It would load a few hundred perhaps, along with a couple fonts
and perhaps dozens of *.wav files. The main thread has video
and sound going.

I quickly looked at the file and I think you’ll really have a hard time
getting it to work, there are too many global variables, whereas I
couldn’t spot any mutex. Threaded programming must be thought about at
the beggining…

What, the tuxpaint.c file? The font thread was supposed to do
a very isolated task. The globals in use by the font thread
would not be touched by the main thread while the font thread
was running, except for reading one “volatile long” that the
font thread sets to indicate completion. The main thread would
do other stuff for a while, then display a progress bar while
waiting for font thread completion. When the font thread is done,
the main thread joins with it before using the global data.

That all worked pretty well, except for memory corruption.
Since the threads don’t race on shared globals, there should
not be any need for a mutex. (where would you put it?)

Well, there is a known memory corruption bug in libSDL_ttf
involving italic fonts. It normally doesn’t cause trouble,
but maybe the altered memory layout of the threaded solution
(different stack location, etc.) made the problem worse.
Perhaps Tux Paint even has a similar problem, though it
tolerates linking with Electric Fence (a malloc bounds checker)
just fine.

It’s looking now like I will use fork(), with a socket to
send back the list of font files. Windows is without fork(),
so the Windows users will just have to wait 20 seconds for
Tux Paint to start. (nothing unusual for them I think)

Taking a shot in the dark here… Since no one can know what you know we
have to take shots in the dark.

On linux, the C/C++ standard libraries may or may not be thread safe, it
all depends on the version you use. If you are on 3.0 or better then the
standard libraries are thread safe (at least that is what the docs I
have found say and it seems to work). For Windows help ask someone else.
OTOH, SDL is not thread safe. All graphics must be done in the main
thread and all keyboard, mouse, and joystick input must be done in the
main thread. You can do anything you want in other threads.

But, like I said, SDL is not thread safe, neither are any of the SDL add
on libraries. If you want to use them in threads you have to build
wrappers for the code you want to use. The wrappers must enforce serial
access to each function call. The libraries have there own global
variables and are coded assuming that only a single thread will run in
them at a time.

What does that mean to an SDL programmer? It means that you can have 10
threads reading 10 files, but only one thread can be decoding a ttf font
at a time.

Now, the thing to consider is that unless you have a hyperthreading or
multicore CPU you can’t actually have more than one thread active at a
time anyway. And, having multiple threads reading from a single disk may
or may not improve disk reading performance. After all, there is only
one disk to perform the reads. So, this month you might not see the
performance improvement you expect to see. Next month you might see a
dramatic performance improvement :slight_smile:

For an example of a thread safe wrapper library used with SDL take a
look at Net2 and Fast Events at
http://gameprogrammer.com/programming.html I had to solve this problem
to do multithreaded network I/O handling in SDL.

Just and editorial comment or two if you don’t mind: Thread programming
in user space is very different from thread programming in system space.
The main difference is that in system space everything has been coded
assuming multiple threads because of interrupt processing. In use space
almost no code is written assuming multiple threads. Most libraries will
turn there toes up and die if you try to use them in a multithreaded
application without writing a serializing wrapper library.

One last editorial comment: A funny thing happened a couple or three
years ago. Processors aren’t getting a lot faster any more. Starting in
about 2001 the rate at which processors get faster dropped dramatically.
OTOH, Moore’s law has been cranking right a long giveing us more and
more transistors on a chip. The processor manufactures have been forced
to redirect the benefits of Moore’s law into things like hyperthreading
and multicore processors rather than into faster processors. Unless
there is a breakthrough that allows CPUs to start getting faster we are
facing a world where all performance increases are going to come from
multithreading on parallel processors and not from faster processors.

We’re going to need thread safe versions of all the libraries we use.

Bob PendletonOn Sun, 2005-01-23 at 12:38 -0500, Albert Cahalan wrote:

On Sun, 2005-01-23 at 12:05, Stephane Marchesin wrote:

SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

13

Taking a shot in the dark here… Since no one can know what you know we
have to take shots in the dark.

Yeah, I know. Thank you for trying.

On linux, the C/C++ standard libraries may or may not be thread safe, it
all depends on the version you use. If you are on 3.0 or better then the
standard libraries are thread safe (at least that is what the docs I
have found say and it seems to work).

This is plain C, not C++. I have various gcc versions from 2.95 to 3.4,
with glibc 2.3.2, and old-style Linux threads.

OTOH, SDL is not thread safe. All graphics must be done in the main
thread

“All graphics”, hmmm? Does that mean just video, or allocating
and freeing software surfaces too? I had also been hoping to
load and scale some images in a thread dedicated to clip-art.

But, like I said, SDL is not thread safe, neither are any of the SDL add
on libraries. If you want to use them in threads you have to build
wrappers for the code you want to use.

I think somebody else just said the opposite. :frowning:

What does that mean to an SDL programmer? It means that you can have 10
threads reading 10 files, but only one thread can be decoding a ttf font
at a time.

This is awful. I looked over libSDL_ttf and didn’t find anything
significant. What did I miss? I saw:

a. error text might get corrupted (but no crash)
b. can’t do simultaneous TTF_Init() calls (no problem)

Regarding #a, note that libSDL_ttf does not pass '%'
in error strings. So the unsafe vararg stuff in SDL’s
error handling would not be involved.

Now, the thing to consider is that unless you have a hyperthreading or
multicore CPU you can’t actually have more than one thread active at a
time anyway. And, having multiple threads reading from a single disk may
or may not improve disk reading performance. After all, there is only
one disk to perform the reads. So, this month you might not see the
performance improvement you expect to see. Next month you might see a
dramatic performance improvement :slight_smile:

Raw performance is a tiny bit better. I suppose this is because one
thread can use the disk while the other uses the CPU.

Apparent performance is dramatically improved. The user can start
using the app while fonts continue to load. If the user quickly
tries to use the text tool they’ll see a progress bar and have to
wait a bit, but they can use every other part of the app without
waiting for the font scanner.

Just and editorial comment or two if you don’t mind: Thread programming
in user space is very different from thread programming in system space.
The main difference is that in system space everything has been coded
assuming multiple threads because of interrupt processing. In use space
almost no code is written assuming multiple threads. Most libraries will
turn there toes up and die if you try to use them in a multithreaded
application without writing a serializing wrapper library.

Ouch. I expected this of init functions and functions with an API
that is inherently unsafe (like strtok).

FreeType solves this as a side-effect of being ROMable. There is
no static data in the library.

We’re going to need thread safe versions of all the libraries we use.

Yup.On Mon, 2005-01-24 at 09:03 -0600, Bob Pendleton wrote:

14

Taking a shot in the dark here… Since no one can know what you know we
have to take shots in the dark.

Yeah, I know. Thank you for trying.

On linux, the C/C++ standard libraries may or may not be thread safe, it
all depends on the version you use. If you are on 3.0 or better then the
standard libraries are thread safe (at least that is what the docs I
have found say and it seems to work).

This is plain C, not C++. I have various gcc versions from 2.95 to 3.4,
with glibc 2.3.2, and old-style Linux threads.

As long as you use the same tool chain to build your code as you would
use to build SDL you are fine. SDL, obviously, uses threads and works
just fine.

OTOH, SDL is not thread safe. All graphics must be done in the main
thread

“All graphics”, hmmm? Does that mean just video, or allocating
and freeing software surfaces too?

That means video and anything that may be called by the video code.

I can’t answer that question without a detailed analysis of software
surface code; I’m pretty sure (I have been in that code recently) that
if you put a wrapper around them you can call the those functions safely
in any thread. I would put a wrapper around any graphics code I used in
another thread. The problem is that some low level SDL code uses lower
level graphics code without going through the high level APIs.

I had also been hoping to
load and scale some images in a thread dedicated to clip-art.

But, like I said, SDL is not thread safe, neither are any of the SDL add
on libraries. If you want to use them in threads you have to build
wrappers for the code you want to use.

I think somebody else just said the opposite. :frowning:

One of us has to be right :slight_smile: I’ve found that being pessimistic in cases
like this save me a lot more time than being optimistic.

What does that mean to an SDL programmer? It means that you can have 10
threads reading 10 files, but only one thread can be decoding a ttf font
at a time.

This is awful. I looked over libSDL_ttf and didn’t find anything
significant. What did I miss? I saw:

a. error text might get corrupted (but no crash)
b. can’t do simultaneous TTF_Init() calls (no problem)

Regarding #a, note that libSDL_ttf does not pass '%'
in error strings. So the unsafe vararg stuff in SDL’s
error handling would not be involved.

What, you want to create a thread per font? Not a problem, read the font
file into memory and open it with TTF_OpenFontRW (use a wrapper around
it to serialize use) and you can load the fonts in parallel and you only
get serialization when you open the font. Or, just use a thread the
sequentially call TTF_OpenFont(). So long as it isn’t being called
anywhere else, there is no problem.

Now, the thing to consider is that unless you have a hyperthreading or
multicore CPU you can’t actually have more than one thread active at a
time anyway. And, having multiple threads reading from a single disk may
or may not improve disk reading performance. After all, there is only
one disk to perform the reads. So, this month you might not see the
performance improvement you expect to see. Next month you might see a
dramatic performance improvement :slight_smile:

Raw performance is a tiny bit better. I suppose this is because one
thread can use the disk while the other uses the CPU.

Yep, and if you are trying to read many files at once you might get
better use of the disk.

Apparent performance is dramatically improved. The user can start
using the app while fonts continue to load. If the user quickly
tries to use the text tool they’ll see a progress bar and have to
wait a bit, but they can use every other part of the app without
waiting for the font scanner.

That is an obvious use of threads and should work if the scanning thread
is properly structured and take proper precautions when using SDL and
your own internal code and data.

This use implies that there will be data and code within your own
program that will have to have mutex protection.

Just and editorial comment or two if you don’t mind: Thread programming
in user space is very different from thread programming in system space.
The main difference is that in system space everything has been coded
assuming multiple threads because of interrupt processing. In use space
almost no code is written assuming multiple threads. Most libraries will
turn there toes up and die if you try to use them in a multithreaded
application without writing a serializing wrapper library.

Ouch. I expected this of init functions and functions with an API
that is inherently unsafe (like strtok).

Man, you would be surprised at the amount of local state that is kept in
global variables.

FreeType solves this as a side-effect of being ROMable. There is
no static data in the library.

Which is wonderful. But, so long as you are using the SDL wrappers you
can’t be sure of what the wrappers do. BTW, there have been some recent
fixes in memory handling in the SDL_TTF library. You might want to use
the latest CVS versions of the code.

	Bob PendletonOn Mon, 2005-01-24 at 12:40 -0500, Albert Cahalan wrote:

On Mon, 2005-01-24 at 09:03 -0600, Bob Pendleton wrote:

We’re going to need thread safe versions of all the libraries we use.

Yup.

SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl

15

What does that mean to an SDL programmer? It means that you can have 10
threads reading 10 files, but only one thread can be decoding a ttf font
at a time.

This is awful. I looked over libSDL_ttf and didn’t find anything
significant. What did I miss? I saw:

a. error text might get corrupted (but no crash)
b. can’t do simultaneous TTF_Init() calls (no problem)

Regarding #a, note that libSDL_ttf does not pass '%'
in error strings. So the unsafe vararg stuff in SDL’s
error handling would not be involved.

Found it!

FreeType keeps a buffer around to fast scan line conversion.
This buffer is not static within FreeType, so FreeType itself
is thread-safe. The buffer must be in the FT_Library struct or
hanging off it. The libSDL_ttf code shares this struct with
all fonts that it opens.

I’m including a patch (not tested) that should improve the
situation greatly. Please apply it. Please also be sure to
fully document the remaining problems. I think they are:

a. SDL error messages might get messed up
b. the byteswap setting is obviously global
c. each thread will need a private TTF_Font

(in the man pages, the wiki pages, the info pages…)

BTW, the wide character handling is kind of broken anyway.
Full Unicode version 2.x requires 32-bit characters.
The early-adopter systems using 16-bit have had to switch
to the nasty UTF-16 encoding. Perhaps all the 16-bit stuff
should just be removed. The gcc compiler provides some help;
you can put attribute((deprecated)) in a header file to
warn users about the obsolete interface.

Anyway, here’s the patch:

////////////////////////////////////////////////////////////////////////
diff -Naurd old/SDL_ttf.c new/SDL_ttf.c
— old/SDL_ttf.c 2005-01-24 16:11:26.000000000 -0500
+++ new/SDL_ttf.c 2005-01-24 16:28:21.000000000 -0500
@@ -90,6 +90,7 @@
struct _TTF_Font {
/* Freetype2 maintains all sorts of useful info itself */
FT_Face face;

  • FT_Library library;

    /* We’ll cache these ourselves */
    int height;
    @@ -122,10 +123,7 @@
    int font_size_family;
    };

-/* The FreeType font engine/library */
-static FT_Library library;
-static int TTF_initialized = 0;
-static int TTF_byteswapped = 0;
+static int TTF_byteswapped;

/* UNICODE string utilities */
static inline int UNICODE_strlen(const Uint16 text)
@@ -205,21 +203,10 @@
#endif / USE_FREETYPE_ERRORS */
}

+/* this exists only to maintain the ABI */
int TTF_Init( void )
{

  • int status = 0;On Mon, 2005-01-24 at 12:40 -0500, Albert Cahalan wrote:

On Mon, 2005-01-24 at 09:03 -0600, Bob Pendleton wrote:

  • if ( ! TTF_initialized ) {
  •   FT_Error error = FT_Init_FreeType( &library );

  •   if ( error ) {

  •   	TTF_SetFTError("Couldn't init FreeType engine", error);

  •   	status = -1;

  •   }
  • }
  • if ( status == 0 ) {
  •   ++TTF_initialized;
  • }
  • return status;
  • return 0;
    }

static unsigned long RWread(
@@ -245,11 +232,6 @@
FT_Stream stream;
int position;

  • if ( ! TTF_initialized ) {
  •   TTF_SetError( "Library not initialized" );

  •   return NULL;
  • }
  • /* Check to make sure we can seek in this stream */
    position = SDL_RWtell(src);
    if ( position < 0 ) {
    @@ -267,6 +249,13 @@
    font->src = src;
    font->freesrc = freesrc;
  • FT_Error error = FT_Init_FreeType( &font->library );
  • if ( error ) {
  •   TTF_SetFTError("Couldn't init FreeType engine", error);

  •   free( font );

  •   return NULL;
  • }
  • stream = (FT_Stream)malloc(sizeof(*stream));
    if ( stream == NULL ) {
    TTF_SetError( “Out of memory” );
    @@ -275,7 +264,7 @@
    }
    memset(stream, 0, sizeof(*stream));
  • stream->memory = library->memory;
  • stream->memory = font->library->memory;
    stream->read = RWread;
    stream->descriptor.pointer = src;
    stream->pos = (unsigned long)position;
    @@ -286,7 +275,7 @@
    font->args.flags = FT_OPEN_STREAM;
    font->args.stream = stream;
  • error = FT_Open_Face( library, &font->args, index, &font->face );
  • error = FT_Open_Face( font->library, &font->args, index, &font->face );
    if( error ) {
    TTF_SetFTError( “Couldn’t load font file”, error );
    TTF_CloseFont( font );
    @@ -668,6 +657,7 @@
    if ( font->freesrc ) {
    SDL_RWclose( font->src );
    }
  • FT_Done_FreeType( font->library );
    free( font );
    }

@@ -849,10 +839,6 @@
FT_UInt prev_index = 0;

/* Initialize everything to 0 */
  • if ( ! TTF_initialized ) {
  •   TTF_SetError( "Library not initialized" );

  •   return -1;
  • }
    status = 0;
    minx = maxx = 0;
    miny = maxy = 0;
    @@ -1735,14 +1721,10 @@

void TTF_Quit( void )
{

  • if ( TTF_initialized ) {
  •   if ( --TTF_initialized == 0 ) {

  •   	FT_Done_FreeType( library );

  •   }
  • }
  • return;
    }

int TTF_WasInit( void )
{

  • return TTF_initialized;
  • return 1; // this is a don’t-care value to satisfy the ABI
    }

////////////////////////////////////////////////////////////////////////

16

You might want to repost your patch in a message with a title that says
there is a patch in it. This is the kind of thing that Sam needs to
looks at.

BTW, moving from 16 bit unicode to 32 bit unicode is on the wish list
for version 1.3 or later.

	Bob PendletonOn Mon, 2005-01-24 at 16:46 -0500, Albert Cahalan wrote:

On Mon, 2005-01-24 at 12:40 -0500, Albert Cahalan wrote:

On Mon, 2005-01-24 at 09:03 -0600, Bob Pendleton wrote:

What does that mean to an SDL programmer? It means that you can have 10
threads reading 10 files, but only one thread can be decoding a ttf font
at a time.

This is awful. I looked over libSDL_ttf and didn’t find anything
significant. What did I miss? I saw:

a. error text might get corrupted (but no crash)
b. can’t do simultaneous TTF_Init() calls (no problem)

Regarding #a, note that libSDL_ttf does not pass '%'
in error strings. So the unsafe vararg stuff in SDL’s
error handling would not be involved.

Found it!

FreeType keeps a buffer around to fast scan line conversion.
This buffer is not static within FreeType, so FreeType itself
is thread-safe. The buffer must be in the FT_Library struct or
hanging off it. The libSDL_ttf code shares this struct with
all fonts that it opens.

I’m including a patch (not tested) that should improve the
situation greatly. Please apply it. Please also be sure to
fully document the remaining problems. I think they are:

a. SDL error messages might get messed up
b. the byteswap setting is obviously global
c. each thread will need a private TTF_Font

(in the man pages, the wiki pages, the info pages…)

BTW, the wide character handling is kind of broken anyway.
Full Unicode version 2.x requires 32-bit characters.
The early-adopter systems using 16-bit have had to switch
to the nasty UTF-16 encoding. Perhaps all the 16-bit stuff
should just be removed. The gcc compiler provides some help;
you can put attribute((deprecated)) in a header file to
warn users about the obsolete interface.

Anyway, here’s the patch:

////////////////////////////////////////////////////////////////////////
diff -Naurd old/SDL_ttf.c new/SDL_ttf.c
— old/SDL_ttf.c 2005-01-24 16:11:26.000000000 -0500
+++ new/SDL_ttf.c 2005-01-24 16:28:21.000000000 -0500
@@ -90,6 +90,7 @@
struct _TTF_Font {
/* Freetype2 maintains all sorts of useful info itself */
FT_Face face;

  • FT_Library library;

    /* We’ll cache these ourselves */
    int height;
    @@ -122,10 +123,7 @@
    int font_size_family;
    };

-/* The FreeType font engine/library */
-static FT_Library library;
-static int TTF_initialized = 0;
-static int TTF_byteswapped = 0;
+static int TTF_byteswapped;

/* UNICODE string utilities */
static inline int UNICODE_strlen(const Uint16 text)
@@ -205,21 +203,10 @@
#endif / USE_FREETYPE_ERRORS */
}

+/* this exists only to maintain the ABI */
int TTF_Init( void )
{

  • int status = 0;
  • if ( ! TTF_initialized ) {
  • FT_Error error = FT_Init_FreeType( &library );

  • if ( error ) {

  • 	TTF_SetFTError("Couldn't init FreeType engine", error);

  • 	status = -1;

  • }
  • }
  • if ( status == 0 ) {
  • ++TTF_initialized;
  • }
  • return status;
  • return 0;
    }

static unsigned long RWread(
@@ -245,11 +232,6 @@
FT_Stream stream;
int position;

  • if ( ! TTF_initialized ) {
  • TTF_SetError( "Library not initialized" );

  • return NULL;
  • }
  • /* Check to make sure we can seek in this stream */
    position = SDL_RWtell(src);
    if ( position < 0 ) {
    @@ -267,6 +249,13 @@
    font->src = src;
    font->freesrc = freesrc;
  • FT_Error error = FT_Init_FreeType( &font->library );
  • if ( error ) {
  • TTF_SetFTError("Couldn't init FreeType engine", error);

  • free( font );

  • return NULL;
  • }
  • stream = (FT_Stream)malloc(sizeof(*stream));
    if ( stream == NULL ) {
    TTF_SetError( “Out of memory” );
    @@ -275,7 +264,7 @@
    }
    memset(stream, 0, sizeof(*stream));
  • stream->memory = library->memory;
  • stream->memory = font->library->memory;
    stream->read = RWread;
    stream->descriptor.pointer = src;
    stream->pos = (unsigned long)position;
    @@ -286,7 +275,7 @@
    font->args.flags = FT_OPEN_STREAM;
    font->args.stream = stream;
  • error = FT_Open_Face( library, &font->args, index, &font->face );
  • error = FT_Open_Face( font->library, &font->args, index, &font->face );
    if( error ) {
    TTF_SetFTError( “Couldn’t load font file”, error );
    TTF_CloseFont( font );
    @@ -668,6 +657,7 @@
    if ( font->freesrc ) {
    SDL_RWclose( font->src );
    }
  • FT_Done_FreeType( font->library );
    free( font );
    }

@@ -849,10 +839,6 @@
FT_UInt prev_index = 0;

/* Initialize everything to 0 */

  • if ( ! TTF_initialized ) {
  • TTF_SetError( "Library not initialized" );

  • return -1;
  • }
    status = 0;
    minx = maxx = 0;
    miny = maxy = 0;
    @@ -1735,14 +1721,10 @@

void TTF_Quit( void )
{

  • if ( TTF_initialized ) {
  • if ( --TTF_initialized == 0 ) {

  • 	FT_Done_FreeType( library );

  • }
  • }
  • return;
    }

int TTF_WasInit( void )
{

  • return TTF_initialized;
  • return 1; // this is a don’t-care value to satisfy the ABI
    }

////////////////////////////////////////////////////////////////////////

SDL mailing list
SDL at libsdl.org
http://www.libsdl.org/mailman/listinfo/sdl