Very precise control of display timing

Hello,

I would like to describe an apparently simple project and ask people
on this list if they believe there is a way to achieve it with SDL
(under linux). In a nutshell, I wonder if it is possible to get control
of display duration down to the screen refresh rate.

I want to display at a fast rate a series of pictures on the screen,
and collect mouse clicks timestamped with the best possible accuracy.

Some pictures should remain on the screen for as little as 2 or 3 refresh
cycles (~30, 45 msec, depending on the Vertical Sync Freq).
Actually I need to have a total control on the number of refresh cycles
a picture is displayed.

The user is instructed to click on one the mouse’s buttons when
he/she detects a pretermined target picture in the continous stream of
pictures… At the end, I want to know precisely when the clicks occured.

At such a fast rate, the user will not see consciously some of the
pictures, but this is precisely what I am interested in (his visual system
may have registered them, in a sort of subliminal perception if you like)

Long ago, I wrote a DOS program in Borland Pascal that does just that:

do
wait for the specified number of vertical retrace synchro signal
swap hidden and visual page
prepare next picture in the hidden page
loop

It worked well, but the only (big) proble is that the 10 years old Borland
BGI drivers only knows about plain VGA video mode…

The problem is much more complex under linux, it seems to me, because
of the multitasking OS:

How can I be sure not to miss a vertical retrace signal? (i.e. how can I
be sure that my process is going to be running at this critical moments?)
Plus the granularity of the typical timing functions seems to be around 10
msec.

Any idea?

Cheers,

Christophe Pallier

Pallier Christophe wrote:

I want to display at a fast rate a series of pictures on the screen,
and collect mouse clicks timestamped with the best possible accuracy.

Do an SDL_GetTick() as soon as you have received a mouse click event.
The problem is how to reduce the latency of the event when you are
doing something else.

Some targets will add the mouse event to SDL’s event queue immediately
(through an OS callback or similar); others, such as X11 or DGA2, will
only do so when PumpEvents is called (directly or via PollEvent or
WaitEvent). Thus, if you call GetTick() in an event filter handler,
you may get lower latency.

Beware that WaitEvent may impose a 10 ms latency because of the way it
is implemented, so you may need to busy-wait

How can I be sure not to miss a vertical retrace signal? (i.e. how can I
be sure that my process is going to be running at this critical moments?)
Plus the granularity of the typical timing functions seems to be around 10
msec.

No, SDL_GetTick() is usually accurate down to 1 ms (and you can get even
higher precision using platform-dependent methods, but this is rather
pointless and not really recommended). There is no way to detect
the number of vertical refreshes that have passed. What you can do
(on video targets that support it) is to request a buffer flip at
the next vertical retrace, and then wait for it to happen.
If your machine is fast enough, you could count the frames that way.

Please read the list archives - this has been discussed many many times
before

Hello,

I would like to describe an apparently simple project and ask people
on this list if they believe there is a way to achieve it with SDL
(under linux). In a nutshell, I wonder if it is possible to get control
of display duration down to the screen refresh rate.

Generally, no, not in any reliable way.

[…]

Long ago, I wrote a DOS program in Borland Pascal that does just that:

do
wait for the specified number of vertical retrace synchro signal
swap hidden and visual page
prepare next picture in the hidden page
loop

It worked well, but the only (big) proble is that the 10 years old
Borland BGI drivers only knows about plain VGA video mode…

The problem is much more complex under linux, it seems to me, because
of the multitasking OS:

Right, there’s no way to hit all of the very short-lived retraces by
polling the VGA registers. You need some real time solution to it that
way.

However, if you time the refresh rate with a very accurate timer (such as
the pentium TSC), you can look at the timer to find out how many frames
have elapsed. Resync the timer to actual retraces whenever possible.

Either way, to actually do something at the right time (as opposed to
just finding out when something happened), you still need a real time
solution.

How can I be sure not to miss a vertical retrace signal? (i.e. how can
I be sure that my process is going to be running at this critical
moments?)

Use Linux/lowlatency and a SCHED_FIFO thread for the retrace polling. You
can use the RTC to wake the polling thread up right before the retrace is
expected. The TSC (RDTSC x86 instruction) is useful if you can get away
with just knowing when the retrace occured/will occur.

http://www.uow.edu.au/~andrewm/linux/schedlat.html

Even better, but a little bit more complicated; do the same thing with
RTlinux or RTAI. You can safely wake up just a few 10’s of ?s before the
expected retrace time, poll, write some data (the current frame count,
for example) to an RTL FIFO, and then put your thread to sleep again.
Timing is ?s accurate, and worst case latencies are in the 5-20 ?s range
depending on hardware.

http://www.rtlinux.org/

http://www.aero.polimi.it/projects/rtai/

Note that the solutions above only helps for about half of the problem;
keeping track of the retraces. You still need to use Linux/lowlatency and
carefully selected drivers to solve the rendering part.

You can’t run SDL as an RTL thread, and you can’t rely on all SDL targets
to benefit from SCHED_FIFO, or even work with it! For example, it’s
possible to run X and X applications as SCHED_FIFO, but some applications
will freeze the entire system. X and normal X applications aren’t
designed to run as real time threads without timesharing preemption.

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |--------------------------------------> david at linuxdj.com -'On Wednesday 22 August 2001 09:39, Pallier Christophe wrote:

Here’s how I’m using the yuv overlay:
//-----------------------------------------
SDL_Overlay *overlay;
SDL_Rect rect;

SDL_LockYUVOverlay(overlay);
memcpy into overlay->pixels[]
SDL_UnlockYUVOverlay(overlay);
SDL_DisplayYUVOverlay(overlay,&rect);
//-----------------------------------------

I keep getting this error " Xlib: unexpected async reply (sequence 0x604)!
". It seems to happend when I switch to 2x zoom and back (rect.h * 2,
rect.w *2 ).

I’ve also noticed that some mpeg/divx players (Lamp) based on
SDL produce the same error.

I’m running Mandrake 8 with XFree86 4.10, Nvidia 12.51 drivers,SDL 1.2.1.

Hello,

Many thanks to David Olofson for the detailed answer and the links provided!

If I may add one question: From my readings about the TSC, I understood
it was counting cycles. I intend to run my soft on laptops, some of which feature
“mobile pentiums” that, if I understand correctly, can adjust their internal
frequency depending on the current load of the processor…(I don’t know if this
is automatic or if this is under control of APM functions, but anyway,
I cannot count on users using a kernel with APM disabled).
If this is correct, then one cannot count on the TSC to measure time…
Am I wrong?

Second, I was aware of the RTC (I played with rtctest.c provided in the linux kernel source code),
but I had not found any information about the latency. I was afraid the scheduler may take
a non-negligible amount of time before switching to the process blocked waiting on a “read
/dev/rtc” (the “normal” beat of the scheduler is 100 Hz that is 10 msec). A quick glance
to one link given by David (http://www.uow.edu.au/~andrewm/linux/schedlat.html) suggests
one can get the answer to this question.

Finally, I am using sdl-1.2 where SDL_GetTicks() relies on gettimeofday() in the
linux implementation. Is it still the same code in sdl-1.3? Are there any critical
differences regarding timing between sdl-1.2 and sdl-1.3?

Lastly, is there a search interface to the sdl mailing list?
I am embarrassed to ask questions which may have been discussed in the past.

Cheers,–
Christophe Pallier

Pallier Christophe wrote:

Finally, I am using sdl-1.2 where SDL_GetTicks() relies on
gettimeofday() in the linux implementation. Is it still the same code
in sdl-1.3? Are there any critical differences regarding timing
between sdl-1.2 and sdl-1.3?

there is no compelling reason not to use gettimeofday() for implementing
SDL_GetTicks(), and for your application (measuring human response time
in some context) I would be quite surprised if you could show that 1/1000 s
timing resolution isn’t sufficient. For gaming purposes it never is

Lastly, is there a search interface to the sdl mailing list?

not yet but try using a standard web search service with answers restricted
to the list archive (it might work)

You can use Google (if you get the IE toolbar, it has a "search only on …"
button):

http://www.google.com/search?q=site:www.libsdl.org+latency+problem

Or download the whole 48mb archive and grep it :slight_smile: Really, it would be nice
if there were a gzip/bzip2 version of that…> Lastly, is there a search interface to the sdl mailing list?

Hello,

Many thanks to David Olofson for the detailed answer and the links
provided!

If I may add one question: From my readings about the TSC, I understood
it was counting cycles.

Yes.

I intend to run my soft on laptops, some of
which feature “mobile pentiums” that, if I understand correctly, can
adjust their internal frequency depending on the current load of the
processor…(I don’t know if this is automatic or if this is under
control of APM functions, but anyway, I cannot count on users using a
kernel with APM disabled).

Then you also have some more sersious problems; you can’t assume that
users are going to use lowlatency patched kernels, RTLinux or even that
they’ll start you program as root so that it can activate SCHED_FIFO
policy.

In short, even if it’s possible to keep track of the refresh rate,
there’s no way you can guarantee that you program can control the display
with anything like single frame accuracy.

If this is correct, then one cannot count on the TSC to measure time…
Am I wrong?

Some “clock throttling” solutions change the bus clock (and thus also the
core clock which drives the TSC), but it seems like some don’t. I don’t
know how the mobile intel CPUs do it, but I’m suspecting that you
shouldn’t trust the TSC on unknown machines in general.

Second, I was aware of the RTC (I played with rtctest.c provided in the
linux kernel source code), but I had not found any information about
the latency.

It works like just about any other driver. It’s driven by a hardware
interrupt, which means that it wakes up sleeping threads immediately,
unless there are higher priority threads, or the kernel is holding the
wrong locks. (The latter is the real problem, and that’s what
Linux/lowlatency deals with.)

I was afraid the scheduler may take a non-negligible
amount of time before switching to the process blocked waiting on a
"read /dev/rtc" (the “normal” beat of the scheduler is 100 Hz that is
10 msec).

No, that has nothing to do with it. The driver has it’s own hardware
timebase, so it can invoke the scheduler at any time, regardless of “HZ”.

A quick glance to one link given by David
(http://www.uow.edu.au/~andrewm/linux/schedlat.html) suggests one can
get the answer to this question.

Well, you can get the answer by studying the scheduler and RTC driver
code. The latency measurement tools just let you measure the actual
latencies caused by non-preemptable kernel code getting in the way.

Finally, I am using sdl-1.2 where SDL_GetTicks() relies on
gettimeofday() in the linux implementation. Is it still the same code
in sdl-1.3? Are there any critical differences regarding timing between
sdl-1.2 and sdl-1.3?

Don’t know. Why would it change?

//David Olofson — Programmer, Reologica Instruments AB

.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
----------------------------> http://www.linuxdj.com/maia -' .- David Olofson -------------------------------------------. | Audio Hacker - Open Source Advocate - Singer - Songwriter |--------------------------------------> david at linuxdj.com -'On Friday 24 August 2001 10:38, Pallier Christophe wrote: