Warning to Lua Users

Hey,

Since this a really bizzare bug and not entirely easy to find I thought I’d
make some noise so others might find this info more easily. Not entirely SDL
related but considering the amount of windows games being written with SDL,
it’s pertinent.

If you are creating a hardware accelerated screen with SDL on windows,
chances are that it’ll be a D3D context. Funny thing happens to the floating
point unit when you use D3D (from the Lua user mailing list):On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

Does anyone have some ideas on how to make the floating point
problems between DX and Lua go away that is more constructive than
use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

http://www.virtualdub.org/blog/pivot/entry.php?id=53

Initializing Direct3D with default settings causes the precision
bits in the floating-point control word to be reset such that FPU
calculations always occur with 24-bit precision (single precision
/ float). This is much more serious as it causes roundoff errors
to become much larger, and it means that double-precision math can
no longer represent all valid values of a 32-bit int. For this
reason, if you invoke Direct3D within an application that may not
be expecting it, such as within a video filter, you should set the
D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P

The D3DCREATE_FPU_PRESERVE flag exists for those applications who
want to set their own FP control word and/or stay with
double-precision. You still must be very careful when setting it
to non-standard rounding or enabling exceptions. For games, you
typically don't want the performance hit of having the FP unit
working in double-precision.

[...]

Using this flag with the FPU in double-precision mode will reduce
Direct3D performance.

                                              -Dave Dodge

This will manifest itself in unknown ways, but I assure you that it
may come in extremely obscure forms that are near impossible to track
down. In my case I got an ‘invalid key to next’ error on a perfectly
innocuous piece of Lua code. To the many people who may use SDL and
Lua together; you have to enable the flag mentioned abovet, which will
require you to recompile SDL.

The relevant code is in SDL_renderer_d3d.c at line 442, the
IDirect3D9_CreateDevice call. Considering this is known ‘feature’ of
D3D that may mess with code in highly unexpected ways, it might be a
good idea to add a config macro to enable/disable
D3DCREATE_FPU_PRESERVE, along with a comment which indicates the
reasons why one might do so.

Aaargh. So close and yet so far. Now I’ve got to get 1.3 to compile,
so I can use OpenGL. I miss my Commodore 64.

Ah that’s right, Lua uses “Number” as its int/float type, which is a 64-bit
double by default. Good catch.On Tue, Aug 2, 2011 at 8:50 PM, Jeremy Jurksztowicz wrote:

Hey,

Since this a really bizzare bug and not entirely easy to find I thought I’d
make some noise so others might find this info more easily. Not entirely SDL
related but considering the amount of windows games being written with SDL,
it’s pertinent.

If you are creating a hardware accelerated screen with SDL on windows,
chances are that it’ll be a D3D context. Funny thing happens to the floating
point unit when you use D3D (from the Lua user mailing list):

On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

Does anyone have some ideas on how to make the floating point
problems between DX and Lua go away that is more constructive than
use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

http://www.virtualdub.org/blog/pivot/entry.php?id=53

Initializing Direct3D with default settings causes the precision
bits in the floating-point control word to be reset such that FPU
calculations always occur with 24-bit precision (single precision
/ float). This is much more serious as it causes roundoff errors
to become much larger, and it means that double-precision math can
no longer represent all valid values of a 32-bit int. For this
reason, if you invoke Direct3D within an application that may not
be expecting it, such as within a video filter, you should set the
D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P

The D3DCREATE_FPU_PRESERVE flag exists for those applications who
want to set their own FP control word and/or stay with
double-precision. You still must be very careful when setting it
to non-standard rounding or enabling exceptions. For games, you
typically don't want the performance hit of having the FP unit
working in double-precision.

[...]

Using this flag with the FPU in double-precision mode will reduce
Direct3D performance.

                                              -Dave Dodge

This will manifest itself in unknown ways, but I assure you that it may come in extremely obscure forms that are near impossible to track down. In my case I got an ‘invalid key to next’ error on a perfectly innocuous piece of Lua code. To the many people who may use SDL and Lua together; you have to enable the flag mentioned abovet, which will require you to recompile SDL.

The relevant code is in SDL_renderer_d3d.c at line 442, the IDirect3D9_CreateDevice call. Considering this is known ‘feature’ of D3D that may mess with code in highly unexpected ways, it might be a good idea to add a config macro to enable/disable D3DCREATE_FPU_PRESERVE, along with a comment which indicates the reasons why one might do so.

Aaargh. So close and yet so far. Now I’ve got to get 1.3 to compile, so I can use OpenGL. I miss my Commodore 64.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

Recompiling Lua to use 32bit floats instead of doubles also fixes the
problem. I still think that SDL needs a config macro with a descriptive
comment because this bug can mess with any code using double precision
floats.On Wed, Aug 3, 2011 at 9:56 AM, Patrick Baggett <baggett.patrick at gmail.com>wrote:

Ah that’s right, Lua uses “Number” as its int/float type, which is a 64-bit
double by default. Good catch.

On Tue, Aug 2, 2011 at 8:50 PM, Jeremy Jurksztowicz < @Jeremy_Jurksztowicz> wrote:

Hey,

Since this a really bizzare bug and not entirely easy to find I thought
I’d make some noise so others might find this info more easily. Not entirely
SDL related but considering the amount of windows games being written with
SDL, it’s pertinent.

If you are creating a hardware accelerated screen with SDL on windows,
chances are that it’ll be a D3D context. Funny thing happens to the floating
point unit when you use D3D (from the Lua user mailing list):

On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

Does anyone have some ideas on how to make the floating point
problems between DX and Lua go away that is more constructive than
use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

http://www.virtualdub.org/blog/pivot/entry.php?id=53

Initializing Direct3D with default settings causes the precision
bits in the floating-point control word to be reset such that FPU
calculations always occur with 24-bit precision (single precision
/ float). This is much more serious as it causes roundoff errors
to become much larger, and it means that double-precision math can
no longer represent all valid values of a 32-bit int. For this
reason, if you invoke Direct3D within an application that may not
be expecting it, such as within a video filter, you should set the
D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P

The D3DCREATE_FPU_PRESERVE flag exists for those applications who
want to set their own FP control word and/or stay with
double-precision. You still must be very careful when setting it
to non-standard rounding or enabling exceptions. For games, you
typically don't want the performance hit of having the FP unit
working in double-precision.

[...]

Using this flag with the FPU in double-precision mode will reduce
Direct3D performance.

                                              -Dave Dodge

This will manifest itself in unknown ways, but I assure you that it may come in extremely obscure forms that are near impossible to track down. In my case I got an ‘invalid key to next’ error on a perfectly innocuous piece of Lua code. To the many people who may use SDL and Lua together; you have to enable the flag mentioned abovet, which will require you to recompile SDL.

The relevant code is in SDL_renderer_d3d.c at line 442, the IDirect3D9_CreateDevice call. Considering this is known ‘feature’ of D3D that may mess with code in highly unexpected ways, it might be a good idea to add a config macro to enable/disable D3DCREATE_FPU_PRESERVE, along with a comment which indicates the reasons why one might do so.

Aaargh. So close and yet so far. Now I’ve got to get 1.3 to compile, so I can use OpenGL. I miss my Commodore 64.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

Hi,

Python also uses double floats for its float number.On Wed, Aug 3, 2011 at 3:12 AM, Jeremy Jurksztowicz wrote:

Recompiling Lua to use 32bit floats instead of doubles also fixes the
problem. I still think that SDL needs a config macro with a descriptive
comment because this bug can mess with any code using double precision
floats.

I should mention that FPU flags do not affect SSE2 (Pentium 4 or above, Athlon 64 or above), so 64bit apps are exempt (as SSE2 is the default there), as are apps using SSE2 math as their main floating
point instruction set (gcc -msse2 -mfpmath=sse).

So if your game is already “highend” then this may be an alternative to caring about the D3D context flag.

My vote is that SDL should always be using D3DCREATE_FPU_PRESERVE on its D3D9 context, as a matter of “path of least breakage”.

After all, this is not degrading performance, it is preserving the same precision and performance as one gets on a GL renderer.On 08/02/2011 06:50 PM, Jeremy Jurksztowicz wrote:

Hey,

Since this a really bizzare bug and not entirely easy to find I thought I’d make some noise so others might find this info more easily. Not entirely SDL related but considering the amount of windows
games being written with SDL, it’s pertinent.

If you are creating a hardware accelerated screen with SDL on windows, chances are that it’ll be a D3D context. Funny thing happens to the floating point unit when you use D3D (from the Lua user
mailing list):

On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

Does anyone have some ideas on how to make the floating point
problems between DX and Lua go away that is more constructive than
use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

 http://www.virtualdub.org/blog/pivot/entry.php?id=53

 Initializing Direct3D with default settings causes the precision
 bits in the floating-point control word to be reset such that FPU
 calculations always occur with 24-bit precision (single precision
 / float). This is much more serious as it causes roundoff errors
 to become much larger, and it means that double-precision math can
 no longer represent all valid values of a 32-bit int. For this
 reason, if you invoke Direct3D within an application that may not
 be expecting it, such as within a video filter, you should set the
 D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

 http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P  <http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P>

 The D3DCREATE_FPU_PRESERVE flag exists for those applications who
 want to set their own FP control word and/or stay with
 double-precision. You still must be very careful when setting it
 to non-standard rounding or enabling exceptions. For games, you
 typically don't want the performance hit of having the FP unit
 working in double-precision.

 [...]

 Using this flag with the FPU in double-precision mode will reduce
 Direct3D performance.

                                               -Dave Dodge

This will manifest itself in unknown ways, but I assure you that it may come in extremely obscure forms that are near impossible to track down. In my case I got an’invalid key to next’ error on a perfectly innocuous piece of Lua code. To the many people who may use SDL and Lua together; you have to enable the flag mentioned abovet, which will require you to recompile SDL.

The relevant code is in SDL_renderer_d3d.c at line 442, the IDirect3D9_CreateDevice call. Considering this is known’feature’ of D3D that may mess with code in highly unexpected ways, it might be a good idea to add a config macro to enable/disable D3DCREATE_FPU_PRESERVE, along with a comment which indicates the reasons why one might do so.

Aaargh. So close and yet so far. Now I’ve got to get 1.3 to compile, so I can use OpenGL. I miss my Commodore 64.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org


LordHavoc
Author of DarkPlaces Quake1 engine - http://icculus.org/twilight/darkplaces
Co-designer of Nexuiz - http://alientrap.org/nexuiz
"War does not prove who is right, it proves who is left." - Unknown
"Any sufficiently advanced technology is indistinguishable from a rigged demo." - James Klass
"A game is a series of interesting choices." - Sid Meier