Segfault in D3DRenderer

SDL Development
1

While porting my game to windows, I came across a problem when switching
between window modes. I am using SDL from hg, compiled with mingw64. Here’s
the stacktrace

Thread 1 received signal SIGSEGV, Segmentation fault.
0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220, texture=0x0,
format=390076419, w=96, h=24) at
src/render/direct3d/SDL_render_d3d.c:886
886 if (texture->texture) {
(gdb) bt
#0 0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220,
texture=0x0, format=390076419, w=96, h=24)
at src/render/direct3d/SDL_render_d3d.c:886
#1 0x000000006c7b1bff in D3D_RecreateTexture (renderer=0x2c220,
texture=0x58be7e0) at src/render/direct3d/SDL_render_d3d.c:1007
#2 0x000000006c7b0773 in D3D_Reset (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:425
#3 0x000000006c7b098b in D3D_ActivateRenderer (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:486
#4 0x000000006c7b2adf in D3D_RenderClear (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:1296
#5 0x000000006c7aa470 in SDL_RenderClear_REAL (renderer=0x2c220)
at src/render/SDL_render.c:1403
#6 0x000000006c7901a1 in SDL_RenderClear (a=0x2c220)

From looking at the code, it appears that
((D3D_TextureData *)texture->driverdata)->texture is NULL for some reason.

I have not looked further yet, as I wanted to hear if somebody has any
insight into this issue first. This time, I also did not find any suitable
SDL test program to reproduce the crash. So I don’t want to rule out that
this is a fault on my side, though the program runs fine on both Linux and
OSX (using the OpenGL renderer).

Please let me know if more information is required.

Regards,

Kai

2

Since there is no answer, I started looking further for the issue and also
checked bugzilla for similar reports (none). Debugging further shows:

(gdb) up
#1 0x000000006c7b1bff in D3D_RecreateTexture (renderer=0x32c1c0,
texture=0x5e36360) at src/render/direct3d/SDL_render_d3d.c:1007
1007 if (D3D_RecreateTextureRep(data->device, &texturedata->texture,
texture->format, texture->w, texture->h) < 0) {
(gdb) print texturedata
$1 = (D3D_TextureData *) 0x0

Aha. It’s the texture data itself that is NULL already. Lucky we did not
crash when dereferencing that! But shouldn’t this only be NULL when the
texture was actually deleted via SDL_DestroyTexture? However:

(gdb) print *texture
$3 = {magic = 0x6c8b3a71 <texture_magic>, format = 390076419, access = 1,
w = 96, h = 24, modMode = 0, blendMode = SDL_BLENDMODE_NONE, r = 255 ‘?’,
g = 255 ‘?’, b = 255 ‘?’, a = 255 ‘?’, renderer = 0x32c1c0,
native = 0x5e36410, yuv = 0x0, pixels = 0x5e980c0, pitch = 288,
locked_rect = {x = 0, y = 0, w = 96, h = 24}, driverdata = 0x0, prev =
0x0,
next = 0x5e36410}

That does not look deleted to me.

But it offers a road forward, by adding

if (!texturedata) return 0;

to D3D_RecreateTexture, and indeed, this resolves my problem. However, this
only adresses the symptom, not the root cause. I may be wrong, but I don’t
think that half-created (or half-deleted) textures should ever show up in
SDLs list of textures.

So the main question is, how does it end up there? I have added proper
error checking in my own code whereever it is calling SDL_CreateTexture,
and those calls never failed. Any ideas?

KaiOn Wed, Jun 22, 2016 at 12:18 AM, Kai Sterker <@Kai_Sterker> wrote:

While porting my game to windows, I came across a problem when switching
between window modes. I am using SDL from hg, compiled with mingw64. Here’s
the stacktrace

Thread 1 received signal SIGSEGV, Segmentation fault.
0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220, texture=0x0,
format=390076419, w=96, h=24) at
src/render/direct3d/SDL_render_d3d.c:886
886 if (texture->texture) {
(gdb) bt
#0 0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220,
texture=0x0, format=390076419, w=96, h=24)
at src/render/direct3d/SDL_render_d3d.c:886
#1 0x000000006c7b1bff in D3D_RecreateTexture (renderer=0x2c220,
texture=0x58be7e0) at src/render/direct3d/SDL_render_d3d.c:1007
#2 0x000000006c7b0773 in D3D_Reset (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:425
#3 0x000000006c7b098b in D3D_ActivateRenderer (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:486
#4 0x000000006c7b2adf in D3D_RenderClear (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:1296
#5 0x000000006c7aa470 in SDL_RenderClear_REAL (renderer=0x2c220)
at src/render/SDL_render.c:1403
#6 0x000000006c7901a1 in SDL_RenderClear (a=0x2c220)

From looking at the code, it appears that
((D3D_TextureData *)texture->driverdata)->texture is NULL for some reason.

I have not looked further yet, as I wanted to hear if somebody has any
insight into this issue first. This time, I also did not find any suitable
SDL test program to reproduce the crash. So I don’t want to rule out that
this is a fault on my side, though the program runs fine on both Linux and
OSX (using the OpenGL renderer).

Please let me know if more information is required.

Regards,

Kai

3

Just tried with the SDL2-devel-2.0.4-mingw.tar.gz package from libsdl.org;
same issue:

Thread 1 received signal SIGSEGV, Segmentation fault.

0x000000006c79976b in SDL_LogCritical ()
from C:\msys64\mingw64\usr\local\bin\SDL2.dll
(gdb) bt
#0 0x000000006c79976b in SDL_LogCritical ()
from C:\msys64\mingw64\usr\local\bin\SDL2.dll
#1 0x000000006c799c6e in SDL_LogCritical ()
from C:\msys64\mingw64\usr\local\bin\SDL2.dll
#2 0x000000006c79a267 in SDL_LogCritical ()
from C:\msys64\mingw64\usr\local\bin\SDL2.dll
#3 0x000000000049c135 in screen::clear () at screen.h:140

Filed a bug report: https://bugzilla.libsdl.org/show_bug.cgi?id=3370

For now I’ll keep my own SDL version with the workaround in place, but an
official fix or pointer to what I am doing wrong would be hghly appreciated
:-).

KaiOn Wed, Jun 22, 2016 at 2:42 PM, Kai Sterker <@Kai_Sterker> wrote:

Since there is no answer, I started looking further for the issue and also
checked bugzilla for similar reports (none). Debugging further shows:

(gdb) up
#1 0x000000006c7b1bff in D3D_RecreateTexture (renderer=0x32c1c0,
texture=0x5e36360) at src/render/direct3d/SDL_render_d3d.c:1007
1007 if (D3D_RecreateTextureRep(data->device,
&texturedata->texture, texture->format, texture->w, texture->h) < 0) {
(gdb) print texturedata
$1 = (D3D_TextureData *) 0x0

Aha. It’s the texture data itself that is NULL already. Lucky we did not
crash when dereferencing that! But shouldn’t this only be NULL when the
texture was actually deleted via SDL_DestroyTexture? However:

(gdb) print *texture
$3 = {magic = 0x6c8b3a71 <texture_magic>, format = 390076419, access = 1,
w = 96, h = 24, modMode = 0, blendMode = SDL_BLENDMODE_NONE, r = 255 ‘?’,
g = 255 ‘?’, b = 255 ‘?’, a = 255 ‘?’, renderer = 0x32c1c0,
native = 0x5e36410, yuv = 0x0, pixels = 0x5e980c0, pitch = 288,
locked_rect = {x = 0, y = 0, w = 96, h = 24}, driverdata = 0x0, prev =
0x0,
next = 0x5e36410}

That does not look deleted to me.

But it offers a road forward, by adding

if (!texturedata) return 0;

to D3D_RecreateTexture, and indeed, this resolves my problem. However,
this only adresses the symptom, not the root cause. I may be wrong, but I
don’t think that half-created (or half-deleted) textures should ever show
up in SDLs list of textures.

So the main question is, how does it end up there? I have added proper
error checking in my own code whereever it is calling SDL_CreateTexture,
and those calls never failed. Any ideas?

Kai

On Wed, Jun 22, 2016 at 12:18 AM, Kai Sterker <@Kai_Sterker> wrote:

While porting my game to windows, I came across a problem when switching
between window modes. I am using SDL from hg, compiled with mingw64. Here’s
the stacktrace

Thread 1 received signal SIGSEGV, Segmentation fault.
0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220,
texture=0x0,
format=390076419, w=96, h=24) at
src/render/direct3d/SDL_render_d3d.c:886
886 if (texture->texture) {
(gdb) bt
#0 0x000000006c7b170b in D3D_RecreateTextureRep (device=0xd82220,
texture=0x0, format=390076419, w=96, h=24)
at src/render/direct3d/SDL_render_d3d.c:886
#1 0x000000006c7b1bff in D3D_RecreateTexture (renderer=0x2c220,
texture=0x58be7e0) at src/render/direct3d/SDL_render_d3d.c:1007
#2 0x000000006c7b0773 in D3D_Reset (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:425
#3 0x000000006c7b098b in D3D_ActivateRenderer (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:486
#4 0x000000006c7b2adf in D3D_RenderClear (renderer=0x2c220)
at src/render/direct3d/SDL_render_d3d.c:1296
#5 0x000000006c7aa470 in SDL_RenderClear_REAL (renderer=0x2c220)
at src/render/SDL_render.c:1403
#6 0x000000006c7901a1 in SDL_RenderClear (a=0x2c220)

From looking at the code, it appears that
((D3D_TextureData *)texture->driverdata)->texture is NULL for some
reason.

I have not looked further yet, as I wanted to hear if somebody has any
insight into this issue first. This time, I also did not find any suitable
SDL test program to reproduce the crash. So I don’t want to rule out that
this is a fault on my side, though the program runs fine on both Linux and
OSX (using the OpenGL renderer).

Please let me know if more information is required.

Regards,

Kai