Memory leaks, output with debug SDL lib

SDL Development
1

This email is a continuation of this email:

http://lists.libsdl.org/pipermail/sdl-libsdl.org/2009-May/070474.html

Hello, I dont know if I compiled it right, but the output.txt now changed a little, I will send it in 3 parts, this is part one._________________________________________________________________
Descubra todas as novidades do novo Internet Explorer 8
http://brasil.microsoft.com.br/IE8/mergulhe/?utm_source=MSN%3BHotmail&utm_medium=Tagline&utm_campaign=IE8
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: part1.txt
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20090519/396a4e9b/attachment.txt

2

This is part two._________________________________________________________________
Conhe?a os novos produtos Windows Live! Clique aqui.
http://www.windowslive.com.br
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: part2.txt
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20090519/6977b6e0/attachment.txt

3

This is part three.

Thank you fot attention :)_________________________________________________________________
Novo Internet Explorer 8. Baixe agora, ? gr?tis!
http://brasil.microsoft.com.br/IE8/mergulhe/?utm_source=MSN%3BHotmail&utm_medium=Tagline&utm_campaign=IE8
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: part3.txt
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20090519/a0ca0228/attachment.txt

4

Thank you fot attention :slight_smile:

It’s still a non-debug build of SDL, posting these logs don’t really
help all that much. All we can tell is that “there are leaks in SDL”,
and you said that in your first email. :slight_smile:

Also, the --show-reachable=yes makes it even worse. Normally, you’d
deal with the unreachable first, since those are normally not false
positives, but with --show-reachable=yes, you can get a lot of false
positive.On Tue, May 19, 2009 at 9:35 PM, Bruno Adami <a_j_bruno at hotmail.com> wrote:

–
http://pphaneuf.livejournal.com/

5

Should I worry about it? I changed in the makefile this line:

CFLAGS = -O2

To:

CFLAGS = -g -O0

What else I need to do to compile a debug version? Or should I forget those leaks and wait for next version of SDL? There is any way to supress those leaks so I can see only leaks from my application (discarding SDL leaks)?

Thank you!_________________________________________________________________
Novo Internet Explorer 8. Baixe agora, ? gr?tis!
http://brasil.microsoft.com.br/IE8/mergulhe/?utm_source=MSN%3BHotmail&utm_medium=Tagline&utm_campaign=IE8

6

Should I worry about it? I changed in the makefile this line:

CFLAGS? = -O2

To:

CFLAGS? = -g -O0

That should do it (although when I build SDL 1.2 here, it uses -g
without me having to change anything?), but it looks like your test
was using those from the system instead of your custom-built ones?

==31870== by 0x406DBE2: SDL_VideoInit (in /usr/lib/libSDL-1.2.so.0.11.2)
==31870== by 0x4040987: SDL_InitSubSystem (in /usr/lib/libSDL-1.2.so.0.11.2)
==31870== by 0x40409E6: SDL_Init (in /usr/lib/libSDL-1.2.so.0.11.2)

You can set the LD_LIBRARY_PATH environment variable (on Linux,
right?) to point to your version of the library, something like this:

export LD_LIBRARY_PATH=$HOME/src/SDL/SDL-1.2/build/.libs

Adapt the path as needed, of course, wherever the libSDL-1.2.so.0.11.2
that you built is located. The output of Valgrind should report that
one instead of the one in /usr/lib, and hopefully will have source
file names and line numbers to guide you to the source of those leaks.

What else I need to do to compile a debug version? Or should I forget those
leaks and wait for next version of SDL? There is any way to supress those
leaks so I can see only leaks from my application (discarding SDL leaks)?

You can indeed write a suppression file that you then give to Valgrind
with the --suppressions option. See the documentation on how to create
that file:

http://valgrind.org/docs/manual/mc-manual.html#mc-manual.suppfiles
http://valgrind.org/docs/manual/faq.html#faq.writesupp

As I said before, I strongly suggest that you start using Valgrind
with the most reliable and important stuff, then make your way to
stricter options (that might be mistaken) progressively. Personally, I
don’t even start with --leak-check, the base errors that Valgrind
gives are very important, and rarely wrong (things like “conditional
depends on undefined”). Those won’t just make your program memory
usage balloon, but are actually bugs that can give you incorrect
results! Note that --track-origins=yes can help locate those, BTW.

When those are all fixed, I then make my way to memory leaks, first
just the unreachable that are likely to be true leaks, then once those
are all fixed (or otherwise accounted for with a suppression file),
enable --show-reachable.

If you have unit tests, I suggest running them under Valgrind, with
–error-exitcode=42 (or some other non-zero value), so that you don’t
introduce regressions!

Good luck!On Wed, May 20, 2009 at 6:27 AM, Bruno Adami <a_j_bruno at hotmail.com> wrote:

–
http://pphaneuf.livejournal.com/

7

Thank you for the support and tips!

Now I guess it is using the debug lib of SDL, check the output.txt (attached), I run valgrind with:

valgrind --tool=memcheck --track-origins=yes --leak-check=full ./Testing_________________________________________________________________
Descubra todas as novidades do novo Internet Explorer 8
http://brasil.microsoft.com.br/IE8/mergulhe/?utm_source=MSN%3BHotmail&utm_medium=Tagline&utm_campaign=IE8

8

I forgot to attach the file again, sorry >.<_________________________________________________________________
Deixe suas conversas mais divertidas. Baixe agora mesmo novos emoticons. ? gr?tis!
http://specials.br.msn.com/ilovemessenger/pacotes.aspx
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed…
Name: output.txt
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20090520/9655bb9d/attachment.txt

9

Does Valgrind give you any way to notify it of what’s an expected “memory leak” that’s supposed to last as long as the app runs and then die with it? That would eliminate a lot of the confusion, but I’m not sure if there’s any good way to do that with an external tool.

That’s a standard feature on Delphi’s memory manager: you can set it up to notify you of memory leaks, and call the “RegisterExpectedMemoryLeak()” function to let it know what not to tell you about. But I don’t think you can do that for something that’s not compiled into the program without tracing the execution flow manually. It would require some sort of metadata that you’re not likely to find support for in a native-code language.>________________________________

From: Bruno Adami <a_j_bruno at hotmail.com>
Subject: Re: [SDL] Memory leaks, output with debug SDL lib

What else I need to do to compile a debug version? Or should I forget those leaks and wait for next version of SDL? There is any way to supress those leaks so I can see only leaks from my application (discarding SDL leaks)?

Thank you!

10

Now I guess it is using the debug lib of SDL, check the output.txt
(attached), I run valgrind with:

valgrind --tool=memcheck --track-origins=yes --leak-check=full ./Testing

Ah, that looks actually manageable. :slight_smile:

The first one looks like some internal buffer of Xlib, not much to do
about it, you should generate a suppression for it.

The second one depends on whether SDL calls XDestroyIC to match that
XCreateIC. If it does, then it’s probably (again) an internal Xlib
buffer, you’ll need a suppression. If there is not a matching
XDestroyIC call, then you might have found an actual leak in SDL. :slight_smile:

The third one is documented in the man page for XSetLocaleModifiers
(“The returned modifiers string is owned by Xlib and should not be
modified or freed by the client. It may be freed by Xlib after the
current locale or modifiers are changed. Until freed, it will not be
modified by Xlib.”). It’s possible that calling it again with a NULL
would have it free the memory, but it’s probably not worth fussing
over it with some not-too-well documented behaviour and just generate
a suppression for it…

Once you have the appropriate suppressions in place, you’ll be able to
use it with your real program in the same way, and find the bugs in
your code.On Wed, May 20, 2009 at 1:43 PM, Bruno Adami <a_j_bruno at hotmail.com> wrote:

–
http://pphaneuf.livejournal.com/

11

Does Valgrind give you any way to notify it of what’s an expected “memory leak” that’s supposed to last as long as the app runs and then die with it? ?That would eliminate a lot of the confusion, but I’m not sure if there’s any good way to do that with an external tool.

That’s a standard feature on Delphi’s memory manager: you can set it up to notify you of memory leaks, and call the “RegisterExpectedMemoryLeak()” function to let it know what not to tell you about. ?But I don’t think you can do that for something that’s not compiled into the program without tracing the execution flow manually. ?It would require some sort of metadata that you’re not likely to find support for in a native-code language.

There’s actually an API for Valgrind that you use in client code (it’s
basically a “magic” byte pattern prefixed by a jump over it, that
Valgrind recognizes and acts upon, but that doesn’t affect anything
when run normally), but curiously, there’s no API to tell it that a
given block is going to be leaked on purpose. I guess you’re just
supposed to use suppressions…

You can find the client requests API in the valgrind/memcheck.h header:

http://www.google.com/codesearch/p?hl=en#AirWIk8dwNA/usr/include/valgrind/memcheck.h

I’ll file a feature request for that.On Wed, May 20, 2009 at 1:53 PM, Mason Wheeler wrote:

–
http://pphaneuf.livejournal.com/

12

Does Valgrind give you any way to notify it of what’s an expected “memory leak” that’s supposed to last as long as the app runs and then die with it? That would eliminate a lot of the confusion, but I’m not sure if there’s any good way to do that with an external tool.

That’s a standard feature on Delphi’s memory manager: you can set it up to notify you of memory leaks, and call the “RegisterExpectedMemoryLeak()” function to let it know what not to tell you about. But I don’t think you can do that for something that’s not compiled into the program without tracing the execution flow manually. It would require some sort of metadata that you’re not likely to find support for in a native-code language.

There’s actually an API for Valgrind that you use in client code (it’s
basically a “magic” byte pattern prefixed by a jump over it, that
Valgrind recognizes and acts upon, but that doesn’t affect anything
when run normally), but curiously, there’s no API to tell it that a
given block is going to be leaked on purpose. I guess you’re just
supposed to use suppressions…

OK. So what does this magic byte pattern do instead, and what’s a suppression?>----- Original Message ----

From: Pierre Phaneuf
Subject: Re: [SDL] Memory leaks, output with debug SDL lib
On Wed, May 20, 2009 at 1:53 PM, Mason Wheeler <@Mason_Wheeler> wrote:

13

OK. ?So what does this magic byte pattern do instead, and what’s a suppression?

The magic pattern (all hidden by macros) allows your code to interact
with Valgrind when running under it. The simplest one is
RUNNING_ON_VALGRIND, which evaluates to 0 if not running on Valgrind,
and non-zero if it is. Other ones allow you to tell Valgrind if you
have your own allocator, give it hints about memory pools, mark memory
as undefined (useful if you implement your own containers that reuse
memory) or inaccessible, tell it about multiple stacks (for threading,
say), etc…

Valgrind can be passed suppression files (as a command-line parameter)
that indicate locations where particular errors or leaks should be
ignored. The nice thing with those is that you do not need modifying
the code at all, so you can work with third-party libraries (such as
SDL or Xlib) while focusing only on your own bugs.On Wed, May 20, 2009 at 2:29 PM, Mason Wheeler wrote:

–
http://pphaneuf.livejournal.com/

14

OK. So what does this magic byte pattern do instead, and what’s a suppression?

Valgrind can be passed suppression files (as a command-line parameter)
that indicate locations where particular errors or leaks should be

So it’s a lot like the RegisterExpectedMemoryLeak API, but in an
external file?

ignored. The nice thing with those is that you do not need modifying
the code at all, so you can work with third-party libraries (such as
SDL or Xlib) while focusing only on your own bugs.

Yeah. Ideally a library’s supposed to call RegisterExpectedMemoryLeaks
for its own memory and not leave it for consumers to worry about.>----- Original Message ----

From: Pierre Phaneuf
Subject: Re: [SDL] Memory leaks, output with debug SDL lib
On Wed, May 20, 2009 at 2:29 PM, Mason Wheeler <@Mason_Wheeler> wrote:

15

So it’s a lot like the RegisterExpectedMemoryLeak API, but in an
external file?

Something like that. The good thing is that the code can be written
without caring for Valgrind at all, so it makes it usable in a wide
variety of situations, but the downside is that it’s keyed per code
location rather than per-allocation, so it could be somewhat improved
(it does work fine for 99% of situations, though).

Yeah. Ideally a library’s supposed to call RegisterExpectedMemoryLeaks
for its own memory and not leave it for consumers to worry about.

Yep, that’s great as long as libraries are diligent about it, but
sometimes, you’ve got the straggler that didn’t or something… Nice
to have a fallback solution.

Valgrind does work with Delphi programs, too, it’s just a bit trickier
(has to be run on Wine, on Linux!). ;-)On Wed, May 20, 2009 at 3:08 PM, Mason Wheeler wrote:

–
http://pphaneuf.livejournal.com/

16

Yep, that’s great as long as libraries are diligent about it, but
sometimes, you’ve got the straggler that didn’t or something… Nice
to have a fallback solution.

Valgrind does work with Delphi programs, too, it’s just a bit trickier
(has to be run on Wine, on Linux!). :wink:

If this program’s so useful, why hasn’t anyone ported it to Windows?
Because honestly, as admirable as Linux is–and I’m not saying it’s
not–on technical and philosophical merit, it still has less than 1%
market share in desktop systems, which means that a Linux-only
programming tool isn’t in a position to do all that much good to
very many people.>----- Original Message ----

From: Pierre Phaneuf
Subject: Re: [SDL] Memory leaks, output with debug SDL lib

17

People will still buy leaky software because most don’t know any
better. If someone can make money without the effort involved in
making an application memory-safe, then the platform becomes what
Windows is today (or, at least, has the reputation for). I’m sure
there are a lot of commercial libraries like Valgrind though, for
those who do care and have the capital.

The other point to be made is that the 1% market share is for
consumers, not developers. Who does such a library help more? I do
think Valgrind should be ported, but it is a very heavy task for such
a tool.

Jonny DOn Wed, May 20, 2009 at 3:29 PM, Mason Wheeler wrote:

If this program’s so useful, why hasn’t anyone ported it to Windows?
Because honestly, as admirable as Linux is–and I’m not saying it’s
not–on technical and philosophical merit, it still has less than 1%
market share in desktop systems, which means that a Linux-only
programming tool isn’t in a position to do all that much good to
very many people.

18

If this program’s so useful, why hasn’t anyone ported it to Windows?
Because honestly, as admirable as Linux is–and I’m not saying it’s
not–on technical and philosophical merit, it still has less than 1%
market share in desktop systems, which means that a Linux-only
programming tool isn’t in a position to do all that much good to
very many people.

It’s mainly because it’s an x86 emulator in user-space, so it has a
lot of knowledge about the specifics of the Linux kernel (and of the
libc, as well). But to use valgrind+wine on Linux, your users don’t
need to switch to Linux, you just need VMware or a dual boot, and
you’re off to the races…On Wed, May 20, 2009 at 3:29 PM, Mason Wheeler wrote:

–
http://pphaneuf.livejournal.com/

19

Yep, that’s great as long as libraries are diligent about it, but
sometimes, you’ve got the straggler that didn’t or something… Nice
to have a fallback solution.

Valgrind does work with Delphi programs, too, it’s just a bit
trickier (has to be run on Wine, on Linux!). :wink:

If this program’s so useful, why hasn’t anyone ported it to Windows?

A programs usefulness isn’t defined by being available for Windows.
Contrary to popular belief, Windows isn’t the be-all & end-all of
systems for many people, and things can be relatively popular even if
they’re not on Windows.

Because honestly, as admirable as Linux is–and I’m not saying it’s
not–on technical and philosophical merit, it still has less than 1%
market share in desktop systems, which means that a Linux-only
programming tool isn’t in a position to do all that much good to
very many people.

Another way of looking at this would be; why are so many developers tied
to Windows when Linux/UNIX has such wonderful development tools?
Perhaps if people moved to a more secure OS where these tools are
available, there would be much less viruses and application problems??

Sorry if this comes across as flamebait, but I hate the idea that an
application must exist for Windows before it’s considered ‘real’ and
’successful’.On May 20, 2009 04:59:07 pm Mason Wheeler wrote:

----- Original Message ----
From: Pierre Phaneuf
Subject: Re: [SDL] Memory leaks, output with debug SDL lib

20

If this program’s so useful, why hasn’t anyone ported it to Windows?
Because honestly, as admirable as Linux is–and I’m not saying it’s
not–on technical and philosophical merit, it still has less than 1%
market share in desktop systems, which means that a Linux-only
programming tool isn’t in a position to do all that much good to
very many people.

Oh, also, there’s something like Valgrind that runs on Windows, called
Purify, but it’s commercial and works slightly differently (you have
to run your executable through it, and it generates an instrumented
binary from your normal one). I’ve heard much good from it too, but
from what I hear, while the executables generated by Purify are
slightly faster (but not much) than Valgrind, there’s a lot of time
spent waiting while generating the instrumented binaries, so people
tend to like Valgrind better, typically.On Wed, May 20, 2009 at 3:29 PM, Mason Wheeler wrote:

–
http://pphaneuf.livejournal.com/