Opaque Objects: summary

Eddy, David,

I’ve re-read carefully the previous posts concerning Opaque Objects.
I also understand both of your point of view.

If I may, here is my conclusion, mixing David’s point of view, yours, and some personal notes:

  • In a team, not all developers have the same level of competence,
    so to avoid problems, a stronger API would be necessary so that
    you don’t spend too much time finding hard-to-debug errors (like
    memory errors)

  • SDL is aimed at simplicity. It is coded in C, and adding extra checks
    or rewriting most functions would lead to “heaviness” which most
    SDL developer just don’t want.

  • If your team uses C/C++, then you ought to write your own wrapper
    aimed at less skilled programmers. Maybe it can be a good manner
    for them to get into SDL’s philosophy, and use it without the wrapper
    in the future.

  • If you can use a higher level language, the problem is almost solved.

  • You cannot impose a whole revolution in the SDL API, the answers you got
    (maybe not as professional as they should have been, I apologize for that)
    cleary show the reluctance to go in such a direction, although your
    motivation is completly well-founded.
    The reason is that you can bring more security on top of a low-level API
    so that both “hard coders” and “light coders” can find their interest, but
    the contrary would prevent “hard coders” from beeing satisfied.

Hope I don’t miss a major point of the discussion and that you, Eddy, can
find a suitable technical solution to your problem.

Best Regards,
Julien

----- Message d’origine ----De : Edward Cullen
? : A list for developers using the SDL library. (includes SDL-announce)
Envoy? le : Lundi, 21 Juillet 2008, 3h42mn 54s
Objet : Re: [SDL] Opaque Objects / Pointers

David Olofson wrote:

As a user of an API that ALL you generally care about; does it do
the job I want? Is it easy to use? The rest, you trust the developer
to make sensible choices on your behalf.

Indeed, but what I’m talking about here is not whether the application
developer really cares about the actual details of the
implementation, but rather, what impact the API design has on the
performance of the implementation. I know all too well that lots of
developers couldn’t care less about performance, but that’s a luxury
many other developers don’t have.

What’s been discussed so far is nothing that would have any
significant performance impact, but in a large system or application,
these things tend to build up. One needs to keep an eye on that.

Absolutely! That’s why I asked for peoples views on the use of opaque
objects. As an engineer, I deal in trade-offs; the trade-off here, is
(potential) performance cost against (potential) benefit to the
’majority’ of users (that is, in my experience ‘good’ programmers are in
the minority).

This is another important aspect of my motivation; ‘the majority’ are
(unfortunately) the the ones who get things done. So as the
minority, the most productive use of our time, is to try to make it
harder for the majority to do things badly. Yes, you may refer to me as
CynicalEddy.

[…]

Well, if you step back and look at the problem: When do you want
to clone an SDL_Surface? If you ever want to do it, why?

It turns out that it’s pretty much a non-issue, because the few
times you actually do want a copy, you actually want a slightly
different version, such as a version suitable for fast blitting to
the screen (SDL_DisplayFormat*()), or a temporary version in some
specific, handy format for some direct pixel level manipulation.
So, if it’s a rare thing, you’re not really gonna be fussed if it
isn’t particularly ‘quick’ are you? But when you DO need it, you’re
probably gonna be pissed that it’s not there at all.

Performance isn’t the issue here. Library size is.

Why isn’t this in the FAQ? (Or am I just blind?)

Q: Can you please add function SDL_Xyz() to SDL?
A: If function SDL_Xyz() can be implemented in a separate
add-on library (ie doesn’t depend on SDL internals), it
most probably does not belong in the SDL core library.

Most of us really like the idea of keeping the SDL core library as
lean and mean as possible, without impairing it’s usability.

As do I. I fully appreciate the ‘thin’ nature of SDL. From what I
understand, keeping the API minimalistic is ‘good design’.

Personally, I sometimes think that it would be very handy with some
general SDL_tools library with all sorts of handy stuff. However, it
seems that SDL developers (like most developers) tend to roll their
own library of handy tools, or just keep a copy-paste library of
handy code.

Either there isn’t enough interest in creating and mantiaining an
SDL_tools library, or we can’t agree on how to do it, or both. So, it
doesn’t happen - and most of us are happy campers anyway. :slight_smile:

Yes. This goes back to the balance; how much is enough for the API to be
’complete’, without being over-burdened.

Actually, that’s a good question - is there a certain minimal API that
you should always have, even if some of the functions would be used
very rarely?

That is, if copying a particular structure is a valid operation, then,
perhaps, one should always provide a function to do it, even if it’s
only ever used in 1:10 programs?

One interesting, related, issue is symmetry in APIs (and in general).
Ever done any Python? It freaks the HELL out of me; it’s asymetrical…
I SOOO want to put a } or something at the end of a function definition
or other block.

As a concrete example - one of the most common ‘beginner’ questions on
the group is ‘why do I segfault when I SDL_FreeSurface() the display
surface’ - programmers (well, C programmers) seem to expect symmetry,
so there is potentially benefit to providing certain operations that
maybe aren’t strictly necessary…

But what about using function-style macros?

Yeah, that, or (if we assume C99) even inline functions - but to me,
and apparently, quite a few others on this list, it’s just extra
typing for the application coder.

Besides, if it’s implemented like that, there’s no binary level
abstraction, so there goes (IMHO) the only real advantage. Sure, it’s
as fast as what we do now, but the only “bonus” is us doing some
extra typing, so… :wink:

True, but then, let’s face it, you’re working at such a low level,
that virtually any change you make to the implementation risks
breaking binary compatibility. The only change you can make behind an
opaque object is to add members to the end of the struct, which you
can’t do if you have open structs.

How do you actually hide pointers when dealing with a C API in
C? :wink:

(Yeah, I know it’s possible, and I’ve even tried it in some of my
older APIs - but it tends to cause more problems than it solves.)
How so?

Well, this is actually going one step further, so it’s not really
about opaque pointers.

Anyway, I’ve done a few APIs using opaque handles (internal table
indices and the like), and they’re kind of handy in small, simple
applications; you can do things like passing -1 for “all objects” and
stuff, and it’s trivial to deal with ranges of objects.

Beyond that however, they offer little more than making certain
mistakes easier to track. Actually, with proper debugging tools (like
Valgrind), this design makes applications harder to debug than
direct pointers, while offering nothing that these tools cannot do
anyway.

So… I can’t really think of any good reason to recommend this kind
of design for anything. It doesn’t really help C and the like, and a
high level language can usually offer much better solutions anyway.

Hmm… I’ve looked into this. I think that, as long as your handles are
actually pointers (rather than arbitrary or obfuscated values) then
modern debuggers can cope. For example, in GDB, you can have an opaque
struct and as long as you have full debugging info (in the lib + app)
you can examine the members of the structures as if the object were open.

GDB also supports ‘invoking’ of preprocessor macros, so you can
comfortably examine the members of an opaque structure.

[…]

It’s a problem C was never designed to solve, so it’s kind of hard
to solve it in C without adding overhead or making things
cumbersome… I just don’t think it’s worth the effort. :slight_smile:
Again, goes back to the library trade-off and the shifting
responsibilities thing; if 99% users of your library are going to do
something you didn’t do because of ‘efficiency’ concerns, then
you’re not actually saving anything.
[…]

Well, let’s turn it around a bit: Who needs to access the information
at all? And, what level of “safety” is required for those who do
need access?

As most people won’t need to touch the fields of an SDL_Surface
anyway, why not replace the SDL_Surface pointers throughout
the “normal” API with an opaque pointer type. (Give them
SDL_SurfaceWidth() etc; only what’s really needed.)

Then offer a typecast macro that returns essentially a pointer to the
current SDL_Surface. The declarations for this could be in a separate
header that isn’t included in “normal” applications.

That way, us hardcore low level guys can have it our way, and you can
have your build scripts yell att untrusted programmers if they ever
try to use that typecast macro. :wink:

Yeah, that’s the scheme used in Xlib… I think that providing SDL_Debug
macros or a including a header with the full definition would be a fair
compromise, given the low-level nature of SDL.

I think this discussion has run its course, so I’m gonna crack-on with
it and see what comes out of the other end…

Eddy


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

  _____________________________________________________________________________ 

Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr

julien CLEMENT wrote:

  • In a team, not all developers have the same level of competence,
    so to avoid problems, a stronger API would be necessary so that
    you don’t spend too much time finding hard-to-debug errors (like
    memory errors)

Change ‘In a team’ to ‘In general’.

Change ‘a stronger API’ to ‘a more intuitive and robust API’.

  • SDL is aimed at simplicity. It is coded in C, and adding extra checks
    or rewriting most functions would lead to “heaviness” which most
    SDL developer just don’t want.

I believe the interpretation of simplicity here is incorrect.

My understanding is that the majority SDL users want a simple-to-use
API that gives them access to OS services in a consistent way on many
platforms. I doubt whether performance is actually their main concern -
I personally have never profiled SDL itself, have you? I believe that
most people will simply take it on trust that SDL performs reasonably
well, based on the names behind it and some of the products it’s been
used in, such as UT2k4…

Implementing opaque objects to many of the larger structures would NOT
add much, if any overhead, as the majority of the structs and functions
concerned already pass-by-reference; it would be a ‘cosmetic’ change
aimed at reducing the cognitive load placed on the developer. (By
cognitive load, I mean the amount “I need to care about HOW this
works”). Note the word ‘reduce’ is does not mean ‘eliminate’; you can
never get away from the fact that this is a C API, something that I
have always been aware of (in spite of being accused of the opposite
many times).

  • If your team uses C/C++, then you ought to write your own wrapper
    aimed at less skilled programmers. Maybe it can be a good manner
    for them to get into SDL’s philosophy, and use it without the wrapper
    in the future.

But surely, if SDL’s API ‘is just right’ then I wouldn’t do that, I
would spend the effort on training, as the return would be greater?

  • If you can use a higher level language, the problem is almost solved.

Of course you can, but that’s not the point of the discussion, it is
about the C API and how it is best to express it.

  • You cannot impose a whole revolution in the SDL API, the answers you got
    (maybe not as professional as they should have been, I apologize for that)
    cleary show the reluctance to go in such a direction, although your
    motivation is completly well-founded.
    The reason is that you can bring more security on top of a low-level API
    so that both “hard coders” and “light coders” can find their interest, but
    the contrary would prevent “hard coders” from beeing satisfied.

Hmm… no one is trying to impose anything. I asked a question about a
particular technique commonly used in C APIs and whether it would
’improve’ SDLs APIs to make more use of it.

Your talk of revolution is misguided. Much of the discussion was centred
around the SDL_Surface structure. I deliberately chose this as an
example as is already semi-opaque. That is, the documentation for
SDL_Surface states that there are addition, undocumented, fields
(meaning that in order to find out the exact details, you have to refer
to the code). Therefore, moving SDL_Surface from semi-opaque to
fully-opaque would not be a revolution, merely an evolution.

To my mind, the discussion exposed 2 key facts:

  1. NOT knowing the exact layout of a structure can, in some situations,
    make debugging more difficult.

  2. That there are AT LEAST 2 techniques (preprocessor macros,
    ‘debugging’ option that makes object transparent) that can be used to
    mitigate this.

Also, both Xlib and the Win32 APIs make extensive use of opaque objects.
Given that one of SDLs primary goals is to abstract these APIs, then
there is a solid argument for continuing the use of opaque objects if
appropriate
. That is to say, if it weren’t for SDL, you’d be dealing
with opaque objects anyway. Also, one can’t help but acknowledge the
fact that 2 separate APIs BOTH use similar techniques to solve the
same problem, there must be some validity in the technique, or at
least enough to make its use worth exploring.

(And before you say, “yeah, but M$ just copied Xlib’s API in order to
save effort or to encourage UNIX developers to switch to Windows.”, the
point remains valid; if M$ could have come up with a 'better’
technique, then surely they would have used it? If the X developers
could have come up with something better, surely they would have used it
when they went from X10 to X11?)

Hope I don’t miss a major point of the discussion and that you, Eddy, can
find a suitable technical solution to your problem.

My goal was to try it out, to experiment. Because of time constraints,
I wanted to try to cut-down the number of iterations I would have to go
through, by asking about the use of the technique before I start
trying to apply it. The only ‘problem’ I have is that people seem to
discourage any attempt to make a change, no matter how well informed or
proven that change might be!

Eddy

Don’t be surprised if a lot of people disagree with you on that point. If
you want a wrapper that implements your interpretation of simplicity, MAKE
ONE! Don’t expect everyone else to do things “your way”. Tail wags dog.

Sheesh, can we put this to rest now?

JeffOn Mon July 21 2008 12:16, Edward Cullen wrote:

julien CLEMENT wrote:

  • SDL is aimed at simplicity. It is coded in C, and adding extra checks
    or rewriting most functions would lead to “heaviness” which most
    SDL developer just don’t want.

I believe the interpretation of simplicity here is incorrect.