SDL_movie

Hello all,

Is there such a thing as an SDL_movie project?
If not, what’s the best GPL-like solution for a movie player?

cheers,
K.

Kostas Kostiadis <kos climaxgroup.com> writes:

Is
there such a thing as an SDL_movie project?
If
not, what’s the best GPL-like solution for a movie player?

Have you looked into http://www.icculus.org/smpeg/ ? It can play MPEG movies.

Have you looked into http://www.icculus.org/smpeg/ ? It can play MPEG movies.

Is there nothing better than SMPEG? How about ffmpeg or whatnot?

–ryan.

Have you looked into http://www.icculus.org/smpeg/ ? It can play
MPEG movies.

Is there nothing better than SMPEG? How about ffmpeg or whatnot?

There’s nothing SDL-based that wraps ffmpeg. I’m currently doing a
project that involves movies also, and that’s what I’m using. It’s
not too bad to use just on its own, though I’ve unfortunately found a
striking lack of decent documentation for it. Most of what I’ve done
I’ve had to learn through example code I’ve seen, and also through
browsing the source code directly.

–ScottOn Jan 24, 2007, at 11:53 AM, Ryan C. Gordon wrote:

How about libtheora? ( http://theora.org/ )
Ogg Theora is nearing its 1.0 final release, which is a good thing for
us game developers because we’ll finally have a solid video equivalent
to Ogg Vorbis (ie. free as in free speech).On 1/24/07, Ryan C. Gordon wrote:

Have you looked into http://www.icculus.org/smpeg/ ? It can play MPEG movies.

Is there nothing better than SMPEG? How about ffmpeg or whatnot?

  • SR

Is there such a thing as an SDL_movie project?

SDL_Mpeg is about the only extension I can think off, although not
necessarily a movie (media) player solution.

If not, what’s the best GPL-like solution for a movie player?

Mplayer (http://www.mplayerhq.hu/) is probably the best GPLed media
player out there, and it can utilize SDL for video/audio output.On 1/24/07, Kostas Kostiadis wrote:

MPlayer has no library API though, which is what I believe he is looking for.On 1/24/07, JDE wrote:

On 1/24/07, Kostas Kostiadis wrote:

Is there such a thing as an SDL_movie project?

SDL_Mpeg is about the only extension I can think off, although not
necessarily a movie (media) player solution.

If not, what’s the best GPL-like solution for a movie player?

Mplayer (http://www.mplayerhq.hu/) is probably the best GPLed media
player out there, and it can utilize SDL for video/audio output.

  • SR

MPlayer has no library API though, which is what I believe he is looking for.

MPlayer is the software built on top of the FFMPEG library, which is the
best resource for playing video in my opinion.

FFMPEG (see in MPlayer) can play pretty much anything, it can also
encode into a variety of formats including some top of the line ones
like H264’s brother: x264.

It’s devellopment is moving a lot and the mailing list is one of the
busyiest I’m subscribed to.

I recommend it, however, the big downside of this is (last time I
checked) the documentation was either poor or simply not there, you
might have to dig into the source code to find out. Although, I once
saw a nice “read and play” tutorial and another for encoding, tell me if
you’re interested I’ll try to find the link again.

Simon

There’s nothing SDL-based that wraps ffmpeg.

Hmmm, that’s a crazy good idea man!

I’m currently doing a
project that involves movies also, and that’s what I’m using. It’s
not too bad to use just on its own, though I’ve unfortunately found a
striking lack of decent documentation for it. Most of what I’ve done
I’ve had to learn through example code I’ve seen, and also through
browsing the source code directly.

Yep, I agree, poor docs, and poor us for that! There were 2 tutorials
that were very nice, well documented within the code (as to why do this
now, etc…), but other than that, it’s a lib for expert i would say!

Simon

Simon wrote:

There’s nothing SDL-based that wraps ffmpeg.

Hmmm, that’s a crazy good idea man!

If someone wants to put together something that does that, I’ll be happy
to provide hosting for the project on icculus.org. There’re way too many
questions recently about decoding movies with SDL, and the existing APIs
are all really complex when all you want is basic playback…this might
be a really good project for someone to tackle.

–ryan.

Simon wrote:

There’s nothing SDL-based that wraps ffmpeg.

Hmmm, that’s a crazy good idea man!

If someone wants to put together something that does that, I’ll be
happy
to provide hosting for the project on icculus.org. There’re way too
many
questions recently about decoding movies with SDL, and the existing
APIs
are all really complex when all you want is basic playback…this
might
be a really good project for someone to tackle.

I would (having already STARTED to figure out the libav* api for my
own project) love to be involved in this, but I would need someone to
mail me (on or off list) about some of the more basic things involved
in maintaining proper coding style consistency with SDL, decent
library design, etc… But having this would mesh quite well with my
current project which is basically using SDL and ffmpeg for video
playback.

In short, I’ve never worked on/contributed to a big open source
project like this and would like someone to hold my hand for a little
while.

–ScottOn Jan 24, 2007, at 3:16 PM, Ryan C. Gordon wrote:

I totally agree with Ryan on this one…
I’m glad to see Scott Harper is already on his way with SDL+ffmpeg.
Scott don’t worry about creating the final lib…Once you have it all
working, creating a simple unit test, documenting and packaging it all up
should be the easy part :wink: You can just look at other libs (SDL_mixer,
SDL_ttf, etc.) to see how other people have done it.

I found the link posted by Simon Roby to theora.org quite interesting…
Since someone else is looking at SDL+ffmpeg I may go the SDL+theora way and
see what happens.

I think it’s fair to say that an SDL_movie project would be quite useful for
a lot of people, so I’m glad the ball is rolling.
Watch this space :wink:

BTW, when I originaly posted this, I was interested in a lightweight
SDL_movie player for simple playback of movies for my project(s).
I’m assuming people are not looking for a fully fledged VLC type player
(i.e. with DVD support, VCD support, streaming servers, etc.)

Cheers,
Kos> ----- Original Message -----

From: sdl-bounces@lists.libsdl.org [mailto:sdl-bounces at lists.libsdl.org] On
Behalf Of Ryan C. Gordon
Sent: 24 January 2007 22:17
To: A list for developers using the SDL library. (includes SDL-announce)
Subject: Re: [SDL] SDL_movie

Simon wrote:

There’s nothing SDL-based that wraps ffmpeg.

Hmmm, that’s a crazy good idea man!

If someone wants to put together something that does that, I’ll be happy
to provide hosting for the project on icculus.org. There’re way too many
questions recently about decoding movies with SDL, and the existing APIs
are all really complex when all you want is basic playback…this might
be a really good project for someone to tackle.

–ryan.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

Actually Xine is better. Mplayer’s codebase is a disaster, such that it
can’t even play DVDs with menus. The only good thing to come out of
mplayer… is ffmpeg. Which Xine uses, among other things.
-------------- next part --------------
A non-text attachment was scrubbed…
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20070125/bbd22a75/attachment.pgpOn Wed, 2007-01-24 at 11:16 -0800, JDE wrote:

Mplayer (http://www.mplayerhq.hu/) is probably the best GPLed media
player out there, and it can utilize SDL for video/audio output.

I’m glad to see Scott Harper is already on his way with SDL+ffmpeg.
Scott don’t worry about creating the final lib…Once you have it all
working, creating a simple unit test, documenting and packaging it all up
should be the easy part :wink: You can just look at other libs (SDL_mixer,
SDL_ttf, etc.) to see how other people have done it.

Heck, just give us source to something that works and we’ll clean it up
into a simple API. :slight_smile:

I found the link posted by Simon Roby to theora.org quite interesting…
Since someone else is looking at SDL+ffmpeg I may go the SDL+theora way and
see what happens.

While there were very good reasons for the way it turned out, I think
it’s fair to say that smpeg made the mistake in specializing. Ideally
something else does the work of “here’s a buffer of movie, figure it
out” and the SDL-specific parts are very very small.

SMPEG was just a wrapper over a freeware MPEG-1 decoder. Ideally the
backend handles whatever movie, with the ability to build in and out
just the codecs you need (which I believe ffmpeg allows), and we don’t
get seperate libraries named SDL_mpeg, SDL_theora, SDL_h263, etc.

But that’s just me…still, I think everything from QuickTime to libav
to DirectShow to gstreamer has proven this is a good way to go…all you
really want is to push data through and get back something SDL-like. We
shouldn’t bring anything more to the table than SDL compatibility.

–ryan.

?? Wed, 24 Jan 2007 16:52:37 -0500
Simon <simon.xhz at gmail.com> ???:

FFMPEG (see in MPlayer) can play pretty much anything, it can also
encode into a variety of formats including some top of the line ones
like H264’s brother: x264.
ffmpeg cannot do H264 AFAIK, since x264 is a different project, although
opensource as well. It supports H263 though.

However, playing with all these formats has a catch: most of the movie
formats are patented: MPEG2 (DVD) is patented, MPEG4 (XviD/DivX) is
patented, H264 (x264) is patented as well, not speaking of WMV. As
such these codecs (even the decoder part) cannot be legally used if you
don’t buy a license without risking to be sued.

Now you must decide why you would need SDL_movie: as a universal
wrapper around all existing opensource codecs, to make it suitable to
write programs like movie players that support lots of formats, or just
to provide support for movie playback in games and multimedia
applications.

If later, I would suggest looking at the Ogg Theora codec which is 100%
free from any legal issues. Its homepage is http://www.theora.org/ and
it provides a compression level comparable to MPEG4, but doesn’t have
square block artifacts (instead, when the bitrate is not enough the
image becomes kind of ‘more smooth’, e.g. loses small details).

In any case, even if you choose to use ffmpeg as well I would suggest
you make it either a separate package (e.g. SDL_movie and SDL_moviex or
something like that) with SDL_movie containing only free en/decoders
and SDL_moviex containing patent-encumbered codecs like ffmpeg. The
reason is that SDL_movie could be included into most Linux
distributions (most of SDL is a standard part of any decent Linux
distribution), and SDL_moviex left out to “3rd-party
package repositories”, like livna.org for Fedora Core.–
Greetings,
Andrew

I’m glad to see Scott Harper is already on his way with SDL+ffmpeg.
Scott don’t worry about creating the final lib…Once you have it
all working, creating a simple unit test, documenting and packaging
it all up should be the easy part :wink: You can just look at other
libs (SDL_mixer, SDL_ttf, etc.) to see how other people have done
it.

Heck, just give us source to something that works and we’ll clean it
up into a simple API. :slight_smile:

I have thought up this thing about 8 months now. Well the litle time I
have had from my other activities.

The basic requirements for the lib, which I called SDL_Video, are these:
It had to be easy to use, as avcodec and companies definitely are not
that. So what is needed then? Video streams are composed with
interleaving packets, which in turn form different data streams.
The basic stream types are video and audio. These I set as mandatory,
but I also thought that having support for subtitles wouldn’t hurt. So
I kind of thought that would my choices prevent me from adding
subtitles later if I wanted to. The solution where I ended adding
subtitles would be quite easy so I went on. While I studied all this
two quite big issues came out that IMHO are very important.

So here they are:

First:
There has to be an easy way to control AV sync or the library is not
very usefull.

–> My Solution

Timestamping the video frames and audio buffers outputted from the
decoder thread.

I eventually got to these SDL_Video types:
SDLV_Stream will be the container type for the video stream.
This data structure is returned when a video file is opened.
This contains the stream structures decoder thread id’s etc.
This also controls everything related to the video like looping etc.
SDLV_Buffer will contain audio data with the start time.
SDLV_Frame will contain one video frame with the display time.

SDLV_Frame can be extended a litle bit so that it can be used to support
subtitles in the future.

Second:
There are three different use cases for the video frames that I could
think as viable.

  1. Copying the frame to SDL_Surface.
  2. Copying the frame to SDL_Overlay.
  3. Copying the frame to OpenGL Texture.

–>My Solution

Some preliminary API for video frame usage.
Case 1:

Sint32
SDLV_CopyFrameToSurface(
SDLV_Frame frame,
SDL_Surface* to,
SDL_Rect* src,
SDL_Rect* dst );

Case 2:

Sint32
SDLV_CopyFrameToOverlay(
SDLV_Frame frame,
SDL_Overlay* to,
SDL_Rect* src,
SDL_Rect* dst );

Case 3: This one is kind of two folded, because I want
the possibility of OpenGL shader rendering and normal texture rendering.

Sint32
SDLV_CopyFrameToTexture(
SDLV_Frame frame,
GLuint texture_id,
SDL_Rect* src,
SDL_Rect* dst,
Sint32 texture_type,
Sint32 texture_data_type );

char* SDLV_GetGLSLShaderSource(
Sint32 texture_type,
Sint32 glsl_max_version );

The reason I don’t want one format is the unnecessary conversions. The
Overlay for example might need some YUV conversion, but that is only
between different YUV formats. That is a lot faster than converting to
RGB. In OpenGL we have even more alternatives. We might want to convert
the frame as an normal RGB texture or we might want full blown (I mean
one of each yuv-component per pixel) YUV texture with color matrix
conversion. The most advanced one is the shader version. It is very
easy to write a function with couple of parameters that will calculate
the filtered RGB value of the given texture coordinates and the number
of the used texture unit.

So the GLSL shader source emited will contain function with the
following GLSL API:

either this:
vec3 SDLV_GetTexturePixel( Sampler2D src, vec2 coords );
or this:
vec3 SDLV_GetTextureRectanglePixel( SamplerRect2D src, vec2 coords );
depending on the texture_type used.

So there might be some bad types, because I didn’t check wheter they
were correct or not, but this was only to give some base of my ideas.
So if there is someone, who has more ideas etc this is quite good
starting point.

I found the link posted by Simon Roby to theora.org quite
interesting… Since someone else is looking at SDL+ffmpeg I may go
the SDL+theora way and see what happens.

While there were very good reasons for the way it turned out, I think
it’s fair to say that smpeg made the mistake in specializing. Ideally
something else does the work of “here’s a buffer of movie, figure it
out” and the SDL-specific parts are very very small.

SMPEG was just a wrapper over a freeware MPEG-1 decoder. Ideally the
backend handles whatever movie, with the ability to build in and out
just the codecs you need (which I believe ffmpeg allows), and we
don’t get seperate libraries named SDL_mpeg, SDL_theora, SDL_h263,
etc.

libavcodec can decode most of the used video formats so it is good
way for SDL video library. It would be good if SDL_Video could be
extended to also do some simple on the fly encoding, but I wouldn’t
lose my sleep if that would need a completly unrelated library.

But that’s just me…still, I think everything from QuickTime to
libav to DirectShow to gstreamer has proven this is a good way to
go…all you really want is to push data through and get back
something SDL-like. We shouldn’t bring anything more to the table
than SDL compatibility.

As I stated there is one thing that states against this and it is the
needed conversion. Because there are viable paths were the conversion
can be eased a lot or be skiped all together.

We agree in that that the frames has to be as easy to use as
SDL_Surfaces, but because of their unique needs like sync to audio they
IMHO need own container type.

So from that I concluded to the SDLV_Frame and SDLV_Buffer types.
The Buffer type is quite poorly named, but that was not so important for
me when I thought all this up.

So what everybody think about this?On Thursday 25 January 2007 20:29, Ryan C. Gordon wrote:

MPlayer has no library API though, which is what I believe he is
looking for.

MPlayer is the software built on top of the FFMPEG library, which is
the best resource for playing video in my opinion.

FFMPEG (see in MPlayer) can play pretty much anything, it can also
encode into a variety of formats including some top of the line ones
like H264’s brother: x264.

The x264 that you are referring to is encoding library.
The codec type is still h264.

Mencoder uses the x264-library for encoding and ffmpeg allows this too,
although it has it’s own h264 encoder.

First two lines of the libavcodec/x264.c of ffmpeg
/*

  • H.264 encoding using the x264 library

It’s devellopment is moving a lot and the mailing list is one of the
busyiest I’m subscribed to.

I recommend it, however, the big downside of this is (last time I
checked) the documentation was either poor or simply not there, you
might have to dig into the source code to find out. Although, I once
saw a nice “read and play” tutorial and another for encoding, tell me
if you’re interested I’ll try to find the link again.

Most of the tutorials are hopelesly outdated. At least the ones that I
found out.

The following files ffmpeg.c, ffplay.c & output_example.c
in this page
http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/files.html
are interesting as is the rest of that documentation.On Wednesday 24 January 2007 23:52, Simon wrote:

The x264 that you are referring to is encoding library.
The codec type is still h264.

Mencoder uses the x264-library for encoding and ffmpeg allows this too,
although it has it’s own h264 encoder.

First two lines of the libavcodec/x264.c of ffmpeg
/*

  • H.264 encoding using the x264 library

Ahhh!.. Thanks for bringing a correction, I’m simply an amateur when
it comes to video!

Most of the tutorials are hopelesly outdated. At least the ones that I
found out.

The following files ffmpeg.c, ffplay.c & output_example.c
in this page
http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/files.html
are interesting as is the rest of that documentation.

The big problem with FFMPEG still is that you need to “invest” so much
time (think of an amateur like me) just in order to get the basics, but
your not yet getting started.

I believe having prior knowledge of what exactly is a codec, how it’s
made, its purpose, what are the different processes involved in decoding
and encoding video would greatly help to understand the docs and
sources… sometimes, it seems experts address themselves to a expert
public (it probably saves them time, where we all have to pay). Doesn’t
bother me too much, just my opinion… just think how much popular
FFMPEG could become with a doc like SDL’s or PHP’s!

Simon

such these codecs (even the decoder part) cannot be legally used if you
don’t buy a license without risking to be sued.

Btw, anybody know how they cost and what’s involved? Just to give an
idea of how much “commercial” a game would have to be to pay for such a
licence? ;D

Now you must decide why you would need SDL_movie: as a universal
wrapper around all existing opensource codecs, to make it suitable to
write programs like movie players that support lots of formats, or just
to provide support for movie playback in games and multimedia
applications.

I think we have to look broad.

Times are coming where the user is generating movies more and more,
cameras are attached to many (if not most) computers today and a game
could definately take advantage of that.

You could have you in-game central station showing all you MMO players
the mission of the day, recorded by the game master at his house. (He
dressed like a general, said the message in front of his camera, clicked
OK in the game and the game analysed, then recompressed the movie and
sent it to the server, etc).

Also, SDL is not just used for gaming. For example, I use it as a
nicier substitute for GLUT (as I do heavy OpenGL). And I could decide
to make a new chatting program like MSN, including webcam stuff.

You know just like me inspiration is endless and movies will
definately be part of our future gaming experience, and this, for me, is
the only reason I need to believe movie encoding and decoding are both
necessary.

If later, I would suggest looking at the Ogg Theora codec which is 100%
free from any legal issues. Its homepage is http://www.theora.org/ and
it provides a compression level comparable to MPEG4, but doesn’t have
square block artifacts (instead, when the bitrate is not enough the
image becomes kind of ‘more smooth’, e.g. loses small details).

This is something to be looked into. My first FFMPEG project was to
first render specific movies in POV-Ray: movies that would try to
produce artefacts in different codecs and compare the decoded frame with
the original, thus finding the best codec for this kind of movie. Kinds
of movies are what you see all the time on the web, colors slowly fading
to another color, but the artifact doesn’t show the fade and just
abruptly changes, squares artifacts, etc…

Obviously, my aim was to make a benchmark of this as well and then
provide the results showing overall, in any kind of video situation,
which is the best codec… (h264, yea maybe, i wanna make sure!)

In any case, even if you choose to use ffmpeg as well I would suggest
you make it either a separate package (e.g. SDL_movie and SDL_moviex or
something like that) with SDL_movie containing only free en/decoders
and SDL_moviex containing patent-encumbered codecs like ffmpeg. The
reason is that SDL_movie could be included into most Linux
distributions (most of SDL is a standard part of any decent Linux
distribution), and SDL_moviex left out to “3rd-party
package repositories”, like livna.org for Fedora Core.

Sounds good, I would give it a very different name to avoid the
confusion too.

Simon

The basic requirements for the lib, which I called SDL_Video, are these:
It had to be easy to use, as avcodec and companies definitely are not
that. So what is needed then? Video streams are composed with
interleaving packets, which in turn form different data streams.
The basic stream types are video and audio. These I set as mandatory,
but I also thought that having support for subtitles wouldn’t hurt. So
I kind of thought that would my choices prevent me from adding
subtitles later if I wanted to. The solution where I ended adding
subtitles would be quite easy so I went on. While I studied all this
two quite big issues came out that IMHO are very important.

So here they are:
[…]

You’re ideas seem to follow a correct path (at least for me at this time
of the day, or night actually).

Let me bring an idea though…

You have defined the name of some container, this is perfect, though, I
think we should then define how the content of the container is going to
be “expressed”.

For example, I want to stream all that through a server I already made
with SDL_net. The client would receive the packets and put them one by
one in the container. And is it going to play as it receives (then how
much time/buffer should it wait for) or play once ready? Let say I want
it to play fullscreen; easy, just output it using SDL directly to the
video, but if I’m using opengl? And worse, what if I want a 3d TV in a
3d world to display the movie?

So I think it is important to abstract the destination of all stream
types (sound, video, subtitles, custom streams(with data?) and other
streams).

Of course, once abstract, it might become very difficult to make the TV
example happen so it can be done later. If we can concentrate on
fullscreen playback without subtitles (which can be made by the
programer the way they prefer: SDL_ttf or other). But at least this
way, we have a core and all the tuff stuff about video decoding is done,
next is the video displaying, on which more people can work as we’re all
used at dealing with the screen’s pixels!

Finally, I’d like to take a line or two to say a warm Good Luck to this
project as I really look “forward” to “play” with it, and even if I
don’t have much time to dedicate to any C/C++ projects, I’ll try to help
in my own way. I’m already willing to do the documentation or
translation of it in french. =)

Cheers,
Simon