Limitations of video overlay?

SDL Development
1

Hi,

I’m evaluating different possibilities for the implementation of a Linux multi-window video viewer application. At the moment, this looks like an “xvideo or sdl” decision to me. I never used SDL before, so I’ve little/no knowledge about it’s limitations.

The viewer application has to meet the following requirements:

  • MDI Application with QT GUI
  • simultaneous display of multiple videos, each in its own MDI subwindow.
  • arbitrary resolutions of video sources.
    ( At least, 5-8 MPixel resolutions have to be considered, but most commonly, resolutions around 0.5-2 MPixel will be used)
  • video sources may have RGB or YUV pixelformat, uncompressed.
  • Primary target platform is Linux, Secondary platform may be OSX.

In short, my question would be: What limitations of SDL do I have to expect, and how bad are they compared to XVideo?

I know that these requirements are quite high, and I expect some problems with them.

(XVideo-)Issues I’m already aware of are:

  • XVideo only supports a certain resolution for overlay video sources, which depends on the graphics card. I’m planning to work around this issue with a dynamic fallback to slower standard drawing mechanisms.
  • XVideo is limited by the number of xv ports provided by the graphics card driver. I also need a fallback solution, when the number of ports is exceeded.

Finally, these are the SDL issues I’m unsure about:

  • (Most Important) I’ve read posts about SDL not supporting reentrancy, which is said to make multiple overlays impossible. Is this true, without exception? I could think of having a global image consumer, which synchronizes ‘new frame’ events from multiple sources, so that only one overlay surface is updated at a time. Would you call this a reasonable approach?
  • I’ve been told that SDL does not have the xvideo size limitation. But SDL uses xv, when possible, right? Any detailled information (or link) about this topic would be great. I’m wondering about questions like: What is the magnitude of cpu utilization I have to expect, depending on the image size? What is the (default) fallback mechanism of SDL, when the allowed overlay size is exceeded?
  • Let’s assume that I’ve already finished my Qt/SDL-based application with all requirements met. (at least all that weren’t impossible to fulfill.) Would you expect difficulties porting the SDl part to OSX?

Thanks for anny comment!

Philipp Beyer

2

Hi,

I’m evaluating different possibilities for the implementation of a Linux multi-window video viewer application. At the moment, this looks like an “xvideo or sdl” decision to me. I never used SDL before, so I’ve little/no knowledge about it’s limitations.

The viewer application has to meet the following requirements:

  • MDI Application with QT GUI
  • simultaneous display of multiple videos, each in its own MDI subwindow.
  • arbitrary resolutions of video sources.

This won’t be possible, this is a limitation of the underlying video
driver. All video driver Xv implementations that I know of have a size
limit. If you really need support for playing huge videos, you should
craft your own hardware environment.

( At least, 5-8 MPixel resolutions have to be considered, but most commonly, resolutions around 0.5-2 MPixel will be used)

  • video sources may have RGB or YUV pixelformat, uncompressed.
  • Primary target platform is Linux, Secondary platform may be OSX.

In short, my question would be: What limitations of SDL do I have to expect, and how bad are they compared to XVideo?

I know that these requirements are quite high, and I expect some problems with them.

(XVideo-)Issues I’m already aware of are:

  • XVideo only supports a certain resolution for overlay video sources, which depends on the graphics card. I’m planning to work around this issue with a dynamic fallback to slower standard drawing mechanisms.
  • XVideo is limited by the number of xv ports provided by the graphics card driver. I also need a fallback solution, when the number of ports is exceeded.

Finally, these are the SDL issues I’m unsure about:

  • (Most Important) I’ve read posts about SDL not supporting reentrancy, which is said to make multiple overlays impossible. Is this true, without exception? I could think of having a global image consumer, which synchronizes ‘new frame’ events from multiple sources, so that only one overlay surface is updated at a time. Would you call this a reasonable approach?

Yeah, with SDL 1.2 it’s the only approach possible: create multiple
processes, each one opens its own window, and sychronize them by hand.

  • I’ve been told that SDL does not have the xvideo size limitation. But SDL uses xv, when possible, right? Any detailled information (or link) about this topic would be great. I’m wondering about questions like: What is the magnitude of cpu utilization I have to expect, depending on the image size? What is the (default) fallback mechanism of SDL, when the allowed overlay size is exceeded?

SDL uses Xv to implement YUV support under X. If Xv creation fails
(because of an unsupported video format, because the size is too
big…) you get a software overlay: the YUV data is converted to RGB
in software by SDL and the sent to a “normal” window.

  • Let’s assume that I’ve already finished my Qt/SDL-based application with all requirements met. (at least all that weren’t impossible to fulfill.) Would you expect difficulties porting the SDl part to OSX?

Thanks for anny comment!

I should add that the primary speed limitation of video playback
usually comes from the bus transfer speed. Therefore, you should
absolutely not transfer anything as RGB, and keep the data as YUV
instead. SDL has this YUV -> RGB software conversion code which works
around size/format limitations, but you don’t want to use it because
it incurs RGB transfers over the bus. Your average YV12/NV12 data is
12 bits per pixel, RGB will be 24 bits at least, and the hardware
probably needs 32 bits per pixel (because of padding), so we’re
talking 2.6 times as much data.

That is, unless you have no real time playback constraint, in which
case software conversion could be fit.

StephaneOn Thu, Aug 14, 2008 at 1:55 PM, Beyer, Philipp <Philipp.Beyer at alliedvisiontec.com> wrote:

3
  • (Most Important) I’ve read posts about SDL not supporting reentrancy, which is said to make
    multiple overlays impossible. Is this true, without exception? I could think of having a global
    image consumer, which synchronizes ‘new frame’ events from multiple sources, so that only one
    overlay surface is updated at a time. Would you call this a reasonable approach?

Yeah, with SDL 1.2 it’s the only approach possible: create multiple processes, each one opens its
own window, and sychronize them by hand.

Is there also a ‘one overlay per process’ limitation, in addition to the ‘no reentrancy’ limitation?
Would it be ok to have one thread updating multiple overlays on a rotating basis?

4

SDL has a “one window per process” limitation, which implies that you
only get a single overlay window.

However, if you bang the Xv api directly, you can have multiple
windows/overlays per process.

StephaneOn Thu, Aug 14, 2008 at 2:47 PM, Beyer, Philipp <Philipp.Beyer at alliedvisiontec.com> wrote:

  • (Most Important) I’ve read posts about SDL not supporting reentrancy, which is said to make
    multiple overlays impossible. Is this true, without exception? I could think of having a global
    image consumer, which synchronizes ‘new frame’ events from multiple sources, so that only one
    overlay surface is updated at a time. Would you call this a reasonable approach?

Yeah, with SDL 1.2 it’s the only approach possible: create multiple processes, each one opens its
own window, and sychronize them by hand.

Is there also a ‘one overlay per process’ limitation, in addition to the ‘no reentrancy’ limitation?
Would it be ok to have one thread updating multiple overlays on a rotating basis?

5

SDL has a “one window per process” limitation, which implies that you only get a single overlay window.

However, if you bang the Xv api directly, you can have multiple windows/overlays per process.

Stephane

Thanks a lot!

6

“Stephane Marchesin” <stephane.marchesin at wanadoo.fr> wrote:

SDL has a “one window per process” limitation, which implies that you
only get a single overlay window.

However, if you bang the Xv api directly, you can have multiple
windows/overlays per process.

use xvinfo to find out what your hardware supports. mine for example
shows:

love ~ # xvinfo
X-Video Extension version 2.2
screen #0
Adaptor #0: “Intel® Textured Video"
number of ports: 16
port base: 89
operations supported: PutImage
supported visuals:
depth 24, visualID 0x23
depth 24, visualID 0x24
depth 24, visualID 0x25
depth 24, visualID 0x26
depth 24, visualID 0x27
depth 24, visualID 0x28
depth 24, visualID 0x29
depth 24, visualID 0x2a
number of attributes: 2
"XV_BRIGHTNESS” (range -128 to 127)
client settable attribute
client gettable attribute (current value is 0)
“XV_CONTRAST” (range 0 to 255)
client settable attribute
client gettable attribute (current value is 0)
maximum XvImage size: 1920 x 1088
Number of image formats: 4
id: 0x32595559 (YUY2)
guid: 59555932-0000-0010-8000-00aa00389b71
bits per pixel: 16
number of planes: 1
type: YUV (packed)
id: 0x32315659 (YV12)
guid: 59563132-0000-0010-8000-00aa00389b71
bits per pixel: 12
number of planes: 3
type: YUV (planar)
id: 0x30323449 (I420)
guid: 49343230-0000-0010-8000-00aa00389b71
bits per pixel: 12
number of planes: 3
type: YUV (planar)
id: 0x59565955 (UYVY)
guid: 55595659-0000-0010-8000-00aa00389b71
bits per pixel: 16
number of planes: 1
type: YUV (packed)

best regards …
clemens

7

I’m evaluating different possibilities for the implementation of a Linux
multi-window video viewer application. At the moment, this looks like an
"xvideo or sdl" decision to me. I never used SDL before, so I’ve little/no
knowledge about it’s limitations.

The viewer application has to meet the following requirements:

  • MDI Application with QT GUI

Look at Qt 4.4’s Phonon module - I believe it will do everything you
want without having to use SDL at all, and probably only 100 or so
lines of code. On X11 Qt will use xvideo to accelerate YUV->RGB
conversion or even use an OpenGL shader if your graphics card has
suitable support.

Under OSX, Qt will use the Quicktime API for acceleration. It really
should be very simple to do.

Cheers,

Tom

8

Is there also a ‘one overlay per process’ limitation, in addition to the ‘no reentrancy’ limitation?

No, you can have multiple overlays, you just can’t have multiple top level
windows in SDL 1.2.

It sounds like if you’re using Qt, Phonon is the way to go. :slight_smile:

See ya!
-Sam Lantinga, Lead Software Engineer, Blizzard Entertainment