Real-time Video Overlay

I am trying to find a library that will allow me to easily place an overlay
that is generated on a laptop computer on a live video stream that is coming
into the laptop (compressed or uncompressed). The final product will be
output to an LCD. Will SDL do that for me? What would a typical latency be
for that? The key is low latency in my application. With that in mind, I
believe uncompressed video will be best, but correct me if I am wrong.

Anyway, thanks for the information!–
Jonathan R. Haws
Student Engineer
Space Dynamics Laboratory
Jonathan.Haws at sdl.usu.edu
(435)797-4629

It depends…

Firstly how fast is the network connection?
How much of the total bandwidth is dedicated to this video feed?
How fast the memory/CPU/GPU is in the showing/sending machine?

In some cases it’s faster to use fast and light
compression/decompression.
Some environments will be faster without compression.

For example 1GB-10GB networks where the transfer time for bigger amounts
of data will go down and the decompression latency could even exceed
the transfer time of the uncompressed frame.

In practice you need to test your environment and find out the faster
way of doing it.

PS. I mention the GPU because it can use what ever image format your
sender is producing unlike video overlays. So no conversions needed →
lower latency.On Friday 15 February 2008, Jonathan Haws wrote:

I am trying to find a library that will allow me to easily place an
overlay that is generated on a laptop computer on a live video stream
that is coming into the laptop (compressed or uncompressed). The
final product will be output to an LCD. Will SDL do that for me?
What would a typical latency be for that? The key is low latency in
my application. With that in mind, I believe uncompressed video will
be best, but correct me if I am wrong.

Anyway, thanks for the information!

Firstly how fast is the network connection?
How much of the total bandwidth is dedicated to this video feed?
How fast the memory/CPU/GPU is in the showing/sending machine?

The RF link currently can transmit the video with very low latency. I don’t have the exact numbers right now, but I do know that there aren’t any latency issues for the current video system (coming down from the helicopter and being displayed). Enough bandwidth is allocated on the RF link for the video. All video processing is done on the laptop, thus the latencies that will kill the app are on the laptop.
The laptop computer currently is a 1.7GHz Intel Centrino with 1GB of RAM. The GPU is on an ATI Mobile Radeon (I don’t have the exact specs with me right now). However, the laptop will be replaced, so I can get pretty much anything I want/need in it to do this.

For example 1GB-10GB networks where the transfer time for bigger amounts
of data will go down and the decompression latency could even exceed
the transfer time of the uncompressed frame.

We definitely don’t have 1GB on the RF link. The video feed will come directly into the laptop from the RF receiver. Since the connection is fairly slow (we are using FreeWave radios) compared to high speed data connections I believe that the uncompressed transfer time will be negligible.

PS. I mention the GPU because it can use what ever image format your
sender is producing unlike video overlays. So no conversions needed →
lower latency.

Since you mention the GPU I have another, related question. I was intending on using the GPU to handle most of the processing. The idea was that the GPU would hold the frame in memory and I would send my overlay to it. At that point the GPU would mix the overlay and the frame and output it to the LCD. Is that viable? I would think that method would give the lowest latency. Also, I wouldn’t even have to have it overlay every frame - even just every other or every third. Anyway, if I did that would SDL still work like that or would I be on my own?

Thanks for the help and answers! I appreciate it!–
Jonathan R. Haws
Jonathan.Haws at sdl.usu.edu


From: sdl-bounces@lists.libsdl.org on behalf of Sami N??t?nen
Sent: Mon 2/18/2008 4:08 AM
To: A list for developers using the SDL library. (includes SDL-announce)
Subject: Re: [SDL] Real-time Video Overlay

On Friday 15 February 2008, Jonathan Haws wrote:

I am trying to find a library that will allow me to easily place an
overlay that is generated on a laptop computer on a live video stream
that is coming into the laptop (compressed or uncompressed). The
final product will be output to an LCD. Will SDL do that for me?
What would a typical latency be for that? The key is low latency in
my application. With that in mind, I believe uncompressed video will
be best, but correct me if I am wrong.

Anyway, thanks for the information!

It depends…

Firstly how fast is the network connection?
How much of the total bandwidth is dedicated to this video feed?
How fast the memory/CPU/GPU is in the showing/sending machine?

In some cases it’s faster to use fast and light
compression/decompression.
Some environments will be faster without compression.

For example 1GB-10GB networks where the transfer time for bigger amounts
of data will go down and the decompression latency could even exceed
the transfer time of the uncompressed frame.

In practice you need to test your environment and find out the faster
way of doing it.

PS. I mention the GPU because it can use what ever image format your
sender is producing unlike video overlays. So no conversions needed →
lower latency.


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

-------------- next part --------------
A non-text attachment was scrubbed…
Name: not available
Type: application/ms-tnef
Size: 6342 bytes
Desc: not available
URL: http://lists.libsdl.org/pipermail/sdl-libsdl.org/attachments/20080218/7e2d3e7d/attachment.bin

This is viable, but you should use OpenGL or DirectX for it. SDL
doesn’t accelerate enough of the pipeline consistently enough for
this.
-:sigma.SBOn Mon, Feb 18, 2008 at 8:38 AM, Jonathan Haws <Jonathan.Haws at sdl.usu.edu> wrote:

Since you mention the GPU I have another, related question. I was intending on using
the GPU to handle most of the processing. The idea was that the GPU would hold the
frame in memory and I would send my overlay to it. At that point the GPU would mix the
overlay and the frame and output it to the LCD. Is that viable? I would think that method
would give the lowest latency. Also, I wouldn’t even have to have it overlay every frame -
even just every other or every third. Anyway, if I did that would SDL still work like that or
would I be on my own?