IMG_Load performance was Problem combining sw_scale withblit operations on surface

Donny Viszneki wrote:

A little about what I’m trying to do:
I have a v4l program that captures 640x480 frames at 30fps from a
security camera. When the program detects motion, it compresses the
frames into individual JPEG files. This all works great. I’m trying
to build a simple media player type application to review and/or
animate the images at up to 30 frames/second. This means I have to
accomplish everything for each frame in less than 33mS.

This is a silly way to go about things. Don’t wait for motion and then
capture a JPEG, just capture to a good VBR video, and write out extra
data about when motion occurs. You can get higher image quality at
higher compression ratios than in your current scheme.

You say that, but I believe that it’s only because you don’t fully know what
I want to accomplish here. On top of that, I’m tinkering in the manner that
I do which isn’t always what others [experts] perceive as sensible.
Regardless, it’s how I learn and I find that it works pretty well for me.
:slight_smile: The ultimate goal for me here is to learn as much as possible about how
stuff works. Eventually I’ll move on to something else and this
project/obsession will die of neglect.

I don’t want anything captured most of the time, only during motion (as I
define it at capture time). The JPEG frames are less than 100K and are of
good quality. This represents a compression ration of 10:1 since capture
frames are nearly 1MB, which I find to be quite acceptable. libJPEG
compresses most images in under 15mS on my machine leaving plenty of time
for my motion analyzer to run on each frame in real time. I will be writing
extra information while I’m capturing, but only to identify a [very small]
subset of the captured frames.

This brings us to the main requirement: the smallest amount of delay
possible from capture to display on a remote X desktop when viewing the most
recent data, while still allowing very responsive VCR type control (pause,
instantly jumping back and forth by varying amounts, etc). So far I’ve been
able to keep things well below 500mS. I also want the ability to easily put
together small video segments with ffmpeg that start/stop at any capture
frame. Each frame has text that was overlayed at capture time, and every
frame has to be able to be reconstituted later on with this information
intact and readable.

Don’t use SDL for this, use GStreamer!

Thanks, I hadn’t found that yet. That’s the trouble I have with pursuing
previously unexplored avenues on Linux these days, finding the most
recent/advanced packages. Too much dead wood lying around on the internet.> On Thu, Apr 9, 2009 at 10:12 AM, michael brown <@michael_brown> wrote:

Mason Wheeler wrote:>> ----- Original Message ----

From: Sami N??t?nen <sn.ml at keijukammari.fi>
Subject: Re: [SDL] IMG_Load performance was Problem combining
sw_scale with blit operations on surface

On my machine, the command line ffmpeg program can process 180
frames per second into an mp4, but IMG_Load takes 15-30mS to load
one image to a surface. All the other SDL calls I make to create
the alpha blended text layer take a total of 5mS on my machine.
Any ideas why is IMG_Load so slow or what I can do about it?

Maybe this a related bug to the PNG loading bug, where the code
opened and
closed the PNG library for every image loaded. For JPEG loading I
would
imagine the speed loss being relatively higher than for PNG loading.

It’s not exactly a bug, just a missing feature. Try adding my
IMG_Init patch.
It affects PNGs, JPEGs and TIFFs, and ought to speed up the loading by
a bit. Just run
IMG_Init(IMG_INIT_JPG);
before you load up any JPEG files, and see if that doesn’t help
things…

I don’t know if it matters on that, but I’m using Linux. I looked at the
source code and it at least appears that it keeps track of whether the
library has been opened. I took out IMG_Load and replaced it with some
libJPEG direct calls to load the image and it’s taking about 15mS to open an
image. That is roughly the same time it takes me to create a JPEG file on
disk in the capture process. I would have thought that uncompressing a JPEG
image would have taken far less time than creating one, but that appears to
be wrong. I’m still wondering how ffmpeg can open (and process) 3 files in
the time it takes me to open 1. I guess it has tricks to accomplish this.

michael brown wrote:

Mason Wheeler wrote:

sw_scale with blit operations on surface

On my machine, the command line ffmpeg program can process 180
frames per second into an mp4, but IMG_Load takes 15-30mS to load
one image to a surface. All the other SDL calls I make to create
the alpha blended text layer take a total of 5mS on my machine.
Any ideas why is IMG_Load so slow or what I can do about it?

Maybe this a related bug to the PNG loading bug, where the code
opened and
closed the PNG library for every image loaded. For JPEG loading I
would
imagine the speed loss being relatively higher than for PNG loading.

It’s not exactly a bug, just a missing feature. Try adding my
IMG_Init patch.
It affects PNGs, JPEGs and TIFFs, and ought to speed up the loading by
a bit. Just run
IMG_Init(IMG_INIT_JPG);
before you load up any JPEG files, and see if that doesn’t help
things…

I don’t know if it matters on that, but I’m using Linux. I looked at
the source code and it at least appears that it keeps track of whether
the library has been opened. I took out IMG_Load and replaced it with
some libJPEG direct calls to load the image and it’s taking about 15mS
to open an image. That is roughly the same time it takes me to create a
JPEG file on disk in the capture process. I would have thought that
uncompressing a JPEG image would have taken far less time than creating
one, but that appears to be wrong. I’m still wondering how ffmpeg can
open (and process) 3 files in the time it takes me to open 1. I guess
it has tricks to accomplish this.

I’m guessing asynchronous I/O or multiple threads might help –
don’t let anything block. (I assume the usual uses synchronous
I/O.) Dealing with many small files usually has a far lower
throughput than dealing with a single big file – for example,
when running MD5 with sync I/O digesting a large directory tree,
the CPU load is usually far below maximum.>>> ----- Original Message ----

From: Sami N??t?nen <sn.ml at keijukammari.fi>
Subject: Re: [SDL] IMG_Load performance was Problem combining


Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

SDL is not dead wood!On Thu, Apr 9, 2009 at 2:25 PM, michael brown wrote:

Donny Viszneki wrote:

Don’t use SDL for this, use GStreamer!

Thanks, I hadn’t found that yet. ?That’s the trouble I have with pursuing
previously unexplored avenues on Linux these days, finding the most
recent/advanced packages. ?Too much dead wood lying around on the internet.


http://codebad.com/