Generate vídeo with SDL2

SDL Development
1

Hi!

Someone knows how to generate vídeo with SDL2?

I was thinking on saving frames as bmp and from them generate video with the ffmpeg command on my Linux. But, there is something more “multiplatform way” of make this task?

Thanks!

2

There’s a couple of options that I’m aware of, none of them are perfect.

  • Your option is valid.The downside of saving individual frames is high storage cost. Reducing frame rate and resolution size would help. Using PNGs to store your image files should also help. Though some versions of BMP allow for compression, I don’t think SDL takes advantage of those options.

  • You could call ffmpeg from within your program then delete the stored images. That would at least reduce the final storage requirement and help to automate the task.

  • You could learn how to use a video encoder/decoder API like libtheora (video) and libvorbis (audio) to create an ogv video from a stream. (My personal annoyance with this is that all images need to be converted to YUV format while everything I work with is in ARGB format). Icculus has some example code for decoding libtheora and libvorbis here, it might help you get comfortable with the library even though you would need to learn encoding instead.

  • At this stack overflow, OpenCV seems to be able to write an uncompressed stream to mp4. (I don’t know how large OpenCV is, but it seems like using a cannon to swat a fly in this situation)

1 Like
3

Why not feeding the ffmpeg directly with frames, without saving them on drive?

1 Like
4

Thanks man! I searched a lot but without success and so the option of saving all frames was the temporary solution. But, I will evaluate your tips!

5

Is that possible? But how?
I will study the options given by the @Sward and if you knows how to feed the ffmpeg without saving on disc, please, tell me.

6

Using pipes — this the standard way to do this. But you have to keep in mind that, for this to work, the frames ready to be written to video must be in RAM so they can be passed to ffmpeg via a pipe.

And here’s the problem: if you’re rendering frames on the GPU side, you’ll need to transfer their pixels from VRAM to RAM, and that’s computationally expensive. Therefore, the higher the resolution and frame rate, the more data you’ll need to transfer, and thus the more CPU time will be required. But this is still a much better solution than saving frames to files, because it eliminates the need to interact with the hard drive, and the amount of free disk space doesn’t matter (since everything happens in memory).

For simple applications, you can use ffmpeg and directly feed it with game frames, but if you want high performance, you should use a hardware video encoder (such as NVENC) instead of ffmpeg, just as OBS does, for example. But doing that won’t be easy.

If you’re interested in adding a gameplay recorder to your game, there are really three main options:

  1. ffmpeg and feeding it with frames — this is a relatively simple solution, but it has its limitations, and its performance won’t be very high.

  2. Using a hardware video encoder — this is difficult to implement; it requires a lot of expertise and a significant amount of time, and it also requires using different SDKs depending on the encoder (Intel, NVIDIA, AMD). Overall, it’s pretty hardcore.

  3. recording input, not video — It doesn’t require any video decoder; all that’s needed is for the game’s main loop to run deterministically. After recording the gameplay input, to play it back, simply start a new game session that uses the input data from the file instead of testing the input devices. CPU and RAM usage is negligible because it doesn’t depend on screen resolution and doesn’t require the transfer of large amounts of data, and the frame rate has a negligible impact on the amount of data to be saved.

The third solution is the most interesting, relatively simple to implement, very efficient, and does not require a lot of memory. I’ve done something like this before — I wrote a “recorder” for Tetris games, whose task was to scan the FCEUX emulator’s screen and record the input so that I could later render a visualization of the gameplay without using the emulator, in the form of a video player, with the option to skip to any frame (instant jump, without any buffering).

Everything was done purely in software, without touching the GPU. Each frame represents the state of the input and sound, the stack state, various counters, flags etc. (234 bytes per frame). At 60 fps, a 12-minute gameplay recording took up about 130kB (data compressed using the ZIP algorithm). If I had used data structures and bit fields instead of classes, the recording file would have taken up significantly less space. This could be optimized even further by writing data only for those frames in which the input state has changed.

So yeah, you can easily implement something like this in any game and then replay the gameplay based solely on the recorded input.

2 Likes
7

True, but you can reduce the storage cost a little by using SDL_PIXELFORMAT_UYVY which is the native format of some uncompressed video files. Still a lot bigger than MPEG-coded video, for example, but manageable for short clips, and SDL is easily capable of streaming that at full rate.

8

@FlowCRANE: I like your option 3, Input Recording.

  • Low CPU impact and storage cost. (potentially smaller files than MP4’s best compression options)
    • The file size could be further reduced if the level/game state can be determined by one or two random seeds that “set the board”. Then you just need to track change of input states and those timestamps. If the player jams down the acceleration button for a minute straight, then nothing is recorded in the file for that minute until they let up that button or press something else.
  • Lossless compression is effective, video can basically always play at the game’s best resolution despite small file sizes.
  • The file’s replay can only be implemented by your programs. (Both pro and con)
  • Gives the player the ability to watch instant replay without including any new libraries.
  • Can easily become a ghost player that the player can compete against.
  • The player can still use their own screen recorders or HDMI splitters to create their own videos.
    A downside is that the player might expect the replay to prove their capabilities in the game, but it’s too easy for a person to use it to create a Tool-Assisted Speedrun.

I’m pretty certain that libtheora is entirely CPU bound, and it looks like it can run in it’s own separate thread… But you are correct for both this option and the piping to ffmpeg one, getting video images from the GPU VRAM to RAM is considered slow either way. If SDL_RenderReadPixels is used, try to use it once per frame or less.

9

Thanks man, I will evaluate the use of pipes!