Does this also mean that SDL 3.3 natively supports some compression format? Png compression is usually loss-less if I understand correctly.
Say I have some human-readable character data that I don’t want the user to be editing in notepad. Or maybe a binary plugin that would benefit from compression.
Would it be possible to run that data through SDL’s core png compression algorithm both to save space and add a layer of security?
Well, I just had to see what would happen if I forced data into a png file.
I had some success, code below. I loaded an std::string into a surface, called SDL_SavePNG on that surface, then reloaded the same data using SDL_LoadPNG.
I see a reduction in size of about 50% on a “lucky” sample, 30-40% reduction in many other samples. I can’t guarantee the same results on all types of data input.
Obviously PNG compression is not and should not be expected to be highly optimized for text. But an average of 30-50% reduction of larger files might be worth the effort.
I can not warn enough, this code is me being stupid and trying out dumb things. I’m not an expert, just a hobbyist. I just made a major bug fix in a recent edit that fixed segfaults.
I was able to run the current version with 130MB of generated text without issue, but there was an upper limit without crash when I tried to go 140MB. (This limit could also be different on other operating systems, I’m on Ubuntu Linux)
In the code, I’m skipping error checks for brevity. Please add that to your own version if you choose to tinker with this.
#include <SDL3/SDL.h>
#include <string>
#include <stdlib.h>
std::string loadData(const std::string &filename)
{
SDL_Surface * container = SDL_LoadPNG(filename.c_str());
SDL_Surface * conversion = SDL_ConvertSurface(container, SDL_PIXELFORMAT_RGBA8888);
std::string data = static_cast<char *> (conversion->pixels);
SDL_DestroySurface(container);
SDL_DestroySurface(conversion);
return data;
}
// this is probably not very safe, I don't know if it might be grabbing from your system RAM past the string's length and saving that in the output file when the string is not aligned to the width.
// width is likely causing the data limit at 130MB, it might better fit the contents of the string using a sqrt() of data length?
void saveData(const std::string & filename, const std::string & data)
{
int width = 512;
int height = 1 + data.length() / (4 * width) ;
// wait, adjust for smaller string lengths
if(data.length()/4 < width)
{
width = 1 + data.length()/4;
height = 1;
}
SDL_Surface * container = SDL_CreateSurface(width, height, SDL_PIXELFORMAT_RGBA8888);
void * hold = container->pixels; // need to hold this pointer to properly destroy surface at the end.
container->pixels = static_cast<void *>(const_cast<char *>(data.data()));
SDL_SavePNG(container, filename.c_str());
container->pixels = hold;
SDL_DestroySurface(container);
}
int main()
{
SDL_Init(SDL_INIT_VIDEO);
std::string testStr = "This is a string of data that I would like to compress. The character's name is Robbie, his attack is 15. I don't know how long this sentence should be, but I'm curious what happens as it gets longer, so...";
for(int i = 0; i < 400; i ++)
{
testStr += 'a' + (rand() % 26);
}
testStr += "||| <- sorry, a lot of random letters.";
//std::string testStr = "Small test";
saveData("testPNG.png", testStr);
std::string retStr = loadData("testPNG.png");
SDL_Log("This is the returned str: %s", retStr.c_str());
SDL_Log("The string is %ld letters long, which is also that many bytes", retStr.length());
SDL_Quit();
}
In Summary;
I can’t fully recommend this as a work-around mostly because I don’t trust my own knowledge on the safety. (See first comment above saveData function)
There is also a lot of data copying in my current work-around thanks to loading it into a surface, then converting it to the correct format, then casting that all back into a string, etc.
I would be very interested to see some form of compression API in native SDL. It feels like we’re just a stone’s throw away from having it.
Finally, there are plenty of external libs designed specifically for this purpose, some with compression almost down to 1/10th of original file size.
Most likely I should just leave this topic be and get over it.
Edit: Since the generated text in the code is randomized and PNG is specialized for compressing repetitive rasterized pixels, I’m actually throwing worst-case scenarios at the algorithm in the above test code, so I’m back to being somewhat impressed that any compression occurred in the above test.
Incidentally, I recently read a few chapters into the user-manual for a popular drawing program “Krita”, and this is pretty much how they store additional information for their brushes. Technically they describe it as xml saved in the PNG’s meta data alongside the brush’s image, so perhaps there is a table in the format describing where that text/data is found in the file and how large, etc, but it’s in there being compressed along with the image.
Anyhow, I don’t quite see how this would be set up in the official core SDL except to have a SDL_PNG struct which includes the SDL_Surface, char pointer and length, along with SDL_GetPNGText and SDL_SetPNGText helper functions.
I understand that the team/community is already on edge about bloat with PNG being included in the project as it is, so I get it.
But I really want this.
SDL’s PNG loading support comes via stb_image, a simple header-only library that doesn’t support anything beyond image data loading. Both the SDL and the stb_image docs warn not to use it to load images you don’t trust (aka only use it for loading your own images that you ship with your game/program).
Similarly, SDL’s PNG writing support comes from miniz, another header-only library that doesn’t support PNG’s more esoteric features like adding custom metadata chunks.
If you want to support the full range of PNG features, use libpng.
If you want to compress stuff to take up less space on disk, just use miniz or zlib directly. Be advised that compression isn’t security; this will stop someone from poking around with Notepad, but that’s it.