Nearest neighbor scaling works strange

SDL Development
1

I have set scaling mode to nearest neighbor with SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "nearest"); and upscale sprites with passing dstrect of higher size than srcrect to SDL_RenderCopyEx.

Seems like scaling result changes a bit when changing dstrect.x and dstrect.y while srcrect being untouched, i. e scaling result depends on image’s position on screen. The same thing happens when changing srcrect.x and srcrect.y while image inside srcrect being unchanged. But the result should only depend on input image and final width / height. Any way to fix it?

2

I have not been able to reproduce this behavior.
Can you share a minimal code+sprite example?
For now, i can suggest you to cache the scaled sprite, if you don’t need to scale it differently.

3

Here is the code example:

#include "SDL.h"
#include "SDL_image.h"

int main(int argc, char* args[])
{
    SDL_Init(SDL_INIT_EVERYTHING);
    IMG_Init(IMG_INIT_PNG);
    SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "nearest");

    SDL_Window *window = SDL_CreateWindow("test", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 1280, 720, SDL_WINDOW_SHOWN);
    SDL_Renderer *renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);

    SDL_Surface *surface = IMG_Load("sprite.png");
    SDL_Texture *texture = SDL_CreateTextureFromSurface(renderer, surface);
    SDL_FreeSurface(surface);

    int frame = 0;

    while(true) {
        SDL_Event event;
        bool quit = false;
        while(SDL_PollEvent(&event)) {
            if(event.type == SDL_QUIT) {
                quit = true;
                break;
            }
        }
        if(quit) {
            break;
        }

        SDL_RenderClear(renderer);
        frame ++;

        SDL_Rect srcrect, dstrect{0, 0, 360, 360};
        if(frame % 60 < 30) {
            srcrect.x = 0;
            srcrect.y = 0;
        }
        else {
            srcrect.x = 48;
            srcrect.y = 0;
        }
        srcrect.w = srcrect.h = 48;

        SDL_RenderCopy(renderer, texture, &srcrect, &dstrect);
        SDL_RenderPresent(renderer);
    }

    SDL_DestroyTexture(texture);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);

    return 0;
}

sprite.png:
sprite
Images in [0; 0; 48; 48] and [48; 0; 48; 48] are equal, but scaling results are different.

4

Yes, I can definitely see the problem with Linux X11, even though it is not present on Windows.
Your problem is unlikely to be solved properly.
The “fix” is to use software scaling (slow), you need to keep loaded SDL_Surface to achieve this, and implement the scaling algorithm.

5

I have just tested it on Windows, and the same problem here.

6

Not appears with SDL2 2.26.5 from MSYS2 repo on my machine.
Anyways, the root of problem is the accelerated scaling, can be caused by anything, so your workaround could be to use software scaling, or do not use scaling at all.

7

I don’t see any difference on Linux (X11) with SDL 2.0.5.

I checked the renderer name and it is “opengl”.

Update: Tested on another computer with Linux and SDL 2.0.14 and can’t see a difference there either.

8

I noticed that some of the upscaled pixels (in the same image) are not the same size. This is necessary because 360 is not evenly divisible by 48. So I guess there is more than one “correct” way to scale this.

Does the problem disappear if you make sure to scale by an exact multiple, e.g. by changing 360 to 384?

SDL_Rect srcrect, dstrect{0, 0, 384, 384};
9

There is no problem when upscaling to integer factor, but screen resolution is not always divisible by game’s base resolution. Scaling result would not be perfect in this case, but at least it should be deterministic for image and scale factor.

I use SDL2 2.26.5 built from source with project. I have tried switching to the system library in /usr/bin, but it didn’t help.

10

A more robust solution might be to render everything to an intermediate texture of the same size as the “game’s base resolution” (without scaling) and when that’s done you render the intermediate texture to the screen (with scaling).

I think that would remove all inconsistencies because rendering to the intermediate texture does not involve any scaling and when rendering the intermediate texture to the screen the “dstrect” and “srcrect” are always the same.

That said, I think non-square pixels of varying sizes looks kind of ugly and would personally prefer to upscale to an integer multiple and just show a black border around it.

Alternatively the graphics could be scaled up to the largest integer multiple that fits using nearest neighbour interpolation as before and then you upscale that to the screen resolution (while still keeping the aspect ratio) using bilinear interpolation. This way you keep the pixelated look and only add as little blur as possible to fill the screen. I think this looks much better than having pixels of varying sizes. I’ve done this in “software” before but I don’t see why it wouldn’t work with textures as well.

1 Like
11

Then if you need constant scaling, there is one more way: use the virtual resolution.
Here is your code rewritten to use it:

#include "SDL.h"
#include "SDL_image.h"

int main(int argc, char* args[])
{
    SDL_Init(SDL_INIT_EVERYTHING);
    IMG_Init(IMG_INIT_PNG);
    SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "nearest");

    SDL_Window *window = SDL_CreateWindow("test", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 1280, 720, SDL_WINDOW_SHOWN);
    SDL_Renderer *renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);

    int w = 1280.f / 360 * 48;
    int h = 720.f / 360 * 48;

    SDL_RenderSetLogicalSize(renderer, w, h);

    SDL_Surface *surface = IMG_Load("sprite.png");
    SDL_Texture *texture = SDL_CreateTextureFromSurface(renderer, surface);
    SDL_FreeSurface(surface);

    int frame = 0;

    while(true) {
        SDL_Event event;
        bool quit = false;
        while(SDL_PollEvent(&event)) {
            if(event.type == SDL_QUIT) {
                quit = true;
                break;
            }
        }
        if(quit) {
            break;
        }

        SDL_RenderClear(renderer);
        frame++;

        SDL_Rect srcrect, dstrect{0, 0, 48, 48};
        if(frame % 60 < 30) {
            srcrect.x = 0;
            srcrect.y = 0;
        }
        else {
            srcrect.x = 48;
            srcrect.y = 0;
        }
        srcrect.w = srcrect.h = 48;

        SDL_RenderCopy(renderer, texture, &srcrect, &dstrect);
        SDL_RenderPresent(renderer);
    }

    SDL_DestroyTexture(texture);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);

    return 0;
}

The glitch also will appear but on my system i can barely see it.

12

SDL_SetRenderTarget should work.

Also OP can just use software rendering at the SDL side, then the example code will render as intended (but the pixel sizes will still differ).
The original problem is that the pixel size will be different depending on X and Y:

I think if the source resolution is bigger than target one, the blur can eliminated? Looks like a solution for OP.

13

One problem with “software rendering” is that it can be slow but if you do it at low resolution it’s usually not a problem.

The way I have done this in the past, when using SDL 1.2 which doesn’t have textures, was to draw everything to a 320×200 SDL_Surface and then only upscale the “dirty rectangles” (in software) to the screen surface and then use SDL_UpdateRects to send the updated rectangles to the GPU. The last step was important because if I had used SDL_Flip to send the whole screen surface each frame the performance would have been too poor.

Notes:
The SDL2 equivalent of SDL_UpdateRects is SDL_UpdateWindowSurfaceRects.
The SDL2 equivalent of SDL_Flip is SDL_UpdateWindowSurface.

In SDL2 I suspect it’s better to use SDL_UpdateTexture. Then you don’t need to do the scaling in software (slow) and can instead only upload the graphics at low resolution to the GPU and do the scaling on the GPU (fast). It’s possible to pass a list of rectangles to SDL_UpdateTexture to only update parts of it but I don’t think that should be necessary as long as the graphics is at low resolution (i.e. the number of pixels to transfer is small).

I have experimented with this too. Scale to an integer multiple of the “game’s base resolution” (using nearest neighbour) so that it becomes equal to or greater than the screen resolution, and then you scale down to the screen resolution (using bilinear).

This indeed gives a less blury result but this is mostly noticeable with small scaling factors so it has to be weighed against the added performance demands and extra memory usage that it requires.

1 Like
14

Yep that’s depends.

I believe OP doesn’t need to scale sprites differently each time, so just scaling each sprite from the spritesheet once (using software way) on game startup is enough, but I also think SDL_RenderSetLogicalSize is the very good way to handle pixel graphics. The example I posted before renders far better than original solution (but it’s not perfect) and much simpler to handle.

I agreed with your pre-rendered frames technique as more stable and robust solution, but personally, I inclined to use more simpler methods :grin:.

15

I have to admit that I’m not very familiar with SDL_RenderSetLogicalSize but it does seem like an easy way, assuming it works.

If you don’t mind not filling up the whole screen, I found you can use SDL_RenderSetIntegerScale to force it to scale to an integer multiple which I think should fix any issues with varying pixel sizes.

SDL_RenderSetIntegerScale(renderer, SDL_TRUE);

Disclaimer: I have no idea how the above will interact with DPI scaling.