Is the texture data of SDL_Texture placed in VRAM or RAM?

XCY · September 18, 2022, 2:01pm

I’ve found that many explanations about SDL_Texture says it store texture in VRAM. However, I found that SDL_Texture's data is actually stored in regular RAM, and is copied to GPU in SDL_RenderPresent after drawing it with SDL_RenderCopy for the first time. A simple example is as follows.

#include <cstdio>
#include <chrono>

#include <SDL2/SDL.h>
#include <SDL2/SDL_image.h>

using namespace std;
using namespace std::chrono;

int main()
{
	SDL_Init(SDL_INIT_EVERYTHING);

	auto window_ptr = SDL_CreateWindow("SDL2", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 1920, 1080, NULL);
	auto renderer_ptr = SDL_CreateRenderer(window_ptr, -1, NULL);

	auto texture_ptr = IMG_LoadTexture(renderer_ptr, /*a 1920x1080 image*/);

	SDL_Rect src = { 0, 0, 1920, 1080 }, dst = { 0,0,1920,1080 };

	SDL_Event event;
	bool running = true;
	while (running)
	{
		SDL_PollEvent(&event);
		switch (event.type)
		{
		case SDL_QUIT:
			running = false;
			break;

		default:
			break;
		}

		SDL_RenderCopy(renderer_ptr, texture_ptr, &src, &dst);

		auto&& t0 = system_clock::now();
		SDL_RenderPresent(renderer_ptr);
		auto&& t1 = system_clock::now();

		printf("%dus\n", duration_cast<microseconds>(t1 - t0).count());
	}

	SDL_DestroyRenderer(renderer_ptr);
	SDL_DestroyWindow(window_ptr);
	SDL_Quit();
}

Before IMG_LoadTexture, the memory occupied by the program was low. After loading the texture, the memory increased rapidly and an instance as big as the image (1920x1080x4) appeared. Shouldn’t the data of the texture be stored in GPU?

Then comes the texture rendering. I turned off vertical synchronization so I could see how long each rendering took. The first SDL_RenderPresent took a full 17ms to render, while the others took only about 1ms. I believe that the texture data was not copied to GPU until the first call to SDL_RenderPresent.

So what does "SDL_Texture stores data in VRAM" mean? Is there any parameter I have not set correctly? If it’s true that textures can only be copied to GPU in SDL_RenderPresent, what should I do with large textures in game development? In the game I’m developing, there is a 2544x2016 size image, and the first time it renders would take a full 30ms, which directly causes a frame drop.

sjr · September 19, 2022, 4:01am

So, first the image must be loaded into RAM. Then that gets handed off to the GPU driver, and what happens after that depends on the API and the driver itself, with the image data eventually being uploaded to VRAM.

On Windows, ages ago some OpenGL drivers (IDK if it they also did this with DirectX) used to “cheat” and not actually copy the texture to VRAM until it was used, to claim faster loading times in games etc. AFAIK this isn’t the case anymore, and with modern APIs like DirectX 12, Metal, and Vulkan, it shouldn’t be possible.

It’s also possible that there’s just some first-time setup happening on the first frame. You can test this by rendering a few empty frames and then rendering your texture.

XCY · September 19, 2022, 4:30am

Thanks for the reply. I changed the code to first draw a hundred of empty frames and then start rendering texture. The first call to SDL_RenderPresent after start rendering texture still cost much more time than the others.

size_t count = 0;
while (running)
{
	/* event handling */

	if (count == 100)
		printf("----start rendering the texture----\n");
	if (count >= 100)
		SDL_RenderCopy(renderer_ptr, texture_ptr, &src, &dst);
	++count;

	auto&& t0 = system_clock::now();
	SDL_RenderPresent(renderer_ptr);
	auto&& t1 = system_clock::now();

	printf("%dus\n", duration_cast<microseconds>(t1 - t0).count());
}

sjr · September 19, 2022, 5:06am

Weird. Does the delay change with the size of the texture? Is it possible that SDL just needs to create some buffers etc. and waits until you actually draw something to do it? What happens if, instead of drawing completely empty frames before you draw the texture, you draw a filled rectangle for those frames instead?

If the GPU driver really is waiting until the texture is actually used before uploading it to VRAM, one thing you could do is render all your textures into an offscreen render target at load time. This would force them to be uploaded to VRAM ahead of time. Then just discard the render target. Something like

SDL_Texture *renderTarget = SDL_CreateTexture(renderer_ptr, SDL_PIXELFORMAT_RGBA32, SDL_TEXTUREACCESS_TARGET, 800, 600);
SDL_SetRenderTarget(renderer_ptr, renderTarget);
// Loop over your textures and do a SDL_RenderCopy() call for each one
SDL_SetRenderTarget(renderer_ptr, nullptr);
SDL_DestroyTexture(renderTarget);

after you’ve loaded your textures.

edit: what render driver is being used?

SDL_RendererInfo info;
SDL_GetRendererInfo(renderer_ptr, &info);
printf("Render driver: %s\n", info.name);

XCY · September 19, 2022, 8:03am

What happens if, instead of drawing completely empty frames before you draw the texture, you draw a filled rectangle for those frames instead?

I changed the code to call SDL_RenderDrawRect in the first ten frames. The time cost by the first call to SDL_RenderPresent after start rendering texture was shortened a bit, but still much longer than the others. The following tests are based on this change.

Does the delay change with the size of the texture?

The delay does change with the size of the texture but I can’t find a pattern. I created four image with similar content at sizes of 1280x720, 1920x1080, 2560x1440 and 3840x2160. The time cost by the first call to SDL_RenderPresent after rendering each texture is not certain. For example, when the order of rendering is 720p, 1080p, 1440p, 2560p, the time taken for the first rendering of each is 10295us, 8242us, 6418us, 8400us. However, when the order of rendering is 2560p, 1440p, 1080p, 720p, the time taken for the first rendering of each becomes 21320us, 4188us, 2718us, 1122us.

One thing you could do is render all your textures into an offscreen render target at load time.

This is very useful. Thank you!

What render driver is being used?

The code you offered print “direct3d”.

Then I tried different render driver, and something confusing happened, as shown below.

`HINT_RENDER_DRIVER`	`RendererInfo`	render time
Not set	`"direct3d"`	As shown above.
`"direct3d"`	`"direct3d"`	Texture’s first rendering time is reduced, but still slightly higher than the others.
`"direct3d11"`	`"direct3d11"`	Texture’s first rendering time is no different from the others.
`"direct3d12"`	`"direct3d12"`	Texture’s first rendering time is no different from the others, but randomly some frames take longer time to render.

I’m using SDL2 2.24.0 installed by vcpkg. So what’s this default displayed-as-“direct3d” render driver? Maybe I should create a new topic about this. Anyway, it seems like I can manually specify render driver to prevent the render delay. I may need to read the source code for the exact reason.