Why is compositing disabled on normal x11 windows?

I’m on KDE Plasma (on EndeavourOS), which uses x11. If I instantiate a window with the following code:

m_window = SDL_CreateWindow(
	"windowname", 
	SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED,
	426, 240,
	SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE
);

the whole screen turns black for a while until the window pops up, and the second monitor turns black for the duration of the window’s existence! Window effects like Wobbly Windows stop having any effect on any windows whatsoever during this time too. If I had downloaded a game that did this, I’d become very suspicious very quickly.
I’ve found I can fix all these problems by adding SDL_WINDOW_UTILITY to the flags:

m_window = SDL_CreateWindow(
	"windowname", 
	SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED,
	426, 240,
	SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE | SDL_WINDOW_UTILITY
);

Now it behaves like all other windows on my system without any weird hiccups, except for the fact that it doesn’t appear in the taskbar! It is also excluded from the alt-tab menu. This is probably intended behaviour for a window called a “utility window”, so I definitely want to make it work without SDL_WINDOW_UTILITY. For similar reasons, I also don’t want SDL_WINDOW_POPUP_MENU and SDL_WINDOW_TOOLTIP.

I looked through the source code for the X11 windows and pinpointed the problem to a peculiar line (593) in SDL_x11window.c where compositing is set to false for normal windows. From a quick search of what compositing means, that sounds exactly like the cause of my woes. Judging by the comment on that line, however, it seems completely intentional!

What is going on here? Is there a reason that I’m unable to make a regular window composite? If so, how can I enable compositing anyway?

It’s for performance, since compositing introduced additional delay and GPU/CPU usage when rendering the window.

We could add a hint here though, for people who want compositing enabled.

1 Like

There’s actually already an (undocumented) hint to disable this behaviour: SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR.

So you could stop SDL from disabling the compositor with:

SDL_SetHint(SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR, "0");

Or to test, just set the environment variable SDL_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR=0 before running the program.

We probably should add this hint to the Wiki documentation at some point, or possibly implement a less platform-specific variant of it.

3 Likes

Oooh thank you so much! This made the second monitor issue, along with desktop effects work without any issues whatsoever.
Now I am wondering, there is no way the issues I mentioned are intended behaviour for any type of program, right? Is that something I can expect on other devices or is it more probable that I’ve just got some configuration set up weirdly on my machine? Namely, I could very well imagine it being an issue with the fact I’m using a laptop with Optimus. If it isn’t exclusive to my machine though, wouldn’t it make more sense to have compositing be the default, with the ability to disable it via a hint?

The disabling of effects like wobbly-windows is an intended side effect: they rely on the compositor, and so can incur a significant performance penalty. Indeed, many window managers would automatically disable compositing if they detected (particularly fullscreen) games running.

The issue with the black screen on another monitor is definitely a bug with your system (X, the window manager, or the graphics drivers), not SDL: disabling compositing shouldn’t cause issues like that. Optimus could definitely be involved.

Ultimately, Wayland should rid us of this problem completely: it doesn’t support disabling the compositor, and in the future it should be able to use separate planes to get a similar performance boost on some hardware (without the downsides).

If your application isn’t performance sensitive, I’d definitely use the SDL_SetHint() call to re-enable compositing. Otherwise, suggesting that users can re-enable it themselves via the environment variable or through their compositor (in KDE, you can add a “window rule” to prevent a window from being able to disable compositing) if they encounter issues could be worth having in the documentation somewhere.

1 Like

Ah, I see. Thank you for taking the time to answer so thoroughly! I’m quite new to the Linux ecosystem(s), so I really appreciate the patience.

I’ll definitely have to have a look at my system then to see how I can safely disable the compositor without causing everything to go haywire, but in the meantime I’m quite satisfied with the hint solution.

No worries.

FYI: If I remember correctly, on KDE, you can quicky enable/disable compositing with the Alt+Shift+F12, which might help when testing.

1 Like

Oh, you’re right, that’s gonna help a lot! Interestingly, disabling compositing that way keeps the second monitor working just fine: it’s only when a SDL requests to disable it that things break down. Something to wish for when Wayland becomes more supported, I suppose. I’ll keep experimenting with all this in the meantime!