SDL: SetDisplayMode: Call XRRSetScreenSize before setting CRTC config

From 16e3bfe807dfdd48b6833da4cd78e789a920e2ec Mon Sep 17 00:00:00 2001
From: Austin Shafer <[EMAIL REDACTED]>
Date: Mon, 28 Jun 2021 11:29:16 -0400
Subject: [PATCH] SetDisplayMode: Call XRRSetScreenSize before setting CRTC

X11_SetDisplayMode currently calls X11_XRRSetCrtcConfig alone. This results
in the monitor's viewport getting changed, but the underlying screen dimensions
stay the same.

The spec indicates that RRSetCrtcConfig only changes the crtc mode and has no effect
on the screen dimensions, only mentioning that the new crtc must fit entirely within the
screen size. For the size to change, RRSetScreenSize also needs to be called.

This affects Metro Exodus on Linux, when changing the resolution in the in-game settings
Metro gets stuck in a loop waiting for the size of its vulkan surface to change. Because
XRRSetScreenSize is not called the screen size is never changed, the vulkan surface dimensions
do not change, and Metro hangs forever watching for a surface size update that will
never come.

This change disables the CRTC, calls XRRSetScreenSize, and then updates the
CRTC configuration. This fixes changing the resolution from the Metro settings.

Tested with:
Metro Exodus, Portal 2
 src/video/x11/SDL_x11modes.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/src/video/x11/SDL_x11modes.c b/src/video/x11/SDL_x11modes.c
index e14c90e83f..09707a5bc2 100644
--- a/src/video/x11/SDL_x11modes.c
+++ b/src/video/x11/SDL_x11modes.c
@@ -1002,6 +1002,7 @@ X11_SetDisplayMode(_THIS, SDL_VideoDisplay * sdl_display, SDL_DisplayMode * mode
     Display *display = viddata->display;
     SDL_DisplayData *data = (SDL_DisplayData *) sdl_display->driverdata;
     SDL_DisplayModeData *modedata = (SDL_DisplayModeData *)mode->driverdata;
+    int mm_width, mm_height;
     viddata->last_mode_change_deadline = SDL_GetTicks() + (PENDING_FOCUS_TIME * 2);
@@ -1030,10 +1031,23 @@ X11_SetDisplayMode(_THIS, SDL_VideoDisplay * sdl_display, SDL_DisplayMode * mode
             return SDL_SetError("Couldn't get XRandR crtc info");
+        X11_XGrabServer(display);
+        status = X11_XRRSetCrtcConfig(display, res, output_info->crtc, CurrentTime,
+          0, 0, None, crtc->rotation, NULL, 0);
+        if (status != Success) {
+            goto setCrtcError;
+        }
+        mm_width = mode->w * DisplayWidthMM(display, data->screen) / DisplayWidth(display, data->screen);
+        mm_height = mode->h * DisplayHeightMM(display, data->screen) / DisplayHeight(display, data->screen);
+        X11_XRRSetScreenSize(display, RootWindow(display, data->screen), mode->w, mode->h, mm_width, mm_height);
         status = X11_XRRSetCrtcConfig (display, res, output_info->crtc, CurrentTime,
           crtc->x, crtc->y, modedata->xrandr_mode, crtc->rotation,
           &data->xrandr_output, 1);
+        X11_XUngrabServer(display);