iOS 16. Metal. Strange window behaviour

rmg.nik · September 23, 2022, 8:09am

Hi
In our game engine, we have implemented a change of screen orientation.
For Android we reused Android_JNI_SetOrientation, and for iOs we made small changes to SDL_uikitviewcontroller (some code, not all):

- (NSUInteger)supportedInterfaceOrientations
{
    return self.isPortrait ? UIInterfaceOrientationMaskPortrait | UIInterfaceOrientationMaskPortraitUpsideDown : UIInterfaceOrientationMaskLandscape;
}

-(void) setIsPortrait:(BOOL)isPortrait
{
    _isPortrait = isPortrait;
    [UIViewController attemptRotationToDeviceOrientation];
}

And it worked fine until iOS 16. For the Metal backend, we started getting strange window events after rotation::

2022-09-23 10:42:22.235361; SDL; INFO; SDL EVENT: SDL_DISPLAYEVENT (timestamp=0 display=0 event=SDL_DISPLAYEVENT_ORIENTATION data1=3)
2022-09-23 10:42:22.242446; ENGINE; DEBUG; WindowSubsystem.GetDisplayBounds: x=0 y=0 w=390 h=844
2022-09-23 10:42:22.243042; ENGINE; INFO; WindowSubsystem.CreateWindow: creating window [0] {x, y, w, h} = {0, 0, 390, 844}
2022-09-23 10:42:22.261544; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=26 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:42:22.279646; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=45 windowid=1 event=SDL_WINDOWEVENT_ENTER data1=0 data2=0)
2022-09-23 10:42:22.280151; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=45 windowid=1 event=SDL_WINDOWEVENT_FOCUS_GAINED data1=0 data2=0)
2022-09-23 10:42:22.281494; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=46 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:42:22.281882; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=47 windowid=1 event=SDL_WINDOWEVENT_SHOWN data1=0 data2=0)
2022-09-23 10:42:22.282258; ENGINE; INFO; WindowSubsystem.WindowSubsystem: subsystem successfully created
2022-09-23 10:42:22.282611; ENGINE; DEBUG; WindowSubsystem.SetScreenOrientation: 0
2022-09-23 10:42:22.283474; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=48 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:42:22.287792; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=53 windowid=1 event=SDL_WINDOWEVENT_HIDDEN data1=0 data2=0)
2022-09-23 10:42:22.296940; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=62 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=844 data2=390)
2022-09-23 10:42:22.297635; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=63 windowid=1 event=SDL_WINDOWEVENT_RESIZED data1=844 data2=390)
2022-09-23 10:42:22.299674; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=65 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=844 data2=390)
2022-09-23 10:42:22.310153; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=75 windowid=1 event=SDL_WINDOWEVENT_FOCUS_GAINED data1=0 data2=0)
2022-09-23 10:42:22.311767; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=77 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:42:22.312526; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=77 windowid=1 event=SDL_WINDOWEVENT_RESIZED data1=390 data2=844)
2022-09-23 10:42:22.313160; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=78 windowid=1 event=SDL_WINDOWEVENT_SHOWN data1=0 data2=0)

Here we can see that at some point the window size changed from (data1=390 data2=844) to (data1=844 data2=390), but almost instantly it changed back to (data1=390 data2=844).

Same log for OpenglES2 backend, where it works correctly:

2022-09-23 10:51:23.075676; SDL; INFO; SDL EVENT: SDL_DISPLAYEVENT (timestamp=0 display=0 event=SDL_DISPLAYEVENT_ORIENTATION data1=3)
2022-09-23 10:51:23.080988; ENGINE; DEBUG; WindowSubsystem.GetDisplayBounds: x=0 y=0 w=390 h=844
2022-09-23 10:51:23.081484; ENGINE; INFO; WindowSubsystem.CreateWindow: creating window [0] {x, y, w, h} = {0, 0, 390, 844}
2022-09-23 10:51:23.102000; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=26 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:51:23.120127; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=45 windowid=1 event=SDL_WINDOWEVENT_ENTER data1=0 data2=0)
2022-09-23 10:51:23.120634; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=45 windowid=1 event=SDL_WINDOWEVENT_FOCUS_GAINED data1=0 data2=0)
2022-09-23 10:51:23.121923; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=46 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=390 data2=844)
2022-09-23 10:51:23.122360; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=47 windowid=1 event=SDL_WINDOWEVENT_SHOWN data1=0 data2=0)
2022-09-23 10:51:23.123661; ENGINE; INFO; WindowSubsystem.WindowSubsystem: subsystem successfully created
2022-09-23 10:51:23.080339; ENGINE; DEBUG; WindowSubsystem.SetScreenOrientation: 0
[Assert] A new orientation transaction token is being requested while a valid one already exists. reason=Root View Controller Setup: <SDL_uikitviewcontroller: 0x7fb96303b200>; windowOrientation=landscapeRight; sceneOrientation=portrait; existingTransaction=<_UIForcedOrientationTransactionToken: 0x600003f3d260; state: committing; originalOrientation: portrait (1)>
2022-09-23 10:51:23.183057; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=107 windowid=1 event=SDL_WINDOWEVENT_SIZE_CHANGED data1=844 data2=390)
2022-09-23 10:51:23.183685; SDL; INFO; SDL EVENT: SDL_WINDOWEVENT (timestamp=108 windowid=1 event=SDL_WINDOWEVENT_RESIZED data1=844 data2=390)

What could be the problem?

Sylvain_Becker · September 28, 2022, 7:24am

@rmg.nik
not sure, but I have also an issue with landscape app + metal +IOS 16

rmg.nik · September 28, 2022, 7:37am

Thanks!
Yes, it seems to be the same problem.

Sylvain_Becker · October 7, 2022, 7:47am

@rmg.nik
for me, this is fixed by not using SDL_WINDOW_OPENGL for the window creation. SDL_CreateRender would pick anyway an accelerated metal back-end.

rmg.nik · October 7, 2022, 9:02am

Thank you! But it didn’t help me as I change the orientation in runtime. So I got an error and crash.

-[MTLDebugRenderCommandEncoder setScissorRect:]:3703: failed assertion `Set Scissor Rect Validation
(rect.x(140) + rect.width(2249))(2389) must be <= render pass width(1170)
'
-[MTLDebugRenderCommandEncoder setScissorRect:]:3703: failed assertion `Set Scissor Rect Validation
(rect.x(140) + rect.width(2249))(2389) must be <= render pass width(1170)
'

Sylvain_Becker · October 7, 2022, 9:15am

strange …
what I know it that METAL validate input parameter,
this should be be here for the scissor command:

github.com

libsdl-org/SDL/blob/main/src/render/metal/SDL_render_metal.m#L1234

      
        
                /* Set Scissor Rect Validation: w/h must be <= render pass */
                SDL_zero(output);
            
            
    if (renderer->target) {
                    output.w = renderer->target->w;
                    output.h = renderer->target->h;
                } else {
                    METAL_GetOutputSize(renderer, &output.w, &output.h);
                }
            
            
    if (SDL_IntersectRect(&output, &clip, &clip)) {
                    MTLScissorRect mtlrect;
                    mtlrect.x = clip.x;
                    mtlrect.y = clip.y;
                    mtlrect.width = clip.w;
                    mtlrect.height = clip.h;
                    [data.mtlcmdencoder setScissorRect:mtlrect];
                }
            
            
    statecache->cliprect_dirty = SDL_FALSE;
            }

some parameter maybe wrong at that time …