Displaying HDR on macOS and iOS from Vulkan and OpenGL rendering

SDL Development
1

I have apps using Vulkan or OpenGL and SDL that want to display HDR content. My Vulkan app, which uses SDL_Vulkan_CreateSurface, successfully displays HDR on macOS but not on iOS. My iOS device, an M4 MacBook Pro, is capable of HDR display.

My OpenGL app, which creates an RBG16F format window and renderbuffer, confirmed by this call

glGetFramebufferAttachmentParameteriv(GL_FRAMEBUFFER, attachment,
                                      GL_FRAMEBUFFER_ATTACHMENT_COMPONENT_TYPE,
                                      &type);

does not display HDR on either macOS or iOS.

Various sources have told me I need to set the wantsExtendedDynamicRangeContent property on the CAMetalLayer to YES/true. So 4 questions:

  1. Does SDL have a video-device independent way to set to set this property.
  2. If the answer to 1 is no, how do I obtain the CAMetalLayer that is backing the VulkanSurface. I see a low level function Cocoa_Metal_GetLayer which is set to a Metal_GetLayerfunction pointer on the video device but I haven’t yet found a way to access this from my c++ app via the Window or Vulkan surface.
  3. Same question for OpenGL. How do I get the CAMetalLayer that is, presumably, backing the SDL_Window that was created with the opengl flag set?
  4. How can I set this CAMetalLayer property from my c++ app. Do I have to write an Objective C wrapper?
2

What do you mean your iOS device is a Macbook? Are you referring to the iOS Simulator? Mac Catalyst?

A framebuffer object you create yourself OpenGL as HDR floating point doesn’t mean the display mode is set to HDR. Have you tried requesting a floating point display buffer with SDL_GL_SetAttribute()? Something like

    SDL_GL_SetAttribute(SDL_GL_RED_SIZE, 16);
    SDL_GL_SetAttribute(SDL_GL_GREEN_SIZE, 16);
    SDL_GL_SetAttribute(SDL_GL_BLUE_SIZE, 16);
    SDL_GL_SetAttribute(SDL_GL_FLOATBUFFERS, 1);

And then confirm it after the GL context has been created with SDL_GL_GetAttribute().

For changing properties of the underlying CAMetalLayer yourself, you’ll have to do it in Objective-C or Swift.

There may not reliably be a CAMetalLayer backing OpenGL on MacOS. On iOS, and ARM-based Macs, OpenGL is implemented as a layer on top of Metal, but on Macs with an Intel CPU (if you still care about those) OpenGL is implemented separately.

3

Typo. Sorry. My iOS device is an M4 iPad Pro.

I have set the SDL_GL attributes the way you describe but, as stated, I confirmed the result via glGetFramebufferAttachmentParameteriv for the framebuffer created by SDL.

Intel based Mac’s are not a concern.

Is there any example code that shows how to get the CAMetalLayer via the SDL public API? Is there a public API to call the function pointers on the display device?

4

I don’t see a way to directly get the Metal layer using only SDL. There’s SDL_Metal_GetLayer() but it requires you to already have obtained the Metal view, which SDL doesn’t seem to provide an API to access directly unless you’re using SDL_Renderer.

If you’re using SDL3, you can get it indirectly with properties and some Objective-C code.

  1. Call SDL_GetWindowProperties() to get the PropertyID
  2. Get the tag number for the Metal view (SDL_GetNumberProperty() paired with either SDL_PROP_WINDOW_UIKIT_METAL_VIEW_TAG_NUMBER or SDL_PROP_WINDOW_COCOA_METAL_VIEW_TAG_NUMBER)
  3. Get the underlying OS window pointer (SDL_GetPointerProperty() and SDL_PROP_WINDOW_[UIKIT/COCOA]_WINDOW_POINTER)
  4. Loop through the window’s views until you find one with the tag that matches the Metal view’s tag
  5. Get the view’s CALayer (CALayer *myLayer = view.layer) and cast that to CAMetalLayer (should probably check to see if it is a CAMetalLayer, something like if([MyMetalLayer isKindOfClass:[CAMetalLayer class]]))
  6. Do whatever

For SDL2 it’d have to be something like

  1. Call SDL_GetWindowWMInfo()
  2. Use that to get the window
  3. Loop through its views until you find the Metal view. Maybe by checking to see if its CALayer is a CAMetalLayer
  4. Profit

In Objective-C.

Given that Vulkan only works on iOS/macOS via MoltenVK, I wonder if that provides a way to get at the underlying Metal view/layer.

Also, I wouldn’t be surprised if iOS has some special property you need to set in your app’s info.plist to enable HDR.

6

I started work on this. I am using SDL3. I have two questions: on no. 3, what is the type of an underlying OS Windows pointer in SDL; on no. 4 how do you loop through the views?

7

For Windows, I have no idea.

For #4, on macOS, the NSWindow has a contentView property, which gives you an NSView that encompasses the whole window. Each NSView has a subviews property that is an NSArray of its subviews you can loop through.

iOS is similar, but with one less step. UIWindow is a subclass of UIView, so you don’t need to go through a contentView property and can instead access its subviews property directly (an NSArray of UIViews you can loop through).

Interestingly, the docs for SDL_GetWindowProperties() say you can query whether HDR is enabled (read-only property SDL_PROP_WINDOW_HDR_ENABLED_BOOLEAN and a few others) but there is no corresponding property you can set to enable HDR in SDL_CreateWindowWithProperties().

8

I implemented this after checking that `SDL_GetNumberProperty()` for `SDL_PROP_WINDOW_{UIKIT,METAL}_VIEW_TAG_NUMBER` returned a non-zero value. However it turns out that neither the NSWindow on macOS nor the UIWindow on iOS have a view with that tag number nor do they have a view with a CAMetalLayer layer. The NSWindow on macOS has no subviews or layers.

It therefore seems impossible to display an HDR rendering from an OpenGL application created using SDL.

9

I have given up on HDR display from my OpenGL app as the SDL window created for an OpenGL context does not have a CAMetalLayer backing it.

I am now investigating why I am not getting HDR display from my Vulkan app on iOS. It works on macOS. I can see a CAMetalLayer being created in SDL_Vulkan_CreateSurface, specifically byUIKit_Metal_CreateView which is called via UIKit_Vulkan_CreateSurface.

UIKit_Metal_CreateViewcalls [metalview setSDLWindow:window] towards the end. I get a bit lost in SetSDLWindow. A comment in UIKit_Vulkan_CreateSurface states that the metalview is a subview of the window but when I call the suggested Obj-C code to look at the subviews on the window after SDL_Vulkan_CreateSurface returns the subviews array has only one view and it is not the metalview.

    for (UIView* view in window.subviews) {
        //if (view.tag == tag_number) {
            CALayer* layer = view.layer;
            if ([layer isKindOfClass:[CAMetalLayer class]]) {
              if (@available(iOS 16.0, *)) {
                ((CAMetalLayer*)layer).wantsExtendedDynamicRangeContent = enable;
              } else {
                // Fallback on earlier versions
              }
            }
            //break;
        //}
    }

How can I find from the SDL_Window the CAMetalLayer that was created by SDL_Vulkan_CreateSurface?

At this point I’m just trying to see if wantsExtendedDynamicRangeContent is set on the CAMetalLayer. When I printed the variable metalview in the debugger when stopped in UIKit_Metal_CreateView I did not see a field with that name which I don’t understand. I am running iPadOS 26 on an M4 iPad Pro. Because of this I want to run code that references the field.

I also do not see a field called tag in UIKitView.

10

UIView definitely has a tag property: Apple docs for UIView

I’d forgotten UIView has a -viewWithTag: function, so you can do something like

UIView *metalView = [window viewWithTag:metalViewTag];
if(metalView != nil) {
    // whatever
} else {
    // oops
}
11

I have found the reason for the issue with HDR in the Vulkan app on iPadOS.

wantsExtendedDynamicRangeContentis not being set on the CAMetalLayer on iPadOS. On macOS, it is set during vkCreateSwapchainKHR when an HDR color space has been selected as the surface format from those returned by vkGetPhysicalDeviceSurfaceFormatsKHR(part of the VK_EXT_swapchain_color_space extension).

For the iPadOS (UIKit) case, I added the following Obj-C code to set wantsExtendedDynamicRangeContent to YES. I call this after creating the swap chain,

#include <UIKit/UIKit.h>
#include <QuartzCore/QuartzCore.h>
#include <SDL3/SDL.h>

void
setWantsExtendedDynamicRangeContent(SDL_Window* window, bool enable)
{
    SDL_PropertiesID wprops = SDL_GetWindowProperties(window);
    NSInteger tag =
      SDL_GetNumberProperty(wprops, SDL_PROP_WINDOW_UIKIT_METAL_VIEW_TAG_NUMBER, 0);
    UIWindow* uiwindow =
      (UIWindow*)SDL_GetPointerProperty(wprops, SDL_PROP_WINDOW_UIKIT_WINDOW_POINTER, nullptr);

    UIView *metalView = [uiwindow viewWithTag:tag];
    if (metalView != nil) {
        CALayer* layer = metalView.layer;
        if ([layer isKindOfClass:[CAMetalLayer class]]) {
            if (@available(iOS 16.0, *)) {
              ((CAMetalLayer*)layer).wantsExtendedDynamicRangeContent = enable;
            }
        }
    }
}

With this I see HDR on my iPad’s OLED screen. It looks like a MoltenVK bug. I will check for updates.

Note that wantsExtendedDynamicRangeContent does not show up in the debugger when I p *(CAMetalLayer*)layer. All I see is a field called _priv. I have to explicitly tell the debugger to display the value.

There is considerable difference in appearance between the 3 linear HDR color spaces returned by vkGetPhysicalDeviceSurfaceFormatsKHR on Apple devices: VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT, VK_COLOR_SPACE_DISPLAY_P3_LINEAR_EXTand VK_COLOR_SPACE_BT2020_LINEAR_EXT. I suspect that, to get consistent color, I need to add primaries conversion to the app’s fragment shaders to convert from the app’s BT.709 to whatever the selected color space is. I have not found any information about how Vulkan apps are supposed to handle the color spaces provided by VK_EXT_swapchain_color_space.

Thank you @sjr for your invaluable help.

1 Like