Cross compiling toolchain in Github codespace

I am trying to set up a github codespace and compile SDL3 from source (since codespace distro does not come with precompiled SDL3 dev packages). I followed directions from https://wiki.libsdl.org/SDL3/README-cmake and successfully built SDL3 as well as a proof of concept Linux executable.

cmake -S . -B build-linux -DSDL_TESTS=OFF -DSDL_EXAMPLES=OFF -DSDL_SHARED=ON -DSDL_STATIC=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build-linux --config Release
sudo mkdir /usr/local/sdl-linux
sudo cmake --install build-linux --prefix /usr/local/sdl-linux

The CMakeLists.txt of my project is:

cmake_minimum_required(VERSION 3.16)
project(sdl_hello LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

set(SDL3_DIR "/usr/local/sdl-linux/lib/cmake/SDL3")
find_package(SDL3 REQUIRED CONFIG)

add_executable(sdl_hello
    src/main.cpp
)

target_link_libraries(sdl_hello PRIVATE SDL3::SDL3)

I intend to support building both for Linux and for Windows, from the start of the project. What I tried is

sudo apt install mingw-w64
cmake -S . -B build-windows -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ -DSDL_TESTS=OFF -DSDL_EXAMPLES=OFF -DSDL_SHARED=ON -DSDL_STATIC=ON -DCMAKE_BUILD_TYPE=Release
sudo cmake --install build-windows --prefix /usr/local/sdl-windows

At this point I have to admit that I am at a loss. I am new to cmake, and I do not even know if what I have done so far is optimal, or even correct for that matter. In addition I would like a cmake file which would support both Linux compilation through g++, and Windows through mingw64. Blindly following the README worked for Linux but no such documentation exists for cross compiling (at least according to my search).
How do I continue from here?

It looks like this post should help:
Build SDL3.DLL from Linux
Make sure that your SDL3 and sub libraries (ttf, mixer, image) source files are all recently updated.

I should point out that the binaries are available now, so building from scratch is only necessary if you want to have the latest updated version possible. Otherwise just download the developer mingw versions found on github:

SDL 3.4.8
SDL Image
SDL TTF
SDL Mixer

Edit: I think from here you just create a local copy into your project that you can link against in code space. (I get a bit shaky in my understanding at this point since I’ve never used github codespace)