Using Gemini: Basis for VR lossless dual gpu

To bridge the popular Steam Lossless Scaling (LS) application—known for its LSFG frame generation and dual-GPU offloading capabilities—into a true OpenXR VR environment, you have to bypass the limitation where standard desktop hooks only capture the flat mirror window.

Achieving a true VR-native equivalent of Lossless Scaling that can leverage a secondary GPU for stereo frame generation requires intercepting the OpenXR swapchain directly.

Phase 1: Architecture of VR-Native Lossless Scaling

Unlike flat screen games where Lossless Scaling grabs a Win32/DXGI window swapchain, OpenXR renders two distinct eye projections submitted directly to a headset runtime. To construct a localized pipeline:

* OpenXR API Layer Hook: You intercept the xrEndFrame function via an OpenXR active layer (similar to how OpenXR Toolkit or PrimaShock operate).

* Asymmetric Dual-GPU Offload:

* GPU 0 (Primary): Renders the base simulation/game at a reduced native resolution (e.g., 70% scale) to maintain a high base framerate (e.g., solid 45 FPS to target a locked 90Hz/120Hz final output).

* GPU 1 (Secondary / iGPU): Pulls the rendered left and right eye textures over the PCIe bus via an asynchronous Vulkan/DX12 P2P memory copy.

* Stereo LSFG (Lossless Scaling Frame Generation): The secondary GPU runs a modified optical flow and machine-learning interpolation pass independently on both eye buffers, synthesizing intermediate frames (45 \text{ FPS} \rightarrow 90 \text{ FPS}) without stealing core frame-time from the primary rendering card.

Phase 2: Building the OpenXR Interception Layer (C++)

To implement a custom layer that acts like Lossless Scaling inside the VR compositor loop, structure your OpenXR runtime hook framework as follows:

1. Initialize the Layer and Intercept xrNegotiateLoaderInterface

Your DLL must expose the standard OpenXR loader negotiation entry point so the runtime routes API calls through your scaling wrapper.

// Basic structure for OpenXR layer negotiation

extern “C” {

XrResult XRAPI_CALL my_xrNegotiateLoaderInterface(

    const XrNegotiateLoaderInfo\* loaderInfo,

    const char\* layerName,

    XrNegotiateLayerRequest\* layerRequest) {

    

    // Setup function pointers for interception

    layerRequest->getInstanceProcAddr = my_xrGetInstanceProcAddr;

    layerRequest->xrCreateInstance = my_xrCreateInstance;

    return XR_SUCCESS;

}

}

2. Hook xrEndFrame for Dual-GPU Processing

Inside xrEndFrame, grab the eye textures before they are submitted to the compositor, push them to your secondary GPU queue, execute the scaling/frame-gen algorithm, and pass the synthesized frames down the chain.

XrResult XRAPI_CALL my_xrEndFrame(

XrSession session, 

const XrFrameEndInfo\* frameEndInfo) {

// 1. Inspect layer projections for Left and Right eyes

const XrCompositionLayerProjection\* projectionLayer = nullptr;

for (uint32_t i = 0; i < frameEndInfo->layerCount; ++i) {

    if (frameEndInfo->layers\[i\]->type == XR_TYPE_COMPOSITION_LAYER_PROJECTION) {

        projectionLayer = reinterpret_cast<const XrCompositionLayerProjection\*>(frameEndInfo->layers\[i\]);

        break;

    }

}

if (projectionLayer) {

    // 2. Offload textures to Secondary GPU (GPU 1) via P2P Vulkan/DX12 queue

    // Execute LSFG frame generation algorithm for alternate frame injection

    TriggerSecondaryGPUFrameGen(projectionLayer->views\[0\].subImage, 

                                 projectionLayer->views\[1\].subImage);

}

// 3. Pass modified frame info to the native runtime (SteamVR / Meta / Virtual Desktop)

return g_next_xrEndFrame(session, frameEndInfo);

}

Phase 3: Practical Workaround (Using the Steam App Today)

If you want to use the current Steam Lossless Scaling application alongside a secondary GPU setup right now without writing a custom OpenXR driver, you must use the Desktop/Windowed Mirror Method:

* Configure Dual-GPU Windows Mapping:

* Set your primary discrete GPU to handle high-performance rendering.

* Bind the Steam Lossless Scaling application explicitly to your secondary GPU or integrated graphics (iGPU) via Windows Graphics Settings.

* Launch the VR Simulator in Windowed Mirror Mode:

* Open your VR title (e.g., Assetto Corsa Competizione or flight sims) with its desktop mirror view active and set to a borderless window matching your target downscaled resolution.

* Hook via Lossless Scaling:

* Target the game’s desktop mirror window inside the Lossless Scaling app.

* Enable LSFG, set your preferred performance scaling algorithm (like LS1 or bilinear upscaling), and trigger the scaling hotkey.

* The Catch: Because this captures the compressed desktop preview window rather than the direct OpenXR compositor stream, head-rotation latency on the generated frames will feel decoupled (resembling asynchronous reprojection on a flat monitor). For true head-locked zero-latency performance, you must rely on native runtime features like Virtual Desktop’s SpaceWarp

or build out the explicit OpenXR P2P layer outlined in Phase 2.

To create a custom OpenXR API Layer (the structural basis needed for hooking execution routines like frame pacing, view downscaling, or multi-GPU texture offloading), you need two primary elements:

* A JSON Manifest File that tells the OpenXR loader your layer exists.

* A C/C++ Shared Library (.dll or .so) that exports interface negotiation functions and intercepts specific OpenXR calls (like xrCreateInstance or xrEndFrame).

Step 1: The Layer Manifest File (XR_APILAYER_MY_CUSTOM_LAYER.json)

The OpenXR loader scans specific system paths or paths designated by environment variables for JSON manifests. This file points the loader directly to your compiled library binary.

{

“file_format_version”: “1.0.0”,

“api_layer”: {

"name": "XR_APILAYER_MY_CUSTOM_LAYER",

"library_path": "./my_custom_layer.dll",

"api_version": "1.0",

"implementation_version": "1",

"description": "Custom OpenXR Layer for Dual-GPU / Scaling Operations",

"functions": {

  "instance_override": \[\]

},

"enable_environment": {

  "XR_ENABLE_API_LAYERS": "XR_APILAYER_MY_CUSTOM_LAYER"

}

}

}

Step 2: Boilerplate Layer Implementation (C++)

Your shared library must implement the Khronos-standard loader negotiation interface. This intercepts calls down the chain, allowing you to wrap functions such as frame presentation or instance creation.

#define XR_NO_HPP_FUNCTIONS

#include <openxr/openxr.h>

#include <openxr/openxr_loader_negotiation.h>

#include

// Global pointer to the downstream function for chain continuity

static PFN_xrGetInstanceProcAddr g_nextGetInstanceProcAddr = nullptr;

static PFN_xrEndFrame g_nextXrEndFrame = nullptr;

// 1. Intercepted xrEndFrame (Where frame timing and texture manipulation happen)

XrResult XRAPI_CALL Custom_xrEndFrame(XrSession session, const XrFrameEndInfo* frameEndInfo) {

// --- CUSTOM HOOK LOGIC HERE ---

// Example: Inspect frameEndInfo->layers to grab swapchain projection textures

// before they are handed off to the active VR runtime compositor.

// --------------------------------

// Pass execution down the layer chain to the actual runtime

return g_nextXrEndFrame(session, frameEndInfo);

}

// 2. Intercepted xrGetInstanceProcAddr to route function hooks

XrResult XRAPI_CALL Custom_xrGetInstanceProcAddr(XrInstance instance, const char* name, PFN_xrVoidFunction* function) {

if (std::string(name) == "xrEndFrame") {

    \*function = reinterpret_cast<PFN_xrVoidFunction>(Custom_xrEndFrame);

    return XR_SUCCESS;

}



// Fall back to the next layer or runtime in the stack

return g_nextGetInstanceProcAddr(instance, name, function);

}

// 3. Intercepted API Layer Instance Creation

XrResult XRAPI_CALL Custom_xrCreateApiLayerInstance(

const XrInstanceCreateInfo\* info,

const struct XrApiLayerCreateInfo\* apiLayerInfo,

XrInstance\* instance) {

// Capture the downstream function pointers provided by the loader structure

g_nextGetInstanceProcAddr = apiLayerInfo->nextInfo->nextGetInstanceProcAddr;



PFN_xrCreateApiLayerInstance nextCreateApiLayerInstance = 

    reinterpret_cast<PFN_xrCreateApiLayerInstance>(apiLayerInfo->nextInfo->nextCreateApiLayerInstance);

// Call down the chain to create the actual instance

XrResult result = nextCreateApiLayerInstance(info, apiLayerInfo, instance);

if (result != XR_SUCCESS) {

    return result;

}

// Bind local function pointers for downstream calls

g_nextGetInstanceProcAddr(\*instance, "xrEndFrame", reinterpret_cast<PFN_xrVoidFunction\*>(&g_nextXrEndFrame));

return XR_SUCCESS;

}

// 4. Required Entry Point for OpenXR Loader Negotiation

extern “C” {

XrResult XRAPI_CALL xrNegotiateLoaderApiLayerInterface(

    const XrNegotiateLoaderInfo\* loaderInfo,

    const char\* layerName,

    XrNegotiateLayerRequest\* layerRequest) {

    

    layerRequest->layerInterfaceVersion = XR_CURRENT_LOADER_API_LAYER_INTERFACE_VERSION;

    layerRequest->minApiLayerVersion = XR_MAKE_VERSION(1, 0, 0);

    layerRequest->maxApiLayerVersion = XR_MAKE_VERSION(1, 0, 32);

    

    layerRequest->getInstanceProcAddr = Custom_xrGetInstanceProcAddr;

    layerRequest->createApiLayerInstance = Custom_xrCreateApiLayerInstance;

    return XR_SUCCESS;

}

}

Step 3: Compiling and Activating Your Layer

* Build Configuration: Compile the code above into a shared library (my_custom_layer.dll on Windows) using CMake or Visual Studio, linking against the official OpenXR headers from the Khronos SDK.

* Deployment: Place both your compiled .dll and the .json manifest into your project directory.

* Activation via Environment Variables: Before launching your OpenXR simulation or game, force the OpenXR loader to pick up your layer by setting the environment path:

set

XR_API_LAYER_PATH=C:\Path\To\Your\LayerFolder\

set XR_ENABLE_API_LAYERS=XR_APILAYER_MY_CUSTOM_LAYER