Revitalized Godot VR Enhancements for Developers in Mixed Reality

The Godot VR enhancements have taken a significant leap forward, offering developers new tools and features that dramatically improve virtual reality (VR) and mixed reality applications. With these latest innovations, creators can design more immersive and engaging experiences for users, marking a remarkable shift in the capabilities of the Godot Engine.

If you are new to the Godot platform, it’s worth noting that it serves as a free and open-source alternative to widely used game engines like Unity and Unreal. Governed by the non-profit Godot Foundation, development occurs openly and transparently, which fosters a dynamic atmosphere for rapid innovation and enhancements.

In a recent update, Bastiaan Olij, one of the core developers focusing on XR (extended reality) support for Godot, offered a comprehensive look into these upgrades. A noteworthy aspect is the financial and technical support from Meta, which has positively impacted the latest features. As a result, developers can expect streamlined experiences not just with Meta Quest devices but with various other XR platforms too.

This announcement coincides with Meta releasing a port of the Godot editor for Horizon OS, enabling developers to create and iterate on games specifically for Quest 3 and Quest Pro headsets independently.

Critical Enhancements in XR Architecture

Enhanced Frame Timing for Improved Tracking

Previously, the initial integration of OpenXR in Godot 3 faced limitations that hindered overall performance. The rendering engine had tightly coupled communication with OpenXR, causing Godot to rely heavily on predictive algorithms for tracking entities like controllers. However, the release of Godot 4.3 significantly addressed these issues. The engine now effectively communicates with OpenXR when a new frame begins processing, enhancing tracking accuracy remarkably.

Standardization of Passthrough AR and Seethrough Features

Passthrough technology allows VR headsets to use video cameras to capture and display the real-world environment as a background. Unfortunately, device-specific implementations posed a challenge for developers targeting multiple platforms. Thankfully, Godot has standardized this process by utilizing OpenXR’s environment blend modes and placing device-specific code into OpenXR vendor plugins. This advancement enables developers to streamline their development efforts and reduce duplication in coding.

Unified Hand, Face, and Body Tracking

Godot has refined its hand tracking capabilities to be more cross-platform compatible, aligning closely with OpenXR and WebXR standards. The prior OpenXRHand node has been replaced by the more advanced XRHandModifier3D node, which is compatible across all XR systems. Additionally, support for face tracking has been introduced, adhering to the Unified Expressions standard for facial data management, along with body tracking that works with various devices using Godot’s retargeting logic.

Advancements in Vulkan Foveated Rendering

Foveated rendering boosts performance by allocating resources preferentially to the user’s primary field of view. Godot now effectively utilizes OpenXR’s foveated rendering support. At the same time, the Vulkan renderer incorporates Variable Rate Shading, optimizing the density used in rendering. This means users enjoy a sharper visual experience, especially in their focal areas.

Expanding WebXR Support

WebXR support in Godot keeps evolving, now featuring hand tracking capabilities along with Multi-Sample Anti-Aliasing (MSAA). Furthermore, various smaller issues and bugs have been addressed, boosting the robustness of the WebXR implementation and enhancing its capability to support augmented reality (AR) applications.

Specific Enhancements Tailored for Meta

Advanced Features for Passthrough and Hand Tracking

In addition to standardizing passthrough capabilities, improvements specific to Meta’s systems have been integrated into the Godot OpenXR Vendors plugin. Developers can now apply multiple filters to passthrough imagery, adjusting brightness, contrast, and saturation or mapping passthrough onto custom geometries. Enhanced hand tracking also offers access to hand meshes and collision shapes, streamlining gesture recognition tasks.

New Support for Composition Layers

Composition layers in OpenXR allow developers to render sharp 2D content onto floating panels, which enhances clarity compared to traditional textures applied to 3D shapes. This technique proves especially useful for creating in-game menus, media displays, and heads-up interfaces, making small text more readable. Both core implementations and specific extensions for Meta have been established, refining the overall user experience.

Scene Discovery and Spatial Anchors Features

Meta’s Scene Discovery feature enables Quest users to scan their real-world surroundings, making it easier to integrate virtual elements seamlessly into available spaces. Moreover, spatial anchors make it possible for these placements to be persistent, allowing users to revisit and interact with previously positioned virtual objects effortlessly.

Expanding Support for XR Devices

Pico 4 Integration

The level of support for Pico headsets has seen significant enhancement recently. Pico has adopted the official Khronos Android Loader for OpenXR, transitioning from a custom loader, a move that has received full backing from Godot.

Magic Leap 2 Compatibility

Godot’s vendor plugin has integrated support for the Magic Leap ecosystem, which is particularly exciting due to Magic Leap’s nature as an optical AR device using AMD hardware. It necessitates the Vulkan mobile renderer for peak performance, a requirement effectively addressed by Godot.

What to Expect in the Future

Looking ahead, Olij has hinted at more exciting features and enhancements on the way. Notably, numerous key upgrades are currently under development.

Separate Rendering Thread Under Development

Plans to establish a separate rendering thread in Godot 4.4 are progressing. Major code cleanup has been completed to ensure smooth and effective threading during rendering. This enhancement aims to ensure a consistently smooth framerate, even on standalone devices, significantly boosting user experience for XR applications.

Logitech MX Ink Support in the Pipeline

The impending Godot 4.4 release is set to include support for the Logitech MX Ink, a positionally tracked stylus compatible with Meta Quest devices. The development team is exploring integration options for this feature into a future patch release for version 4.3 as well.

Support for Standalone HTC Vive Headsets

HTC has provided the Godot team with hardware units from the HTC Vive XR Elite, enabling them to ensure compatibility with standalone HTC devices. This collaboration focuses on enhancing the user experience for these headsets, further cementing Godot’s position in the VR landscape.

https://docs.godotengine.org/en/stable/tutorials/xr/index.html