VR development in Unity

VR development shares common workflows and design considerations with any real-time 3D development in Unity. However, distinguishing factors include:

• Richer user input: in addition to “traditional” button and joystick controllers, VR devices provide spatial head, controller, and (in some cases) hand and finger tracking.
• More “intimate” interaction with the environment: in conjunction with the possibilities of richer input, VR raises the expectations of much closer and “physical” interaction with the environment than typical 3D games and applications. Users expect to be able to pick things up and interact with objects in the environment. With head tracking, the cameraA component which creates an image of a particular viewpoint in your scene. The output is either drawn to the screen or captured as a texture. More info
  See in Glossary can get much closer to the walls and other boundaries of the environment – even passing through them.
• User comfort concerns: many people experience motion sickness in VR when camera movement doesn’t match the movement of their head. You can mitigate the causes of motion sickness by maintaining a high frame rate, offering a range of locomotion options so that users can choose a mode they are comfortable with, and avoiding moving the camera independently of the user’s head tracking.

To get started with VR development, use the XR Plug-in Management system to install and enable XR provider plug-ins for the devices you want to support. See XR Project set up for more information.

Basic VR scene elements

A basic VR sceneA Scene contains the environments and menus of your game. Think of each unique Scene file as a unique level. In each Scene, you place your environments, obstacles, and decorations, essentially designing and building your game in pieces. More info
See in Glossary should contain an XR Origin, which defines the 3D origin for tracking data. This collection of GameObjectsThe fundamental object in Unity scenes, which can represent characters, props, scenery, cameras, waypoints, and more. A GameObject’s functionality is defined by the Components attached to it. More info
See in Glossary and components also contains the main scene Camera and the GameObjects representing the user’s controllers. See Set up an XR scene for instructions on setting up a basic VR scene.

Beyond the basics, you typically need a way for the user to move around and to interact with the 3D world you have created. The XR Interaction Toolkit provides components for creating interactions like selecting and grabbing objects. It also provides a customizable locomotion system. You can use the Input System in addition to or instead of the XRAn umbrella term encompassing Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) applications. Devices supporting these forms of interactive applications can be referred to as XR devices. More info
See in Glossary Interaction Toolkit.

VR packages

Most of the features and APIs used for VR development in Unity are provided through packages. These packages include:

• Provider plug-ins
• XR Interaction Toolkit
• XR Core Utilities
• Input System
• VR project template
• Hand tracking

VR provider plug-ins

To build VR apps in Unity, use the XR Plug-inA set of code created outside of Unity that creates functionality in Unity. There are two kinds of plug-ins you can use in Unity: Managed plug-ins (managed .NET assemblies created with tools like Visual Studio) and Native plug-ins (platform-specific native code libraries). More info
See in Glossary Management system to add and enable provider plug-ins for the devices you want to support. See XR Project set up for instructions.

The VR provider plug-ins supported by Unity include:

• Apple visionOS XR Plugin for the Apple Vision Pro.
• Oculus for Oculus Rift, Meta Quest 2, and Quest Pro.
• OpenXR for any device with an OpenXR runtime, including Meta headsets, VIVE headsets, Valve SteamVR, HoloLens, Windows Mixed RealityMixed Reality (MR) combines its own virtual environment with the user’s real-world environment and allows them to interact with each other.
  See in Glossary, and others.
• PlayStation VR (available to registered PlayStation developers) for Sony PS VR and PS VR2 devices. Visit PlayStation Partners for more information.
• Mock HMD for simulating a VR headset in the Unity Editor Play mode view.

Note: Many headset makers are working toward using the OpenXR runtime as a standard. However, this process is not complete and there can be feature discrepancies between OpenXR and a headset maker’s own provider plug-in or SDK

XR Interaction Toolkit

The XR Interaction Toolkit can make it easier and faster to develop VR applications. The XR Interaction Toolkit provides:

• An XR Origin set up with controllers.
• XR controller setups with Input System presets for basic interactions like select and grab.
• Interactor and Interactable components for creating object manipulation.
• A configurable locomotion system.
• XR UI input.

XR Core Utilities

The XR Core Utilities package contains software utilities used by other Unity XR plug-ins and packages. Typically, this package gets installed in your project as a dependency of other XR packages.

Input System

The Unity Input System package not only supports accessing user input from VR controller buttons and joysticks, but also provides access to XR tracking data and haptics. The Input System package is required if you use the XR Interaction Toolkit or the OpenXR provider plug-in.

Hand tracking

Hand tracking is a feature that allows users to interact with a VR application using their hands. Hand tracking is supported by the XR Hands package.

The Hands package provides:

• A standard hand data model.
• An API for accessing hand tracking data.
• The XR Hand Skeleton Driver component, which maps a set of Transforms to their corresponding hand jointsA physics component allowing a dynamic connection between Rigidbody components, usually allowing some degree of movement such as a hinge. More info
  See in Glossary and updates those Transforms as tracking data is received.
• The XR Hand Mesh Controller, which enables and disables a meshThe main graphics primitive of Unity. Meshes make up a large part of your 3D worlds. Unity supports triangulated or Quadrangulated polygon meshes. Nurbs, Nurms, Subdiv surfaces must be converted to polygons. More info
  See in Glossary as hand tracking is acquired or lost.
• A HandVisualizer sample that demonstrates how to use the hand tracking API.