mixed-reality
Mixed reality feature samples and apps
Every development journey starts with a look back at what other developers have successfully built - mixed reality is no different. Currently, all of our tutorials and sample apps are built in Unity or Unreal. As we develop content for other engines and platforms, you’ll find them under the relevant heading in the Table of Contents.
Sample application case studies
Feature samples
For each of the developer scenarios listed below, there are feature samples that correspond to specific implementations that are covered in our documentation and cover a range of development platforms and hardware devices.
| Scenario | Feature sample | Engine | Description |
|---|---|---|---|
| Build basic Unity mixed reality scenarios | OpenXR with Unity samples | Unity C# | Get started with cross-platform developer tools using the latest Unity 2020.LTS and OpenXR plugin. |
| Anchoring Strategies | Local anchor | Persist and share spatial anchors across application sessions and across devices. See the Spatial anchors article. | |
| Azure Spatial Anchors samples | Build spatially aware mixed reality applications with the essential capabilities of Azure Spatial Anchors. | ||
| QR Codes | Unity C# | Detect QR Codes in the environment. | |
| Collaboration in mixed reality | User identity | Unity C# | Set up your HoloLens 2 device using Azure Active Directory (AAD) credentials. |
| Azure Spatial Anchors samples | Build spatially aware mixed reality applications with the essential capabilities of Azure Spatial Anchors. | ||
| Spatial interaction | Basic hologram sample | Windows 10 C++ | Render a spinning cube in Windows Mixed Reality. |
| Scene/Object understanding | Scene understanding samples | Unity C# | Help design environmentally-aware mixed reality applications. |
| Azure Object Anchors samples | Unity C# | Detect an object in the physical world using a 3D model and estimate its 6DoF pose with Azure Object Anchors. | |
| Contextual data overlays | QR Codes | Unity C# | Detect QR Codes in the environment. |
| Poster tracker sample | Unity C# | Align a hologram with a real world object. | |
| Build mixed reality digital twins | Unity C# | Learn how to create a mixed reality application using Azure Digital Twins and Unity, a real-time 3D platform. | |
| Capturing camera data | Mixed reality capture sample | Windows 10 C++ | Capture the first-person experience of mixing real and digital worlds as a photo or video. |
| Research Mode samples | Windows 10 C++ | Access to key sensors on a HoloLens device for research applications. | |
| Holographic Remoting | Holographic Remoting player | Windows 10 C++ | Stream holographic content from a PC to your Microsoft HoloLens in real time by using a Wi-Fi connection. |
| Azure Remote Rendering samples | Unity C# | Test the Azure Remote Rendering service by using the resources for sample data listed here. | |
| Task management and guidance | Dynamics 365 Remote Assist | Collaborate more efficiently by working together from different locations with Dynamics 365 Remote Assist on HoloLens, HoloLens 2, Android, or iOS devices. | |
| Dynamics 365 Guides | Help operators learn during the flow of work by providing holographic instructions when and where they’re needed. | ||
| World Locking holograms | World locked physics sample | Unity C# | Explore a few virtual physics experiences that are enabled by World Locking Tools’ world-locked coordinate system. |
| Space pin sample | Unity C# | Move closer to the inner workings of a real-world application that needs to align a large object or objects with real-world features. The Space Pin sample offers a simplified and more focused view of the Space Pin feature. | |
| Ray pins example | Unity C# | Demonstrate how to set up Space Pins by manually manipulating marker objects into position using MRTK affordances. | |
| World Locking Tools with Azure Spatial Anchors sample | Unity C# | Provide a stable coordinate system that can be persisted across sessions and shared across devices in your application. This is made possible when combining World Locking Tools for Unity (WLT) with Azure Spatial Anchors (ASA). | |
| Managing Power and Thermals | Managing Power & Thermals | Unity C#, Win32 C++ | When the HoloLens 2 is running in warm environments or with heavy performance requirements (CPU/GPU usage, peripheral usage, etc.), PowerThermalNotification SDK can be used to subscribe to notification events, allowing the device to operate longer. See the PowerThermalNotification API reference. |
Build basic OpenXR scenarios
If you’re new to building basic mixed reality scenarios, these samples will help you to getting started.
For developers targeting Unity 2020 to build HoloLens 2 or mixed reality applications, the OpenXR Plugin can be used instead of the Windows XR Plugin for better cross-platform compatibilities. The Mixed Reality OpenXR Plugin also works well with the latest version of the Mixed Reality Toolkit (2.7.x).
| Sample | Reference article | Platform | Description |
|---|---|---|---|
| Mixed Reality OpenXR with Unity samples | Using the OpenXR plugin | Unity C# | These sample projects showcase how to build Unity applications for HoloLens 2 or Mixed Reality headsets using the Mixed Reality OpenXR plugin. The following sample scenarios are covered: <ul> <li> Using hand tracking joints and mesh inputs in Unity </li> <li> Using anchor and anchor persistence </li> <li> Using ARPlaneManager on HoloLens2 </li> <li> Using ARRaycastManager on HoloLens2 </li> <li> Using eye tracking on HoloLens2 </li> <li> Using locatable camera in unity </li> |
| OpenXR MRTK Base Unity project | See sample readme | Unity C# | This repo contains a Unity project that is set up with the Microsoft Mixed Reality Toolkit Foundations and Standard Assets packages and the Microsoft OpenXR Plugin package. |
| Using Unity 2020.3 and MRTK 2.7.2 | What is MRTK | Unity C# | MRTK-Unity is a Microsoft-driven project that provides a set of components and features that are used to accelerate cross-platform MR app development in Unity. |
| OpenXR Explorer | See sample readme | C++, Windows, Linux, OpenXR | OpenXR Explorer is a handy debug tool for OpenXR developers. It allows for easy switching between OpenXR runtimes, shows lists of the runtime’s supported extensions, and allows for inspection of common properties and enumerations, with direct links to relevant parts of the OpenXR specification! |
| OpenXR Samples for Mixed Reality Developers | What is OpenXR | C++ | These OpenXR samples use C++17 and Direct3D 11. The same source code works across UWP applications running on HoloLens 2 and Win32 applications running on Windows Desktop with Windows Mixed Reality immersive headsets. |
Collaboration in mixed reality
In mixed reality, people come together virtually to share experiences and collaborate. Samples listed here demonstrate some features that make such collaboration possible.
User identity
This sample sets up your HoloLens 2 device using Azure Active Directory (AAD) credentials, and then configures the device to use iris login.
| Sample | Reference article |
|---|---|
| AAD Login on HoloLens 2 | Overview of Microsoft identity platform |
Spatial interaction - basic hologram sample
This sample runs on Windows Mixed Reality and renders a spinning cube. You can interact with the cube by placing it in a new position, and various input methods are allowed. This sample works on PCs with headset devices attached, and also on Microsoft HoloLens.
| Sample | Reference article |
|---|---|
| Windows Universal samples - basic hologram | See sample readme |
Scene understanding
Scene understanding provides Mixed Reality developers with a structured, high-level environment representation. Scene understanding is designed for developing intuitive and environmentally aware applications, by combining the power of existing mixed reality runtimes. These runtimes are the highly accurate but less structured spatial mapping and new AI driven runtimes.
| Sample | Reference article | Platform | Description |
|---|---|---|---|
| Mixed Reality Scene Understanding samples (SU SDK) | Scene understanding | Unity C# and SU SDK | A Unity-based sample application that showcases Scene Understanding on HoloLens 2. |
| Scene Understanding sample (MRTK + SU SDK) | Scene understanding observer (MRTK) | Unity C#, MRTK + SU SDK | MRTK + Scene Understanding SDK sample. |
Contextual data overlays
Contextual data is the background information that provides a broader understanding of an event, person, or item. With Augmented Reality (AR), this information can be displayed and precisely aligned with physical objects to provide insights, instructions, service records, and other important data.
QR codes
HoloLens 2 can detect QR codes in the environment around the headset, establishing a coordinate system at each code’s real-world location.
| Sample | Reference article |
|---|---|
| QR code tracking in Unity | QR codes |
Poster tracker sample
It is often helpful to be able to align a hologram to a real-world object, or align multiple HoloLens devices to a common set of world coordinates, so that everyone sees the same holograms in the same location. For instance, in your Unity scene, you can add a “poster” where you want to anchor your scene (perhaps a game board), and then add holograms on or around it. Then you can print the poster, lay it on a table, and run the calibration/alignment tool, which will move the holographic version of the poster so that it aligns with the physical version of the poster. This moves all of the linked holograms to the correct alignment.
| Sample | Reference article |
|---|---|
| Poster calibration sample | See sample readme |
Build mixed reality digital twins
In this sample, you’ll learn how to create a mixed reality HoloLens 2 application using Azure Digital Twins and Unity, a real-time 3D platform.
| Sample | Reference article |
|---|---|
| Building mixed reality digital twins with Azure Digital Twins and Unity | Full learning path |
Camera captures
Unstructured environment sensor data that your Mixed Reality device captures are converted into powerful abstract or holographic representations of the physical world around us.
Holographic Mixed Reality Capture
Mixed reality capture (MRC) captures the first-person experience of the combined real and digital worlds as a photo or video and shares what you see with others in real time.
| Sample | Reference article |
|---|---|
| Mixed Reality Capture samples | Mixed Reality Capture |
Research Mode
Research Mode was introduced in the first-generation HoloLens to give access to key sensors on the device, specifically for research applications that are not intended for deployment. The sample applications below are examples for accessing and recording Research Mode streams and using the intrinsic and extrinsic.
| Sample application | Reference article |
|---|---|
| HoloLens (first gen) | Research Mode |
| HoloLens 2 | Research Mode |
Holographic Remoting
The Holographic Remoting Player is a companion app that connects to PC apps and games that support Holographic Remoting. Holographic Remoting streams holographic content from a PC to your Microsoft HoloLens in real time using a Wi-Fi connection, and is supported on HoloLens (first gen) and HoloLens 2.
| Sample | Reference article |
|---|---|
| Holographic Remoting samples | Holographic Remoting Overview |