The Windows Mixed Reality home is the starting point where users land before launching applications. When designing your application for Windows Mixed Reality headsets, use a 3D model as an app launcher and place 3D deep links into the Windows Mixed Reality home. This article outlines the guidelines for creating 3D models compatible with the Windows Mixed Reality home.
When creating 3D models for Windows Mixed Reality, there are some requirements that all assets must meet:
[!NOTE] The 3D Viewer app supports different formats and resolutions, but ultimately converts models to .glb/glTF before displaying them in the Mixed Reality home.
The rest of this article includes a detailed overview of these requirements and extra guidelines to ensure your models work well with the Windows Mixed Reality home.
The Windows Mixed Reality home expects 3D assets to be delivered using the .glb file format with embedded images and binary data. Glb is the binary version of the glTF format, which is a royalty free open standard for 3D asset delivery maintained by the Khronos group. As glTF evolves as an industry standard for interoperable 3D content, so will Microsoft’s support for the format across Windows apps and experiences. If you haven’t created a glTF asset before you can find a list of supported exporters and converters on the glTF working group github page.
Windows expects assets to be generated using the following modeling guidelines to ensure compatibility with the Mixed Reality home experience. When modeling in your program of your choice, keep in mind the following recommendations and limitations:
The Windows Mixed Reality home doesn’t support models with more than 10,000 triangles. It’s recommended you triangulate your meshes before exporting to ensure they don’t exceed this count. Windows MR also supports optional geometry levels of detail (LODs) to ensure a performant and high-quality experience. The WindowsMRAssetConverter will help you combine 3 versions of your model into a single .glb model. Windows determines which LOD to display based on the amount of screen real estate the model is taking up. Only 3 LOD levels are supported with the following recommended triangle counts:
| LOD Level | Recommended Triangle Count | Max Triangle Count |
|---|---|---|
| LOD 0 | 10,000 | 10,000 |
| LOD 1 | 5,000 | 10,000 |
| LOD 2 | 2,500 | 10,000 |
The Windows Mixed Reality home doesn’t support models with more than 64 nodes or 32 submeshes per LOD. Nodes are a concept in the glTF specification that define the objects in the scene. Submeshes are defined in the array of primitives on the mesh in the object.
| Feature | Description | Max Supported | Documentation |
|---|---|---|---|
| Nodes | Objects in the glTF Scene | 64 per LOD | Here |
| Submeshes | Sum of primitives on all meshes | 32 per LOD | Here |
Textures should be prepared using a PBR metal roughness workflow. Begin by creating a full set of textures including Albedo, Normal, Occlusion, Metallic, and Roughness. Windows Mixed Reality supports textures with resolutions up to 4096x4096 but it’s recommended that you author at 512x512. Textures should be authored at resolutions in multiples of 4. This is a requirement for the compression format applied to textures in the exporting steps outlined below. When generating mip maps or a texture, the lowest mip must be a maximum of 4x4.
| Recommended Texture Size | Max Texture Size | Lowest Mip |
|---|---|---|
| 512x512 | 4096x4096 | max 4x4 |
Raw color with no lighting information. This map also contains the reflectance and diffuse information for metal (white in the metallic map) and insulator (black in the metallic map) surfaces respectively.
Tangent Space Normal map
Describes the microsurface of the object. White 1.0 is rough Black 0.0 is smooth. This map gives the asset the most character, as it truly describes the surface. For example, scratches, fingerprints, smudges, grime, and so on.
Value scale map showing areas of occluded light, which blocks reflections
Tells the shader if something is metal or not. Raw Metal = 1.0 white Non-metal = 0.0 black. There can be transitional gray values that indicate something covering the raw metal such as dirt, but in general this map should be black and white only.
Windows Mixed Reality home offers a series of optimizations on top of the core glTF spec defined using custom extensions. These optimizations are required on Windows versions <= 1709 and recommended on newer versions of Windows. You can easily optimize any glTF 2.0 model using the Windows Mixed Reality Asset Converter available on GitHub. This tool will perform the correct texture packing and optimizations as specified below. For general usage, we recommend using the WindowsMRAssetConverter, but if you need more control over the experience and would like to build your own optimization pipeline then you can refer to the detailed specification below.
[!NOTE] For a definitive list of what the possibilities are for exact model limits, refer to the 3D model optimization article for use in Dynamics 365 applications.
To improve asset loading time in Mixed Reality environments Windows MR supports rendering compressed DDS textures packed according to the texture packing scheme defined in this section. DDS textures are referenced using the MSFT_texture_dds extension. Compressing textures is highly recommended.
HoloLens-based mixed reality experiences expect textures to be packed using a 2-texture setup using the following packing specification:
| glTF Property | Texture | Packing Scheme |
|---|---|---|
| pbrMetallicRoughness | baseColorTexture | Red (R), Green (G), Blue (B) |
| MSFT_packing_normalRoughnessMetallic | normalRoughnessMetallicTexture | Normal (RG), Roughness (B), Metallic (A) |
When compressing the DDS textures the following compression is expected on each map:
| Texture | Expected Compression |
|---|---|
| baseColorTexture, normalRoughnessMetallicTexture | BC7 |
PC-based Windows Mixed Reality experiences for immersive (VR) headsets expect textures to be packed using a 3-texture setup using the following packing specification:
| glTF Property | Texture | Packing Scheme |
|---|---|---|
| pbrMetallicRoughness | baseColorTexture | Red (R), Green (G), Blue (B) |
| MSFT_packing_occlusionRoughnessMetallic | occlusionRoughnessMetallicTexture | Occlusion (R), Roughness (G), Metallic (B) |
| MSFT_packing_occlusionRoughnessMetallic | normalTexture | Normal (RG) |
When compressing the DDS textures the following compression is expected on each map:
| Texture | Expected Compression |
|---|---|
| normalTexture | BC5 |
| baseColorTexture, occlusionRoughnessMetallicTexture | BC7 |
| glTF Property | Texture | Packing Scheme |
|---|---|---|
| pbrMetallicRoughness | baseColorTexture | Red (R), Green (G), Blue (B) |
| MSFT_packing_occlusionRoughnessMetallic | roughnessMetallicOcclusionTexture | Roughness (R), Metallic (G), Occlusion (B) |
| MSFT_packing_occlusionRoughnessMetallic | normalTexture | Normal (RG) |
When compressing the DDS textures the following compression is expected on each map:
| Texture | Expected Compression |
|---|---|
| normalTexture | BC5 |
| baseColorTexture, roughnessMetallicOcclusionTexture | BC7 |
Windows MR uses geometry node LODs to render 3D models in different levels of detail depending on on-screen coverage. While this feature is technically not required, it’s recommended for all assets. Currently Windows supports 3 levels of detail. The default LOD is 0, which represents the highest quality. Other LODs are numbered sequentially, for example, 1, 2 and get progressively lower in quality. The Windows Mixed Reality Asset Converter supports generating assets that meet this LOD specification by accepting multiple glTF models and merging them into a single asset with valid LOD levels. The following table outlines the expected LOD ordering and triangle targets:
| LOD Level | Recommended Triangle Count | Max Triangle Count |
|---|---|---|
| LOD 0 | 10,000 | 10,000 |
| LOD 1 | 5,000 | 10,000 |
| LOD 2 | 2,500 | 10,000 |
When using LODs always specify 3 LOD levels. Missing LODs will cause the model to not render unexpectedly as the LOD system switches to the missing LOD level. glTF 2.0 doesn’t currently support LODs as part of the core spec. LODs should be defined using the MSFT_LOD extension.
LODs are displayed in Windows Mixed Reality based on a system driven by the screen coverage value set on each LOD. Objects that are currently consuming a larger portion of the screen space are displayed at a higher LOD level. Screen coverage isn’t a part of the core glTF 2.0 spec and must be specified using MSFT_ScreenCoverage in the “extras” section of the MSFT_lod extension.
| LOD Level | Recommended Range | Default Range |
|---|---|---|
| LOD 0 | 100% - 50% | 0.5 |
| LOD 1 | Under 50% - 20% | 0.2 |
| LOD 2 | Under 20% - 1% | 0.01 |
| LOD 4 | Under 1% | - |
[!NOTE] This feature was added as part of Windows 10 April 2018 Update. On older versions of Windows these animations won’t play back, however, they will still load if authored according to the guidance in this article.
The mixed reality home supports animated glTF objects on HoloLens and immersive (VR) headsets. If you wish to trigger animations on your model, you’ll need to use the Animation Map extension on the glTF format. This extension lets you trigger animations in the glTF model based on the user’s presence in the world, for example trigger an animation when the user is close to the object or while they’re looking at it. If you glTF object has animations, but doesn’t define triggers the animations won’t be played back. The section below describes one workflow for adding these triggers to any animated glTF object.
First, download the following tools if you don’t have them already. These tools will make it easy to open any glTF model, preview it, make changes and save back out as glTF or .glb:
Start by opening up the glTF model in VSCode by dragging the .glTF file into the editor window. If you’ve a .glb instead of a .glTF file you can import it into VSCode using the glTF Tools addon that you downloaded. Go to “View -> Command Palette” and then begin typing “glTF” in the command palette and select “glTF: Import from glb”, which will pop up a file picker for you to import a .glb with.
Once you’ve opened your glTF model, you should see the JSON in the editor window. You can also preview the model in a live 3D viewer using the by right-clicking the file name and selecting the “glTF: Preview 3D Model” command shortcut from the right click menu.
Animation triggers are added to glTF model JSON using the Animation Map extension. The animation map extension is publicly documented here on GitHub (NOTE: THIS IS A DRAFT EXTENSION). To add the extension to your model just scroll to the end of the glTF file in the editor and add the “extensionsUsed” and “extensions” block to your file if they don’t already exist. In the “extensionsUsed” section, you’ll add a reference to the “EXT_animation_map” extension and in the “extensions” block you’ll add your mappings to the animations in the model.
As noted in the spec you define what triggers the animation using the “semantic” string on a list of “animations”, which is an array of animation indices. In the example below we’ve specified the animation to play while the user is gazing at the object:
"extensionsUsed": [
"EXT_animation_map"
],
"extensions" : {
"EXT_animation_map" : {
"bindings": [
{
"semantic": "GAZE",
"animations": [0]
}
]
}
}
The following animation triggers semantics are supported by the Windows Mixed Reality home.
Once you’ve made the changes to your glTF model, you can save it directly as glTF. You can also right-click the name of the file in the editor and select “glTF: Export to GLB (binary file)” to export a .glb.
Animations can’t be longer than 20 minutes and can’t contain more than 36,000 keyframes (20 mins at 30 FPS). Additionally when using morph target-based animations don’t exceed 8192 morph target vertices or less. Exceeding these counts will cause the animated asset to be unsupported in the Windows Mixed Reality home.
| Feature | Maximum |
|---|---|
| Duration | 20 minutes |
| Keyframes | 36,000 |
| Morph Target Vertices | 8192 |
Windows MR doesn’t support flipping geometry using negative scales. Geometry with negative scales will likely result in visual artifacts.
The glTF asset MUST point to the default scene using the scene attribute to be rendered by Windows MR. Additionally the Windows MR glTF loader before the Windows 10 April 2018 update requires accessors:
The following material properties are used from core glTF 2.0 spec but not required:
The following material properties are ignored from core spec:
Windows MR doesn’t support primitive mode lines and points.
Only a single UV vertex attribute is supported.