Holographic Remoting combines advanced technologies and uses many terms, which could introduce some ambiguities. Therefore, the following section will list some of the terminology related to Holographic Remoting.
[!Note] The descriptions of these terms are rough and simple explanations targeted on the use with Holographic Remoting, it is not a technical description.
| Term | Description |
|---|---|
| Player | The Player is the application that runs on your head-mounted display. The Player sends poses to the Remote application and receives video frames from the Remote in exchange, all in real time. You can find the Holographic Remoting Player in the Windows Store. The store Player provides the general functionality to use Holographic Remoting. You can use the store Player for various Remote applications and there is no need for you to create a Player application. If you need extended functionality in the Player application, you can write your own Player. For more information, see Writing a custom Player app |
| Remote | The Remote is the application that runs on a desktop computer or on a virtual machine in the cloud. The Remote receives poses from the Player, performs the calculation-intense rendering, and sends video frames back to the Player. You can find C++ Remote examples in our Holographic Remoting samples github repository. If you use Holographic Remoting within Unity or Unreal, the game engines are the Remote application. |
| Server | A server listens for an incoming connection. With Holographic Remoting, it’s possible that either the Player or the Remote is the server depending on your needs. This functionality is the reason we use the extra terms Player and Remote. For example, the store Player runs as a server, which waits for a connection from a Remote client. If your custom Player should be the client, you can configure the Remote app as server. |
| Client | A client connects to a server. As stated before, with Holographic Remoting, it’s possible that either the Player or the Remote is the client. If your Player has to be the client, you can create a custom Player. For more information, see Writing a custom Player app. |
| Hostname | The hostname is used to identify a device in the network. To establish a Holographic Remoting connection, you have to provide the server’s hostname (for example, the IP address) on the client side. With the hostname, the client knows where it can find the server in the network. |
| Ports | Ports are used by many Internet protocols, such as TCP or UDP, to assign packets to the correct processes. For the Holographic Remoting connection you also have to provide the port that the client and the server should use. The default port is 8265 on both sides. |
| TCP | The Transmission Control Protocol (TCP) is a common network protocol. TCP creates a connection between client and server by executing a three-way handshake. TCP is a “reliable” protocol–it detects errors and performs a retransmission if packets are lost. Unfortunately, the reliability comes with the cost of extra latency. |
| UDP | The User Datagram Protocol is a connectionless and “unreliable” protocol. UDP doen’t resend packets if they’re lost. One advantage of UDP is that the latency is lower compared to TCP. For real-time applications, such as Holographic Remoting, reducing latency is important. Luckily, not all the data has to be sent in a reliable fashion; this allows the use of UDP. For example, the Player sends the pose in real time with high frequency to the Remote. If one of the packets that contain the pose data gets lost, waiting for a retransmission would take so long that the pose is already outdated. In this case, the Remote can just use one of the subsequent new poses. |
| Firewall | A firewall protects a system from unwanted network accesses. Depending on your firewall settings, you have to allow Holographic Remoting applications and the used ports in order to connect successfully. |
| Data Channel | Data channels are used to send certain data between the Player and the Remote. Holographic Remoting uses various data channels, such as video and audio. If you need to send custom data between your Player and Remote apps, you can use a custom data channel. All the data channels, including the custom data channels, share the available bandwidth. For more information, see Custom data channels with the OpenXR API or Custom data channels with the Windows Mixed Reality API. |
| Bandwidth | In general, bandwidth expresses how many bits per second can be transferred. For Holographic Remoting, the available bandwidth between the Player and the Remote application is critical for the experience. The maximum bandwidth Holographic Remoting should use can be configured on the Remote application. The actual available bandwidth depends on various factors such as the physical connection or other traffic in the network. Holographic Remoting compensates for changes in the available bandwidth as well as possible and the video encoder will account for these changes by adjusting the quality of the video stream. |
| Video Encoder | The Remote application uses hardware accelerated video-encoding to compress the rendered image into a video stream. This compression is needed because transmitting the uncompressed video data is impossible in real time. In your Remote application, you can select which video codec should be used for encoding. For more information, see Writing a remote app using the OpenXR API or Writing a remote app using the Windows Mixed Reality API. |
| Video Decoder | The video decoder is used to decode the encoded image on the Player side. The video decoding is also hardware accelerated to minimize latency between the Player and Remote. |
| Latency | Latency is the amount of time data takes to pass from one side to the other. Holographic Remoting is used for real-time applications. Therefore, the latency between the Player and the Remote application plays an important role in the experience. The experience won’t be great if you turn your head and you don’t instantly see the changes in your HMD. Holographic Remoting is highly optimized and the latency is reduced to a minimum; it can be unnoticeable that an image was rendered on another machine. |
| Windows Mixed Reality API | The Windows Mixed Reality API (sometimes called “HolographicSpace API”) is an API introduced in Windows 10 that allows allows access to Windows Mixed Reality devices. With Holographic Remoting, you can use the Windows Mixed Reality API to stream in real time to Windows Mixed Reality devices. |
| OpenXR API | OpenXR is a new open standard that provides access to XR platforms and devices across various vendors. Holographic Remoting provides an OpenXR runtime that allows real-time streaming to XR devices. With Holographic Remoting, it’s also possible to use a Windows Mixed Reality Player with a OpenXR Remote. |