The term 360° video, often used interchangeably with virtual reality (VR) video, describes video media in which the consumer chooses the viewing direction. Depending on the client platform, PC, tablet, or head mounted device (HMD), this may be achieved by dragging, swiping, or moving the device – or by turning one’s head. The media needs to be captured by special cameras or camera rigs in which all angles are captured synchronously.
Cameras for capturing high quality 360° are readily available to both consumer and professional markets. The output of professional rigs is usually a high resolution (currently 4K or 8K) rectangular projection of the 360° view. But the high resolution is not the only problem. Compression artefacts, which may be tolerable or completely invisible at the normal viewing distance for TV, are significantly more noticeable when the display is closer to the eye. This calls for higher bitrates to deliver 360° video at broadcast quality. Delivering 360° media to the consumer whilst maintaining quality presents a significant challenge.
The limitation on resolution is currently dictated by the display device. We demonstrated at NAB Show 2017 that an 8K full 360° image has benefits over a 4K image on an HD display (1/2 HD per eye) as used on the Oculus Rift. As HMD resolution increases (with up to 10K per eye being proposed) camera capture resolution will need to increase to take full advantage. The result being that full frame delivery of the content will not be sustainable for two main reasons:
- Bandwidth – HMD viewing requires very high resolution and high quality
- Decoder requirements – HEVC is currently only standardised up to 8K and that requires a hardware decoder (meaning it can’t be done on a mobile device for example)
But why send full frame 360° video to every end user when only a fraction, the viewport (approximately 15% of the full frame) is ever viewed? Why not deliver to each user only what they are looking at?
This, of course, has issues related to latency. The motion of the device must result in an immediate update of the image to avoid nausea, and high-resolution imagery should be presented within 20-50 msec. for a great user experience! For most networks, this precludes having the motion to image delivery in a single loop. One solution is to deliver a lower resolution “backdrop” image and only deliver high quality images in the viewport. These can then be rendered in such a way that the image is updated quickly in response to head motion but the quality may lag. In this way, we can deliver much higher effective resolution, 8K or more, in a bandwidth dependent only on the HMD resolution and the desired quality.
For IBC 2017, we have teamed up with Tiledmedia to demonstrate the use of a tiled 360° video delivery solution using Ericsson’s industry leading HEVC software encoding technology. The Tiledmedia solution takes a full resolution 360° video and produces one low-resolution sequence and a set of spatially constrained tiles. Each tile segment is then encoded using Ericsson’s HEVC software encoding technology. The encoded tile segments are made available to the client on the tablet or HMD via a (standard) HTTP server which may be on premises or in the cloud/CDN. The client on the mobile device then contains the logic for downloading the low-resolution image and the required high-quality tiles in response to the gaze vector.
But our IBC demonstration isn’t just about the technology. We want to open a dialog with customers about 360° content, who will watch it and how, and of course, how it can be monetized.
It seems unlikely that 360° will replace traditional broadcast or that long form programming and live sports will be consumed by HMD users in the near future Indeed, watching sport is often a social experience with friends and family which does not necessarily lend itself to wearing a VR headset. The Ericsson demo, which we have developed with fellow members of the Virtual Reality Industry Forum (VRIF), shows how 360° content can be used to enhance the linear viewing experience. The 360° video content is delivered simultaneously and synchronously with a standard linear broadcast. Imagine watching a motor race and being able to put on an HMD or pick up a tablet to have a live 360° video experience from inside the car or in the pits during a pit-stop. Or being able to see parts of a football match from the managers’ dugout or from the crowd. This technology makes it possible.
