Live ultra HD services are on the verge of commercial deployments after a series of trials and growing consensus over some of the most important extensions such as HFR (High Frame Rate), 10-bit colour depth and HDR (High Dynamic Range). There is still work to do agreeing standards for some of these extensions, especially HDR, but one of the biggest challenges lies in live encoding. This is because the high resolutions of 4K are very computationally intensive, involving trade-offs between cost, perceived video quality, degree of compression/bitrate, latency and optimization of the subsequent decode.
When compressing streams offline for on-demand delivery (VoD, SVoD) real-time processing is not required, which means that maximum compression can be achieved with less powerful and less expensive CPUs while still ensuring that the subsequent decode is as computationally easy as possible. The latter is essential given the need to make UHD content available not just to big screen Ultra HD TVs or powerful PCs, but also mobile devices such as smartphones where lack of processing power, memory and battery capacity are all at a premium.
But live encoding has to take place in real-time, with minimum latency. This is a particular challenge for live sports, which exerts a major pull for Ultra HD, because the impact there on perceived quality, immersion and the overall viewer experience is disproportionately greater than say for a studio interview. Live sports programs are particularly demanding because they are shot at least at 50 or 59.94 frames per second and involve fast moving action, which requires extra computation or higher bit rates to encode. To make matters worse, there is an especially low tolerance of latency among consumers watching sport, since they do not want to hear their neighbour cheering a goal or a great tennis shot before it has played on their UHD screen.
This challenge is hard enough with basic Ultra HD but will get even worse when new and attractive schemes are added, such as higher display frame rates (HFR) and HDR. Until recently network operators and broadcasters have used an un-optimized high density server approach to meet these challenges. The production challenge is now to reduce the equipment footprint and its power dissipation using more optimized encoding algorithms simultaneously supporting more advanced encoding schemes (e.g. HDR up to 12-bit colour depth and HFR).
Fortunately there is also a vast scope for optimization of the HEVC encoding process, even more than there was with its predecessor H.264/AVC and MPEG-2 before that. This has been a major focus for French vendor BBright, which was acutely aware from the outset that making HEVC encoding efficient was critical for the ultimate success of UHD. This start-up company is adamant that live UHD trials must remain affordable if they are to happen in the first place, and lead on to commercial deployments. BBright is working with Tier-1 Telcos and broadcasters on UHD trials with its encoders and quad 3G-SDI baseband players.
A case in point can be seen in some of the worldâ€™s most advanced UHD trials to date centred on the French Open tennis championships held in Paris and described in detail (in French) here. After a gradual ramp up which started in 2013, France TÃ©lÃ©visions took the lead for live shooting, producing and broadcasting in Ultra HD themselves, for the whole duration of the 2015 tournament. The central court action was shot with Grass Valley 4K cameras, which were launched at NAB only this year. BBright supplied both the live HEVC encoder and UHD playback server for transmission of live matches over the stadiumâ€™s private IPTV network. Ateme took care of the HEVC encoding for the satellite channel (DVB-S2 distribution with Eutelsat) while Envivio added Dolby AC-4 audio to the terrestrial channel (DVB-T2 distribution with TDF).
For the first time during a tennis event, audio recording and processing was done using a binaural technique and encoded in AAC stereo. Binaural recording uses a model human head with external ears and ear canals and a microphone placed where each eardrum would be. Playback through headphones reproduces the exact audio experience with spatial discrimination so viewers can hear what part of the tennis court the sound is coming from.
Samsung TV sets received direct DVB-S2 and DVB-T2 signals over the air and processed the HEVC UHD decoding. A STmicroelectronic set top box was added for the AC-4 audio decoding. IP set top boxes in lounges and around the RG Labs were provided by Technicolor and based on STMicroelectronicsâ€™ UHD / HEVC decoding chipset for the live IP decoding. Live UHD video streaming was reliably demonstrated for 10-bit HEVC video, showing that fast moving tennis action could successfully be live-encoded and distributed over at full quality in a 16 Mbps data stream including next generation audio. This now makes delivery to client devices of a 24/7 live UHD service feasible over a larger and growing number of broadband networks. BTâ€™s initiative in the UK is a good example with live UHD over IP, which started streaming service early August.
BBright also reckons that there is further to go and that UHD technology will continue to mature for some time yet. Both physical interfaces and HEVC codec features are still developing, so any UHD platform must be field upgradable. For this reason broadcasters and operators will increasingly rely on scalable software. Any live encoder sold today must enable upgradable and scalable solutions for 10/12-bit HDR and higher frame rates up to 120 fps (even for 1080p but also Ultra HD).
Software-based HEVC encoding remains the most flexible and fast turn-around technology allowing evolving standards to be implemented as they emerge within bodies. The new Ultra HD Forum members define the interworking of the UHD delivery ecosystem. The whole Ultra HD industry is watching these technical initiatives and consensus is growing that these software centric encoder appliances will be able to support the many relevant features as they come along in the future.