Versatile Video Coding (VVC) has emerged as the latest and arguably the greatest codec so far, but is being bogged down by the same licensing quagmire that has held back deployment of its predecessor, HEVC (High Efficiency Video Coding). It has established a clear lead over rivals in benchmarks for efficiency and bit rate reduction but even its advocates concede it will not gain widespread deployment unless royalty issues are satisfactorily resolved.
There is also confusion in the name, since VVC was proposed as a successor to HEVC early in 2018 by the Joint Video Exploration Team (JVET), set up for this purpose by MPEG and the ITU. The project and reference software was called Joint Exploration Model (JEM), but then renamed VVC to reflect improved versatility and the ability to take greater advantage of varying picture complexity for both inter and intra frame compression.
The new codec is still at an early stage of development, with completion scheduled for the end of 2020. In setting that timescale, MPEG and ITU are breaking with the tradition of bringing out new codecs about every 10 years, each roughly doubling efficiency or halving bit rate for a given video quality. VVC is scheduled to arrive around six years after HEVC was introduced, reflecting accelerating demand for compression efficiency driven by two factors. One is the rise in streaming of both live and on-demand content over broadband networks with varying bit rate constraints. The second is the growth in Ultra HD content at 4K resolutions, with the prospect of higher frame rates exerting even more pressure on bandwidth.
MPEG and the ITU were also motivated by the groundswell behind AV1, the open source and supposedly royalty free codec developed as a successor to Google’s VP9 by the Alliance for Open Media (AOM). This powerful consortium’s governing members include Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel, Microsoft and Netflix.
AV1 had been edging ahead of HEVC in some performance benchmarks, while gaining traction through its promise of being royalty free. Furthermore, to counter the possibility of some patent infringement cases being brought against AV1, the AOM has set up a legal defence fund to pick up the tab if any AV1 users are drawn in.
Against this backdrop, JVET felt it had to move fast with VVC and, from the technical perspective, has succeeded in laying down a strong marker, even while the codec is still under development.
Technically, VVC is a progression from HEVC with some significant enhancements. It has the same basic partitioning structure in which input video is divided into blocks called Coding Tree Units, which in HEVC were extended to allow 64×64, 32×32 or 16×16, with a larger pixel block size increasing coding efficiency. These CTUs are in turn cleaved into one or more coding units (CUs). One of the improvements in VVC is greater flexibility in this sub-structure of CUs, which is now arranged in a nested-tree like arrangement, allowing each CU to be rectangular as well as square, for example.
While such fundamental architectural aspects of VVC have been defined, specific details of the codec syntax, covering aspects such as sequence of events during execution, as well as detailed partitioning of the image into CTUs at the high level, have yet to be determined. That being the case, VVC’s early benchmark figures are remarkably good, with recent tests conducted by the BBC indicating a 27% reduction in bit rate compared with HEVC for a given video quality and 25% over AV1. This is about in line with several other tests and suggests that the target of 50% reduction over HEVC is well within reach.
Most codec experts agree that technically VVC is on track, one being Ken McCann, Founder and Director at UK-based technology consultancy Zetacast and former Chairman of the DVB technical group responsible for defining audio-visual coding standards. McCann, though, argues that the behaviour by some licence holders has rendered HEVC ill-suited to some key market segments, such as video streaming applications, and that the same doubts hang over VVC. In contrast to its predecessors – H.264 and before that, MPEG-2 – HEVC is covered by three competing patent pools: from MPEG-LA, HEVC Advance and Velos. This presents a confusing and risky picture for potential users.
Another streaming consultant, Jan Ozer, who specialises in the application of H.264, H.265 and VP9 encoding for live and on-demand production, goes further. He warns of VVC suffering from “HEVC syndrome” to the extent that it makes no sense for MPEG to launch another codec without resolving what he calls the “royalty land grab” that is still going on over five years after the launch of HEVC. This then, is a more urgent and difficult challenge for MPEG than the technical one of pushing the boundaries of video compression further.