"Perceptual Video Compression"
Dr. Sebastian Schwarz, BBC, UK
Title: Perceptual Video Compression
Date: October 9, 2015 Place: L429, Campus Sundsvall, Mid Sweden University
It is estimated that around 70% to 90% of all neurons in the human brain are dedicated to vision. Such numbers emphasise our aptitude to be entertained and informed by visual information. Video technology is omnipresent, playing a crucial role in our everyday lives, be it for entertainment, information, education or communication. Nowadays, with pervasive video services and increasing demands on quality and mobility, the need for technical advances in the video compression field and the skilled researchers to deliver such advances has never been greater.
Since the late 1980’s, all major video compression standards have been based on the same hybrid motion-compensated block transform coding model. The latest iteration of this approach, High Efficiency Video Coding (HEVC), delivers convincing coding performance. However, both the increasing use of video on mobile devices and the demand for higher quality, such as Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR) or Wide Colour Gamut (WCG), will soon push us against existing physical bandwidth limits.
Significant technical advances are expected through the emerging area of ‘perceptually-oriented video compression’. This approach is driven by the assumption that, in most cases, the target is to provide good subjective quality rather than to minimize the mathematical error between the original and encoded pictures. Typical video encoding frameworks, optimised using per-pixel distortion measures, do not necessarily correlate with the way humans perceive visual information.
A better understanding of the subjective perception of visual signals could yield significant technical advances in video compression. Encoding may be driven by modelling scene content instead, and mathematical distortion would no longer be a valid optimisation metric. The emphasis would shift from rate-distortion optimisation (RDO) to rate-quality optimisation (RQO).
This lecture will provide an introduction to the field of perceptual video compression, addressing the underlying principles of human vision, as well as the distinction to ‘traditional’ video encoding. The lecture will focus in particular on four topics: Visual Attention Modelling, Advanced Motion Modelling, Content Analysis & Synthesis, and Perceptual Fidelity Metrics. Recent trends and developments in the field will be presented and an outlook on future video compression applications will be provided.
Sebastian Schwarz received his Dipl.-Ing. degree in media technology from the Technical University of Ilmenau, Germany, in 2009. In 2010 he joined the Realistic 3D Research group at Mid Sweden University under supervision of Prof. Mårten Sjöström. He received his Licentiate and his Ph.D. degree in computer and system science for his work on Time-of-Flight video capture, in 2012 and 2014, respectively. The day after his Ph.D. defense Sebastian got married and the next day he moved to London to join the BBC Research & Development department in London, working on next generation video compression algorithms.
Sebastian holds a Marie Skłodowska-Curie fellowship as “experienced researcher” within the MSCA Initial Training Network PROVISION (PeRceptually Optimized VIdeo compresSION), where he is leading the work package responsible for defining the requirements and benchmarks for perceptually optimized video. Furthermore he was co-chairing the MPEG AdHoc group on Future Video Coding Standardization Challenges, organised and chaired a special session on perceptual video compression during the IEEE International Conference on Image Processing 2015 (that would be “last week” at the date of the presentation J) and is an associated researcher with the Realistic 3D research group here at MIUN.