HuifangSun
`
`Yun Q. Shi
`
`|PR2018—01413
`
`Sony EX1008 Page 1
`
`'
`,
`IMAGE and
`COMPRESSION I
`
`for MULTIMEDIA
`
`W F
`
`undamentals,
`Algorithms, and Standards
`
`
`
`
`
`
`IPR2018-01413
`Sony EX1008 Page 1
`
`

`

`
`
`|PR2018—01413
`
`Sony EX1008 Page 2
`
`IPR2018-01413
`Sony EX1008 Page 2
`
`

`

`
`
`|PR2018—01413
`
`Sony EX1008 Page 3
`
`IPR2018-01413
`Sony EX1008 Page 3
`
`

`

`
`
`|PR2018—01413
`
`Sony EX1008 Page 4
`
`IPR2018-01413
`Sony EX1008 Page 4
`
`

`

`IMAGE and VIDEO
`
`COMPRESSION
`
`for MULTIMEDIA
`
`ENGINEERING
`
`Fundamentals,
`Algorithms, and Standards
`
`|PR2018—01413
`
`Sony EX1008 Page 5
`
`IPR2018-01413
`Sony EX1008 Page 5
`
`

`

`IMAGE PROCESSING SERIES
`
`Scrics Editor: Phillip A. Laplante
`
`Forthcoming Titles
`
`Adaptive Image Processing: A Computational Intelligence
`Perspective
`Ling Guan, I-Iau-San Wong‘ and Stuart William I‘L‘rry
`
`Shape Analysis and Classification: Theory and Practice
`Luciano dn Fontoum Costa and Roburto Mnrcmuius (Sugar, Ir.
`
`|PR2018—01413
`
`Sony EX1008 Page 6
`
`IPR2018-01413
`Sony EX1008 Page 6
`
`

`

`m c
`
`onmssron
`
`for MULTIMEDIA
`
`W F
`
`undamentals,
`Algorithms, and Standards
`
`Yun Q. Shi
`New Jersey Institute of Technology
`Newark NI
`
`Huifang Sun
`Mitsubishi Electric Information Technology Center
`America Advanced Television Laboratory
`New Providence, NJ
`
`CRC Press
`
`Boca Raton London NewYork Washington. D.C.
`
`|PR2018—01413
`
`Sony EX1008 Page 7
`
`IPR2018-01413
`Sony EX1008 Page 7
`
`

`

`Library of Congress Cataloging-in-Publication Data
`
`3 Video compression
`
`Shi. Yun Q.
`Image and video compression for multimedia engineering : fundamentals. algorithms. and standards r
`Yuri Q. Shi. l-luifang Sun.
`p.
`cm.
`Includes bibliographical references and indeit.
`ISBN {1-3493-3491-8 talk. paper]
`1. Multimedia systems.
`2. Image compression.
`QA?6.S?5.5555 I999
`006.?—-dc2l
`
`l. Sun. Huifong.
`
`1|. Titlc.
`
`9904“}?
`
`This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with
`permission. and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish
`reliable data and information. but the author and the publisher cannot assume responsibility for the validity of all materials
`or for the consequences of their use.
`Neither this book not any pan may be reproduced or transmitted in any form or by any means. electronic or tltcClFlIllt-‘fll-
`including photocopying. microfilming. and recording. or by any information storage or retrieval sysrem. withttul PTIO"
`permission in writing from the publisher.
`The consent of CRC Press LLC does not extend to copying for general distribution. for promotion. for creating new
`works. or for resale. Specific permission must bl: obtained in writing from CRC Press LLC for such copying.
`Direct all inquiries to CRC Press LLC. 2000 Corporate Blvd. N.W.. Boca Raton. Florida 334?- I.
`
`Trademark Notice: Product or corporate names may be trademarks or registered trademarks. and are only used for
`identification and explanation. without intent to infringe.
`
`E.) 2000 by CRC Press LLC
`
`No claim to original US. Government works
`International Standard Book Number 0-8493-3491-8
`
`Library of Congress Card Number 99-04713?
`Printed in the United States ofAmcrica l
`2 3 4 5 6 't 8 9 0
`Printed on acid-free paper
`
`|PR2018—01413
`
`Sony EX1008 Page 8
`
`IPR2018-01413
`Sony EX1008 Page 8
`
`

`

`h P
`
`reface
`
`It is well known that in the IQoOs the advent of the semiconductor computer and the space program
`swiftly brought the field ol‘ digital
`image processing into public locus. Since then the lield has
`experienced rapid grovvlh and has entered into every aspect of modern technology. Since the early
`03805. digital
`image sequence processing has been an attractive research area because an image
`sequence. as a collection ol' images. may provide more information than a smgle image frame. The
`increased computational complexity and memory space required for image sequence processing
`are becomlng more attainable. This is due to more ad vanced. achievable computational capability
`resulting from the continuing progress made in technologies. especially those associated with the
`VLSI industry and information processing.
`
`In addition to image and image sequence processing in the digitized domain. facsimile trans—
`mission ltas switched from analog to digital since the l9?0s. However. the concept of high definition
`television (HDTV) when proposed in the late 1970s and early l9805 continued to be analog. This
`has since changed. In the U 5.. the first digital system proposal for HDTV appeared in 1990. The
`Advanced Television Standards Committee {ATSCL formed by the television industry, recom-
`mended the digital HDTV system developed jorntly by the seven Grand Alliance members as the
`standard. which was approved by the Federal Com mt: nication Commission (FCC) in 199?. Today's
`worldwide prevailing concept of HDTV is digital. Digital televtsron (D'I'V) provides the signal that
`can be used in computers. Consequently.
`the marriage of TV and computers has begun. Direct
`broadcasting by satellite (DBS). digital video disks (DVD). video-on-dcinand (VOD). video games.
`and other digital video related media and services are available now. or soon will be.
`As in the case ol‘ image and video transmission and storage. audio transmission and storage
`through some media have changed I'rom analog to digital. Examples include entertainment audio
`on compact disks {CD} and telephone transmission over long and medium distances. Digital TV
`signals. mentioned above. provide another example since they include audio signals. ‘I‘ransmission
`and storage of audio signals through some other media are about to change to digital. Examples
`ol‘ this Include telephone transitiission through local area and cable TV.
`Although most signals generated l'rom various sensors are analog in nature. the switching from
`analog to digital is motivated by the superiority of digital signal processing and transmission over
`their analog counterparts. The principal advantage of the digital signal
`is its robustness against
`various noises. Clearly, this results I'rom the fact that only blnary digits exist iii digital format and
`it is much easier to distinguish one state from the other than to handle analog signals.
`Another advantage of being digital is case ot'signal manipulation. In addition to the deveIOpment
`ot' a variety ol‘digital Signal processing techniques (including image. video. and audio) and specially
`designed software and hardware that may be well known, the Following development is an example
`01' this advantage. The digitized information format. in. thc bitstrcam. ol‘tcrt
`in a compressed
`version.
`is a revolutionary change in the video industry that enables many manipulations which
`are either impossible or very complicated to execute in analog t‘onnat. For instance. video. audio,
`and other data can be lirst compressed to separate bitslrcams and then combined to l’orm a signal
`bitstream. thus providing a multimedia solution for many practical applications. Information from
`dil‘l‘crcnt sources and to dil'l'ercnt devices can be multiplexed and dcmultiplcscd in terms 01’ the
`bitstrearn. Bitsti'cam conversion in terms of hit rate conversion. resolution conversion. and syntax
`
`in digital video. content-based coding. retrieval. and manipulation
`conversion becomes feasible.
`and the ability to edit video in the compressed domain become feasible. All systcin-timing signals
`
`|PR2018—01413
`
`Sony EX1008 Page 9
`
`IPR2018-01413
`Sony EX1008 Page 9
`
`

`

`in the digital systems can be included in the bitstrearn instead of being transmitted separately as
`in traditional analog systems.
`The digital fermat is well suited to the recent development of modern telecon‘nnunicalion
`structures as exemplified by the Internet and World Wide Web (WWW). Therefore. we can see. that
`digital computers. eonSutner electronics (including television and video games), and telecommu—
`nications networks are combined to produce an information revolution. By combining audio. video,
`and other data. multimedia becomes an indispensable element ol‘ modern lil‘e. While the pace and
`the future of this revolution cannot be predicted. one thing is certain:
`this process is going to
`drastically change many aspects of our world in the next several decades.
`One of the enabling technologies in the information revolution is digital data compression,
`since the digitization of analog signals causes data expansion.
`In other words. storage and/or
`transmission of digitized signals require more storage space :tndr'or bandwidth than the original
`analog signals.
`The focus of this book is on image and video compression encountered in multimedia engi-
`neering. Fundamentals. algorithms. and standards are the three emphases ol’ the book. it is intended
`to serve as a seniorfgraduate-leve] and Its material is sul'tieient For a one—semester or one—quarter
`graduate course on digital image and video coding. For this put‘pt‘tsc. at the end of each chapter
`there is a section of exercises containing problems and projects for practice. and a section ol‘
`references For further reading.
`Based on this book. a short course entitled "Image and Video Compression for Multitttcdia."
`was conducted at Nanyang Technological University, Singapore in March and April, [999. The
`response to the short course was overwhelmingly positive.
`
`|PR2018—01413
`
`Sony EX1008 Page 10
`
`IPR2018-01413
`Sony EX1008 Page 10
`
`

`

`
`
`Authors
`
`Dr. Yon Q. Shi has been a professor with the Department of Electrical and Computer Engineering
`at the New Jersey Institute 01‘ Technology. Newark. NJ since 1937. Bcl'ore that he obtained his 13.3.
`degree in Electronic Engineering and MS. degree in Precision Instrumentation from the Shanghai
`Jitto Tong University. Shanghai. China and his PhD. in Electrical Engineering from the University
`of Pittsburgh. His research interests include motion analysis from image sequences. video coding
`and transmission. digital
`image watermarking. computer vision. applications of digital
`image
`processing and pattern recognition to industrial automation and biomedical engineering. robust
`stability. spectral factorization. multidimensional systems and signal processing. Prior to entering
`graduate school. he worked in a radio factory as a design and test engineer in digital control
`manufacturing and in electronics.
`
`He is the author or coauthor of about 90 journal and conference proceedings papers in his
`research areas and has been a Formal reviewer ol' the i‘l’httt’tcntttticrt! Reviews since I987. an l'EEE
`
`senior member since 1993. and the chairman of Signal Processing Chapter of [BEE North Jersey
`Section since [9%. He was an associate editor for IEEE irons-notions on Signed Processing
`responsible l'or Multidimensional Signal Processing from 1994 to I999. the guest editor ol’ the
`special issue on image Sequence Processing for the International Jonrttrtl‘ ofi'ntoging .S'vt-stetns and
`Technology. published as Volumes 9.4 and 9.5 in 1998, one 01" the contributing authors in the area
`of Signal and Image Processmg to the. Comprehensive Dictionary of Electrical Engineering, pub-
`lished by the CRC Press LLC in 1998. His biography has been selected by Marquis Who's Who
`I'or inclusion in the 2000 edition of Who's Who in Science and Engineering.
`
`Dr. Huifang Sun received the BS. degree in Electrical Engineering from Harbin Engineering
`institute‘ Harbin. China, and the PhD. in Electrical Engineering from University 01' Ottawa. Ottawa.
`Canada. In l986 he jointed Fairleigh Dickinson University, Teaneck. NJ as an assistant professor
`and was promoted to an associate professor in electrical engineering. Front I990 to 1995. he was
`with the David Sarnol'l' Research Center (Sarnoff Corp.) in Princeton as a member of technical
`staff and later promoted to technology leader of Digital Video Technology where his activities
`included MPEG video coding. AD-HDTV, and Grand Alliance HDTV development. He Joined the
`Advanced Television Laboratory, Mitsubishi Electric Information Technology Center America
`UTA), New Providence, NJ in 1995 as a senior principal technical staff and was promoted to deputy
`director in 1997 working in advanced television development and digital video processing. He has
`been active in MPEG video standards for many years and holds 10 US. patents with several
`pending. He has authored or coauthored more than 80 journal and conference papers and obtatned
`the I993 best paper award ol' IEEE Transactions on Consumer Electronics. and 1997 best paper
`award at" International Conference on Consumer Electronics. For his comributions to HDTV
`development. he obtained the I994 Sarnoff technical achievement award. He is currently the
`associate editor of {EEE Transactions on Circuits and bit-'é’tents for Video Technm‘ogv.
`
`|PR2018—01413
`
`Sony EX1008 Page 11
`
`IPR2018-01413
`Sony EX1008 Page 11
`
`

`

`
`
`Acknowledgments
`
`We are pleased to express our gratitude here for the support and help we received in the course of
`writing this book.
`The first author thanks his friend and former colleague. Dr. C. Q. Shu. for fruitful technical
`discussions related to some contents of the book. Sincere thanks also are directed to several of his
`
`friends and former students. Drs. .l. N. Pan. X. Xia, S. Lin, and Y. Shi, for their technical contri-
`
`butions and computer simulations related to some subjects of the book. He is grateful to Ms. L.
`Fitton for her English editing of l
`I chapters. and to Dr. 2.. F. Chen for her help in preparing many
`graphics.
`The second author expresses his appreciation to his colleagues. Anthony Velro and Ajay
`Divakaran. for fruitful technical discussion related to some contents of the book and for proofreading
`nine chapters. He also extends his appreciation to Dr. Xiaobing Lee for his help in providing some
`useful references. and to many friends and colleagues of the MPEGers who provided wonderful
`MPEG documents and tutorial materials that are cited in some chapters of this hook. He also would
`like to thank Drs. Tommy Poor. Jim Foley, and Toshiaki Sakaguchi for their continuing support
`and encouragement.
`Both authors would like to express their deep appreciation to Dr. 2. F. Chen for her great help
`in formatting all the chapters of the book. They also thank Dr. F. Chichester for his help in preparing
`the book.
`
`Special thanks go to the cditor—in-chicf of the Image Processing book series of CRC Press.
`Dr. P. Laplante. for his constant encouragement and guidance. Help from the editors at CRC Press.
`N. Konopka. M. Mogck, and other staff, is appreciated.
`The first author acknowledges the support he received associated with writing this book from
`the Electrical and Computer Engineering Department at the New Jersey Institute of Technology.
`In particular. thanks are directed to the department chairman. Professor R. Haddad, and the associate
`chairman, Professor K. Sohn. He is also grateful to the Division of information Engineering and
`the Electrical and Electronic Engineering School at Nanyang Technological University (NTU).
`Singapore for the support he received during his sabbatical
`leave.
`It was in Singapore that he
`finished writing the manuscript. In particular. thanks go to the dean of the school, Professor Er
`Meng Hwa, and the division head. Professor A. C- Kot. With pleasure. he expresses his appreciation
`to many of his colleagues at the NTU for their encouragement and help. In particular, his thanks
`go to Drs. G. Li and J. S. Li. and Dr. G. A. Bi. Thanks are also directed to many colleaguefi.
`graduate students, and some technical staff from industrial companies in Singapore who attended
`the short course which was based on this book in March/April
`|999 and contributed their enthu-
`siastic support and some fruitful discussion-
`Last but not least. both authors thank their families for their patient support during the course
`of the writing. Without their understanding and support we would not have been able to complete
`this book.
`
`Yun Q. Shi
`
`Huifang Sun
`
`|PR2018—01413
`
`Sony EX1008 Page 12
`
`IPR2018-01413
`Sony EX1008 Page 12
`
`

`

`
`
`Content and Organization
`of the Book
`
`The entire book consists of 20 chapters which can be grouped into four sections:
`
`I. Fundamentals.
`
`II. Still image Compression.
`HI. Motion Estimation and Compensation. and
`IV. Video Compression.
`
`In the following. we summarize the aim and content of each chapter and each part. and the
`relationships between some chapters and between the four parts.
`Section 1 includes the first six chapters. It provides readers with asolid barns for understanding
`the remaining three parts of the book.
`In Chapter 1,
`the practical needs for image and video
`compression is demonstrated. The feasibility of image and video compression is analyzed. Specif-
`ically. both statistical and psychovisual redundancies are analyzed and the removal of these redun—
`dancies leads to image and video compression. In the course of the analysis. some fundamental
`characteristics of the human visual system are discussed. Visual quality measurement as another
`important concept in the compression is addressed in both subjective and objective quality measures.
`The new trend in combining the virtues ol' the two measures also is presented. Some information
`theory results are presented as the final subject of the chapter.
`Quantization. as a crucial step iit lossy compression, is discussed in Chapter 2. It is known that
`quantization has a direct impact on both the coding bit rate and quality of reconstructed frames.
`Both uniform and nonuniform quantization are covered. The issues of quantization distortion.
`optimum quantiaation. and adaptive quantization are addressed. The final subject discussed in the
`chapter is pulse code modulation (PCM) which. as the earliest, best-established. and most frequently
`applied coding system normally serves as a standard against which other coding techniques are
`compared.
`Two efficient coding schemes, differential coding and transform coding (TC). are discussed in
`Chapters 3 and 4, re3pectively. Both techniques utilize the redundancies discussed in Chapter 1,
`thus achieving data compression. In Chapter 3. the formulation of general differential pulse code
`modulation (DPCM) systems is described first, followed by discussions of optimum linear predic-
`tion and several implementation issues. Then. delta modulation (DM). an important. simple. special
`case of DPCM. is presented. Finally. application of the differential coding technique to interframe
`coding and information—preserving differential coding are covered.
`Chapter 4 begins with the introduction of the Hotelling transform, the discrete version of the
`optimum Karhunen and Loeve transform. Through statistical. geometrical, and basis vector (image)
`interpretations. this introduction provides a solid understanding of the transform coding technique.
`Several linear unitary transforms are then presented, followed by performance comparisons between
`these transforms in terms ofencrgy compactness. mean square reconstruction error. and computa-
`tional complexity. It is demonstrated that the discrete cosine transform (DCT) performs better than
`others,
`in general. In the discussion of bit allocation. an efficient adaptive scheme is presented
`using thresholtling coding devised by Chen and Pratt in 1984. which established a basis for the
`international still image coding standard. Joint Photographic (image) Experts Group UPEG}. The
`
`|PR2018—01413
`
`Sony EX1008 Page 13
`
`IPR2018-01413
`Sony EX1008 Page 13
`
`

`

`comparison between DPCM and TC is given. The combination of these two techniques (hybrid
`transformlwavefonn coding). and its application in image and video coding also are described.
`The last two chapters in the first part cover some coding (codeword assignment) techniques.
`In Chapter 5.
`two types of variable-length coding techniques. Huffman coding and arithmetic
`coding. are. discussed. First. an introduction to some basic coding theory is presented, which can
`be viewed as a continuation of the information theory results presented in Chapter I. Then the
`Huffman code. as an optimum and instantaneous code. and a modified version are covered. Huffman
`coding is a systematic procedure for encoding a source alphabet with each source symbol having
`an occurrence probability. As a block code (a fixed codeword having an integer number of bits is
`assigned to a source symbol). it is optimum in the sense that it produces minimum coding redun~
`dancy. Some limitations of Huffman coding are analyzed. As a stream-based coding technique.
`arithmetic coding is distinct from and is gaining more popularity than Huffman coding. It maps a
`string of source symbols into a string of code symbols. Free of the integer~bits-pcr-source-synibol
`restriction, arithmetic coding is more efficient. The principle of arithmetic coding and some of its
`implementation issues are addressed.
`While the two types of variable-length coding techniques introduced in Chapter 5 can be
`classified as fitted‘length to variable-length coding techniques. both run-length coding {RLCl and
`dictionary coding. discussed in Chapter 6. can be classified as variable-length to fixed—length coding
`techniques. The. discrete Markov source model (another portion of the information theory results]
`that can be used to characterize l-D RLC. is introduced at the beginning of Chapter 6. Both l«D
`RLC and 2—D RLC are then introduced. The comparison between ID and 2-D RLC is made in
`terms of coding efficiency and transmission error effect. The digital facsimile coding standards
`based on 1-D and 2-D RLC are introduced. Another locus of Chapter 6 is on dictionary coding.
`Two groups of adaptive dictionary coding techniques- the L27? and L273 algorithms. are presented
`and their applications are discussed. At the end of the chapter. a discussion of international standards
`for lossless still image compression is given. For both lossless bilevel and multilevel still image
`compression. the respective standard algorithms and their performance comparisons are provided.
`Section II ofthe book {Chapters 7. 8. and 9) is devoted to still image compression. In Chapter '1'.
`the international still image coding standard. JPEG. is introduced. Two classes of encoding: lossy
`and losslcss; and four modes of operation: sequential DCT-based mode. progressive DCT—based
`mode. lossless mode. and hierarchical mode are covered. The discussion in the first part of the
`book is very useful in understanding what is introduced here for JPEG.
`Due to its higher coding efficiency and superior spatial and quality scalability features over the
`DCT coding technique. the discrete wavelet transform (DWTJ coding has been adopted by JPEG—
`2000 still image coding standards as the core technology. Chapter 8 begins with an introduction to
`wavelet transform (WT). which includes a comparison between WT and the short-time Fourier
`transform (STFT). and presents WT as a unification of several existing techniques known as filter
`bank analysis. pyramid coding. and subband coding. Then the DWT for still
`image coding i5
`discussed. In particular. the embedded zerotree wavelet {EZW} technique and set partitioning in
`hierarchical trees (SPIHT) are discussed. The updated PEG-2000 standard activity is presented.
`Chapter 9 presents three nonstandard still image coding techniques: vector quantization (VQ).
`fractal. and model-based image coding. All three techniques have several important features such
`as very high compression ratios for certain kinds of images. and very simple decoding procedures.
`Due to some limitations. however. they have not been adopted by the still image coding standards.
`On the other hand. the facial model and face animation technique have been adopted by the MPEG4
`vrdeo standard.
`
`Section ITI. consisting of Chapters l0 througlt 14. addresses the motion estimation and motion
`compensation — key issues in modern video compression. In this sense. Section III is a prerequisite
`to Section IV, which diseussas various video coding standards. The first chapter in Section Ill.
`Chapter 10. introduces motion analysis and compensation in general- The chapter begins with the
`concept of imaging space. which characterizes all images and all image sequences in temporal and
`
`|PR2018—01413
`
`Sony EX1008 Page 14
`
`IPR2018-01413
`Sony EX1008 Page 14
`
`

`

`image sequences are special proper subsets of the
`spatial domains. Both temporal and spatial
`imaging space. A single image becomes merely a speeitic cross section of the imaging space. Two
`techniques in video compression utilizing intert‘rame correlation. both developed in the late 19605
`and early 19705:. are presented. Frame replenishment is relatively simpler in modeling and imple—
`mentation. However. motion compensated coding achieves higher coding cl'ltciency and hetter
`quality in reconstructed frames with :1 ID displacement model. Motion analysis is then viewed
`from the signal processing perspective. Three. techniques in motion analysis are briefly discussed.
`They are block matching. pel
`recursion. and optical
`llow. which are presented in detail
`in
`Chapters 1 l. 12. and 13. respectively. Finally. other applications of motion compensation to image
`sequence processing are. discussed.
`Chapter It addresses the block matching technique. which presently is the most frequently
`used motion estimation technique. The chapter first presents the original block matching technique
`proposed by Jain and Jain. Several different matching criteria and search Strategies are then
`discussed. A thresholding multiresolution block mulching algorithm is described in some detail so
`as to provide an insight into the technique. Then. the limitations of block matching techniques are
`analyzed. from which several new improvements are presented. They include hierarchical block
`matching. multigrid block matching. predictive motion field segmentation. and overlapped block
`matching. All of these techniques modify the nonoverlappcd. equally spaced.
`[ix-sized. small
`rectangular block model proposed by Jain and Jain in some way so that the motion estimation is
`more accurate and has fewer block artifacts and less overhead side information.
`
`The pel recursive technique is discussed in Chapter [2. First, determination of 2-D displacement
`vectors is converted via the use ofthe displaced frame difference (DFD) concept to a minimization
`problem. Second. descent methods In optimualion theory are discussed. In particular. the steepest
`descent method and Newton—Raphson method are addressed in terms of algorithm. convergence.
`and implementation issues such as selection of stcp~sizc and initial value. Third.
`the first pel
`recursive techniques proposed by Netravali and Robbins are presented. Finally. several improvement
`algorithms are described.
`Optical (low. the third technique in motion estimation for video coding. is covered in Chapter 13.
`First. some fundamental issues in motion estimation are addressed. They include the difference
`
`the aperture problem. and the ill-posed
`and relationships between 2-D motion and optical flow.
`nature of motion estimation. The gradient-based and correlation-based approaches to optical flow
`determination are tltcn discussed in detail. For the former,
`the Horn and Schunck algorithm is
`illustrated as a representative technique and some other algorithms are briefly introduced. For the
`latter. the Singh method is introduced as a representative technique. In particular. the concepts of
`conservation information and neighborhood information are emphasized. A correlation-feedback
`algorithm is presented in detail to provide an insight into the correlation technique. Finally, multiple
`attributes for conservation information are discussed.
`
`Chapter 14. the last chapter in Section 111. provides a further discussion and summary of 2-D
`motion estimation. First. a few features common to all
`three major techniques discussed in
`Chapters 11, 12. and 13 are addressed. They are the aperture and ill-posed inverse problems.
`conservation and neighborhood information. occlusion and disocelusion. rigid and nonrigid motion.
`Second. a variety ol'differertt classifications ofmotion estilttnlion techniques is presented. Frequency
`domain methods are discussed as well. Third. a performance comparison between the three major
`techniques in motion estimation is made. Finally. the new trends in motion estimation are presented.
`Section IV. discussing varioth video coding standards. is covered in Chapters [5 through 20.
`Chapter 15 presents fundamentals of video coding. First. digital video representation is discussed.
`Second. the rate distortion function of the video signal
`is covered -— the fourth portion of the
`information theory results presented in this book. Third. various digital video formats are discussed.
`Finally.
`the current digital
`imagei’video coding standards are summarized. The full names and
`abbreviations of some organizations.
`the completion time, and the major features of various
`imageivideo coding standards are listed in two tables.
`
`|PR2018—01413
`
`Sony EX1008 Page 15
`
`IPR2018-01413
`Sony EX1008 Page 15
`
`

`

`Chapter 16 is devoted to video coding standards MPEG-lfl. which are the most widely used
`video coding standards at the present. The basic technique of MPEG-lfl is a I'ull-ntotion-contpcn-
`sated DCT and DPCM hybrid coding algorithm. The Features of MPEG-1 (including layered data
`Structure) and the MPEG-2 enhancements (including field/frame modes For supporting the interlaced
`video input and scalability extension) are described. issues of rate control. optimum mode decision.
`and multiplexing are discussed.
`Chapter 17 presents several application examples of MPEG- ”2 video standards. They are the
`ATSC DTV standard approved by the FCC in the U.S.. transcoding. the down-conversion decoder.
`and error concealment. Discussion of these applications can enhance the understanding and mas-
`tering of MPEGJIZ standards. Some research work is reported that may be helpful For graduate
`students to broaden their knowledge of digital video processing — an active research field.
`Chapter 13 presents the MPEG-4 video standard. The predominant feature ol'MPEG-4. content—
`based manipulation,
`is emphasized. The underlying concept of audiot'visual objects (AVOs) is
`introduced. The important functionalities of MPEG-4: content-based interactivity (including bit-
`stream editing. synthetic and natural hybrid coding [SM-1CD. content—based coding efficiency. and
`universal access (including content-based scalability). are discussed. Since neither MPEG-1 nor
`MPEG-2 includes synthetic video and content-based coding.
`the most important application of
`MPEG-4 is in a multimedia environment.
`
`Chapter l9 introduces ITU-T video coding standards H.26l and H.263, which are utilized
`mainly for Videophony and videoconferencing. The basic technical details of H.261. the earliest
`video coding standard, are presented. The technical improvements by which H.263 achieves high
`coding efficiency are discussed. Features of H.263+. H.263++. and H.26L are presented.
`Chapter 20 covers the systems part of MPEG — multiplcxing/demultiplexing and synchronizing
`the coded audio and video as well as other data. Specifically. MPEG-2 systems and MPEG-4
`sySIems are introduced. In MPEG2 systems, two forms: Program Stream and Transport Stream.
`are described. In MPEG-4 systems, some multimedia application related issues are discussed.
`
`|PR2018—01413
`
`Sony EX1008 Page 16
`
`IPR2018-01413
`Sony EX1008 Page 16
`
`

`

`
`
`Contents
`
`Section 1 Fundamentals
`
`Chapter 1
`
`Introduction
`
`1.4
`
`1.]
`1.2
`
`Practical Needs for Image and Video Compression ................................................................. 4
`Feasibility of Image and Video Compression .......................................................................... 4
`1.2.1
`Statistical Redundancy .................................................................................................. 4
`1.2.2
`Psychovisual Redundancy .............................................................................................9
`1.3 Visual Quality Measurement .................................................................................................. 18
`1.3.1
`Subjective Quality Measurement“)
`1.3.2 Objective Quality MeasutemanZU
`Information TheoryResultsl4
`1.4.1
`Entropy.....................
`. 24
`1.4.2 Shannon's Noiseless Source Coding Theorem ........................................................... 25
`1.4.3
`Shannon‘s Noisy Channel Coding Theorem ..............................................................26
`1.4.4
`Shannon’s Source Coding Theorem ...........................................................................27
`1.4.5
`In formation Transmission Theorem ............................................................................ 27
`Summary .......................................................