`
`
`
`
`TK 6680 °
`
`
`69
`
`1996
`
`5 M
`
`
`
` Disiitall
`Helernizelie)
`
`
`_elpjelairels
`ec
`4
`ms |
`
`
`
`
`MPEG vipeo
`ram)el
`
`STANDARD Joan L. Mitchell,
`
`William B. Pennebaker,
`Chad E. Fogg, and
`Didier ]. LeGall
`
`Page 1 of 498
`
`GOOGLEEXHIBIT 1018
`
`Page 1 of 498
`
`GOOGLE EXHIBIT 1018
`
`

`

`Digital systems/Electrical engineering
`
`MPEG
`VIDEO
`COMPRESSION
`STANDARD
`
`
`
`
`
`
`
`
`
`
`LIBRARY OF CONGRES*:
`
`
`
`
`
`
`
`
`
`
`
`
`
`0006 1397437.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Forall those interested in high definition television,
`
`
`
`
`
`
`multimedia, and image compression, this unique
`
`
`
`
`
`
`
`
`reference will be an essential tool.
`It provides
`
`
`
`
`
`
`
`the first comprehensiveintroduction to this field,
`
`
`
`
`
`incorporating material ranging from basic concerns
`
`
`
`
`
`
`
`of newcomers to thefield through sophisticated
`
`
`
`
`
`
`reviews of cutting edge technical issues.
`
`
`
`
`
`
`
`
`
`
`
`
`
`Written by acknowledged experts in the field, MPEG
`
`
`
`
`
`Vicleo Compression Standard offers important bene-
`
`
`
`
`fits to readers including:
`
`
`
`
`
`
`
`
`
`Detailed information on MPEG modes of operation, signaling conven-
`
`
`
`
`
`
`
`tions, and structure of MPEG compressed data
`© Eachsection of the bookis labeled by level of technicaldifficulty,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`allowing less technical readers to skip higher level sections andstill gain
`
`
`
`
`
`
`
`
`
`
`
`a broad understanding of the subject while guiding advanced reaclers to
`
`
`
`
`
`the in-depth coverage they require
`
`
`
`
`
`
`
`
`
`
`With its comprehensive coverage of MPEG video compression, this impor-
`
`
`
`
`
`
`
`
`
`
`
`
`
`tant book meets the needsof those working to develop the standard as
`
`
`
`
`
`
`
`
`
`
`
`well as those who use MPEG in their work. Electrical engineers, multimedia
`
`
`
`
`
`
`
`
`
`
`
`
`producers, computerscientists, as well as all those interested in this fast
`
`
`
`
`
`
`
`
`
`
`growing field will find MPEG Video Compression Standard essential in
`their work.
`
`
`
`
`
`
`
`
`ABOUT THE AUTHORS
`
`
`
`
`
`
`
`
`
`
`
`
`Joan L. Mitchell is a Research Staff Member at the IBM Thomas J. Watson
`Research Center.
`
`
`
`
`
`
`William B. Pennebakeris a consultant and former Research Staff Member,
`
`
`
`
`
`
`
`
`
`
`
`
`IBM Thomas J. Watson Research Center.
`
`
`
`
`
`
`Chad E. Fogg, Cascade Design Automation
`
`
`
`
`
`
`
`
`
`
`Didier J. LeGalll is Vice President of Research and Development, C-cube
`
`Microsystems.
`
`
`
`
`
`
`
`Chapman & Hall
`
`
`
`115 Fifth Avenue, New York, NY 10003
`
`
`
`
`
`
`
`2-6 Boundary Row, London SE1 8HN
`
`
`
`
`
`Printed in U.S.A,
`
`
`
`
`
`
`
`
`ISBN
`
`
`
`O-4Yle-Oar?1-s
`90000
`
`
`
`
`
`
`a.
`
`
`
`
`
`
`
`
`
`JOIN US ON THE INTERNET WWW:http://www.thomson.com
`
`
`
`
`
`
`
`Visit Chapman & Hall's Internet Resource Center for information on
`
`
`
`
`
`
`
`
`
`
`our new publications, links to useful sites on the World Wide Web and
`
`
`
`
`
`
`
`
`
`
`
`
`
`an opportunity to join our e-mail mailing list.
`
`
`
`
`
`
`
`
`Point your browserto: http://www.chaphall.com/chaphall.html or
`
`
`
`
`
`http://www.chaphall.com/chaphall/electeng.htmlfor Electrical
`
`
`
`Engineering
`
`
`9°780412°08/7714
`
`
`
`
`
`
`
`
`
`
`
`Page 2 of 498
`
`
`
`A service of 1(T)P°
`: Se.ee
`
`Page 2 of 498
`
`

`

`
`
`itp
`
`Mitchell
`
`
`
`
`Pennebakel
`
`
`i0=U
`
`| D
`WI
`
`
`
`O30IA9]34W
`
`
`
`
`
`
`
`aw
`
`i
`ae
`m4
`
`Ut
`
`|e
`
`4a
`
`e
`|e
`ee
`ae
`
`
`
`
`
`Page 3 of 498
`
`Page 3 of 498
`
`

`

`
`
`eeeee
`
`
`
`fou{psaAhanttnatare
`
`
`
`
`
`
`
`
`Join Us on the Internet
`
`WWW:_
`
`
`EMAIL:
`
`http://www.thomson.com
`
`
`findit@kiosk.thomson.com
`
`
`
`|
`
`it
`
`1
`
`_
`
`
`
`
`
`
`
`
`
`
`
`thomson.com is the on-line portal for the products, services and
`
`
`
`
`
`
`
`resources available from International Thomson Publishing (ITP).
`
`
`
`
`
`
`
`
`
`
`
`This Internet kiosk gives users immediate access to more than 34 ITP
`
`
`
`
`
`
`
`
`publishers and over 20,000 products. Through thomson.com Internet
`
`
`
`
`
`
`
`
`users can search catalogs, examine subject-specific resource centers
`
`
`
`
`
`
`
`
`
`
`and subscribeto electronic discussion lists. You can purchase ITP
`
`
`
`
`
`
`
`
`
`products from yourlocal bookseller, or directly through thomson.com.
`
`
`
`
`
`
`
`
`
`
`
`
`
`apman & Hall’s Internet Resource Center for information on our new publications,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`to useful sites on the World Wide Web and an opportunity to join our e-mail
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`“ mailing list. Point your browserto: http://www.chaphall.com/chaphall.htmlor
`
`
`
`
`
`
`
`~ http://www.chaphall.com/chaphall/electeng.html for Electrical Engineering
`
`
`
`
`
`
`
`A service of I(1)P
`
`
`
`
`®
`
`
`
`
`
`Page 4 of 498
`
`Page 4 of 498
`
`

`

`
`
`
`
`nt
`CHAPMAN & HALL
`
`
`IP INTERNATIONAL THOMSONPUBLISHING
`
`
`
`f
`
`A
`
`‘4
`
`f
`
`Page 5 of 498
`
`®
`
`
`
`
`
`
`
`New York + Albany * Bonn + Boston + Cincinnati + Detroit » London - Madrid « Melbourne
`
`
`
`
`
`
`
`
`
`
`Mexico City * Pacific Grove » Paris « San Francisco + Singapore + Tokyo + Toronto + Washington
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 5 of 498
`
`

`

`
`cee
`
`
`7 OF CONC
`
`"Es
`
`TK el&tO
`\AMie
`
`5N
`
`
`
`
`
`Chapman & Hall
`
`
`
`2-6 Boundary Row
`
`
`
`London SE1 8HN
`
`England
`
`
`
`
`
`
`Chapman & Hall GmbH
`
`
`
`Postfach 100 263
`
`
`D-69442 Weinheim
`
`Germany
`
`
`
`International Thomson Publishing-Japan
`
`
`
`
`Hirakawacho-cho Kyowa Building, 3F
`
`
`1-2-1 Hirakawacho-cho
`
`
`
`Chiyoda-ku, 102 Tokyo
`
`Japan
`
`
`
`
`
`
`
`Cover design: Curtis Tow Graphics
`
`
`
`
`
`
`
`
`Copyright © 1996 by Chapman & Hall
`
`
`
`
`
`
`Printed in the United States of America
`
`
`
`
`
`
`
`
`Chapman & Hall
`
`
`
`115 Fifth Avenue
`
`
`
`New York, NY 10003
`
`
`
`
`
`
`
`Thomas Nelson Australia
`
`
`
`102 Dodds Street
`
`
`
`South Melbourne, 3205
`
`
`Victoria, Australia
`
`
`International Thomson Editores
`
`
`
`
`CamposEliseos 385, Piso 7
`
`
`Col. Polanco
`
`
`
`11560 Mexico D.F
`
`Mexico
`
`
`
`
`
`
`
`
`International Thomson Publishing Asia
`
`
`
`
`221 Henderson Road #05-10
`
`
`Henderson Building
`
`
`Singapore 0315
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`All rights reserved. Nopart of this book covered by the copyright hereon may be reproduced or
`
`
`
`
`
`
`
`
`
`
`
`
`
`used in any form or by any means—graphic,electronic, or mechanical, including photocopying,
`
`
`
`
`
`
`
`
`
`
`
`
`recording, taping, or information storage and retrieval systems—withoutthe written permission of
`
`
`the publisher.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`23456789 10 XXX 01 00 99 98 97
`
`
`
`
`
`
`
`Library of Congress Cataloging-in-Publication Data
`
`
`
`
`
`
`
`
`
`MPEGvideo : compression standard / Joan L. Mitchell ... [et al.].
`
`
`cm.
`p.
`
`
`
`
`
`Includes bibliographical references and index.
`
`
`
`
`ISBN 0-412-08771-5 (alk. paper)
`
`
`
`
`
`
`
`
`3. Sound -
`1, Digital video.
`2. Video compression ~ Standards.
`
`
`
`
`
`
`- Recording and reproducing -- Digital techniques -- Standards.
`
`
`
`
`
`
`
`4. Coding theory.
`I. Mitchell, Joan L,
`
`
`TK6680.M64
`1996
`
`621.388--de20
`
`
`
`
`
`
`
`
`96-31124
`
`CIP
`
`
`
`
`
`
`
`
`
`British Library Cataloguing in Publication Data available
`
`
`
`
`
`
`
`
`
`
`
`
`
`To order this or any other Chapman & Hall book, please contact International Thomson
`
`
`
`
`
`
`
`
`
`
`
`Publishing, 7625 Empire Drive, Florence, KY 41042. Phone: (606) 525-6600 or 1-800-842-3636.
`
`
`
`
`
`Fax: (606) 525-7778. e-mail: order@chaphall.com.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`For acompletelisting of Chapman & Hall titles, send your request to Chapman & Hall, Dept. mY
`
`
`
`
`
`
`
`115 Fifth Avenue, New York, NY 10003.
`
`
`
`
`
`
`
`Page 6 of 498
`
`Page 6 of 498
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`To
`
`
`
`
`
`
`
`Nancy, Don, and Sandy
`
`
`
`
`Margaret, Betsy, Patty, and Erik
`
`
`
`Lori and John.
`
`
`
`Dedication
`
`
`
`
`
`Page 7 of 498
`
`
`
`Page 7 of 498
`
`

`

`
`
`
`Series Preface
`
`
`
`This bookinitiates a new digital multimedia standards series. The purpose
`
`
`
`
`
`
`
`
`
`
`of the series is to make information about digital multimedia standards
`
`
`
`
`
`
`
`
`
`
`readily available. Both tutorial and advanced topics will be covered in the
`
`
`
`
`
`
`
`
`
`
`
`series, often in one book. Our hopeis that users will find the series helpful
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`in deciding what standards to support and use while implementors will dis-
`
`
`
`
`
`
`
`
`
`
`
`cover a wealth of technicaldetails that help them implement those standards
`
`
`
`
`
`
`
`
`
`
`
`correctly.
`In today’s global economy standards are increasingly important. Yet
`
`
`
`
`
`
`
`
`until a standard is widely used, most of the benefits of standardization are
`
`
`
`
`
`
`
`
`
`
`
`
`not realized. We hope that standards committee chairpeople will organize
`
`
`
`
`
`
`
`
`
`and encourage a bookin this series devoted to their new standard. This can
`
`
`
`
`
`
`
`
`
`
`
`
`be a forum to share and preserve some of the “why” and “how” that went
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`into the developmentof the standard and, in the process, assist in the rapid
`
`
`
`
`
`
`
`
`
`
`
`
`
`adoption of the standard.
`.
`
`
`
`
`Already in production for this series are books titled Digital Video: In-
`
`
`
`
`
`
`
`
`
`
`
`troduction to MPEG-2 and Data Compression in Systems.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`vii
`
`
`
`
`
`Page 8 of 498
`
`Page 8 of 498
`
`

`

`
`
`
`
`
`
`Foreword
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The reduction of bitrate of digitised television signals has been an R&D
`
`
`
`
`
`
`
`
`
`
`
`
`
`topic in research centres and academia for more than 30 years. Many were
`
`
`
`
`
`
`
`
`
`
`
`
`the purposes for which research was funded, but all shared the basic ra-
`
`
`
`
`
`
`
`
`
`
`
`
`
`tionale. If you had a digital or digitised delivery medium and could bring
`the more than 200 Mbit/s bitrate of PCM television down to a value that
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`could be handled economically by signal processing technology then you
`
`
`
`
`
`
`
`
`
`would achieve the goal of substantially improving existing products/services
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`or even creating new ones. Thebitrate of up to 1.5 Mbit/s was the value
`
`
`
`
`
`
`
`
`
`
`
`
`around which three industries shared a common interest towards the end of
`
`
`
`
`
`
`
`
`
`the ’80s: consumerelectronics with compact disc, broadcasting with digi-
`
`
`
`
`
`
`
`
`
`tal radio and telecommunications with Narrowband ISDN and ASDL. The
`
`
`
`
`
`
`
`
`
`
`
`
`merit of MPEG wasits ability to identify the opportunity and to bring
`
`
`
`
`
`
`
`
`
`disparate industries to work together. The unprecedented degree of interna-
`
`
`
`
`
`
`
`
`tional collaboration involving hundreds of researchers produced a standard-
`
`
`
`
`
`
`
`
`
`
`
`
`MPEG-1 - of very high technical quality. Even today, after several years and
`
`
`
`
`
`
`
`
`
`
`
`countless implementations not a single error was found in the specification.
`
`
`
`
`
`
`
`
`
`
`
`
`MPEG-1 devices are now counted by the millions in all continents. MPEG-1
`
`
`
`
`
`
`
`
`
`
`
`
`encoded content can be found everywhereandis the first implementation of
`
`
`
`
`
`
`
`
`
`
`
`delivery media independence of content. The beginning of the ’90s provided
`
`
`
`
`
`
`
`
`
`
`
`the next opportunity with a general interest in a new standard-- MPEG-2 .
`- for applications spanning all industries: broadcasting over satellite, ter-
`
`
`
`
`
`
`
`
`restrial and CATV networks, Broadband ISDN,digital discs and tapesetc.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The availability of another MPEG standards promised - and delivered - as
`
`
`
`
`
`
`
`
`
`
`
`
`scheduled was not a small contribution to the multimedia revolution we are
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`living today. The book by Drs. Le Gall, Fogg, Mitchell and Pennebakeris a
`
`
`
`
`
`
`
`
`
`
`recommended reading for those who want to have a thorough understanding
`
`
`
`
`
`
`
`
`
`
`
`
`of the Video parts of MPEG-1 and MPEG-2. The concepts that were devel-
`oped in those standards and presented in the book are a necessary reading
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`for anybody who needs to work on them or go beyond them.
`
`
`
`
`
`
`
`
`Leonardo Chiariglione
`Convenor, ISO/IEC JTC1/SC29/WG11 (MPEG)
`
`
`
`
`
`
`
`
`
`
`Page 9 of 498
`
`Page 9 of 498
`
`

`

`
`
`Acknowledgments
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Several hundred technical experts worked hard, attended meetings all over
`
`
`
`
`
`
`
`
`
`
`
`the world, and spent many, many hours of personal and professional time
`
`
`
`
`
`
`
`
`
`
`
`before reaching a consensus on the MPEG-1 standard. We want to specifi-
`
`
`
`
`
`
`
`
`
`
`cally acknowledge their dedicated work even though they are too numerous
`
`
`
`
`
`to be listed by name.
`
`
`
`
`
`
`
`
`
`
`JM wishes to thank a former manager, Cesar Gonzales, for permission
`
`
`
`
`
`
`
`
`
`
`
`
`
`to start work on this book. She wishes to thank her current IBM manage-
`ment and in particular, Fred Mintzer and Howard Sachar, for giving her
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`permission to take a two-year leave of absence from IBM. Herparticipation
`
`
`
`
`
`
`
`
`
`
`
`
`in writing this book would have been much more difficult without this extra
`
`
`
`
`
`
`
`
`
`
`freedom. She also acknowledges the support of the Computer and Electri-
`
`
`
`
`
`
`
`
`
`cal Engineering Department and the Beckman Institute at the University
`
`
`
`
`
`
`
`
`
`
`
`of Illinois, Urbana,IIl., for their 1996 part-time Visiting Professor and Vis-
`iting Scientist positions which she held during thefinal stages of finishing
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`this book. She also wants to thank Christiana Creighton; Sandy and David
`
`
`
`
`
`
`
`
`
`
`Creighton; Anne Gilligan; Sandy Grover and Dorothy Cullinan of the Cut-
`
`
`
`
`
`
`
`
`
`
`
`
`leaf Maples Motel, Arlington, VT; Jackie and Bruce Kelman; Joan and Don
`Lucas; Sherri and Peter McCormick; Doris, Bill, and Carol Mitchell; Nancy
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and Don Mitchell; Margaret and Bill Pennebaker; Ruth Redden; Frances
`
`
`
`
`
`
`
`
`
`
`
`and Charles Riggs; and Norma and Don Vance for encouragement, meals,
`
`
`
`
`
`
`
`
`and lodging during the writing of this book.
`
`
`
`
`
`
`
`
`
`
`
`
`WP wishes to thank his wife, Margaret, for her patience and active
`
`
`
`
`
`
`
`
`
`
`
`
`support during the preparation of this book and the implementation of a
`
`
`
`
`
`
`
`
`
`
`
`MPEG-1 video encoder and decoder. Acknowledgment is also given to the
`
`
`
`
`
`
`
`
`
`
`
`
`
`neighbors who took the subjective quality tests, results of which are in this
`
`book.
`
`
`
`
`
`
`
`
`
`
`
`
`We also want to thank the vendors who supplied information for this
`book; Cliff Reader, who contributed the IPR chapter; our many editors at
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Van Nostrand Reinhold and later at Chapman and Hall; Nancy Mitchell,
`
`
`
`
`
`
`
`
`
`
`
`
`
`who reviewed this book for consistency at the easiest technical level; and all
`of the other people who reviewed this book and helped uscorrect ourerrors
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`before publication.
`
`
`
`al
`
`
`
`
`
`Page 10 of 498
`
`
`
`Page 10 of 498
`
`

`

`
`
`Contents
`
`
`
`
`
`
`Dedication
`
`Series Preface
`
`Foreword
`Acknowledgments
`
`
`
`List of Figures
`
`
`
`List of Tables
`
`Trademarks
`
`
`
`1
`
`
`
`
`
`
`Introduction
`
`
`
`
`............
`1.1 Why compress?O
`
`
`
`
`
`1.2 Why standardize video compression? O
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`13 VocabulayO ............00.
`
`
`
`
`Sequences, pictures, and samplesO
`1.3.1
`
`
`
`
`1.3.2 Framesand fieldsO
`.......
`
`
`
`
`
`1.3.3 PelsandpixelsO ...... lo,
`
`
`
`1.3.4 Compression vocabulary O
`Le
`
`
`
`1.3.5 Standards vocabularyO
`.....
`
`
`
` ......
`1.3.6 MPEG vocabularyO
`
`
`
`
`
`
`
`1.4 A quick look at MPEG compressionO
`
`
`
`
`
`1.5 Compact pseudocode notation in MPEGO
`
`
`
`1.6 MPEG applicationsO
`..........
`
`
`
`
`
`1.7 Organization of the bookO
`.......,
`
`
`
`
`1.8 Level of dificultyO
`...........
`
`
`
`
`1.9 Aninternational collaborationO
`....
`
`
`
`
`1.10 Anevolving standardO
` .........
`
`
`
`
`
`
`
`2 Overview of MPEG
`....-......
`2.1 MPEG system layerO
`
`
`
`
`
`System.target decoderO
`2.1.1
`Lea
`
`
`
`
`
`
`
`2.1.2
`System layer syntaxO ......
`
`
`
`
`2.2 MPEG audio® .............4,
`
`
`
`
`2.33 MPEG videooO
`..............
`
`
`
`
`2.3.1 MPEG video layersO
`......,
`
`xiii
`
`
`
`
`
`
`
`Page 11 of 498
`
`
`
`
`
`
`vii
`
`ix
`
`xi
`
`
`
`
`xxiii
`XXXi
`XXXvii
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`eebobo
`
`
`
`
`
`
`
`
`
`SCNUDGERGURA
`
`
`
`
`
`
`
`Page 11 of 498
`
`

`

`
`
`XIV
`
`
`
`
`
`
`
`
`«2 ww we ee ee a 23
`2.3.2 Display and coding orderO
`
`
`
`
`23
`Biase
`.Macroblock)
`~.a0li liao a eke wada ae ea
`
`
`
`
`Be SNC
`aA a een Een ee ee ees dcta Ts 23
`
`
`
`
`
`
`
`
`
`2.3.5 The discrete cosine transform in MPEGO
`YO
`dona It
`\Quantivntion®)
`occ a4 oe casg te dwt bh we
`27
`9:36:
`
`
`
`
`
`
`2.3.7 Motion compensationO
`............000. 28
`
`
`
`
`
`2.3.8 MPEG coding modelsO ...........02.004 29
`
`
`
`
`
`2.3.9 Encoder decision strategiesO ............, 30
`
`
`
`GA! EPRGRAGS
`at eld a hdd ter 2a, alee ea lore beara 30
`
`
`
`BF MPEGME!
`or jis aca okies ees ace
`ta oa ah WP ees Si
`
`
`|
`
`
`
`
`
`|
`
`
`
`
`
`
`33
`3 The Discrete Cosine Transform
`
`
`
`
`
`
`3.1 The one-dimensional cosine transformO ........... 33
`
`
`
`
`
`
`
`3.2
` 1-D discrete cosine transform (DCT)O
` ........... 37
`
`
`
`
`
`SUN EET.
`alts
`ony oie op am i thet Bg) De ie Be
`id?
`42
`ad,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a4 Why usethe DOTTO.
`240668 wa de pee new bas
`44
`
`
`
`
`
`
`....... 44
`3.4.1 Decorrelation and energy compactionO
`
`
`
`
`
`3.4.2
`Efficient DCT algorithmsO .............. 45
`
`
`
`
`
`
`
`
`3.5 Precision of the DOTO«paawe da be a wane bas 45
`
`
`
`
`
`
`3.6 Quantization ofthe DCTO 2.504655 e eee wae
`46
`
`
`
`
`
`
`3.6.1 Rounding conventions for quantizationO
`...... 46
`
`
`
`
`
`
`
`Scaling of quantizer tablesO
`............. 48
`3.6.2
`
`
`
`
`
`
`3.6.2.1 Quantization ofthe FDCT® ........ 48
`
`
`
`
`
`
`3.6.2.2 Dequantization of the IDCT® ....... 49
`
`
`
`
`
`
`3.7 MismatchofIDCTsO 2... .. ee ee 49
`
`
`
`
`
`
`
`
`51
`4 Aspects of Visual Perception
`
`
`
`.. 2.0... ee ce tee es 52
`4.1 ColorrepresentationsO
`
`
`
`
`4.1.1 TrichromatictheoryO ............00005 52
`
`
`
`
`
`4.1.2 Luminance and chrominanceO
`.,......4-5.% 54
`
`
`
`
`4.1.3 Brightness perceptionO .............00% 54
`
`
`
`
`41.4 GammacorrectionO ..,....00 0.000 ee eae 55
`
`
`
`
`
`
`Luma and gamma-corrected luminanceO
`...... 58
`4.1.5
`
`
`
`
`
`
`
`
`
`4.1.6 YUV and YCbCr color systems and chromaO
` ...
`58
`
`
`
`
`
`
`
`4.1.7 Opponent model of human vision®
`........., 60
`
`
`
`
`4.1.8 Chromaticity coordinates® .............. 61
`
`
`
`
`
`
`43:9|Center-of:gravity(R s.. jeu eee ae ee 62
`
`
`
`
`4.1.10 Coordinate transformations®
`............ 63
`
`
`
`
`211) GIG promaries@®.
`nck aun ee i AEM pele ps
`64
`
`
`
`
`
`4.1.12 CIELUV color space@®
`66
` ........,020000.
`
`
`
`
`
`4.2 Resolving spatial detaillO
` ........ 0,00 eee eee 67
`
`
`
`
`
`4.2.1. MPEG chrominance samplingOQ
` ........... 68
`
`
`
`
`
`4.2.2 Varying precisionO
`«fk ee 69
`
`
`Page 12 of 498
`
`

`

`
`
`
`
`
`
`
`
`
`4.2.3
`Frequency weighting and quantizationO ....... 69
`
`
`
`
`
`
`4.2.4 Activity masking and facilitationO
` ......... 70
`
`
`
`
`
`
`4.3 PerceptionofmotionO
`........... 0000000 72
`
`
`
`
`
`
`4.3.1 Critical flicker frequencyO
` ............0.4 72
`
`
`
`
`
`
`
`Spatial resolution of time-varying scenesO
`..... 74
`4.3.2
`
`
`
`
`
`
`
`4.3.3 Motion sensingO
`........... 0000000 75
`
`
`
`
`4.3.4 Temporal maskingO. ..............000. 76
`
`
`
`
`
`
`
`4.3.5 Tracking of moving objecttsO
`............4. 78
`
`
`
`
`
`4.3.6 Coding time-varying scenesO ..........004.
`78
`
`
`
`
`
`
`81
`5 MPEGCoding Principles
`
`
`
`
`
`
`
`
`
`
`
`
`2... we ee ee 81
`5.1 Coding system structureO
`
`
`
`
`
`
`
`
`
`
`
`...............4. 82
`Isolating themodelO
`5.1.1
`
`
`
`
`
`
`5.1.2
`Symbols and statistical modelingO
`......... 83
`
`
`
`
`
`
`
`
`
`
`
`
`
`5.2 Emtropy codingO 3... ee 84
`
`
`
`
`
`
`Principles of entropy codingO
` .. 1... ee eee 84
`5.2.1
`
`
`
`
`
`
`5.2.2 Mathematics of entropy coding®
`.......... 85
`
`
`
`
`5.2.8 HuffmancodingO
`.............2.00004 86
`
`
`
`
`
`
`.5.2.4 Assigning Huffman codes®
`.............. 86
`
`
`
`
`
`5.2.5 Adaptive entropy coding®
`.............. 87
`
`
`
`
`Symbol probabilitiese® ......... ke eee
`5.2.6
`88
`
`
`
`
`
`
`
`
`
`5.3 Statistical modelbO
`6 2. ee 88
`6
`
`
`
`
`
`
`
`
`
`
`
`5.4 Coding modelsO «2.1... ee 89
`
`
`
`
`
`
`I-, P-, and B-picturesO
`..........0.020.4 90
`5.4.1
`
`
`
`
`
`5.4.2 MPEG coding model8sO
`.............0.4. 90
`
`
`
`
`
`5.4.3 Quantization in MPEG-10
`.............. 91
`
`
`
`
`
`
`5.4.4 Coding of I-picturess®
`...............0. 91
`
`
`
`
`
`
`
`Coding DC coefficients in I-picturess® ...
`92
`5.4.4.1
`
`
`
`
`
`
`
`5.4.4.2 Coding AC coefficients in -picturese® ...
`94
`
`
`
`
`
`
`5.4.5 Coding of P- and B-picturs® ........ ee OO”
`
`
`
`
`
`
`
`5.5 Encoder and decoder block diagrams®
`........... 100
`
`
`
`
`5.5.1 Reconstruction module®
`............... 100
`
`
`
`
`
`5.5.2 Encoder block diagram®
`............... 100
`
`
`
`
`
`5.5.3 Decoder block diagram®
`............0004 102
`
`
`
`
`
`
`
`
`
`105
`6 Pseudocode and Flowcharts
`
`
`
`
`
`...........2.2.00005 106
`Pseudocodepresentation®
`6.1
`
`
`
`
`..............00000. 106
`6.2 Flowchart primitives®
`
`
`
`
`6.2.1 Decision notation®
`............2.2. 0004 108
`
`
`
`
`6.2.2 Test notation ® eee eee Ve eee 108
`
`
`
`
`6.3 Operator notation® .. 2... . ee ee. 110
`
`
`
`
`
`6.3.1 Branches and loops®
`............0000% 111:
`
`
`
`
`
`6.4 Data element procedures®
` ...........0.0 0008 113
`
`Page 13 of 498
`
`Page 13 of 498
`
`

`

`
`
`
`
`
`
`peeweeseiterecenetoteenennensrienmestttngne
`
`
`
`
`
`
`
`
`Page 14 of 498
`
`
`
`XV1
`
`
`
`
`
`
`
`«2.1... ee es 114
`6.5 Nextbits() procedure®
`
`
`
`
`
`6.6 Encoder versus decoder operation®
`............. 115
`
`
`
`
`6.7 Other low-level procedures®
`................. 115
`
`
`
`
`
`
`
`
`
`
`117
`7 MPEG System Syntax
`
`
`
`7.1 Start codesO 2...ee 117
`
`
`
`
`7.1.1 Next.startcode() function®
`............. 117
`
`
`
`
`
`
`
`7.1.2
`System and video start codess®
`.....,....., 119
`
`
`
`
`7.2 System overviewO
`...........00 000002 bee 119
`
`
`
`
`
`ISO/IEC 11172 streaam®
`7.3
`......,...... see 121
`
`
`
`
`
`7.4 Packlayer®
`.............0.0 00000000004 121
`
`
`
`
`System header® .................00. 123
`7.4.1
`
`
`
`
`7.5 Packet layerr®
`......... ee 127
`
`7.6 Summary of system data elements®
`............. 131
`
`
`
`
`
`
`
`
`
`
`
`7.7 SCR calculation examplese@
`.......,.......,0.. 133
`
`
`
`
`
`
`
`135
`8 MPEG-1 Video Syntax
` ............. 2000. 135
`- 8.1 Video sequence overviewO
`
`
`
`
`
`
`
`
`8.2 Start codes® .....ee 137
`
`
`
`
`
`8.3 Video sequence layer®
`«wwe ee ee 138
`
`8.4 Group of pictures layer® Lee ee es 145
`
`
`
`
`
`
`
`
`
`
` -8.5 Picture layer®
`©... 6... ee ee 147
`
`
`
`.... ee ee 150
`8.6 Slicelayerr®
`
`
`
`a. Macroblock layerr®
`=... 2... ee ee 151
`
`
`
`
`
`
`“8.7.1 Skipping macroblocks® ................, 160
`
`8,3. Block layer®
`«6 2.ee 160
`
`
`
`
`
`
`
`
`
`
`
`
`
`~ 8.9 Summary of data element syntax®
`............. 165
`
`
`8.10 MPEG-1 video data stream example@
`.........., 166
`
`
`
`
`
`
`
`
`_.
`'
`
`
`
`
`171
`9° MPEG-2 Overview
`
`
`
`
`
`
`
`
`
`
`9.1 Advantages of MPEG-2 over MPEG-1O ........... 171
`
`
`
`9.2 MPEG-2 applicationsO
`............ 0000000. 172
`
`
`
`
`
`
`9.3
`Joint development with ITU-TO ............0.0.. 173
`
`
`
`
`9.4 MPEG-2systemsO
`....... 0.00.00 04eee 173
`
`
`
`
`
`
`9.5 Fields, frames, and picturesO ..............004 174
`
`
`
`
`9.6 Chrominance samplingO
`...............0004 175
`
`
`
`
`9.7 VideoformatsO
` ....... 0... eee es 175
`
`
`
`......0..0 0.000000 00000004 177
`9.8 ScalabilityO
`
`
`
`
`2... .....0.0.0....0084 177
`SNRescalabilityO
`9.8.1
`
`
`
`
`9.8.2 Spatial scalabilityO
`........, Dee ee eee 178
`
`
`
`
`9.8.3° Temporal scalabilityO ..............004 178
`
`
`
`
`
`9.8.4 Data partitioning extensionO
`............ 178
`
`
`
`9.9 ProfleseO 2.2... eee 178
`
`
`
`
`Page 14 of 498
`
`

`

`9.10 Levels® ..........0.. eee 179
`
`
`
`
`
`9.11 Main profile video sequence overviewO
` ........... 183
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`9.12 Differences between MPEG-1 and MPEG-2 video®
`... . 185
`
`Xvii
`
`
`
`
`
`
`
`
`
`187
`10 MPEG-2 Main Profile Syntax
`
`
`
`
`
`10.1 MPEG-2 start codes® 2... ee ee ee 187
`
`
`
`
`
`6 ww we ee ee 189
`10.2 MPEG-2 video sequence®
`
`
`
`
`
`10.2.1 MPEG-2 sequence header®
`............. 189
`
`
`
`
`
`............ 192
`10.2.2 MPEG-2 sequence extension@®
`
`
`
`
`
`
`.10.2.3 MPEG-2 optional sequence extensions®
`...... 195
`
`
`
`
`
`
`
`
`
`
`
`
`
`10.2.4 MPEG-2 user data®@®
`...... Se 196
`
`
`
`
`
`
`10.2.5 MPEG-2 sequence display extension® ........ 196
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10.3 MPEG-2 GOP header@®
`................0004 198
`
`
`
`
`
`
`
`.......... 202
`10.4 MPEG-2 picture header and extensions®
`
`
`
`
`
`
`10.4.1 MPEG-2 picture coding extension®
`......... 203
`
`
`
`
`
`
`
`
`10.4.2 MPEG-2 picture extensions and user data®@
`2... 242
`
`
`
`
`
`
`10.4.3 MPEG-2 quant matrix extension®
`......... 212
`
`
`
`
`
`
`10.4.4 MPEG-2 picture display extension® ......... 213
`
`
`
`
`
`10.4.5 MPEG-2 copyright extension® ............ 215
`
`
`
`
`
`
`10.4.6 MPEG-2 picture data function@®
`........... 215
`
`
`
`
`
`10.5 MPEG-2 slice header®
`5. we ee ee es .. 216
`
`
`
`
`
`10.6 MPEG-2 macroblock header@®3..............04. 217
`
`
`
`
`10.6.1 Macroblock modes@®
` ..............0.-. 220
`10.6.2 Motion vectors@®
`.........0...00 0008 222
`
`
`
`
`
`
`
`
`
`
`10.6.3 MPEG-2 coded block patten®
` ........... 224
`
`
`
`
`
`10.7 MPEG-2 block(i) function®
` ..............08. 225
`
`
`
`
`
`
`
`
`
`
`
`
`
`10.8 MPEG-2 MP@ML and SP@MLadditional restrictions @
`. 227
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10.9 MPEG-2 MP@MLvideodata elements®
`.......... 227
`
`237
`11 Motion Compensation
`
`
`
`
`
`
`
`
`
`
`
`
` ..........4.J. 6 237
`11.1 Motion in sequences of picturesO
`............. 240
`11.2 Motion compensation in MPEG-10
`
`
`
`
`
`
`11.3 P-picture motion-compensated prediction® ......... 241
`
`
`
`
`
`
`
`
`
`
`11.4 B-picture motion-compensated prediction® ......... 242
`
`
`
`
`
`11.5 Generic motion displacements®
`.............4. 245
`
`
`
`
`
`
`11.6 Displacement principal and residual®
`............ 245
`
`
`
`
`
`
`11.7 Wraparound of motion displacements®
`........... 248
`11.8 Coding and decoding of motion vectors®
`.......... 250
`
`
`
`
`
`
`
`
`
`
`
`
`
`11.8.1 Decoding motion displacements®
`.......... 251
`
`
`
`
`
`11.8.2 Forward motion vectors® ............00-. 251
`11.8.3 Backward motion vectors® ......... vn 253 »
`
`
`
`
`
`
`
`
`
`
`
`11.8.4 P- and B-picture motion vectors® .......... 253
`
`
`
`
`
`
`
`
`11.8.5 Calculating displacements for YandC® ...... 253
`
`|
`
`.
`
`Page 15 of 498
`
`
`
`
`
`|
`
`|
`it
`
`'
`
`Page 15 of 498
`
`

`

`
`
`
`
`
`Page 16 of 498
`
`
`
`ls
`
`xviii
`
`
`
`
`
`
`
`
`11.9 Strategies for determining motion vectorsO
`
`
`
`
`11.10Motion compensation in MPEG-20
`
`
`
`
` ..............
`11.10.1 MPEG-2 motion vectors@®
`Cr ee
`
`
`
`11.10.2 MPEG-2 dual-prime prediction @
`
`
`11.10.3 Motion-compensated predictions @
`
`
`
`
`
`
`204
`
`206
`
`206
`
`260
`
`
`263
`
`Ca
`6 8 ee ee
`
`fe ee he ew le
`
`eee we
`
`
`
`
`
`12 Pel Reconstruction
`
`
`
`
`
`12.1 Intra-coded block dequantization®
`263
`
`
`
`
`
`
`
`12.2 DC prediction in intra macroblocks®
`265
`
`
`
`
`
`12.3 Nonintra-coded block dequantization®
`268
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`12.4 IDCT mismatch®
` .. 1...
`269
`re er
`12.4.1 IEEE Standard Specification for IDCT®
`
`
`
`
`
`
`
`269
`
`
`
`
`12.5 Limiting error propagation®
`
`272
`
`
`
`
`
`12.6 H.261 and H.263 oddification®
`
`272
`
`
`
`
`12.7 MPEG-2 pel reconstruction® ..............00.
`
`272
`
`
`
`
`
`
`12.7.1 MPEG-2 intra-coded block reconstruction D
`.
`. 274
`
`
`
`
`
`
`12.7.2 MPEG-2 nonintra-coded block reconstructionD
`J. 275
`
`
`
`
`
`
`12.8 IDCT mismatch control@ ..........0... 000085
`275
`eee ee ee ee
`
`
`
`
`
`
`12.8.1 Review of Katayama’s paper @
`277
`
`
`
`
`
`
`' 12.8.2 Review of Yagasaki’s paper®
`279
`
`eee ee ee
`
`
`
`
`
`
`
`
`
`
`13. Motion Estimation
`
`
`
`
`
`
`
`
`
`
`
`3.1 Criterion for block matching®
`284
`
`
`
`
`
`
`
`
`
`
`
`13.2 The search reference picture® «11... . ee ee
`286
`13.3 How large should the search range be? ®
`
`
`
`
`
`
`
`
`
`
`286
`eee
`
`
`
`
`
`
`
`
`
`
`
`13.4 Fractional pel searches®
`290
`"43.5 Correlation between motion vectors®
`
`
`
`
`
`
`
`292
`13.5.1 Adding chromato the distortion measure®
`
`
`
`
`
`
`
`
`292
`eee ee ee
`
`
`
`
`
`
`
`
`
`
`
`
`13.5.2 Tiebreaking®
`295
`
`
`
`
`
`
`
`
`
`
`
`
`13.5.3 Search failure®
` ....... 2.0... eee ee
`296
`
`
`
`
`
`
`
`13.5.4 Search failure at scene changes® ...........
`297
`
`
`
`
`
`
`13.6 Motion displacement search algorithms®
`297
`
`
`
`
`
`
`
`13.7 Fast algorithms for motion estimation®
`300
`
`
`
`
`
`
`
`
`13.8 A survey of fast search algorithms®
`.............
`301
`
`
`
`
`
`
`
`
`
`13.8.1 Fast search algorithms based on sparse sampling®
`. 304
`
`
`
`
`
`
`13.8.2 Variable resolution search techniques®
`
`
`
`
`
`ew ee
`13.8.3 Statistically sparse searches®
`5 ww.
`304
`
`
`
`
`
`
`
`13.8.4 Searches dependent on spatial continuity O
`305
`13.8.5 Telescopic search®
`=... ..... eee le
`
`
`
`
`
`
`
`
`
`
`
`
`305
`ee ee ee ee
`13.8.6 3-D (spatial/temporal) estimation®
`
`
`
`
`
`306
`ee ee ee ee ee
`
`
`
`
`
`13.8.7 Phase correlation searchO
`..............
`13.8.8 Other search techniqus®
`
`
`
`
`
`307
`
`
`
`
`
`
`
`13.8.9 Improving the motion displacement estimate®
`.
`. 807
`
`SY
`
`283
`
`304
`
`306
`
`Page 16 of 498
`
`

`

`
`
`
`
`
`
`
`
`
`13.9 Comparison of fast search algorithms®
`
`
`
`i 307
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`313
`
`314
`315
`
`. 315
`
`
`ee ee eee ee ee
`
`
`
`
`
`
`316
`
`. 317
`
`320
`
`. 321
`
`321
`
`321
`
`322
`
`
`
`
`
`
`
`324
`
`
`327
`
`327
`
`
`
`
`
`
`332
`
`
`
`14 Variable Quantization
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`14.1 Literature review®
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`14.1.1 Still-picture coding®
`14.1.2 Masking functions based on spatial derivative®
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`14.1.2.1 Spatial derivative error®
`
`14.1.2.2 Vertical/horizontal spatial derivative®
`
`
`
`
`
`
`14.1.2.3 Spatial derivative classificationO
`
`
`
`
`
`Masking functions based on block variance®
`
`
`
`Global variance®
`
`
`Scene complexity ®
`
`
`
`Bit allocation®
`
`
`
`Codingartifact classification®
`
`
`
`Perceptual quality ®
`
`
`
`
`Statistical activity classesO
`ee eee ee en
`
`
`
`DCT-based masking functions®
`eee ee ee
`
`
`
`
`14.1.4.1 AC energy®
`14.1.4.2 Minimax DCT coefficient ®
`
`
`
`
`
`
`
`
`14.1.4.3 Coefficient energy sum®
`
`
`
`
`
`14.1.4.4 Macroblock classification using the DCTO
`Overhead in switching quantizer_scale®
`
`
`
`
`
`
`
`
`
`
`1.6 MPEG-2 Test Model 50
`See ee ee
`
`
`
`14.1.7 Coded bits®
`eee ee ee te ee
`
`
`
`
`14.2 Perceptual studies®
`
`
`
`
`
`14.3 Variable quantization and motion®
`
`
`
`14.3.1 Temporal masking®
`14.3.2 Relative motion between adjacent macroblocksO
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`14.4 Summary O
`15 Rate Control in MPEG
`
`333 —
`
`
`
`
`
`
`
`
`
`333
`15.1 Literature review O
`
`
`
`
`
`
`The rate buffer ®
`334
`
`
`
`
`
`
`
`px64 reference model and generalizations®
`334
`
`
`
`
`
`15.1.2.1 S-shaped control characteristic®
`335
`eee eee te ee
`
`
`
`
`
`
`MPEG-2 Test Model 50
`15.1.3.1 Rate control with DCT minimax activity®
`
`
`
`
`
`
`
`336
`
`
`
`
`
`
`Rate control based on recoding ®
`336
`eee eee eee ee ee et
`
`
`
`
`
`337
`VBRrate control®
`
`
`
`
`
`Recursive prediction with recoding ®
`337
`
`
`
`Complexity prediction®
`Se ee tee 338
`
`
`
`Adaptive GOP®
`ee 339
`
`
`
`
`Adaptive bit allocation®
`
`
`
`
`
`
`
`322
`
`322
`
`322
`
`323
`
`323
`
`323
`
`325
`
`328
`
`328
`
`331
`
`331
`
`. 331
`
`
`
`335
`
`
`
`Le 339
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 17 of 498
`
`Page 17 of 498
`
`

`

`|:
`Le
`rr
`:
`
`
`
`
`
`H
`:
`4
`hs
`:
`:
`:
`aie
`|:
`H.
`
`xx
`
`
`
`
`
`15.2 Encoder and decoder bufferingB 2... 2... 340
`
`
`
`
`
`
`15.3 VBV decoding modl® ..........,,,,..02~2~, 342
`
`
`
`
`
`15.3.1 bitrate® 2... 342
`
`
`
`
`
`15.3.2 picturerate®. . 343
`
`15.3.3 vbv_buffersize® 2.0.0... 343
`
`
`
`15.3.4 vbv.delay® .... 344
`
`
`
`
`15.4 Rate contrl® 345
`
`
`
`
`15.5 Buffer occupancy and direct rate control® Se ee 347
`
`
`
`
`
`
`
`
`15.5.1 Rate feedbak®
`......0,.,,000020~2~2~—2! 348
`
`
`
`
`
`15.5.1.1 Errorcarry®
`......0,,,,00 202. 349
`
`
`
`
`
`15.5.1.2 Interleaved rate control systems® ....., 349
`
`
`
`
`
`
`...... 349
`15.5.1.3 Quantizer switching overhead®
`
`
`
`
`
`|
`15.5.2 Effects of quantization®
`......,.,,,00~2 ~~. 349
`
`
`
`
`
`|
`15.5.3 macroblock-type selection®
`.....,,...... 350
`
`
`
`
`
`| 15.5.4 Interactions between picturess®32... 351
`
`
`
`
`
`15.6 Buffer constraints® 2... 351
`
`
`
`
`15.6.1 Rate control profiles® ........,0,,00~... 353
`
`
`
`
`
`15.6.2 Buffer overfow® .......0000,000020~2~2. 354
`
`
`
`
`15.6.3 Buffer underfow® ......00000,0,022 2. 354
`
`
`
`
`7
`. 15.6.4 Adjusting the bit allocation®
`.........~,.. 355
`.
`
`
`
`
`
`
`
`....... 355
`15.6.5 Discarding high frequency coefficients®
`: Lo
`Ji
`
`
`
`
`
`
`
`" St
`15.6.6 Picture repeating®
`...... ek ee ey 355
`
`
`
`
`: