`
`
`
`
`
`
`
`
`
`
`
`
`
`
`I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n
`
`
`
`ITU-T
`
`TELECOMMUNICATION
`STANDARDIZATION SECTOR
`OF ITU
`
`H.264
`
`(03/2005)
`
`SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS
`Infrastructure of audiovisual services – Coding of moving
`video
`
`Advanced video coding for generic audiovisual
`services
`
`ITU-T Recommendation H.264
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 1
`
`
`
`
`
`ITU-T H-SERIES RECOMMENDATIONS
`AUDIOVISUAL AND MULTIMEDIA SYSTEMS
`
`
`
`CHARACTERISTICS OF VISUAL TELEPHONE SYSTEMS
`INFRASTRUCTURE OF AUDIOVISUAL SERVICES
`General
`Transmission multiplexing and synchronization
`Systems aspects
`Communication procedures
`Coding of moving video
`Related systems aspects
`Systems and terminal equipment for audiovisual services
`Directory services architecture for audiovisual and multimedia services
`Quality of service architecture for audiovisual and multimedia services
`Supplementary services for multimedia
`MOBILITY AND COLLABORATION PROCEDURES
`Overview of Mobility and Collaboration, definitions, protocols and procedures
`Mobility for H-Series multimedia systems and services
`Mobile multimedia collaboration applications and services
`Security for mobile multimedia systems and services
`Security for mobile multimedia collaboration applications and services
`Mobility interworking procedures
`Mobile multimedia collaboration inter-working procedures
`BROADBAND AND TRIPLE-PLAY MULTIMEDIA SERVICES
`Broadband multimedia services over VDSL
`
`For further details, please refer to the list of ITU-T Recommendations.
`
`
`
`H.100–H.199
`
`H.200–H.219
`H.220–H.229
`H.230–H.239
`H.240–H.259
`H.260–H.279
`H.280–H.299
`H.300–H.349
`H.350–H.359
`H.360–H.369
`H.450–H.499
`
`H.500–H.509
`H.510–H.519
`H.520–H.529
`H.530–H.539
`H.540–H.549
`H.550–H.559
`H.560–H.569
`
`H.610–H.619
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 2
`
`
`
`
`ITU-T Recommendation H.264
`
`Advanced video coding for generic audiovisual services
`
`
`
`
`Summary
`This Recommendation | International Standard represents an evolution of the existing video coding standards (H.261,
`H.262, and H.263) and it was developed in response to the growing need for higher compression of moving pictures for
`various applications such as videoconferencing, digital storage media, television broadcasting, Internet streaming, and
`communication. It is also designed to enable the use of the coded video representation in a flexible manner for a wide
`variety of network environments. The use of this Recommendation | International Standard allows motion video to be
`manipulated as a form of computer data and to be stored on various storage media, transmitted and received over existing
`and future networks and distributed on existing and future broadcasting channels.
`The revision approved 2005-03 contains modifications of the video coding standard to add four new profiles, referred to
`as the High, High 10, High 4:2:2, and High 4:4:4 profiles, to improve video quality capability and to extend the range of
`applications addressed by the standard (for example, by including support for a greater range of picture sample precision
`and higher-resolution chroma formats). Additionally, a definition of new types of supplemental data has been specified
`to further broaden the applicability of the video coding standard. Finally, a number of corrections to errors in the
`published text have been included. This revision, in addition to enhancing video coding capability, will serve to maintain
`technical alignment with the corresponding jointly-developed ISO/IEC 14496-10 standard.
`Corrigendum 1 to ITU-T Rec. H.264 corrected and updated various minor aspects to bring the ITU-T version of the text
`up to date relative to the April 2005 output status approved as a new edition of the corresponding jointly-developed and
`technically-aligned text ISO/IEC 14496-10. It additionally fixes a number of minor errors and needs for clarification and
`defines three previously-reserved sample aspect ratio indicators.
`This edition includes the text approved 2005-03 and its Corrigendum 1 approved 2005-09.
`
`
`Source
`ITU-T Recommendation H.264 was approved on 1 March 2005 by ITU-T Study Group 16 (2005-2008) under the ITU-T
`Recommendation A.8 procedure. It includes modifications introduced by H.264 (2005) Cor.1 approved on 13 September
`2005 by ITU-T Study Group 16 (2005-2008) under the ITU-T Recommendation A.8 procedure.
`
`
`
`
`
`
`
`
`
`
`
`ITU-T Rec. H.264 (03/2005)
`
`i
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 3
`
`
`
`
`
`FOREWORD
`The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of
`telecommunications. The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of
`ITU. ITU-T
`is responsible for studying
`technical, operating and
`tariff questions and
`issuing
`Recommendations on them with a view to standardizing telecommunications on a worldwide basis.
`The World Telecommunication Standardization Assembly (WTSA), which meets every four years,
`establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on
`these topics.
`The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1.
`In some areas of information technology which fall within ITU-T's purview, the necessary standards are
`prepared on a collaborative basis with ISO and IEC.
`
`
`
`
`NOTE
`In this Recommendation, the expression "Administration" is used for conciseness to indicate both a
`telecommunication administration and a recognized operating agency.
`
`Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain
`mandatory provisions (to ensure e.g. interoperability or applicability) and compliance with the
`Recommendation is achieved when all of these mandatory provisions are met. The words "shall" or some
`other obligatory language such as "must" and the negative equivalents are used to express requirements. The
`use of such words does not suggest that compliance with the Recommendation is required of any party.
`
`
`
`
`INTELLECTUAL PROPERTY RIGHTS
`ITU draws attention to the possibility that the practice or implementation of this Recommendation may
`involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence,
`validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others
`outside of the Recommendation development process.
`As of the date of approval of this Recommendation, ITU had received notice of intellectual property,
`protected by patents, which may be required to implement this Recommendation. However, implementors
`are cautioned that this may not represent the latest information and are therefore strongly urged to consult the
`TSB patent database.
`
`
`
`
` ITU 2005
`All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the
`prior written permission of ITU.
`
`ii
`
`ITU-T Rec. H.264 (03/2005)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 4
`
`
`
`
`
`CONTENTS
`
`
`Page
`Foreword ..........................................................................................................................................................................xiv
`0
`Introduction ................................................................................................................................................................1
`0.1
`Prologue ...............................................................................................................................................................1
`0.2
`Purpose.................................................................................................................................................................1
`0.3
`Applications ..........................................................................................................................................................1
`0.4
`Publication and versions of this specification ......................................................................................................1
`0.5
`Profiles and levels.................................................................................................................................................2
`0.6 Overview of the design characteristics .................................................................................................................2
`0.6.1
`Predictive coding ..........................................................................................................................................3
`0.6.2
`Coding of progressive and interlaced video..................................................................................................3
`0.6.3
`Picture partitioning into macroblocks and smaller partitions........................................................................3
`0.6.4
`Spatial redundancy reduction........................................................................................................................3
`0.7 How to read this specification ..............................................................................................................................3
`Scope ............................................................................................................................................................................4
`1
`2 Normative references..................................................................................................................................................4
`3 Definitions....................................................................................................................................................................4
`4 Abbreviations............................................................................................................................................................12
`5 Conventions...............................................................................................................................................................12
`5.1
`Arithmetic operators...........................................................................................................................................13
`5.2
`Logical operators................................................................................................................................................13
`5.3
`Relational operators ...........................................................................................................................................13
`5.4
`Bit-wise operators...............................................................................................................................................13
`5.5
`Assignment operators .........................................................................................................................................14
`5.6
`Range notation....................................................................................................................................................14
`5.7 Mathematical functions.......................................................................................................................................14
`5.8
`Variables, syntax elements, and tables ...............................................................................................................15
`5.9
`Text description of logical operations ................................................................................................................16
`5.10 Processes ............................................................................................................................................................17
`Source, coded, decoded and output data formats, scanning processes, and neighbouring relationships..........17
`6.1
`Bitstream formats................................................................................................................................................17
`6.2
`Source, decoded, and output picture formats .....................................................................................................17
`6.3
`Spatial subdivision of pictures and slices ...........................................................................................................22
`6.4
`Inverse scanning processes and derivation processes for neighbours................................................................23
`6.4.1
`Inverse macroblock scanning process.........................................................................................................23
`6.4.2
`Inverse macroblock partition and sub-macroblock partition scanning process...........................................24
`6.4.2.1
`Inverse macroblock partition scanning process ......................................................................................25
`6.4.2.2
`Inverse sub-macroblock partition scanning process................................................................................25
`6.4.3
`Inverse 4x4 luma block scanning process...................................................................................................26
`6.4.4
`Inverse 8x8 luma block scanning process...................................................................................................26
`6.4.5
`Derivation process of the availability for macroblock addresses................................................................26
`6.4.6
`Derivation process for neighbouring macroblock addresses and their availability.....................................27
`6.4.7
`Derivation process for neighbouring macroblock addresses and their availability in MBAFF frames ......27
`6.4.8
`Derivation processes for neighbouring macroblocks, blocks, and partitions..............................................28
`6.4.8.1 Derivation process for neighbouring macroblocks .................................................................................29
`6.4.8.2 Derivation process for neighbouring 8x8 luma block.............................................................................29
`6.4.8.3 Derivation process for neighbouring 4x4 luma blocks ...........................................................................30
`6.4.8.4 Derivation process for neighbouring 4x4 chroma blocks .......................................................................30
`6.4.8.5 Derivation process for neighbouring partitions.......................................................................................31
`6.4.9
`Derivation process for neighbouring locations ...........................................................................................33
`6.4.9.1 Specification for neighbouring locations in fields and non-MBAFF frames ..........................................33
`6.4.9.2 Specification for neighbouring locations in MBAFF frames..................................................................34
`Syntax and semantics ...............................................................................................................................................36
`7.1 Method of describing syntax in tabular form......................................................................................................36
`7.2
`Specification of syntax functions, categories, and descriptors ...........................................................................37
`7.3
`Syntax in tabular form ........................................................................................................................................38
`7.3.1
`NAL unit syntax..........................................................................................................................................38
`7.3.2
`Raw byte sequence payloads and RBSP trailing bits syntax.......................................................................39
`7.3.2.1 Sequence parameter set RBSP syntax.....................................................................................................39
`
`7
`
`6
`
`
`
`
`
`
`
`ITU-T Rec. H.264 (03/2005)
`
`iii
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 5
`
`
`
`
`
`7.3.2.1.1 Scaling list syntax ............................................................................................................................40
`7.3.2.1.2 Sequence parameter set extension RBSP syntax..............................................................................40
`7.3.2.2 Picture parameter set RBSP syntax.........................................................................................................41
`7.3.2.3 Supplemental enhancement information RBSP syntax...........................................................................42
`7.3.2.3.1 Supplemental enhancement information message syntax ................................................................42
`7.3.2.4 Access unit delimiter RBSP syntax ........................................................................................................43
`7.3.2.5 End of sequence RBSP syntax................................................................................................................43
`7.3.2.6 End of stream RBSP syntax....................................................................................................................43
`7.3.2.7 Filler data RBSP syntax..........................................................................................................................43
`7.3.2.8 Slice layer without partitioning RBSP syntax.........................................................................................43
`7.3.2.9 Slice data partition RBSP syntax ............................................................................................................43
`7.3.2.9.1 Slice data partition A RBSP syntax..................................................................................................43
`7.3.2.9.2 Slice data partition B RBSP syntax..................................................................................................44
`7.3.2.9.3 Slice data partition C RBSP syntax..................................................................................................44
`7.3.2.10 RBSP slice trailing bits syntax..............................................................................................................44
`7.3.2.11 RBSP trailing bits syntax ......................................................................................................................44
`7.3.3
`Slice header syntax .....................................................................................................................................45
`7.3.3.1 Reference picture list reordering syntax .................................................................................................46
`7.3.3.2 Prediction weight table syntax ................................................................................................................47
`7.3.3.3 Decoded reference picture marking syntax.............................................................................................48
`7.3.4
`Slice data syntax .........................................................................................................................................49
`7.3.5 Macroblock layer syntax.............................................................................................................................50
`7.3.5.1 Macroblock prediction syntax.................................................................................................................51
`7.3.5.2 Sub-macroblock prediction syntax..........................................................................................................52
`7.3.5.3 Residual data syntax ...............................................................................................................................53
`7.3.5.3.1 Residual block CAVLC syntax........................................................................................................54
`7.3.5.3.2 Residual block CABAC syntax........................................................................................................55
`7.4
`Semantics............................................................................................................................................................56
`7.4.1
`NAL unit semantics ....................................................................................................................................56
`7.4.1.1 Encapsulation of an SODB within an RBSP (informative) ....................................................................58
`7.4.1.2 Order of NAL units and association to coded pictures, access units, and video sequences....................59
`7.4.1.2.1 Order of sequence and picture parameter set RBSPs and their activation........................................59
`7.4.1.2.2 Order of access units and association to coded video sequences .....................................................60
`7.4.1.2.3 Order of NAL units and coded pictures and association to access units ..........................................60
`7.4.1.2.4 Detection of the first VCL NAL unit of a primary coded picture ....................................................62
`7.4.1.2.5 Order of VCL NAL units and association to coded pictures............................................................63
`7.4.2
`Raw byte sequence payloads and RBSP trailing bits semantics .................................................................63
`7.4.2.1 Sequence parameter set RBSP semantics ...............................................................................................63
`7.4.2.1.1 Scaling list semantics .......................................................................................................................68
`7.4.2.1.2 Sequence parameter set extension RBSP semantics ........................................................................69
`7.4.2.2 Picture parameter set RBSP semantics ...................................................................................................70
`7.4.2.3 Supplemental enhancement information RBSP semantics .....................................................................73
`7.4.2.3.1 Supplemental enhancement information message semantics...........................................................73
`7.4.2.4 Access unit delimiter RBSP semantics ...................................................................................................73
`7.4.2.5 End of sequence RBSP semantics...........................................................................................................73
`7.4.2.6 End of stream RBSP semantics...............................................................................................................73
`7.4.2.7 Filler data RBSP semantics.....................................................................................................................73
`7.4.2.8 Slice layer without partitioning RBSP semantics ...................................................................................73
`7.4.2.9 Slice data partition RBSP semantics.......................................................................................................74
`7.4.2.9.1 Slice data partition A RBSP semantics ............................................................................................74
`7.4.2.9.2 Slice data partition B RBSP semantics.............................................................................................74
`7.4.2.9.3 Slice data partition C RBSP semantics.............................................................................................74
`7.4.2.10 RBSP slice trailing bits semantics.........................................................................................................74
`7.4.2.11 RBSP trailing bits semantics.................................................................................................................75
`7.4.3
`Slice header semantics ................................................................................................................................75
`7.4.3.1 Reference picture list reordering semantics ............................................................................................80
`7.4.3.2 Prediction weight table semantics...........................................................................................................81
`7.4.3.3 Decoded reference picture marking semantics........................................................................................82
`7.4.4
`Slice data semantics....................................................................................................................................85
`7.4.5 Macroblock layer semantics .......................................................................................................................85
`7.4.5.1 Macroblock prediction semantics ...........................................................................................................92
`7.4.5.2 Sub-macroblock prediction semantics ....................................................................................................93
`7.4.5.3 Residual data semantics ..........................................................................................................................95
`
`iv
`
`ITU-T Rec. H.264 (03/2005)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1019, p. 6
`
`
`
`
`
`7.4.5.3.1 Residual block CAVLC semantics...................................................................................................96
`7.4.5.3.2 Residual block CABAC semantics ..................................................................................................96
`8 Decoding process.......................................................................................................................................................96
`8.1
`NAL unit decoding process.................................................................................................................................97
`8.2
`Slice decoding process........................................................................................................................................98
`8.2.1
`Decoding process for picture order count ...................................................................................................98
`8.2.1.1 Decoding process for picture order count type 0 ....................................................................................99
`8.2.1.2 Decoding process for picture order count type 1 ..................................................................................100
`8.2.1.3 Decoding process for picture order count type 2 ..................................................................................101
`8.2.2
`Decoding process for macroblock to slice group map ..............................................................................102
`8.2.2.1 Specification for interleaved slice group map type...............................................................................103
`8.2.2.2 Specification for dispersed slice group map type..................................................................................103
`8.2.2.3 Specification for foreground with left-over slice group map type ........................................................104
`8.2.2.4 Specification for box-out slice group map types...................................................................................104
`8.2.2.5 Specification for raster scan slice group map types ..............................................................................105
`8.2.2.6 Specification for wipe slice group map types .......................................................................................105
`8.2.2.7 Specification for explicit slice group map type.....................................................................................105
`8.2.2.8 Specification for conversion of map unit to slice group map to macroblock to slice group map .........105
`8.2.3
`Decoding process for slice data partitioning.............................................................................................105
`8.2.4
`Decoding process for reference picture lists construction.........................................................................106
`8.2.4.1 Decoding process for picture numbers..................................................................................................107
`8.2.4.2
`Initialisation process for reference picture lists.....................................................................................107
`8.2.4.2.1
`Initialisation process for the reference picture list for P and SP slices in frames...........................108
`8.2.4.2.2
`Initialisation process for the reference picture list for P and SP slices in fields.............................108
`8.2.4.2.3
`Initialisation process for reference picture lists for B slices in frames...........................................109
`8.2.4.2.4
`Initialisation process for reference picture lists for B slices in fields.............................................109
`8.2.4.2.5
`Initialisation process for reference picture lists in fields................................................................110
`8.2.4.3 Reordering process for reference picture lists.......................................................................................110
`8.2.4.3.1 Reordering process of reference picture lists for short-term reference pictures.............................111
`8.2.4.3.2 Reordering process of reference picture lists for long-term reference pictures..............................112
`8.2.5
`Decoded reference picture marking process .............................................................................................112
`8.2.5.1 Sequence of operations for decoded reference picture marking process...............................................113
`8.2.5.2 Decoding process for gaps in frame_num.............................................................................................113
`8.2.5.3 Sliding window decoded reference picture marking process................................................................114
`8.2.5.4 Adaptive memory control decoded reference picture marking process ................................................114
`8.2.5.4.1 Marking process of a short-term reference picture as “unused for reference” ...............................114
`8.2.5.4.2 Marking process of a long-term reference picture as “unused for reference” ................................115
`8.2.5.4.3 Assignment process of a LongTermFrameIdx to a short-term reference picture ...........................115
`8.2.5.4.4 Decoding process for MaxLongTermFrameIdx.............................................................................115
`8.2.5.4.5 Marking process of all reference pictures as “unused for reference” and setting
`MaxLongTermFrameIdx to “no long-term frame indices”...............................................................................115
`8.2.5.4.6 Process for assigning a long-term frame index to the current picture ............................................116
`8.3
`Intra prediction process....................................................................................................................................116
`8.3.1
`Intra_4x4 prediction process for luma samples ........................................................................................117
`8.3.1.1 Derivation process for the Intra4x4PredMode......................................................................................117
`8.3.1.2
`Intra_4x4 sample prediction .................................................................................................................119
`8.3.1.2.1 Specification of Intra_4x4_Vertical prediction mode ....................................................................119
`8.3.1.2.2 Specification of Intra_4x4_Horizontal prediction mode................................................................119
`8.3.1.2.3 Specification of Intra_4x4_DC prediction mode ...........................................................................120
`8.3.1.2.4 Specification of Intra_4x4_Diagonal_Down_Left prediction mode ..................................