US009191673B2
`
`(12) United States Patent
`US 9,191,673 B2
`(10) Patent N0.:
`
`Oh et al.
`(45) Date of Patent:
`*Nov. 17, 2015
`
`(54)
`
`IMAGE ENCODING APPARATUS
`
`(71) Applicant: M & K HOLDINGS INC., Seoul (KR)
`.
`.
`.
`.
`InVemors‘ figogcgaiyeosfifggggégj
`g’
`g F
`(73) Assignee: M & K HOLDINGS INC., Seoul (KR)
`
`(72)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`This patent is subject to a teminal dis_
`claimer.
`
`(2014.01)
`(2014.01)
`
`H04N 19/91
`H04N19/13
`(52) US. Cl.
`CPC ............. H04N19/109 (2014.11);H04N19/11
`(2014.11); H04N19/122 (2014.11); H04N
`19/129 (2014.11); H04N19/13 (2014.11);
`H04N19/136 (201411); H04N19/159
`(2014.11); H04N19/176 (2014.11); H04N
`19/18 (2014.11); H04N19/61 (2014.11); H04N
`19/91 (2014.11); H04N 19/96 (2014.11)
`(58) Field of Classification Search
`CPC ......
`.................. H04N 19/18; H04N 19/109
`See application file for complete search history.
`
`(21) Appl. N0.: 14/687,482
`
`(56)
`
`References Cited
`
`(22)
`
`(65)
`
`Filed:
`
`Apr. 15, 2015
`_
`_
`_
`Prlor Publlcatlon Data
`US 2015/0222898 A1
`Aug. 63 2015
`
`Related US Application Data
`(63) Continuation of application No. 13/624,753, filed on
`863131215 2012’ now Pat. NO' 9519056515 Wthh 15 a
`contmuation
`Of
`application
`NO’
`PCT/KR2011/002972, filed on Apr. 22, 2011.
`
`(30)
`
`Foreign Application Priority Data
`
`Apr. 23, 2010
`
`(KR) ........................ 10-2010-0038158
`
`(51)
`
`Int. Cl-
`H04N 19/109
`H04N 19/129
`H04N 19/61
`H041V 19/96
`H04N 19/11
`H04N 19/136
`H041V 19/18
`H04N 19/122
`H04N19/159
`H04N19/176
`
`(2014.01)
`(2014.01)
`(2014.01)
`(2014.01)
`(201401)
`(2014.01)
`(2014.01)
`(2014.01)
`(2014.01)
`(2014.01)
`
`U.S. PATENT DOCUMENTS
`8,976,861 B2 *
`3/2015 Rojals et 31.
`............. 375/24002
`2010/0118971 A1 *
`5/2010 Tanida et al.
`............ 375/24023
`2012/0008683 A1 *
`1/2012 KarczeWICz et 31.
`375/24012
`(Continued)
`Primary Examiner 7 Sath V Perungavoor
`Assistant Examiner D James Pontius
`(74) Attorney, Agent, or Firm 7 Mintz Levin Cohn Ferris
`Glovsky and Popeo, P.C.; Kongsik Kim; Jonathon Western
`
`57
`
`ABSTRACT
`
`)
`(
`Provided is a moving picture coding apparatus for increasing
`a compression ratio of a moving picture signal while main-
`taining high image quality. To minimize the number of bits
`required for a residual block, quantized transform coefficients
`are adaptively divided into a plurality of subsets according to
`a size of a transform block, and non-zero quantized transform
`coefficients of each subset are scanned and encoded. Also, an
`intra prediction mode ofthe current block is determined using
`reference pixels obtained by filtering reference pixels of the
`current block. Accordingly, it is possible to minimize the
`amount of data required for the residual block of the current
`block.
`
`7 Claims, 5 Drawing Sheets
`
`654K733
`'
`
`NO
`
`DIVISION
`32x32
`BLOCK?
`
`
`YES
`16x16
`BLOCK7
`
`
`YES
`
`Mb64_Iypo
`
`Mb16_Typo
`
`Mb32_1ypc DETERMINE
`CODING MODE
`
`
`::1
`
`DETERMINE
`DETERMINE
`CODING MODE
`CODING MODE
`CODING MODE
`AMONG 16x16
`AMONG 32x32
`AMONG 64x64.
`AND
`32x16 AND
`64x32 AND
`16x32
`32X64
`SUB-BLOCKS
`
`CODING IN
`CODING IN
`CODING IN
`ACCORDANCE
`ACCORDANCE
`ACCORDANCE
`WITH
`WITH
`WITH
`DETERMINED
`DETERMINED
`DETERMINED
`CODING MODE
`CODING MODE
`
`mb64,s1<1 ,fla
`::1
`p
`
`9
`
`mb32,skip,fIag
`: :1
`
`mbIB,sI<i
`
`,Ila
`
`p
`
`g
`
`Page 1 of 14
`
`SAMSUNG EXHIBIT 1001
`
`SAMSUNG EXHIBIT 1001
`
`Page 1 of 14
`
`

`

`US 9,191,673 B2
`
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`2012/0170649 A1*
`7/2012 Chenetal.
`............... 375/240.12
`
`2013/0058407 A1*
`3/2013 Sole Rojals et a1.
`375/240.12
`2013/0188724 A1*
`7/2013 Nguyen et a1.
`........... 375/240.18
`
`2013/0343454 A1* 12/2013 Yeo etal.
`2014/0321529 A1* 10/2014 Jung et a1.
`2015/0078432 A1*
`3/2015 Wang et a1.
`2015/0078447 A1*
`3/2015 Gameietal.
`
`................. 375/240.03
`.. 375/240.02
`
`.............. 375/240.02
`............. 375/24012
`
`* cited by examiner
`
`Page 2 of 14
`
`Page 2 of 14
`
`

`

`U.S. Patent
`
`Nov. 17, 2015
`
`Sheet 1 of5
`
`US 9,191,673 B2
`
`64x64
`
`32x32
`
`16x16
`
`Page 3 of 14
`
`Page 3 of 14
`
`

`

`U.S. Patent
`
`NOV. 17, 2015
`
`Sheet 2 of 5
`
`US 9,191,673 B2
`
`
`uz__2mu._.m_m_
`
`
`
`mEOE02500
`
`
`
`mexm:OZO_2<
`
`QZ<
`
`meOJmeDm
`
`oQBIm.BE
`
`9meH
`
`m_Z__>Em_._.m_n_
`
`
`
`mQOE02500
`
`
`
`"mmXNmOZOE<
`
`QZ<weXNm
`
`mmxm:
`
`Z_@2500
`
`Z_@2500
`
`/\
`
`
`
`mm>zo_m_>_o
`
`OZ
`
`N.0_n_
`
`QwZEmEFmQ
`
`
`
`mQOE02500
`
`Dm_2__2mm;m_m
`
`
`
`m_m_0_>_GZEOQ
`
`I._._>>
`
`I._._>>
`
`mablvng $50.5
`
`mafilmng
`
`$meH
`
`m_Z__\/_mm_._.m_m
`
`mQOE©2500
`
`
`
`n4%waOZOE<
`
`DZ<mmxfiw
`
`wamm
`
`Z_@2500
`
`$waH
`
`
`
`zo_m_>_oOzmmemExémm>
`
`
`36%
`
`/\
`
`Page 4 of 14
`
`mOZ<QmOOO<
`92,79quBE
`m02<QmOOO<
`@QrTQEwwmmnE
`MQZ<DEOOO<
`omzwagmwvwgE
`
`Qm_2__2mm_km_m_
`
`
`
`mDOEOZEOO
`
`I._._>>
`
`Page 4 of 14
`
`
`

`

`US 9,191,673 B2
`
`ZO_.r<_2mOn_Z_
`
`
`
`of$023:me
`
`
`
`
`
`20:921cm:mo<mokmmoEszEoo.
`
`
`
`E55520:02
`
`S
`
`a
`
`3$98
`mEoEzmof
`
`5xat$55M\028ng
`
`moo:
`
`
`20:05meI358%.
`
`I<Ez_<Ez_
`
`
` x:23/zmolmz/EtH/2929330m/umE>z_U,/EzmaEmoo:23E32550
`
`
`2929236macs.M/QmN_._.Z<DO\EmOmegnc.©2500
`v./2m0mmz<E
`
`
`
`
`
`S.U
`
`eta
`
`P.moi
`
`tJoEzoonEgoo
`
`<534052004..........................
`
`Page 5 of 14
`
`Page 5 of 14
`
`
`
`
`

`

`US. Patent
`
`Nov. 17, 2015
`
`Sheet 4 015
`
`US 9,191,673 B2
`
`FIG. 4
`
`
`
`FIG. 5
`
`
`
`
`
`
`
`Page 6 of 14
`
`Page 6 of 14
`
`

`

`U.S. Patent
`
`NOV. 17, 2015
`
`Sheet 5 of 5
`
`US 9,191,673 B2
`
`FIG. 6
`
`III-III-
`
`III...//
`III..///
`III.////
`
`%%%%%%II
`%%%%%III
`%%%%llll
`%%%lllll
`%%IIIIII
`%IIIIIII
`III-III-
`
`m-------
`m-------
`
`Page 7 of 14
`
`Page 7 of 14
`
`

`

`US 9,191,673 B2
`
`2
`
`1
`IMAGE ENCODING APPARATUS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation application of applica-
`tion Ser. No. 13/624,753 filed on Sep. 21, 2012, which is a
`continuation of International Application No. PCT/KR2011/
`002972 filed on Apr. 22, 2011, which claims priority to
`Korean Application No. 10-2010-0038158 filed on Apr. 23,
`2010. The applications are incorporated herein by reference.
`
`TECHNICAL FIELD
`
`The present invention relates to an image processing appa-
`ratus and method, and more particularly, to an apparatus for
`reducing the amount of residual signals of an image and
`entropy-coding the residual signals.
`
`BACKGROUND ART
`
`To efiiciently transmit a moving picture signal at a low data
`rate while maintaining high image quality, various digital
`moving picture compression technologies have been pro-
`posed. These moving picture compression technologies
`include H.261, Motion Picture Experts Group (MPEG)-2/
`H.262, H.263, MPEG-4, Advanced Video Coding (AVC)/
`H.264, and so on. The compression technologies include a
`discrete cosine transform (DCT) scheme, a motion compen-
`sation (MC) scheme, a quantization scheme, an entropy cod-
`ing scheme, and so on.
`For image coding, each picture is divided into a plurality of
`slices, and each slice is divided into a plurality of coding
`blocks of a predetermined size. Since an image of a high-
`definition (HD) grade or above has many flat areas, an image
`compression can be improved by encoding the image with
`coding blocks that are larger than a macroblock (MB).
`Therefore, new prediction technique is required and
`changes in transform coding, entropy coding, intra prediction
`and inter prediction is required as the size of the coding unit
`increases to increase image compression ratio without
`increase of the complexity of image compression.
`
`SUMMARY OF THE DISCLOSURE
`
`The present invention is directed to an apparatus for encod-
`ing a moving picture, and more particularly, to an apparatus
`for effectively encoding transformed residual signals of a
`moving picture having a predetermined size or more.
`One aspect of the present invention provides an apparatus
`for encoding a moving picture, including: a coding mode
`determiner configured to divide an input image into coding
`units and determine a prediction mode of the coding unit and
`a size of a predictive coding block of the coding unit; a
`transform/quantization unit configured to transform and
`quantize a residual block between the predictive coding block
`and a prediction block generated through intra prediction or
`inter prediction; an inverse quantization/transform unit con-
`figured to inversely quantize and inversely transform the
`quantized transform block into a frequency domain; a
`deblocking filter configured to apply a deblocking filtering
`process to image data received from the inverse quantization/
`transform decoding unit; a prediction block generator config-
`ured to generate the predictive coding block; and an entropy
`coder configured to encode the quantized transform coeffi-
`cients from the transform coding/quantization unit, wherein,
`when a size of a transform block is equal to or larger than a
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Page 8 of 14
`
`predetermined size, the entropy coder divides the quantized
`transform coefficients into a plurality of subsets, and scans
`and entropy-codes non-zero quantized transform coefficients
`of the each subset.
`
`The present invention enables coding efficiency to be
`enhanced by reducing the amount of residual signals of a
`block to be encoded. Also, by effectively scanning a quan-
`tized transform coefficient other than 0 during entropy cod-
`ing, the number of bits required for the entropy coding is
`minimized, such that the coding efficiency can be improved.
`
`BRIEF DESCRIPTION OF DRAWINGS
`
`FIG. 1 shows a block division structure according to an
`exemplary embodiment of the present invention.
`FIG. 2 illustrates a method of determining a coding mode
`according to the present invention.
`FIG. 3 illustrates an apparatus for encoding a moving pic-
`ture according to the present invention.
`FIGS. 4 to 6 are diagrams illustrating an adaptive block
`division method according to the present invention.
`FIG. 7 is a diagram illustrating a method of encoding a
`residual signal.
`
`DETAILED DESCRIPTION OF THE
`DISCLOSURE
`
`Hereinafter, various embodiments of the present invention
`will be described in detail with reference to the accompanying
`drawings. However, the present invention is not limited to the
`exemplary embodiments disclosed below, but can be imple-
`mented in various types. Therefore, many other modifications
`and variations of the present invention are possible and it is to
`be understood that within the scope of the disclosed concept,
`the present invention may be practiced otherwise than as has
`been specifically described.
`For image coding, each picture is divided into a plurality of
`slices, and each slice is divided into a plurality of coding units
`of a predetermined size. Since an image of a high-definition
`(HD) grade or above has many flat areas, an image compres-
`sion ratio can be improved by encoding the image with coding
`units that are greater than a macroblock (MB) having a size of
`1 6x 1 6.
`
`Coding unit according to the present invention may be a
`block having a size of 32x32 or block having a size of 64x64
`as well as MBs having the size of16><16.Also, a block having
`a size of 8x8 or less may be the coding unit. For convenience,
`the largest coding unit is referred to as a super macroblock
`(SMB). A size of the SMB may be determined according to
`information indicating a size of the smallest coding unit and
`depth information. The depth information indicates a differ-
`ence between a size of the SMB and a size of the smallest
`
`coding unit.
`Thus, coding unit that will be used for encoding all pictures
`of an image sequence may be SMB or sub-block of the SMB.
`The allowable sizes of the coding units may be designated by
`default or in a sequence header. When the allowable sizes of
`coding units are designated in the sequence header, the allow-
`able sizes of the coding units are designated according to the
`size of the smallest coding unit and the depth information.
`Each picture or slice is divided into a plurality of SMB
`units. Each SMB or sub-block ofthe SMB may is intra-coded
`or inter-coded and decoded.
`
`To enable the coding unit (i.e., an SMB or a sub-block of
`the SMB) to be accurately decoded, an encoder should add
`size information on a predictive coding block of the coding
`unit and prediction mode information indicating that the cod-
`
`Page 8 of 14
`
`

`

`US 9,191,673 B2
`
`3
`ing unit has been coded in intra prediction mode or in inter
`prediction mode to a bit stream. To this end, the prediction
`mode information and the information indicating the size of
`the predictive coding block should be included in the bit
`stream of coding unit. The prediction mode varies according
`to a type of the slice.
`When the type of the slice is intra (I), all predictive coding
`blocks in the slice are intra predicted, and a prediction type of
`a predictive coding block can be determined according to a
`size of the predictive coding block. However, when the slice
`type is unidirectional prediction (P) or bidirectional predic-
`tion (B), a prediction type ofa predictive coding block may be
`determined according to prediction mode information and a
`size of the predictive coding block. Thus, it is preferable to
`generate the prediction type ofthe predictive coding block on
`the basis of the slice type, the prediction mode information
`and the information indicating the size of the predictive cod-
`ing block, and insert the generated prediction type in a header
`of the coding unit.
`When the predictive coding block is intra-coded, intra pre-
`diction mode information used for intra prediction as well as
`the prediction type needs to be transmitted to a decoder.
`When the predictive coding block is inter-coded, the pre-
`dictive coding block is encoded by any one of unidirectional
`prediction and bidirectional prediction. In the case of unidi-
`rectional prediction, a bit stream should include information
`on a reference picture and motion vector information used for
`prediction as well as a prediction type ofthe predictive coding
`block for unidirectional prediction. In the case of bidirec-
`tional prediction, a header of the predictive coding block
`should include information on two reference pictures and
`motion vector information used for bidirectional prediction
`as well as a prediction type of the block for bidirectional
`prediction. The motion vector information may include infor-
`mation indicating a residual motion vector and a motion
`vector predictor.
`FIG. 1 shows a hierarchical division structure illustrating
`allowable predictive coding blocks for a SMB having a size of
`64x64 according to an exemplary embodiment of the present
`invention.
`
`When an SMB is used as a coding unit, it is preferable to
`have four sub-block division steps as shown in FIG. 1, but
`block division is not limited to the four sub-block division
`
`steps. When there are four sub-block division steps, a total of
`13 predictive block types (64x64, 64x32, 32x64, 32x32,
`32x16, 16x32, 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4)
`can be defined.
`
`Here, there may be no data to be transmitted for inter
`predictive coding blocks having a greater size than an MB.
`Thus, it is preferable to newly add an MB64_SKIP mode
`when a size of a predictive coding block is 64x64, and an
`MB32_SKIP mode when the size of the predictive coding
`block is 32x32. To transmit the mode information to the
`
`decoder, an MB64_SKIP_flag or an MB32_SKIP_flag may
`be used. When values of these flags are 1, there is no trans-
`mission data of the corresponding predictive coding block.
`Meanwhile, when plural
`successive SMBs are not
`encoded, an MB64_SKIP_flag may be inserted in a first SMB
`only, and may be omitted in the following SMBs. In this case,
`the number of SMBs that are successively skipped may be
`added to the slice or the first SMB. Specifically, when the
`plural of successive SMBs are not encoded, a SMB_
`SKIP_flag ofthe first SMB is set to 1, and may also be applied
`to several successive SMBs in common. In this case, infor-
`mation corresponding to the number of SMBs that are not
`successively encoded (e.g., an SMB_SKIP_number) may be
`added to the slice.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`When a coding unit has a size of32x32, block having a size
`of32x32, 32x1 6 or 16x32 in addition to existing block having
`a size of16><16,16x8,8><16,8><8,8><4,4><8 or4><4 canbeused
`as a predictive coding block.
`In an inter-prediction mode, a prediction type (Mb32_type)
`of a coding unit may indicate a 32x32 partition when the
`prediction type is 0, a 32x16 partition when the prediction
`type is 1, a 16x32 partition when the prediction type is 2, and
`a 16x16 partition when the prediction type is 3.
`When a coding unit is divided into four sub-coding units,
`the four sub-coding units are encoded and transmitted in a
`raster scan order. In this case, a quantization parameter may
`be transmitted for each coding unit, and may be transmitted
`only once in a header of a super-coding unit when the same
`quantization parameter is applied to all the sub-coding units.
`However, when a quantization parameter needs to be changed
`in a sub-coding unit, only a difference value with respect to a
`quantization parameter of the upper layer coding unit or the
`preceding coding unit of the same level may be transmitted.
`Each sub-coding unit may be divided using a quadtree
`method, and a coded block pattern (cbp) and a residual coef-
`ficient may also be transmitted using the quadtree method.
`When a 1-bit cbp is used, a cbp value of 1 may indicate that the
`coding unit has at least one coefficient other than 0, and a cbp
`value of 0 may indicate that all coefficients are 0.
`FIG. 2 illustrates a method of determining a coding mode
`when a size of an SMB is 64x64 according to the present
`invention.
`As illustrated in FIG. 2, when an SMB is a 64x64 block, an
`encoder determines whether or not it is possible to skip the
`64><64 block, and determines a SKIP mode as a coding mode
`when it is possible to skip the 64><64 block. At this time, an
`mb64_skip_flag should be transmitted to the decoder. When
`the 64><64 block has data to be encoded but is not divided into
`
`32x32 blocks, coding size ofthe SMB which is one of64><64,
`64x32 and 32x64 and information about whether the SMB is
`intra-coded or inter-coded is inserted in an SMB header, and
`an SMB data block is generated using encoded data.
`When the 64><64 block has data to be encoded and is
`divided into 32x32 blocks, it is likewise determined whether
`there is data to be encoded in the corresponding 32x32
`blocks. When there is no data to be encoded in the corre-
`
`sponding 32x32 blocks, the SKIP mode is determined as the
`32x32 block mode, and an mb32_skip_flag is transmitted to
`the decoder.
`However, when the 32x32 block has data to be encoded but
`is not divided into 16><16 blocks, coding size of the 32x32
`block which is one of32><32, 32x16 and 16x32 and informa-
`tion about whether the 32x32 blocks are intra-coded or inter-
`coded is inserted in headers of the 32x32 blocks, and a data
`block is generated using encoded data.
`When the 32x32 block has data to be encoded and is
`divided into 16><16 blocks, it is determined whether there is
`data to be encoded in the corresponding 16><16 blocks. When
`there is no data to be encoded in the corresponding 16><16
`blocks, the SKIP mode is determined as the 16><16 block
`mode, and an mb16_skip_flag is transmitted to the decoder.
`On the other hand, when there is data to be encoded in the
`corresponding 16><16 blocks, coding size ofthe 16><16 block
`which is one of 16x16, 16x8 and 8x16 and information about
`whether the 16><16 block is intra-coded or inter-coded is
`inserted in a header of the 16><16 block, and a data block is
`generated using encoded data.
`When blocks having such a variety of sizes are used, block
`size information may vary according to each sequence level,
`picture level, slice level, SMB or sub-coding unit ofthe SMB.
`
`Page 9 of 14
`
`Page 9 of 14
`
`

`

`US 9,191,673 B2
`
`5
`FIG. 3 illustrates an apparatus for encoding a moving pic-
`ture according to the present invention.
`Referring to FIG. 3, an apparatus for encoding a moving
`picture according to the present invention includes a coding
`mode determiner 110, a transform/quantization unit 120, an
`entropy coder 130, an intra predictor 140, a motion compen-
`sator 150, a motion estimator 155, an inverse quantization/
`transform unit 160, a deblocking filter 170, a picture storage
`180, an adder and a subtracter.
`The coding mode determiner 110 analyzes an input video
`signal to divide a picture into coding units, and determines a
`size of a predictive coding block and a prediction mode for
`each coding unit. Also, the coding mode determiner 110
`sends the predictive coding block corresponding to the deter-
`mined size to the subtracter.
`
`The transform/quantization unit 120 determines a size of a
`transform block to transform residual signal received from
`the subtracter. The size of a transform block may be equal to
`or less than that ofthe predictive coding block, but may be set
`differently in an inter mode. In other words, the size of the
`transform block may be greater than that of the predictive
`coding block in the inter mode. In this case, the transform/
`quantization unit 120 receives multiple residual blocks from
`the subtracter and generates one transform block consisting
`of the multiple residual blocks. A size of the transform block
`is equal to or less than that of the coding unit. The transform/
`quantization unit 120 performs a two-dimensional (2D) dis-
`crete cosine transform (DCT) on the transform block to gen-
`erate transform coefficients. The DCT may be an integer
`DCT.
`
`Also, the transform/quantization unit 120 determines a
`quantization step size used for quantizing the transform coef-
`ficients, and quantizes the transform coefficients using a
`quantization matrix determined according to the determined
`quantization step size and a coding mode.
`The inverse quantization/transform unit 160 inversely
`quantizes and inversely transforms the quantization coeffi-
`cients quantized by the transform coding/quantization unit
`120, thereby restoring a residual block in the spatial domain
`from a residual block transformed into the frequency domain.
`The deblocking filter 170 receives image data that is
`inversely quantized and inversely transformed from the
`inverse quantization/transform unit 160 and performs a filter-
`ing process for removing a blocking effect. The deblocking
`filtering process may be applied to a boundary between pre-
`dictive coding blocks and between transform blocks. The
`boundary is an edge of a grid having a predetermined size or
`more and the predetermined size may be 8x8. The deblocking
`filtering process includes a step of determining a boundary to
`be filtered, a step of determining a boundary filtering strength
`to be applied to the boundary, a step of determining whether
`or not to apply a deblocking filter, and a step of selecting a
`filter to be applied to the boundary when it is determined to
`apply the deblocking filter.
`Whether or not to apply the deblocking filter is determined
`according to i) whether or not the boundary filtering strength
`is greater than 0 and ii) whether or not a value indicating the
`difference between boundary pixels of P block and Q block is
`less than a first reference value determined according to a
`quantization parameter.
`Two or more filters may exist. When an absolute value of a
`difference between two pixels adjacent to the block boundary
`is equal to or greater than a second reference value, a weak
`filter is selected. The second reference value is determined by
`the quantization parameter and the boundary filtering
`strength
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`The picture storage 180 receives filtered image from the
`deblocking filter 170, stores the image in picture units. The
`picture may be an image of a frame or an image of a field. The
`picture storage 180 has a buffer (not shown) capable of stor-
`ing a plurality of pictures.
`The motion estimator 155 performs motion estimation
`using at least one reference picture stored in the picture stor-
`age 180, and outputs a reference picture index representing
`the reference picture and a motion vector.
`The motion compensator 150 extracts a prediction block
`corresponding to a block to be encoded from the reference
`picture used for motion estimation among a plurality of ref-
`erence pictures stored in the picture storage 180 according to
`the reference picture index and the motion vector input from
`the motion estimator 155, and outputs the extracted predic-
`tion block.
`
`The intra predictor 140 performs intra-prediction using
`reconstructed pixel values in the same picture. The intra pre-
`dictor 140 receives a current block to be predictively encoded,
`selects one of a predetermined number of intra prediction
`modes according to a size of the current block, and performs
`intra prediction.
`The entropy coder 130 entropy-codes the quantization
`coefficients quantized by the transform/quantization unit 120,
`motion information generated by the motion estimator 155,
`and so on. The quantized transform coefficients are converted
`into one-dimensional (1D) quantized transform information
`by a predetermined scanning method and entropy-coded.
`An intra prediction process will be described below with
`reference to FIG. 3.
`
`First, the intra predictor 140 receives position and size
`information on a coding block to be encoded from the coding
`mode determiner 110.
`
`Next, the intra predictor 140 receives valid reference pixels
`for determining an intra prediction mode of the current pre-
`dictive coding block from the picture storage 180. The refer-
`ence pixels have already been encoded and restored, and are
`adjacent to the current predictive coding block (hereafter,
`referred to as current block). When the current block is posi-
`tioned at an upper boundary of the current picture, pixels
`adjacent to the upper side of the current block are not defined.
`When the current block is positioned at a left boundary ofthe
`current picture, pixels adjacent to the left side of the current
`block are not defined. Also, when the current block is posi-
`tioned at the upper or left boundary of the slice, pixels adja-
`cent to the upper or left side are not defined.
`When there are no pixels adjacent to the left or upper side
`of the current block or there are no pixels that have been
`previously encoded and restored as mentioned above, an intra
`prediction mode ofthe current block can be determined using
`only valid pixels.
`However, the invalid reference pixels may be generated
`using pixels adjacent to the current block or available pixel.
`For example, when pixels of an upper block are not valid,
`pixels on the upper side may be generated using one or more
`available reference pixels on the left side.
`Meanwhile, even when there are pixels on the upper or left
`side of the current block, the pixels may be determined as
`invalid pixels according to a coding mode of a block to which
`the pixels belongs. For example, when a block to whichpixels
`adjacent to the upper side ofthe current block belong has been
`inter-coded and restored, the pixels may be determined as
`invalid pixels. In this case, reference pixels may be generated
`using reference pixel of a block having intra mode.
`
`Page 10 of14
`
`Page 10 of 14
`
`

`

`US 9,191,673 B2
`
`7
`Next, the intra predictor 140 determines the intra predic-
`tion mode ofthe current block using the reference pixels. The
`number of intra prediction modes depends on a size of the
`block.
`
`According to the size of the block, 33, 16 or 2 directional
`modes and at least one non-directional mode are allowed. The
`
`non-directional mode may be a direct current (DC) mode or a
`planar mode.
`Different numbers of intra prediction modes can be
`assigned to blocks having the same size. To indicate that
`different numbers of intra prediction modes are allowed,
`information indicating the number of intra prediction modes
`can be inserted in at least one of a sequence header, picture
`header, slice header and a coding unit header. It is preferable
`to insert the information in a sequence header or a picture
`header.
`
`Next, when the intra prediction mode of the current block
`is determined, a prediction block of the current block is gen-
`erated. The prediction block is generated using reference
`pixels including generated pixel or using a linear combination
`of the reference pixels on the basis of the intra prediction
`mode ofthe current block. For example, in a directional mode
`of a specific direction, a prediction mode can be generated
`using reference pixels on the upper side of the current block
`and those on the left side of the current block.
`
`The valid reference pixels used to generate the prediction
`block may be filtered reference pixels. A filter to be applied to
`the valid reference pixels may be plural in number. Also, the
`plurality of filters may be adaptively applied according to a
`size of the current block and the intra prediction mode.
`Next, a residual block obtained by using the current block
`and the prediction block generated by the intra predictor 140
`are encoded by the transform/quantization unit 120 and the
`entropy coder 130.
`Meanwhile, the intra prediction mode of the current block
`is separately encoded. The intra prediction mode may be
`encoded by the intra predictor 140, a separate intra prediction
`mode coder (not shown) or the entropy coder 130.
`The intra prediction mode of the current block is encoded
`using intra prediction modes of an upper block and a left
`block of the current block.
`
`First, the intra prediction modes of the left and upper
`blocks ofthe current block are derived. When the upper block
`is plural in number, a leftmost upper block or a block having
`the minimum mode number is set as an upper block of the
`current block. Also, when the left block is plural in number, an
`uppermost left block or a block having the minimum mode
`number is set as a left block of the current block. When the
`
`upper block or the left block is not encoded in intra prediction
`mode, the DC mode (mode number 2) may be set as an intra
`prediction mode of the upper block or the left block.
`Next, when the intra prediction mode number of the upper
`or left block is equal to or greater than the number of allow-
`able intra prediction modes of the current block, the intra
`prediction mode of the upper or left block is converted into
`one of the intra prediction modes allowed for the current
`block.
`
`Next, when the intra prediction mode of the current block
`is equal to one of the intra prediction mode of the left block
`and the intra prediction mode of the upper block, a flag
`indicating that the intra prediction mode of the current block
`is equal to one of the intra prediction mode of the left block
`and the intra prediction mode of the upper block, and a flag
`indicating one of the intra prediction modes of the upper and
`left blocks are transmitted. In this case, when the left and
`upper blocks of the current block have the same intra predic-
`tion mode, only the flag indicating that the intra prediction
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`mode of the current block is equal to one of the intra predic-
`tion mode of the left block and the intra prediction mode of
`the upper block may be transmitted. Likewise, when only one
`of the intra prediction modes of the upper and left blocks is
`valid and equal to that of the current block, only the flag
`indicating the intra prediction mode of the current block is
`equal to one of the intra prediction mode of the left block and
`the intra prediction mode of the upper block may be trans-
`mitted.
`
`However, when the intra prediction mode of the current
`block is different from those of the left and upper blocks, it is
`determined whether or not the intra prediction mode number
`ofthe current block is smaller than those ofthe left and upper
`blocks.
`
`When it is determined that both of the intra prediction
`mode numbers of the left and upper blocks of the current
`block are greater than the intra prediction mode number ofthe
`current block, the intra prediction mode ofthe current block is
`determined as a final intra prediction mode. However, when
`only one of the intra prediction mode numbers of the left and
`upper blocks of the current block is not greater than the intra
`prediction mode number of the current block, an intra predic-
`tion mode having a mode number obtained by subtracting 1
`from the intra prediction mode number ofthe current block is
`determined as the final intra prediction mode of the current
`block. Also, when neither of the intra prediction mode num-
`bers ofthe left and upper blocks ofthe current block is greater
`than the intra prediction mode number ofthe current block, an
`intra prediction mode having a mode number obtained by
`subtracting 2 from the intra prediction mode number of the
`current block is determined as the final intra prediction mode
`of the current block.
`
`Next, the final intra prediction mode of the current block is
`encoded. Using different coding tables according to whether
`or not the upper block of the current block has the same intra
`prediction mode as the left block ofthe current block, the final
`intra prediction mode of the current block