US007903735B2
`
`(12) United States Patent
`Cha et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,903,735 B2
`Mar. 8, 2011
`
`(54) METHOD OF EFFECTIVELY PREDICTING
`MULTI-LAYER BASED VIDEO FRAME, AND
`VIDEO CODING METHOD AND APPARATUS
`USING THE SAME
`
`(75) Inventors: Sang-chang Cha, Hwaseong-si (KR);
`Woo-jin Han, Suwon-si (KR)
`
`(73) Assignee: Samsung Electronics Co., Ltd.,
`Suwon-si (KR)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1477 days.
`
`(21) Appl. No.: 11/311,384
`
`(22) Filed:
`
`Dec. 20, 2005
`
`(65)
`
`Prior Publication Data
`US 2006/O1651.71 A1
`Jul. 27, 2006
`
`Related U.S. Application Data
`(60) Provisional application No. 60/646,534, filed on Jan.
`25, 2005.
`
`(30)
`
`Foreign Application Priority Data
`
`Feb. 26, 2005 (KR). 10-2005-OO16270
`
`(51) Int. Cl.
`(2006.01)
`H04N 7/2
`(52) U.S. Cl. .......... 375/240.12; 375/240.15: 375/240.16;
`375/240.24; 375/240.08
`(58) Field of Classification Search ........................ None
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`2003, OO12279 A1*
`1/2003 Chaddha .................. 375,240.12
`11/2003 Zhou
`2003/0206594 A1
`
`
`
`2004/0042549 A1* 3/2004 Huang et al. ............. 375,240.11
`2005. O157784 A1* 7, 2005 Tanizawa et al. ........ 375,240.03
`2005, 0163211 A1* 7/2005 Shanableh ................. 375,240.1
`2005/01693.74 A1* 8/2005 Marpe et al. ...
`375,240.16
`2005/0195896 A1* 9/2005 Huang et al. ............. 375,240.03
`FOREIGN PATENT DOCUMENTS
`O7-222 165
`8, 1995
`08-237.599
`9, 1996
`11-275585
`10, 1999
`1 429 564 A1
`6, 2004
`2001-0080644 A
`8, 2001
`10-2004-0054747. A
`6, 2004
`
`JP
`JP
`JP
`JP
`KR
`KR
`
`OTHER PUBLICATIONS
`Ascenso, Joao; Pereira, Fernando; Drift Reduction for a H.264/AVC
`Fine Grain Scalability with Motion Compensation Architecture;
`2004; International Conference on Image Processing: pp. 2259
`2262.* Yuwen He, Xuejun Zhao, Yuzhuo Zhong, Shiqiang Yang;
`Improved Fine Granular Scalable Coding with Inter-Layer Predic
`tion; 2002; Data Compression Conference; pp. 1-10.*
`* cited by examiner
`Primary Examiner — Mark KZimmerman
`Assistant Examiner — Miya J Cato
`(74) Attorney, Agent, or Firm — Sughrue Mion, PLLC
`(57)
`ABSTRACT
`A method and apparatus are provided for effectively predict
`ing a video frame that use all of the advantages of an intra
`mode and an intra BL mode in multi-layer structure based
`Video coding. The method includes reconstructing the intra
`block of a lower layer using the previously reconstructed first
`neighboring blocks of the intra block; subtracting the first
`neighboring blocks from previously stored the second neigh
`boring blocks of an upper layer corresponding to the first
`neighboring blocks; creating a differential predicted block
`based on a predetermined intra prediction mode by perform
`ing intra prediction using virtual differential neighboring
`blocks that are created as a result of the Subtraction; adding
`the differential predicted block and the reconstructed intra
`block; and Subtracting a predicted block, which is created as
`a result of the addition, from a block of the upper layer
`corresponding to the intra block.
`
`25 Claims, 11 Drawing Sheets
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 1
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 1 of 11
`
`US 7,903,735 B2
`
`F.G. 1
`
`8 directions
`
`(8)
`
`(1)
`
`(6)
`
`+ DC (2)
`
`(3)
`
`(7)
`
`(O)
`
`(5)
`
`(4)
`
`
`
`FIG 2
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 2
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 3
`
`

`

`U.S. Patent
`
`US 7,903,735 B2
`
`
`
`
`
`CN
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 4
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 5
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 5 of 11
`
`US 7,903,735 B2
`
`9
`
`* ?I),
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 6
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 6 of 11
`
`US 7,903,735 B2
`
`FG 7
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`inter
`prediction
`unit 62
`
`intra
`prediction
`unit (63)
`
`intra B
`prediction
`unit (64)
`
`
`
`
`
`
`
`
`
`
`
`WeSe
`quantization
`Unit (71)
`
`VerSe
`transform
`unit (72)
`
`
`
`
`
`
`
`
`
`
`
`VerSe inter
`prediction
`unit (74)
`
`in VerSe
`prediction
`Unit (75)
`
`
`
`Verse intra
`B prediction
`unit 76)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 7
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 7 of 11
`
`US 7,903,735 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(£11
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 8
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 8 of 11
`
`US 7,903,735 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`0T º ?I ?I
`
`(DGI)
`
`Fl-ºf
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 9
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 9 of 11
`
`US 7,903,735 B2
`
`FG 11
`
`y
`f
`- ->,
`l
`(5)
`
`(O)
`
`y
`(7)
`
`Upper
`layer
`
`OWer
`layer
`
`O)
`
`F.G. 12
`
`
`
`(8)
`
`(1)
`
`(6)
`
`3)
`
`(7)
`
`(O)
`
`(5)
`
`(4)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 10
`
`

`

`U.S. Patent
`
`Mar. 8, 2011
`
`Sheet 10 of 11
`
`US 7,903,735 B2
`
`F.G. 13
`82
`
`81
`
`Upper
`layer
`
`83
`
`
`
`OWer
`layer
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 11
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 12
`
`

`

`US 7,903,735 B2
`
`1.
`METHOD OF EFFECTIVELY PREDICTING
`MULTI-LAYER BASED VIDEO FRAME, AND
`VIDEO CODNG METHOD AND APPARATUS
`USING THE SAME
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application claims priority from Korean Patent Appli
`cation No. 10-2005-0016270 filed on Feb. 26, 2005 in the
`Korean Intellectual Property Office, and U.S. Provisional
`Patent Application No. 60/646,534 filed on Jan. 25, 2005 in
`the United States Patent and Trademark Office, the disclo
`sures of which are incorporated herein by reference in their
`entirety.
`
`10
`
`15
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`Apparatuses and methods consistent with the present
`invention relate generally to video coding, and more particu
`larly, to effectively predicting a video frame that use all of the
`advantages of an intra mode and an intra BL mode in multi
`layer structure based-video coding.
`2. Description of the Related Art
`As information and communication technology, including
`the Internet, develops, image-based communication as well
`as text-based communication and Voice-based communica
`tion is increasing. The existing text-based communication is
`insufficient to satisfy various consumers demands. Therefore,
`the provision of multimedia service capable of accommodat
`ing various types of information, such as text, images and
`music, is increasing. Since the amount of multimedia data is
`large, multimedia data require high-capacity storage media
`and require broad bandwidth at the time of transmission.
`Therefore, to transmit multimedia data, including text,
`images and audio, it is essential to use a compression coding
`technique.
`The fundamental principle of data compression is to elimi
`nate redundancy in data. Data can be compressed by elimi
`nating spatial redundancy Such as the case where an identical
`color or object is repeated in an image, temporal redundancy
`Such as the case where there is little change between neigh
`boring frames or identical audio Sound is repeated, or psy
`chovisual redundancy in which the fact that humans visual
`and perceptual abilities are insensitive to high frequencies is
`taken into account.
`For such a moving image compression method, H.264/
`Advanced Video Coding (AVC), which has higher compres
`sion efficiency than Moving Picture Experts Group (MPEG)-
`4, has attracted attention recently. H.264 uses directional
`intra-prediction, which eliminates spatial similarity in each
`frame, as one of the schemes for improving compression
`efficiency.
`Directional intra-prediction is a method of predicting the
`values of a current Sub-block and encoding only a difference
`in Such a way as to perform copying in predetermined direc
`tions using neighboring pixels on the upper and left sides of a
`sub-pixel with respect to the sub-block.
`In H.264, a predicted block with respect to a current block
`is generated based on other blocks having preceding sequen
`tial positions. The difference between the current block and
`the predicted block is encoded. For a luminance component,
`each predicted block is generated on a 4x4 block or 6x16
`macroblock basis. There area total of nine optional prediction
`modes for each 4x4 block, whereas there are a total of four
`optional prediction modes for each 16x16 block. An H.264
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`based video encoder selects the one prediction mode that
`minimizes the difference between the current block and the
`predicted block, from among the prediction modes, for each
`block.
`For prediction modes for the 4x4 block, H.264, as shown in
`FIG. 1, employs a total of nine prediction modes, including a
`total of nine directional modes (modes 0, 1, 3 to 8), and a DC
`mode (mode 2) that uses an average of the values of nine
`neighboring pixels.
`FIG. 2 illustrates an example of labeling to illustrate the
`nine prediction modes. In this example, a predicted block
`(including regions “a” to “p') with respect to a current block
`is generated using previously decoded samples A to M. If
`regions E, F, G and H cannot be previously decoded, regions
`E, F, G and H can be virtually created by copying region D to
`the locations of the regions E, F, G and H.
`With reference to FIG. 3, the nine prediction modes are
`respectively described in detail below. In the case of mode 0.
`the pixels of a predicted block are extrapolated using upper
`samples A, B, C and D in a vertical direction, and in the case
`of mode 1, the pixels are extrapolated using left samples I, J.
`Kand L in a horizontal direction. Furthermore, in the case of
`mode 2, the pixels of the predicted block are uniformly
`replaced by the averages of upper samples A, B, C and Dand
`left samples I, J, K and L.
`In the case of mode 3, the pixels of the predicted block are
`interpolated between a lower-left position and an upper-right
`position in a direction that is inclined at an angle of 45°, and
`in the case of mode 4, the pixels are extrapolated in a direction
`that is inclined toward an upper-left position at an angle of
`45°. Furthermore, in the case of mode 5, the pixels of the
`predicted block are extrapolated in a direction that is inclined
`rightward from a vertical direction at an angle of about 26.6°
`(width/height=/2).
`In the case of mode 6, the pixels of the predicted block are
`extrapolated in a direction that is inclined downward from a
`horizontal direction at an angle of about 26.6°, and in the case
`of mode 7, the pixels are extrapolated in a direction that is
`inclined leftward from a vertical directionatan angle of about
`26.6°. Finally, in the case of mode 8, the pixels of the pre
`dicted block are interpolated in a direction that is inclined
`upward from a horizontal direction at an angle of about 26.6°.
`The arrows of FIG. 3 indicate prediction directions in
`respective modes. In modes 3 to 8, the samples of the pre
`dicted block can be generated from the weighted averages of
`previously decoded reference samples A to M. For example,
`in the case of mode 4, Sampled, which is located in the upper
`left, can be predicted as expressed by the following Equation
`1. In this Equation, the round.(...) function is a function that
`rounds off an input value to an integer position.
`
`Meanwhile, a 16x16 prediction model for luminance com
`ponents includes four modes, that is, mode 0, mode 1, mode
`2 and mode 3. In the case of mode 0, the pixels of a predicted
`block are extrapolated from upper samples H, and in the case
`of mode 1, the pixels of a predicted block are extrapolated
`from left samples V. Furthermore, in the case of mode 2, the
`pixels of a predicted block are calculated using the averages
`ofupper samples H and left samples V. Finally, in the case of
`mode 3, a “plane' function suitable for upper samples H and
`left samples V is used. This mode is more suitable for a region
`in which luminance Smoothly changes.
`Meanwhile, in addition to efforts to improve the efficiency
`of video coding, research into video coding that allows the
`resolution of transmitted video data, a frame rate, and a Sig
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 13
`
`

`

`3
`nal-to-Noise Ratio (SNR) to be adjusted, that is, that supports
`Scalability, is actively being carried out.
`With regard to this scalable video coding technique, stan
`dardization work is in progress in the Moving Picture Experts
`Group (MPEG)-21 PART-13. Of these methods for support
`ing scalability, a multi-layered video coding method is con
`sidered a prominent method. For example, multiple layers,
`including a base layer, a first enhanced layer and a second
`enhanced layer 2, are provided and respective layers have
`different resolutions QCIF, CIF and 2CIF or different frame
`rates.
`In the scalable video coding standard currently in progress,
`besides interprediction and directional intra prediction (here
`inafter simply referred to as intra prediction) used in existing
`H.264 to predict a current block or macroblock, a method of
`predicting a layer to which a current block belongs using the
`correlation between the current block and a corresponding
`lower layer block is additionally introduced. This prediction
`method is referred to as “intra BL (intra BL) prediction' in
`the standard, and the case of performing encoding using Such
`prediction is referred to as “intra BL mode.”
`FIG. 4 is a schematic diagram showing the three prediction
`methods, which illustrates the case of performing intra pre
`diction on a macroblock of a current frame 1 (CD), the case of
`performing interprediction using a frame 2 placed at a tem
`25
`poral location different from that of the current frame 1 (2)),
`and the case of performing intra BL prediction using texture
`data about the region 6 of the frame of a base layer corre
`sponding to a macroblock ((3)).
`As described above, in the scalable video coding standard,
`the advantageous one of the three prediction methods is
`selected for each macroblock, and a corresponding macrob
`lock is encoded using the selected method. That is, for one
`macroblock, inter prediction, intra prediction and intra BL
`prediction are selectively used. However, a differential block
`35
`created using intra BL prediction still has considerable cor
`relation with neighboring differences. Accordingly, it is nec
`essary to develop a prediction technique that takes the advan
`tages of both intra BL prediction and intra prediction into
`account. Although a prediction method that takes the advan
`tages of intra prediction, intra BL prediction and interpredic
`tion into account may be considered, the characteristics of
`intra BL prediction and intra prediction are considerably dif
`ferent from those of inter prediction, so this method is not
`desirable.
`
`10
`
`15
`
`30
`
`40
`
`45
`
`US 7,903,735 B2
`
`4
`According to an aspect of the present invention, there is
`provided a method of efficiently predicting a multi-layer
`based video frame, including reconstructing the intra block of
`a lower layer using previously reconstructed the first neigh
`boring blocks of the intra block; subtracting the first neigh
`boring blocks from the previously stored second neighboring
`blocks of an upper layer corresponding to the first neighbor
`ing blocks; creating a differential predicted block based on a
`predetermined intra prediction mode by performing intra pre
`diction using virtual differential neighboring blocks that are
`created as a result of the subtraction; adding the differential
`predicted block and the reconstructed intra block; subtracting
`a predicted block, which is created as a result of the addition,
`from a block of the upper layer corresponding to the intra
`block; and encoding a residual block that is created as a result
`of the subtraction.
`According to an aspect of the present invention, there is
`provided a method of efficiently predicting a multi-layer
`based video frame, including reconstructing the intra block of
`a lower layer using the previously reconstructed first neigh
`boring blocks of the intra block; subtracting the first neigh
`boring blocks from the previously stored second neighboring
`blocks of an upper layer corresponding to the first neighbor
`ing blocks; creating a differential predicted block based on a
`predetermined intra prediction mode by performing intra pre
`diction using virtual differential neighboring blocks that are
`created as a result of the subtraction; adding the differential
`predicted block and the reconstructed intra block; recon
`structing a block of the upper layer corresponding to the intra
`block; and adding the reconstructed block of the upper layer
`and a predicted block that is acquired as a result of the addi
`tion.
`According to an aspect of the present invention, there is
`provided a multi-layer based video encoder, including a
`means for reconstructing the intra block of a lower layer using
`the previously reconstructed first neighboring blocks of the
`intra block; a means for Subtracting the first neighboring
`blocks from the previously stored second neighboring blocks
`of an upper layer corresponding to the first neighboring
`blocks; a means for creating a differential predicted block
`based on a predetermined intra prediction mode by perform
`ing intra prediction using virtual differential neighboring
`blocks that are created as a result of the Subtraction; a means
`for adding the differential predicted block and the recon
`structed intra block; a means for Subtracting a predicted
`block, which is created as a result of the addition, from a block
`of the upper layer corresponding to the intra block; and a
`means for encoding a residual block that is created as a result
`of the subtraction.
`According to an aspect of the present invention, there is
`provided a multi-layer based video encoder, including a
`means for reconstructing the intra block of a lower layer using
`the previously reconstructed first neighboring blocks of the
`intra block; a means for Subtracting the first neighboring
`blocks from the previously stored second neighboring blocks
`of an upper layer corresponding to the first neighboring
`blocks; a means for creating a differential predicted block
`based on a predetermined intra prediction mode by perform
`ing intra prediction using virtual differential neighboring
`blocks that are created as a result of the Subtraction; a means
`for adding the differential predicted block and the recon
`structed intra block; a means for reconstructing a block of the
`upper layer corresponding to the intra block; and a means for
`adding the reconstructed block of the upper layer and a pre
`dicted block that is acquired as a result of the addition.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The above and other aspects of the present invention will be
`more clearly understood from the following detailed descrip
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a prediction method that
`uses both intra BL prediction and intra prediction.
`Furthermore, the present invention may improve video
`coding efficiency using the prediction method.
`According to an aspect of the present invention, there is
`provided a method of efficiently predicting a multi-layer
`based video frame, including reconstructing the intra block of
`a lower layer using the previously reconstructed first neigh
`boring blocks of the intra block; subtracting the first neigh
`boring blocks from previously stored the second neighboring
`blocks of an upper layer corresponding to the first neighbor
`ing blocks; creating a differential predicted block based on a
`predetermined intra prediction mode by performing intra pre
`diction using virtual differential neighboring blocks that are
`created as a result of the subtraction; adding the differential
`predicted block and the reconstructed intra block; and sub
`tracting a predicted block, which is created as a result of the
`addition, from a block of the upper layer corresponding to the
`intra block.
`
`50
`
`55
`
`60
`
`65
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 14
`
`

`

`US 7,903,735 B2
`
`10
`
`15
`
`5
`tion of the exemplary embodiments taken in conjunction with
`the accompanying drawings, in which:
`FIG. 1 is a diagram illustrating prediction directions for
`respective modes defined by H.264;
`FIG. 2 is a diagram illustrating an example of labeling that
`is used to illustrate the intra prediction modes of FIG. 1;
`FIG. 3 is a diagram illustrating the intra prediction modes
`of FIG. 1 in detail;
`FIG. 4 is a schematic diagram illustrating conventional
`three prediction methods;
`FIG. 5 is a diagram illustrating the concept of a prediction
`method according to an exemplary embodiment of the present
`invention;
`FIG. 6 is a block diagram illustrating the construction of a
`Video encoder according to an exemplary embodiment of the
`present invention;
`FIG. 7 is a block diagram illustrating the schematic con
`struction of an encoding unit that is included in the video
`encoder of FIG. 6;
`FIG. 8 is a block diagram illustrating the schematic con
`struction of a decoding unit that is included in the video
`encoder of FIG. 6;
`FIG. 9 is a block diagram illustrating the construction of a
`Video encoder according to another exemplary embodiment
`of the present invention;
`25
`FIG. 10 is a block diagram illustrating the construction of
`a video encoder according to still another exemplary embodi
`ment of the present invention;
`FIG. 11 is a view illustrating neighboring direction with
`respect to a vertical mode;
`FIG. 12 is a view illustrating neighboring directions with
`respect to 8 intra prediction modes having directionality:
`FIG. 13 is a diagram illustrating the corresponding regions
`between layers when the resolutions of the layers do not
`coincide with each other; and
`35
`FIG. 14 is a block diagram illustrating the construction of
`a video decoder according to an exemplary embodiment of
`the present invention.
`
`6
`which is produced between a video encoder and a video
`decoder, using such closed loop encoding, so that recon
`structed images, which are decoded after being encoded, are
`used as the images of existing blocks that are used to predict
`a current block in the present invention. However, it is appar
`ent to those skilled in the art that the present invention can also
`be applied to an open loop method using the original images
`of existing blocks.
`In a lower layer, a block corresponding to the current block
`10 (hereinafter referred to as a “corresponding block”) refers
`to an intra block 20 that has been encoded and decoded using
`intra prediction as shown in FIG. 3. Furthermore, the neigh
`boring blocks 25 of the intra block 20 can be encoded/de
`coded using any prediction method in the same manner as the
`neighboring blocks 15 of the upper layer. The neighboring
`blocks 25 of the lower layer exist at locations corresponding
`to those of the neighboring blocks 15.
`Meanwhile, the resolution of the upper layer may be iden
`tical to or different from the resolution of the lower layer. If
`the resolutions of both layers are identical to each other, the
`size of the blocks 10 and 15 of the upper layer is identical to
`the size of the blocks 20 and 25 of the lower layer. Otherwise,
`the blocks 10 and 15 of the upper layer may be larger than
`those of the lower layer. For example, if the resolution of the
`upper layer is two times the resolution of the lower layer and
`the blocks 20 and 25 of the lower layer have a 4x4 size, the
`blocks of the upper layer may have an 8x8 size.
`Furthermore, although the present invention is described
`with the term “neighboring block” referring to four blocks,
`that is, left, upper left, upper and upper right blocks, which are
`referred to for intra prediction, the number and locations of
`neighboring blocks may vary if the type of blocks, which are
`referred to for intra prediction, varies.
`As described above, when the neighboring blocks 15 of the
`upper layer and the intra block 20 and neighboring blocks 25
`of the lower layer have been all reconstructed, the predicted
`block 40 of the current block 10 can be created using them. A
`process of creating the predicted block 40 is described in
`detail below.
`First, virtual differential neighboring blocks 35 are created
`by subtracting the reconstructed neighboring blocks 25 of the
`lower layer from the neighboring blocks 15 of the upper layer.
`The subtraction is performed between the corresponding
`blocks of both layers. When directional intra prediction illus
`trated in FIG. 3 is applied to the neighboring blocks 35 cre
`ated as described above, differential predicted blocks 30 can
`be created for nine modes.
`Thereafter, each of the nine created differential predicted
`blocks 30 is added to the intra block 20, and then an optimal
`candidate is selected from nine candidates obtained for the
`predicted block 40. For example, the selection may be per
`formed in Such a way as to select one that minimizes the
`image different between each of the nine candidates and the
`current block 10. Alternatively, for another example, a
`method of predicting the current block 10 using the nine
`candidates for the predicted block 40, encoding prediction
`results and selecting an optimal candidate using the Rate
`Distortion (R-D) cost function may be used. The former
`method has the advantages of a small amount of calculation
`and simple performance, while the latter method has the
`advantage of more accurate selection between candidates.
`When the predicted block 40 is selected from the candi
`dates for the predicted block 40, the current block 10 is
`predicted using the predicted block 40. That is, a residual
`block 50 is created by subtracting the predicted block 40 from
`
`30
`
`DESCRIPTION OF THE EXEMPLARY
`EMBODIMENTS
`
`Reference now should be made to the drawings, in which
`the same reference numerals are used throughout the different
`drawings to designate the same or similar components.
`FIG. 5 is a diagram illustrating the concept of a prediction
`method according to an exemplary embodiment of the present
`invention. In the method, how to create a predicted block 40
`for the original image of a current block 10 (hereinafter sim
`ply referred to as a current block) that belongs to an upper
`layer is an essential point. In the present invention, the term
`“block” may be defined as a block having a size identical to
`that of an intra block of the H.264 standard. Meanwhile, for a
`conventional intra block, 4x4 mode and 16x16 mode exist for
`luminance components and 8x8 mode exists for chrominance
`components and, accordingly, the “block” of the present
`invention may have one of the various sizes.
`In an upper layer, before the prediction of a current block
`10, the neighboring blocks of the current block 10 have
`already been encoded/decoded using a certain prediction
`method. The neighboring blocks 15 may be encoded/decoded
`using any prediction method, Such as inter-prediction, intra
`prediction, or intra BL prediction.
`In order to predict the current block 10 as described above,
`images passed through encoding and decoding (closed loop),
`rather than the original images of the neighboring blocks 15.
`are used. Currently, most codecs eliminate drifting error,
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1026, p. 15
`
`

`

`US 7,903,735 B2
`
`7
`the created current block 10. Thereafter, the fundamental
`operation of the present invention is completed by encoding
`the residual block 50.
`The schematic operation of the present invention is
`described above. The conventional method using an intra BL
`mode is schematically compared to the method of the present
`invention from a conceptual point of view. When the image of
`the current block10 is C1 and the reconstructed intra block 20
`of the lower layer is CO, CO is used as the predicted block of
`C1 in the conventional intra BL mode on the assumption that
`CO is similar to C1, as shown in Equation 2.
`(2)
`C1-COsO
`In contrast, the present invention does not use C0 as the
`predicted block, but instead uses the correlation of the fol
`lowing Equation 3. In Equation 3, INT(...) refers to a function
`that performs directional intra prediction, N1 refers to the
`neighboring blocks of the upper layer and No refers to the
`neighboring blocks of the lower layer.
`
`10
`
`15
`
`8
`forming the inverse of the process of the encoding unit 105,
`that is, inverse prediction, on inversely transformed signals.
`The reconstructed neighboring blocks are input to a Subtrac
`tor 135.
`An intra encoding unit 115 encodes the corresponding
`block image, and the intra decoding unit 120 decodes
`encoded results. It should be noted that in the present inven
`tion, the corresponding block must be encoded using intra
`prediction. However, the neighboring blocks of the current
`block or the neighboring blocks of the corresponding block
`may be encoded using any prediction method.
`Accordingly, the intra encoding unit 115 may beformed of
`the intra prediction unit 63, the transform unit 65 and the
`quantization unit 66, as shown in the block diagram of FIG. 7,
`and the intra decoding unit 120 may be formed of the inverse
`quantization unit 71, the inverse transform unit 72 and the
`inverse intra prediction unit 75, as shown in the block diagram
`of FIG. 8.
`A signal output from the intra decoding unit 120, that is, a
`reconstructed corresponding block (intra block), is input to an
`adder 160 selectively through an up-sampler 150. The up
`sampler 150 is not used when the resolutions of the upper
`layer and the lower layer coincide with each other, and per
`forms up-sampling so that the resolution of the reconstructed
`intra block coincides with that of the upper layer when the
`resolutions do not coincide with each other.
`Meanwhile, the encoding unit 125 encodes the images of
`the neighboring blocks of the corresponding block, and the
`decoding unit 130 decodes the encoded results. The encoding
`unit 125 may have a construction identical to that of FIG. 7,
`and the decoding unit 130 may have a construction identical
`to that of FIG. 8.
`A signal output from the decoding unit 130, that is, recon
`structed neighboring blocks, is input to the subtractor 160
`selectively through the up-sampler 140. The up-sampler 140
`is not used when the resolutions of the upper layer and the
`lower layer coincide with each other, and performs up-Sam
`pling so that the resolution of the reconstructed neighboring
`blocks coincides with that of the upper layer when the reso
`lutions do not coincide with each other.
`The subtractor 135 acquires differential neighboring
`blocks by Subtracting a signal, which is input from the decod
`ing unit 130 or up-sampler 140, from a signal, which is input
`from the decoding unit 110.
`A predicted block creation unit 145 creates a predeter
`mined number of differential predicted blocks for respective
`modes by performing directional intra prediction using the
`differential neighboring blocks. For example, as shown in
`FIG.3, in H.264, a total of nine modes, including eight modes
`having directionality and one DC mode, can be used. Nine
`differential predicted blocks are created for the respective
`modes, and the created differential predicted blocks are pro
`vided to the adder 160.
`The adder 160 adds the differential predicted blocks for the
`respective modes to the signal input from the intra decoding
`unit 120 or from the up-sampler 150. As a result, a number of
`the predicted block candidates equal to the number of modes
`are created.
`The mode selection unit 190 selects an optimal mode (intra
`prediction mode) from the modes that the predicted block
`candidates have, and selects an optimal predicted block from
`the predicted block candidates based on the selected mode.
`The mode selection is performed in Such away as to select the
`mode that minimizes the difference between the predicted
`block candidates and the current block, as described above. In
`
`25
`
`30
`
`35
`
`40
`
`As shown in Equation3, the difference between the current
`block Co and the corresponding lower laye