USOO8135064B2
`
`(12) United States Patent
`Tasaka et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,135,064 B2
`Mar. 13, 2012
`
`(54) INTRA PREDICTION APPARATUS
`
`(56)
`
`References Cited
`
`JP
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. 375,240.26
`
`U.S. PATENT DOCUMENTS
`6,654.420 B1 * 1 1/2003 Snook ...................... 375,240.16
`6,859,559 B2
`2/2005 Boon et al.
`2005: R ck 1858 8. St. et al .
`.
`.
`.
`.
`un et al.
`7, 2004 Dahlhoff et al.
`2004/O136458 A1
`FOREIGN PATENT DOCUMENTS
`2004-304724
`10, 2004
`OTHER PUBLICATIONS
`Advanced video coding for generic audiovisual Services H.264
`S
`H: Aud
`1 and Multimedia Syst
`Infrastruct
`f
`S
`: UCOWSU a
`Ca SS aS CU. O
`audiovisual services—Coding of moving video.
`Japanese Office Action issued in Japanese Patent Application No.JP
`2005-32 1566 dated Jul. 20, 2010.
`* cited by examiner
`Primary Examiner — King Poon
`Assistant Examiner — Neil R McLean
`(74) Attorney, Agent, or Firm — McDermott Will & Emery
`LLP
`ABSTRACT
`(57)
`An intra prediction apparatus includes: a candidate narrow
`ing-down unit which narrows down, from a plurality of intra
`prediction directions, an intra prediction direction candidate
`which is applied to a block of multiple pixels that constitutes
`a video in accordance with a characteristic of the video; and
`an intra prediction execution unit which executes intra pre
`diction in the intra prediction direction narrowed down by
`said candidate narrowing-down unit. Through this, computa
`tional load can be lightened while contributing to improve
`ment in picture quality and encoding efficiency.
`
`(75) Inventors: Kei Tasaka, Ikoma (JP); Hiroshi
`Arakawa, R.g Takashi
`Masuno, Hirakata (JP); Koji Arimura,
`Hirakata (JP); Tatsuro Juri, Osaka (JP)
`(73) Assignee: Panasonic Corporation, Osaka (JP)
`c
`-
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1643 days.
`(21) Appl. No.: 11/291,923
`(22) Filed:
`Dec. 2, 2005
`9
`Prior Publication Data
`US 2006/0120456A1
`Jun. 8, 2006
`Foreign Application Priority Data
`
`(30)
`
`(65)
`
`Dec. 3, 2004 (JP) ................................. 2004-350928
`Nov. 4, 2005 (JP) ................................. 2005-321566
`(51) Int. Cl.
`H04B I/66
`HO)4N 7/12
`H04N 1 1/02
`HO4N II/04
`HO4N 9/12
`G06K 9/46
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`
`(52) U.S. Cl. ......... 375/240.16; 375/240.08: 375/240.24;
`375/240.26; 382/236; 348/392
`(58) Field of Classification Search ............. 375/24O16
`375/240.26, 240.24, 240.08; 382/236; 348/392
`See application file for complete search history.
`
`
`
`Wideo
`Materia
`Input
`
`Int
`na
`ESSEE
`atuS
`pp
`
`24 Claims, 15 Drawing Sheets
`
`Coding Apparatus-l
`
`Auxiliary Information
`
`Conversion
`Factor
`O
`
`15
`
`Inverse
`Quantizing
`Unit
`
`Inverse
`Conversion
`Unit
`
`Bit
`Stream
`Output
`
`Inter Prediction Unit
`
`17O
`
`Motion
`Compensation
`Unit
`
`Motion
`Prediction
`Unit
`
`Motion Wector
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 1
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 1 of 15
`
`US 8,135,064 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sl?Xld 914
`
`
`
`- - - - -
`
`- - - - - - - -
`
`•=. T !
`! ! - - - - - -
`• • •= =
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 2
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 2 of 15
`
`US 8,135,064 B2
`
`(C)
`
`(b)
`
`22.223
`22:32: {O
`2SSS:
`(2GGGO
`2X /N ><>
`
`CK2
`CDCDCD
`CK2 SR
`
`Prediction MOde O
`
`Prediction MOce 1
`
`Prediction MOce 2
`
`
`
`
`
`(f)
`
`Prediction Mode 5
`
`Prediction MOde 6
`
`
`
`Pixels to be
`encoded
`
`
`
`
`
`22.2%
`%2OOO)
`3>>>
`2-222
`29292
`55
`
`(2d
`
`Prediction MOde 7
`
`Prediction MOde 8
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 3
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 3 of 15
`
`US 8,135,064 B2
`
`(a)
`
`16 Pixels
`
`FIG. 3
`
`(b)
`
`16 Pixels
`
`Ç×××××
`××××××??.
`××××××
`
`Prediction Mode O
`(Vertical Prediction)
`
`Prediction Mode 1
`(Horizontal Prediction)
`
`(C)
`
`16 Pixels
`
`(d)
`
`16 Pixels
`
`
`
`
`
`
`
`Prediction Mode 2
`(DC Prediction)
`
`Prediction Mode 3
`(Plane Prediction)
`
`(2)
`
`Pixels already encoded
`in the above block Or
`the left block
`
`O Pixels to be encoded
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 4
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 4 of 15
`
`US 8,135,064 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 5
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 6
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 6 of 15
`
`US 8,135,064 B2
`
`FIG. 6
`
`Coding Control Unit
`
`18
`
`Video Characteristic
`Volume Input Unit
`
`Intra Prediction
`Apparatus
`Candidate Narrowing
`DOWn Unit
`
`11
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Video
`
`1
`
`Intra Prediction
`Execution Unit
`
`112
`
`Difference
`Value
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 7
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 7 Of 15
`
`US 8,135,064 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 8
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 8 of 15
`
`US 8,135,064 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 8
`
`18
`
`Coding Control Unit
`Video Characteristic 181
`Volume Input Unit
`Pixel Output Ratio
`Detection Unit
`
`Intra Prediction
`Apparatus
`Candidate Narrowing
`DOWn Unit
`
`11
`
`
`
`Video
`
`Intra Prediction
`Execution Unit
`
`112
`
`
`
`Difference
`Value
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 9
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 9 Of 15
`
`US 8,135,064 B2
`
`FIG. 9
`
`
`
`
`
`18
`
`Coding Control Unit
`181
`VideO Characteristic
`Volume Input Unit
`Picture Structure
`182
`Determination Unit
`
`Intra Prediction Apparatus
`
`Frame Intra
`Field Intra
`Prediction
`Prediction
`Direction
`Direction
`Candidate
`Candidate
`Candidate Narrowing-Down Unit
`112
`Difference Value
`
`Intra Prediction
`Execution Unit
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Field Intra Prediction
`Frame Intra Prediction
`Direction Candidate Example Direction Candidate Example
`O Vertical
`O Vertical
`1 Horizontal
`1 Horizontal
`2 DC
`X 2 DC
`3 Diagonal-Down-Left
`3 Diagonal-Down-Left
`4 Diagonal-Down-Right
`4 Diagonal-Down-Right
`5 Vertical-Left
`X5 Vertical-Left
`6 Horizontal-Up
`6 Horizontal-Up
`7 Vertical-Right
`X7 Vertical-Right
`8 Horizontal-DOwn
`8 Horizontal-DOwn
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 10
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 10 of 15
`
`US 8,135,064 B2
`
`FIG. 10
`
`O Vertical
`1 Horizontal
`------------------- -------
`2 DC
`3 Diagonal-Down-Left
`/(
`4 Diagonal-Down-Right
`/\ \
`5 Vertical-Left
`f\ \ }
`6 Horizontal-Up
`\, \ |
`|
`7 Vertical-Right
`N
`8 Horizontal-DOwn
`N NA/
`
`f / \
`| / 4.
`
`N-Y.4...............S.-J-2.
`
`5
`
`7
`
`5
`
`7
`
`(a) Prediction Direction
`in Field
`
`(b) Prediction Direction
`Corresponding to
`Original Picture
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 11
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 12
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 12 of 15
`
`US 8,135,064 B2
`
`FIG. 12
`
`18
`Coding Control Unit
`181
`
`Video Characteristic
`Vofume Input Unit
`Motion Vector Input
`Unit
`
`
`
`
`
`
`
`Prediction Mode
`Input Unit
`
`Pixel Computation
`information Input Unit
`
`
`
`
`
`
`
`Intra Prediction
`ApparatuS
`Candidate Narrowing
`DOWn Unit
`
`Intra Prediction
`Execution Unit
`
`112
`
`
`
`Difference
`Value
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Video
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 13
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 13 of 15
`
`US 8,135,064 B2
`
`
`
`s
`
`t 9
`o
`t
`O
`
`to 9
`9 B
`O 9
`O a
`
`V
`s
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 14
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 14 of 15
`
`US 8,135,064 B2
`
`
`
`t 9
`o 9
`
`O
`
`an
`
`O 9
`O All
`
`era
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 15
`
`

`

`U.S. Patent
`
`Mar. 13, 2012
`
`Sheet 15 Of 15
`
`US 8,135,064 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 16
`
`

`

`1.
`INTRA PREDCTION APPARATUS
`
`BACKGROUND OF THE INVENTION
`
`10
`
`15
`
`30
`
`45
`
`1. Field of the Invention
`The present invention relates to an intra prediction appa
`ratus used in moving picture compression and the like, and to
`technology for reducing an amount of computation for intra
`prediction which is one approach to moving picture compres
`Sion.
`2. Description of the Related Art
`In recent years, through the spread of communication
`infrastructures such as broadband, and through the drop in
`cost of personal computers, high-volume recording appara
`tuses such as HDD and DVD, memory cards, and so on, an
`environment in which general consumers edit, store, transmit,
`and carry video has been established, and the number of users
`is expanding.
`While situations in which general consumers handle video
`increase, there are also situations where personal computers
`do not have a level of performance that is sufficient for han
`dling video, even in the present age of high-performance
`personal computers. For example, the information amount of
`Video is extremely high, and it takes a significant amount of
`time to execute moving picture compression even when using
`25
`a high-performance personal computer. Increasing the com
`putational efficiency by processing using the same encoding
`performance but with a smaller computation amount can be
`considered as a solution to this. Additionally, the execution of
`highly efficient computation can be linked to a reduction in
`consumed power, and in a portable filming device, constraints
`placedon filmingtime by battery life can be relaxed. With this
`in mind, highly efficient moving picture compression tech
`nology is becoming more and more necessary.
`Amid this, the newly-developed international moving pic
`35
`ture compression standard H.264 (for example, see Non
`Patent Reference Document 1, “H264 Advanced video cod
`ing for generic audiovisual services. Triceps) uses a plurality
`of moving picture compression tools with the aim of improv
`ing picture quality and encoding efficiency. Additionally,
`40
`H.264 has a feature of sequentially comparing a plurality of
`prediction methods, selecting a prediction method with an
`advanced coding efficiency, and executing coding.
`For example, in the case where coding is executed in a
`16x16 pixel macroblock included in the Intra Coded Picture
`(I-picture) shown in FIG. 1, intra prediction is executed from
`a plurality of directions in each block of 4x4 pixels in the
`macroblock divided into 16 pixels, as well as being executed
`from a plurality of directions in the macroblock.
`The 4x4 intra prediction, which is that representative
`example, selects the prediction method with the highest
`encoding efficiency by comparing a plurality of prediction
`methods.
`In that prediction method, as shown in FIG. 2, there are 9
`modes in all: a prediction mode 0 (vertical) in which a pre
`dicted pixel value is calculated from an above macroblock
`(MB) and the pixel value is predicted in a vertical direction; a
`prediction mode 1 (horizontal) in which a predicted pixel
`value is calculated from an adjacent MB and the pixel value is
`predicted in a horizontal direction; a prediction mode 8 (hori
`Zontal-up), a prediction mode 6 (horizontal down), and a
`prediction mode 4 (diagonal-down-right), in which a pre
`dicted pixel value is calculated from an adjacent MB and the
`pixel value is predicted respectively in a direction shifted
`t22.5 degrees and -44.5 degrees from the horizontal; a pre
`diction mode 5 (vertical right), a prediction mode 7 (vertical
`left), and a prediction mode 3 (diagonal-down-left), in which
`
`50
`
`55
`
`60
`
`65
`
`US 8,135,064 B2
`
`2
`a predicted pixel value is calculated from an adjacent MBand
`the pixel value is predicted in a direction shifted 22.5
`degrees and -44.5 degrees from the vertical; as well as a
`prediction mode 2 (DC) in which prediction is executed from
`an average in pixel values from adjacent MBS.
`In the same manner, 16x16 intra prediction also has a
`plurality of prediction methods. In this 16x16 intra predic
`tion, there are 4 modes of the same type of prediction method.
`For 16x16 intra prediction, as shown in FIG. 3, there is a
`prediction mode 0 (vertical), a prediction mode 1 (horizon
`tal), a prediction mode 2 (DC), and a prediction mode 3
`(plane).
`Because of this, as shown in FIG. 4, when an I-picture is
`inputted in a conventional intra prediction apparatus, a pre
`diction error (absolute value difference sum) in the 9 paths of
`prediction modes in block 0 is calculated. In other words, in
`the 9 types of prediction modes, a differential value between
`a pixel to be encoded and a reference pixel, and an absolute
`value difference sum, are all calculated. Then, the intra pre
`diction apparatus uses the mode with the lowest prediction
`error as the intra prediction.
`When the intra prediction in block 0 finishes, the intra
`prediction apparatus repeats the same processing as men
`tioned above, and executes intra prediction in block 1 to block
`15.
`When the intra prediction in block 15 finishes, a differen
`tial value between a pixel to be encoded and a reference pixel,
`and an absolute value difference sum, are all calculated in the
`4 types of prediction modes in that macroblock. Then, the
`intra prediction apparatus uses the mode with the lowest
`prediction error as the intra prediction; in other words, for
`example, the smaller of i) the total of the absolute value
`difference sum of each block and ii) the absolute value dif
`ference sum of the macroblock. That is to say, in the case
`where the total of the absolute value difference sum of each
`block is Smaller, the intra prediction apparatus outputs a dif
`ference value for each block. On the other hand, in the case
`where the absolute value difference sum of the macroblock is
`Smaller, the intra prediction apparatus outputs a difference
`value for the macroblock.
`Data compression of the I-picture is executed in this man
`.
`However, as the abovementioned examples show, in a con
`ventional intra prediction apparatus, overhead and computa
`tion load are large, because in the H.264 intra prediction that
`is the international standard moving picture compression for
`mat currently disclosed, a predictive picture is created
`through a plurality of prediction methods (9types in 4x4 intra
`prediction, and 4 types in 16x16 intra prediction) for increas
`ing picture quality, and an optimum prediction method is
`selected from a variety of prediction methods; in other words,
`because the processing with a high encoding efficiency is
`selected when executing encoding, an increase in the amount
`of encoding computation cannot be avoided while increasing
`picture quality and encoding efficiency.
`Because of this, a reduction in the amount of encoding
`computation of H.264, which has advanced picture quality, is
`currently in demand in order to realize a moving picture
`compression technology with high picture quality and high
`efficiency.
`
`SUMMARY OF THE INVENTION
`
`Accordingly, an object of the present invention is to pro
`vide an intra prediction apparatus that can lighten the com
`putational load while providing an increase in picture quality
`and coding efficiency.
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 17
`
`

`

`US 8,135,064 B2
`
`3
`To achieve the aforementioned object, the industrious
`research of the present inventor has resulted in that, in the case
`where apixel of agenerally adjacent macroblock (hereinafter,
`MB) is used and prediction is executed, a correlation between
`pixels becomes lower the greater the distance between the
`adjacent MB and the pixel becomes. The present inventor
`noted that in the case where intra prediction is executed using
`all prediction directions without considering the correlation
`between pixels, there are situations where intra prediction is
`executed using prediction directions which do not contribute
`to encoding efficiency, and wasteful processing occurs. Addi
`tionally, the present inventor noted a problem whereinunnec
`essary processing, Such as processing of prediction directions
`which do not contribute to encoding efficiency, is executed
`when uniform intra prediction that is not adapted to the char
`acteristics of the image and so on is executed.
`In addition, the following problem exists in a method
`which systematically processes prediction direction in order
`to reduce intra prediction processing. For example, in 4x4
`intra prediction, prediction in horizontal and vertical direc
`tions is evaluated first, and depending on the result, the next
`prediction direction to be evaluated is determined. The
`present inventor noted a problem in that in this method, even
`in the case where it is already known that prediction in the
`vertical direction will not contribute to encoding efficiency,
`one evaluation in the vertical direction is necessary, and an
`unnecessary prediction in the vertical direction is executed.
`Through these observations, the present inventor arrived at
`the present invention.
`To solve the abovementioned problems, an intra prediction
`apparatus according to the present invention includes: a can
`didate narrowing-down unit which narrows down, from a
`plurality of intra prediction directions, intra prediction direc
`tion candidates in accordance with a characteristic of video
`data, the intra prediction direction candidates to be applied to
`a block of multiple pixels that constitutes the video data; and
`an intra prediction execution unit which executes intra pre
`diction using the intra prediction direction candidates nar
`rowed down by said candidate narrowing-down unit.
`In other words, at the time of intra prediction, the present
`invention omits, in advance, processing with poor encoding
`efficiency and processing with no difference in encoding
`efficiency, in accordance with the characteristics of the video.
`Through this, computational load can be lightened while
`increasing the picture quality and encoding efficiency.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may narrow down the intra prediction direction candidates
`according to a pixel aspect ratio which is the characteristic of
`the video data.
`In addition, in the intra prediction apparatus according to
`the present invention, the aspect ratio may be acquired from
`one of a video signal input device and information obtained
`from inside and outside of a video stream.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may narrow down the intra prediction direction candidates by
`using horizontal and vertical distances between adjacent pix
`els in an original picture as probabilities of prediction in
`horizontal and vertical directions, respectively, of intra pre
`diction according to the pixel aspect ratio.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may narrow down the intra prediction direction candidates
`according to picture structure information which indicates
`one of an interlaced image and a progressive image and which
`is the characteristic of the video data.
`
`40
`
`45
`
`4
`In addition, in the intra prediction apparatus according to
`the present invention, the picture structure information may
`be acquired from one of a video signal input device and
`information obtained from inside and outside of a video
`Stream.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may prioritize an elimination of vertical-right and a vertical
`left directions included in the intra prediction directions as
`being candidates when the picture structure information indi
`cates the interlaced image.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may prioritize an elimination of Vertical-right, vertical-left,
`diagonal down-right, and diagonal down-left directions
`included in the intra prediction directions as being candidates
`when the picture structure information indicates the inter
`laced image.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may prioritize an elimination of a DC direction included in
`the intra prediction directions as being a candidates when the
`picture structure information indicates the interlaced image.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may prioritize an elimination of DC and vertical directions
`included in the intra prediction directions as being candidates
`when the picture structure information indicates the inter
`laced image.
`In addition, in the intra prediction apparatus according to
`the present invention, when pictures are coded using a pict
`level adaptive coding a moving picture compression standard,
`the candidate narrowing-down unit is operable to narrow
`down the intra prediction direction candidates according to
`picture structure information.
`In addition, in the intra prediction apparatus according to
`the present invention, when the picture structure information
`is a field structure or frame structure and, when pictures are
`coded using a MB-level adaptive coding as a moving picture
`compression standard, the candidate narrowing-down unit
`may narrow down the intra prediction direction candidates
`according to information on whether a macroblock pair is the
`field structure or the frame structure when predicting by
`switching between a macroblock pair in the field structure
`and a macroblock pair in the frame structure.
`In addition, in the intra prediction apparatus according to
`the present invention, the candidate narrowing-down unit
`may narrow down the intra prediction direction candidates
`based on a result obtained with respect to an original picture
`or an encoded picture preceding in an encoding order.
`Through this, a candidate for the intra prediction direction
`can be narrowed down easily.
`In addition, in the intra prediction apparatus according to
`the present invention, an original picture or an encoded pic
`ture before a current picture to be encoded may be used as the
`original picture or the encoded picture preceding in the
`encoding order.
`Through this, correlation between the former original pic
`ture or encoded picture and the picture to be encoded becomes
`high, and a candidate for the intra prediction direction can be
`narrowed down accurately.
`In addition, in the intra prediction apparatus according to
`the present invention, the result is a motion vector obtained
`from the encoded picture preceding in the encoding order, and
`the candidate narrowing-down unit may narrow down the
`intra prediction direction candidates based on a direction of
`the motion vector.
`
`10
`
`15
`
`25
`
`30
`
`35
`
`50
`
`55
`
`60
`
`65
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 18
`
`

`

`5
`Through this, in the case where there is motion in the
`picture, a candidate of the intra prediction direction with a low
`correlation can be accurately and efficiently eliminated from
`an already-known motion vector.
`In addition, in the intra prediction apparatus according to
`the present invention, wherein a motion vector in a unit area
`which is indicative of any of a spatially same position as a
`macroblock to be encoded, a position including Surroundings
`of a spatially same position as a macroblock to be encoded,
`and an entire picture, may be used as the motion vector.
`Through this, the precision of a movement vector for a
`macroblock to be encoded can be enhanced, and a candidate
`of the intra prediction direction with a low correlation can be
`accurately and efficiently eliminated from an already-known
`motion vector.
`In addition, in the intra prediction apparatus according to
`the present invention, the result is an intra prediction direction
`obtained in an encoded picture preceding in the encoding
`order, and the candidate narrowing-down unit may narrow
`down the intra prediction direction candidates based on the
`intra prediction direction.
`Through this, regardless of motion in a picture, a candidate
`of the intra prediction direction with a low correlation can be
`accurately and efficiently eliminated from an already-known
`motion vector.
`In addition, in the intra prediction apparatus according to
`the present invention, an intra prediction direction of a mac
`roblock to be encoded and a macroblock of a spatially iden
`tical position and a position including the Surroundings of that
`spatially identical position may be used as the intra prediction
`direction.
`Through this, the precision of a movement vector for a
`macroblock to be encoded can be enhanced, and a candidate
`of the intra prediction direction with a low correlation can be
`accurately and efficiently eliminated from that intra predic
`tion direction.
`In addition, in the intra prediction apparatus according to
`the present invention, the result is an adjacent pixel difference
`of the original picture or the encoded picture preceding in the
`encoding order, and the candidate narrowing-down unit may
`40
`narrow down the intra prediction direction candidates based
`on the adjacent pixel difference.
`Through this, in the case where there is little motion in a
`picture, a candidate of the intra prediction direction with a low
`correlation can be accurately and efficiently eliminated from
`an already-known pixel difference.
`Note that the present invention can be realized not only by
`this kind of intra prediction apparatus, but also can be realized
`by an intra prediction method which makes steps of the char
`acteristic units included in this kind of intra prediction appa
`ratus, as well as by a program which causes a computer to
`execute those steps. It goes without saying that Such a pro
`gram may be distributed via a recording medium such as, for
`example, a CD-ROM Furthermore, it also goes without say
`ing that the present invention can be realized as a picture
`encoding apparatus that includes the abovementioned intra
`prediction apparatus.
`AS has been made clear by the above descriptions, through
`the intra prediction apparatus in the present invention, com
`putational load for intra prediction can be lightened while
`60
`retaining high picture quality and encoding efficiency.
`In other words, intra prediction is a processing which cop
`ies a pixel value from Surrounding pixels and forms a predic
`tive picture. However, because intra prediction selects a pre
`diction mode with a high encoding efficiency using a plurality
`of prediction directions, computation of prediction directions
`which were not finally selected is redundant. Therefore,
`
`6
`omission of prediction directions which do not contribute to
`encoding efficiency or do not produce much of a difference in
`encoding efficiency is valid from the viewpoint of reducing
`the amount of encoding computation. Additionally, with a
`field picture in interlace form, by not using a prediction direc
`tion that produces little difference in encoding efficiency, load
`occurring in intra prediction can be lightened. Additionally,
`by using features of a video, reduction of intra prediction
`processing in accordance to the video is possible, and pro
`cessing efficiency becomes higher than conventional meth
`ods.
`Therefore, the practical value of the present invention is
`extremely high in the present day, where video cameras,
`cellular phones with built-in cameras, and so on have spread
`extensively.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`These and other objects, advantages and features of the
`invention will become apparent from the following descrip
`tion thereof taken in conjunction with the accompanying
`drawings that illustrate a specific embodiment of the inven
`tion. In the Drawings:
`FIG. 1 is a diagram showing a relationship between a
`16x16 pixel macroblock included in an I-picture and each
`4x4 pixel block;
`FIG. 2 is a diagram showing a 4x4 intra prediction direc
`tion;
`FIG. 3 is a diagram showing a 16x16 intra prediction
`diagram;
`FIG. 4 is a sequence diagram showing a prediction pro
`cessing executed by a conventional intra prediction appara
`tus;
`FIG. 5 is a function block diagram showing an overall
`configuration of a picture encoding apparatus 1 applied by an
`intra prediction apparatus according to an embodiment in the
`present invention;
`FIG. 6 is a block diagram showing a function configuration
`of an intra prediction apparatus 11 which uses a video char
`acteristic Volume;
`FIG. 7 is a sequence diagram showing a processing
`executed by each part of an intra prediction apparatus 11 in
`detail;
`FIG. 8 is a block diagram showing a function configuration
`of an intra prediction apparatus 11 in the case where a picture
`structure is used;
`FIG.9 is a block diagram showing a function configuration
`of an intra prediction apparatus 11 in the case where an aspect
`ratio is used;
`FIG. 10 is a diagram showing a difference in an angle
`between an original picture and a field in a 4x4 intra predic
`tion;
`FIG. 11 is a diagram showing a distance relationship
`between a pixel location in a field and a pixel location in an
`original picture, in a 16x16 intra prediction;
`FIG. 12 is a block diagram showing a function configura
`tion of an intra prediction apparatus 11 which uses a motion
`vector, prediction mode, and pixel computation information;
`FIG. 13 is a diagram for describing determination of a
`prediction direction candidate by a motion vector,
`In particular, FIG. 13(a) shows original pictures arranged
`in encoding order, FIG. 13(b) shows encoded pictures
`arranged in encoding order, and FIG. 13(c) shows a relation
`ship between an encoded image ahead in an encoding order
`and a picture to be encoded;
`FIG. 14 is a diagram for describing determination of a
`prediction direction candidate by a prediction mode;
`
`US 8,135,064 B2
`
`10
`
`15
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1039, p. 19
`
`

`

`US 8,135,064 B2
`
`7
`In particular, FIG. 14(a) shows original pictures arranged
`in encoding order, FIG. 14(b) shows encoded pictures
`arranged in encoding order, and FIG. 14(c) shows a relation
`ship between an encoded image ahead in an encoding order
`and a picture to be encoded; and
`FIG. 15 is a flowchart showing an operation of an intra
`prediction candidate narrowing-down processing which nar
`rows down an intra prediction candidate using intra predic
`tion, which is an already-known result of a candidate narrow
`ing-down unit 111.
`
`5
`
`10
`
`15
`
`8
`The movement prediction unit 176 uses the motion vector and
`reference picture index obtained to fetch an optimum picture
`area from a coded picture stored in the frame memory 175 and
`generates a predictive picture.
`By executing variable-length coding in the entropy coding
`unit 16 on the encoding information of the motion vector,
`reference picture index, and residual encoding signal output
`ted through the above series of processing, a bit stream with a
`Small amount of data is outputted through this encoding pro
`cessing.
`The above flow of processing is an operation in the case
`where inter picture prediction encoding is executed, but
`Switching to intra prediction encoding can be executed
`through a switch 114 and a switch 115.
`In the case where intra encoding is executed, generation of
`a predictive image through motion compensation is not
`executed, and in the intraprediction apparatus 11, a predictive
`image of an area to be encoded is generated from a coded area
`of inside the same picture, and by taking the difference a
`difference picture signal is generated. Note that details of this
`are given later. This difference picture signal is, in the same
`manner as the case of inter encoding, converted to a residual
`coding signal in the conversion unit 14 and the quantizing unit
`15. The residual signal is variable-length coded by the
`entropy encoding unit 16, and a bit stream with a small
`amount of data is outputted.