(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2005/0281337 A1
`Kobayashi et al.
`(43) Pub. Date:
`Dec. 22, 2005
`
`US 2005O281.337A1
`
`(54) MOVING IMAGE CODINGAPPARATUS
`(75) Inventors: Satoru Kobayashi, Bunkyou-ku (JP);
`Jun Makino, Kokubunji-shi (JP)
`Correspondence Address:
`Canon U.S.A. Inc.
`Intellectual Property Division
`15975 Alton Parkway
`Irvine, CA 92618-3731 (US)
`(73) Assignee: Canon Kabushiki Kaisha, Ohta-ku (JP)
`(21) Appl. No.:
`11/152,609
`(22) Filed:
`Jun. 14, 2005
`(30)
`Foreign Application Priority Data
`
`Jun. 17, 2004
`Feb. 2, 2005
`
`(JP)...................................... 2004-179933
`(JP)...................................... 2005-026530
`
`Publication Classification
`
`(51) Int. Cl. ............................ H04B 1/66; H04N 11/02;
`H04N 11/04; H04N 7/12
`(52) U.S. Cl. ................................. 375/240.18; 375/240.24
`
`(57)
`
`ABSTRACT
`
`An image coding apparatus determines an image pattern of
`image data and, based on the determined image pattern,
`Selects a prediction mode for generating predicted pixel
`values by predicting pixel values in a frame using pixel
`values in the same frame. Alternatively, based on photo
`graphing information concerning input image data, an image
`coding apparatus Selects a prediction mode for generating
`predicted pixel values by predicting pixel values in a frame
`using pixel values in the same frame.
`
`
`
`600
`
`INPUT IMAGE
`DATA
`
`INTEGER
`TRANSFORM
`
`INVERSE
`QUANTIZATION
`
`NVERSE
`INTEGER
`TRANSFORM
`UNIT
`
`ENTROPY
`COOING UN
`
`CODED
`DATA
`
`INTRA
`PREDICTION
`UNIT
`
`INTER
`PREDICTION
`UNIT
`
`MOTION
`DETECTION
`UNIT
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 1
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 1 of 18
`
`US 2005/0281337 A1
`
`FIG.1
`
`
`
`102
`
`INTRA PREDICTION
`MODE DESIGNATION
`UNIT
`
`?
`
`
`
`
`
`
`
`IMAGE PATTERN
`DETERMINATION
`UNIT
`
`INPUT
`MAGE
`O
`
`SELECTOR
`
`
`
`
`
`VERTICAL
`INTRA PREDICTION
`UNT
`
`HORIZONTA
`INTRA PREDICTION
`UNIT
`
`DC
`INTRA PREDICTION
`UNIT
`
`PREDCTED
`MAGE
`O
`
`SELECTOR
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 2
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 2 of 18
`
`US 2005/0281337 A1
`
`FIG.2
`
`INPUT IMAGE
`
`x DIRECTION
`
`
`
`HADAMARD
`TRANSFORM
`
`HADAMARDTRANSFORM
`COEFFICIENT
`
`FREGUENCY
`IN HORIZONTAL
`low- DIRECTION-HIGH
`LOW
`
`(a)
`
`H G H
`
`(b)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 3
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 3 of 18
`
`US 2005/0281337 A1
`
`INPUT IMAGE
`
`
`
`HADAMARO TRANSFORM
`COEFFICIENT
`
`
`
`
`
`ear
`
`510Yoo so
`ooloo
`oooo
`(b)
`
`
`
`FIG.3B
`
`FIG.3C
`
`59 goo
`?o OSoo
`soooo
`
`N
`
`1.
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 4
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 4 of 18
`
`US 2005/0281337 A1
`
`FIG.4A
`
`WERTICAL
`
`FIG.4B
`
`MABCDEFGH
`H:
`
`33
`
`S
`
`LE:::
`
`HORIZONTAL
`
`FIG.4C
`
`
`
`
`
`
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 5
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 5 of 18
`
`US 2005/0281337 A1
`
`FIG.5
`
`START
`
`INPUT 4X4 PXEL
`BLOCK
`
`S5O2
`HADAWARD
`TRANSFORM
`
`S503
`
`
`
`
`
`DOES
`INPUT IMAGE CONTAIN
`VERTICAL EDGE
`
`YES
`
`
`
`
`
`
`
`
`
`
`
`
`
`DOES
`INPUT IMAGE CONTAIN
`HORIZONTAL
`EDGE
`NO
`
`IS INPUT
`IMAGE FLAT2
`
`S505
`
`
`
`
`
`
`
`OUTPUT
`INPUT IMAGE
`
`
`
`
`
`S506
`SELECT
`VERTICAL INTRA
`PREDICTION MODE
`
`
`
`
`
`SELECT
`HORIZONTAL INTRA
`PREDICTION MODE
`S508
`SELECT DC INTRA
`PREDICTION MODE
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 6
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 6 of 18
`
`US 2005/0281337 A1
`
`
`
`NOI.LV/ZILNVITO
`ESHEAN||
`
`LINT
`
`|MIHO-ISN\/H1
`HEROELNI
`ESHEAN||
`
`LINQ
`
`HE5DELNI
`LINT)
`WHO-ISNV HL
`
`HELTI-||
`dOOT
`
`HELNI
`
`NOLLOIGJElbid
`
`LINTI
`
`VELNI
`LINT)
`NOHLOICIE8 d.
`
`NOLLOIN
`
`LINT)
`
`NOI LOE LEIC]
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 7
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 7 of 18
`
`US 2005/0281337 A1
`
`FIG.7
`
`
`
`700
`
`701 a 7Of
`
`701C
`
`N \ A
`
`CAMERA
`SIGNAL
`WAW PROSESSING
`
`704
`
`CODING
`UNIT
`
`CODED
`
`OUTPUT
`
`O DATA
`
`FOCUS
`DETECTIO
`UNIT
`
`CAMERA
`CONTROL
`UNIT
`
`MOTION
`SENSOR
`
`OPERATION
`UNIT
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 8
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 8 of 18
`
`US 2005/0281337 A1
`
`FIG.8
`
`
`
`PANNING
`DIRECTION
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 9
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 9 of 18
`
`US 2005/0281337 A1
`
`FIG.9
`
`START INTRA PREDICTION
`
`PANNING OR
`TILTING ON?
`
`
`
`
`
`
`
`ACOUIRE
`INFORMATION ON
`PANNING OR TILTING
`
`VERTICAL
`
`SELECT
`PREDICTION
`DIRECTION
`
`DOWN-RIGHT
`
`S904
`
`
`
`
`
`INTRA
`PREDICTION
`MODE = O
`
`
`
`HORIZONTAL
`INTRA
`INTRA
`PREDICTION
`PREDICTION
`MODE 1
`MODE 4
`
`S907
`ORDINARY INTRA
`PREDICTION
`MODE - O TO 8
`
`
`
`END INTRA PREDICTION
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 10
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 10 of 18
`
`US 2005/0281337 A1
`
`FIG.O
`
`
`
`Y
`
`DURING
`ZOOMING
`
`?
`
`Y
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 11
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 11 of 18
`
`US 2005/0281337 A1
`
`FIG.11
`
`START INTRA PREDCTION
`
`
`
`
`
`ACOUIRE
`INFORMATION ON
`ZOOMING
`
`DOWN-LEFT
`
`
`
`
`
`
`
`DETERMINE AREA
`
`CENTER
`
`S1 104
`
`
`
`
`
`
`
`INTRA
`PREDICTION
`MODE - 3
`
`
`
`DOWN-RIGHT
`INTRA
`PREDICTION
`MODE = 4
`
`Sf 106
`
`ORDINARY INTRA
`PREDICTION
`MODE - O TO 8
`
`END INTRA PREDICTION
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 12
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 12 of 18
`
`US 2005/0281337 A1
`
`F.G. 12
`
`
`
`BLURRED
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 13
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 13 of 18
`
`US 2005/0281337 A1
`
`FIG.13
`
`START INTRA PREDICTION
`
`
`
`
`
`
`
`ACOUIRE
`INFORMATION ON
`FOCUSING
`
`
`
`
`
`
`
`
`
`OUT OF FOCUS
`
`S13O2
`DETERMINE
`INFOCUS
`FOCUSING STATE
`
`INTRA
`PREDICTION
`MODE = 2
`
`
`
`
`
`END INTRA PREDICTION
`
`ORONARY INTRA
`PREDICTION
`MODE O TO 8
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 14
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 14 of 18
`
`US 2005/0281337 A1
`
`
`
`HEI™OBILNI
`LINT)
`WHO-ISNV HL
`
`VH1N1
`
`NOILOIGJE!!!!d
`
`LINT)
`
`
`NOI LOICIE?ld
`HELNI
`LINT?
`
`NOLLOIN
`
`NOI 10BLECI
`LIN[]
`
`OTVNOIS
`TOH1NO3)
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 15
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 15 of 18
`
`US 2005/0281337 A1
`
`FIG.15
`
`
`
`MABCDEFGH
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 16
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 17
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 17 of 18
`
`US 2005/0281337 A1
`
`FIG.17
`
`
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 18
`
`

`

`Patent Application Publication Dec. 22, 2005 Sheet 18 of 18
`
`US 2005/0281337 A1
`
`FIG.18
`
`START INTRA PREDCTION
`S1801
`
`S1802
`INTRA PREDICTION
`MODE = i
`
`S1803
`
`SELECT OPTIMAL VALUE
`
`
`
`S1804
`
`i++
`
`S1805
`
`NO
`
`END INTRA PREDICTION
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 19
`
`

`

`US 2005/0281337 A1
`
`Dec. 22, 2005
`
`MOVING IMAGE CODING APPARATUS
`
`BACKGROUND OF THE INVENTION
`0001) 1. Field of the Invention
`0002 The present invention relates to a moving image
`coding apparatus, and more particularly to a technique for
`Selecting an optimal intra prediction mode when the H.264
`coding Standard is used.
`0003 2. Description of the Related Art
`0004 Various coding standards, such as Motion-JPEG,
`MPEG-1 and MPEG-2, have been established as techniques
`for high-efficiency coding of moving images. Manufacturers
`have been developing and marketing image capture appa
`ratuses, Such as digital cameras, digital Video cameras, DVD
`(digital versatile disk) players, etc., which are capable of
`Storing moving images using these coding Standards.
`Accordingly, users are allowed to easily play back moving
`images using these image capture apparatuses, DVD play
`ers, personal computers, or the like.
`0005 Digitized moving images carry large mounts of
`data. Therefore, various coding methods for moving images
`capable of performing more efficient high compression than
`the above coding standards, such as MPEG-1 or MPEG-2,
`have been continuously researched and developed. Recently,
`a new coding algorithm called H.264/MPEG-4 Part 10 AVC
`(hereinafter referred to as the H.264 standard) has been
`standardized by the ITU-T (International Telecommunica
`tion Union-Telecommunication Standardization Sector) and
`the ISO (International Organization for Standardization).
`0006 The H.264 standard requires a large computational
`complexity for coding and decoding as compared to the
`conventional coding Standards, Such as MPEG-1 and
`MPEG-2, but provides a higher coding efficiency. A system
`and proceSS for computation processing using the H.264
`Standard are disclosed, for example, in Japanese Laid-Open
`Patent Application No. 2004-56827.
`0007. The H.264 standard includes a prediction method
`known as intra prediction for predicting pixel values within
`a given frame by using pixel values within the same frame.
`In this intra prediction, there are a plurality of intra predic
`tion modes, which are Selectively used. In this instance, an
`intra prediction mode Suitable for an input image is Selected
`to form coded data that has little deterioration even after
`being Subjected to highly efficient compression.
`0008 AS for the intra prediction, the H.264 standard
`provides nine intra prediction modes to improve the preci
`Sion of prediction. An optimal intra prediction mode is
`generally Selected from the nine intra prediction modes by
`tentatively executing all of the intra prediction modes for an
`input image and, based on a result of the tentative eXecution,
`finding an intra prediction mode capable of obtaining an
`optimal result.
`0009. The reason for employing such a selection method
`is described below with reference to FIG. 17. FIG. 17 is a
`diagram showing a frame, in which an outer quadrilateral
`denotes the entire picture and five inner quadrilaterals
`denote blocks to be Subjected to intra prediction. Although
`blocks are actually Set over the entire picture, only five
`representative blocks are illustrated for the Sake of conve
`nience of description. The arrows in FIG. 17 indicate the
`
`direction of prediction indicated by an intra prediction mode
`that has been determined to be optimal as a result of
`computation in each block. Thus, Since various objects are
`present at the respective locations even within the same
`picture, different intra prediction modes may be determined
`to be optimal for the respective blocks, and more than one
`particular intra prediction mode may be selected. For Such a
`reason, in performing intra prediction, the precision of
`prediction is obtained for all of the intra prediction modes in
`each block, and an intra prediction mode capable of per
`forming optimal prediction is Selected and designated in
`each block.
`0010. However, computation for all of the intra predic
`tion modes in each block to Select an optimal intra prediction
`mode from among them increases computational complexity
`in the H.264 coding process, thus resulting in an excessive
`increase in coding processing time or a wasteful consump
`tion of electric power.
`
`SUMMARY OF THE INVENTION
`0011. The present invention has been made in consider
`ation of the above situation, and an aspect of the present
`invention is to facilitate Selecting an optimal intra prediction
`mode for an input image.
`0012 Another aspect of the present invention is to facili
`tate Selecting an optimal intra prediction mode for an input
`image by using information on photography concerning the
`input image.
`0013 A further aspect of the present invention is to
`implement effective coding by alleviating computation com
`plexity in an image coding apparatus using the H.264
`Standard.
`0014.
`In one aspect of the present invention, an image
`coding apparatus includes an input unit configured to input
`image data, a division unit configured to generate blockS by
`dividing the image data input by the input unit into blockS
`each comprised of a plurality of pixels, a determination unit
`configured to determine an image pattern of image data in
`each block generated by the division unit, a Selection unit
`configured to Select one of a plurality of prediction modes
`based on the image pattern determined by the determination
`unit, and a processing unit configured to output predicted
`pixel values by performing processing for predicting pixel
`values in a picture using pixel values in the same picture
`according to the prediction mode Selected by the Selection
`unit.
`0015. In another aspect of the present invention, an image
`coding apparatus includes an input unit configured to input
`image data to be coded, an acquisition unit configured to
`acquire photographing information concerning the image
`data, a Selection unit configured to Select m prediction
`modes from among n prediction modes (1sms n) using the
`photographing information, and a processing unit configured
`to output predicted pixel values by performing processing
`for predicting pixel values in a picture using pixel values in
`the same picture according to the m prediction modes
`Selected by the Selection unit.
`0016 Other features and advantages of the present inven
`tion will become apparent to those skilled in the art upon
`reading of the following detailed description of embodi
`ments thereof when taken in conjunction with the accom
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 20
`
`

`

`US 2005/0281337 A1
`
`Dec. 22, 2005
`
`panying drawings, in which like reference characters des
`ignate the same or Similar parts throughout the figures
`thereof.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0.017. The accompanying drawings, which are incorpo
`rated in and constitute a part of the Specification, illustrate
`embodiments of the invention and, together with the
`description, Serve to explain the principles of the invention.
`0.018
`FIG. 1 is a block diagram illustrating processing
`for intra prediction according to a first embodiment of the
`invention.
`0.019
`FIG. 2 is a diagram illustrating Hadamard trans
`form.
`0020 FIGS. 3A, 3B and 3C are diagrams illustrating
`examples of transform in Hadamard transform.
`0021
`FIGS. 4A, 4B and 4C are diagrams illustrating
`intra prediction modes.
`0022 FIG. 5 is a flow chart illustrating processing for
`Selecting an intra prediction mode according to the first
`embodiment.
`0023 FIG. 6 is a block diagram of an image coding
`apparatus using the H.264 Standard according to the first
`embodiment.
`0024 FIG. 7 is a block diagram showing the configura
`tion of an image capture apparatus according to a third
`embodiment of the invention.
`0.025
`FIG. 8 is a diagram showing an example of direc
`tions of intra prediction for an image obtained during
`panning.
`0.026
`FIG. 9 is a flow chart illustrating processing for
`Selecting an intra prediction mode according to the direction
`of panning or tilting in an image coding apparatus according
`to the third embodiment.
`0.027
`FIG. 10 is a diagram showing an example of
`directions of intra prediction for an image obtained during
`Zooming.
`0028 FIG. 11 is a flow chart illustrating processing for
`Selecting an intra prediction mode according to Zooming in
`an image coding apparatus according to a fourth embodi
`ment of the invention.
`0029 FIG. 12 is a diagram showing an example of intra
`prediction for a blurred image.
`0030 FIG. 13 is a flow chart illustrating processing for
`an intra prediction mode according to focusing in an image
`coding apparatus according to a fifth embodiment of the
`invention.
`0.031
`FIG. 14 is a block diagram of the image coding
`apparatus according to the third embodiment.
`0.032
`FIG. 15 is a diagram illustrating pixels for use in
`intra prediction.
`0.033
`FIG. 16 is a diagram illustrating nine intra predic
`tion modes.
`0034 FIG. 17 is a diagram showing an example of
`directions of intra prediction for an ordinary image.
`
`0035 FIG. 18 is a flow chart illustrating ordinary pro
`cessing for Selecting an intra prediction mode.
`
`DETAILED DESCRIPTION OF THE
`EMBODIMENTS
`0036) Embodiments of the invention will be described in
`detail below with reference to the drawings.
`
`First Embodiment
`0037 FIG. 6 is a block diagram showing the configura
`tion of an image coding apparatuS 600 according to a first
`embodiment of the invention. The image coding apparatus
`600 includes a subtracter 601, an integer transform unit 602,
`a quantization unit 603, an entropy coding unit 604, an
`inverse quantization unit 605, an inverse integer transform
`unit 606, an adder 607, frame memories 608 and 611, an
`intra prediction unit 609, a loop filter 610, an interprediction
`unit 612, a motion detection unit 613, and a Switch 614. The
`image coding apparatus 600 is configured to perform pro
`cessing for coding input image data to output the coded data.
`0038. The coding process in the image coding apparatus
`600 shown in FIG. 6 is described next. The image coding
`apparatuS 600 performs processing for coding according to
`the H.264 standard. Further, the image coding apparatus 600
`divides input moving image data into blockS and performs
`processing for every block.
`0039. Initially, the subtracter 601 subtracts predicted
`image data from image data input to the image coding
`apparatus 600 (input image data) to output difference image
`data. Generation of the predicted image will be described
`later.
`0040. The integer transform unit 602 orthogonally trans
`forms the difference image data output from the Subtracter
`601 to output transform coefficients. Then, the quantization
`unit 603 quantizes the transform coefficients using prede
`termined quantization parameters.
`0041. The entropy coding unit 604 receives and entropy
`codes the transform coefficients quantized by the quantiza
`tion unit 603 to output coded data.
`0042. The transform coefficients quantized by the quan
`tization unit 603 are also used to generate predicted image
`data for coding Subsequent blockS. The inverse quantization
`unit 605 performs inverse quantization on the transform
`coefficients quantized by the quantization unit 603. The
`inverse integer transform unit 606 performs inverse integer
`transform on the transform coefficients Subjected to inverse
`quantization by the inverse quantization unit 605 to output
`decoded difference data. The adder 607 adds the decoded
`difference data and the predicted image data to output
`reconstructed image data.
`0043. The reconstructed image data is stored into the
`frame memory 608 and is also stored into the frame memory
`611 via the loop filter 610. Data possible to refer to in the
`Subsequent prediction among the reconstructed image data is
`stored for a short period in the frame memory 608 or 611.
`The loop filter 610 is used to remove blocking noise.
`0044) The intra prediction unit 609 performs intra-frame
`prediction using the reconstructed image data Stored in the
`frame memory 608 So as to generate predicted image data.
`The inter prediction unit 612 performs inter-frame predic
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 21
`
`

`

`US 2005/0281337 A1
`
`Dec. 22, 2005
`
`tion based on motion vector information detected by the
`motion detection unit 613 and using the reconstructed image
`data Stored in the frame memory 611 to generate predicted
`image data. The motion detection unit 613 detects motion
`vector information in the input image data and outputs the
`detected motion vector information to the inter prediction
`unit 612 and the entropy coding unit 604.
`004.5 The Switch 614 functions as a selection unit for
`Selecting any one of intra prediction and inter prediction for
`every macroblock. That is, the Switch 614 selects one of the
`output from the intra prediction unit 609 and the output from
`the inter prediction unit 612 and outputs the Selected pre
`dicted image data to the subtracter 601 and the adder 607.
`0.046
`FIG. 1 is a block diagram illustrating processing
`for intra prediction according to the first embodiment. The
`configuration shown in FIG. 1 corresponds to the intra
`prediction unit 609 included in the image coding apparatus
`600.
`0047 Referring to FIG. 1, the intra prediction unit 609
`includes an image pattern determination unit 101, an intra
`prediction mode designation unit 102, a selector 103, a
`Vertical intra prediction unit 104, a horizontal intra predic
`tion unit 105, a DC intra prediction unit 106, and a selector
`107. The image pattern determination unit 101 determines
`an image pattern by performing Hadamard transform on an
`input image. The intra prediction mode designation unit 102
`designates an optimal intra prediction mode from among a
`plurality of intra prediction modes based on the image
`pattern determined by the image pattern determination unit
`101. The Selectors 103 and 107 Select one of the vertical
`intra prediction unit 104, the horizontal intra prediction unit
`105 and the DC intra prediction unit 106, corresponding to
`the intra prediction mode designated by the intra prediction
`mode designation unit 102. The vertical intra prediction unit
`104 performs intra prediction using a vertical intra predic
`tion mode. The horizontal intra prediction unit 105 performs
`intra prediction using a horizontal intra prediction mode.
`The DC intra prediction unit 106 performs intra prediction
`using a DC intra prediction mode. Thus, one of the Vertical,
`the horizontal and the DC intra prediction mode is selected
`and executed according to the selection by the selectors 103
`and 107. As a result, the intra prediction unit 609 outputs a
`predicted image.
`0.048. The essential portions of the intra prediction unit
`609 shown in FIG. 1 are described next in detail. In the
`present embodiment, for example, the image pattern deter
`mination unit 101 performs Hadamard transform on an input
`image that is divided into 4x4 pixel blocks. However, the
`block size and the configuration for determining an image
`pattern are not limited to those described herein.
`0049. The image pattern determination unit 101 divides
`pixel data of an input image into 4x4 pixel blocks, performs
`Hadamard transform on pixel data for each block, and
`determines an image pattern for each block based on Had
`amard transform coefficients obtained by the Hadamard
`transform computation.
`0050 Hadamard transform is described next with refer
`ence to FIG. 2. Hadamard transform is one type of orthogo
`nal transform. FIG. 2 illustrates the manner of 4x4 Had
`amard transform in the image pattern determination unit
`101. In FIG. 2, part (a) illustrates an input image composed
`
`of 4x4 pixels, and part (b) illustrates Hadamard transform
`coefficients obtained by Hadamard transform. Let H be the
`Hadamard transform matrix, X be input image Signals, and
`Y be signals for the Hadamard transform coefficients. Then,
`the Hadamard transform is represented by the following
`equation:
`Y=HLXIIH
`0051) Here, let
`
`(1)
`
`1
`
`1
`
`1
`
`X11 X12 X13 X14
`
`Y11 Y12 Y13 Y14
`
`(2)
`
`(3)
`
`(4)
`
`0052 Thus, the Hadamard transform is defined as fol
`lows:
`
`1
`
`1
`
`1
`
`(5)
`
`0053 Thus, the Hadamard transform can be performed
`with only one division, addition and subtraction. It should be
`noted that Y on the left side of the equation (5) indicates
`a DC component of the input image, and Y to Y indicate
`Hadamard transform coefficients representing AC compo
`nents of the input image.
`0054 FIGS. 3A, 3B and 3C illustrate examples of the
`Hadamard transform for an input image in which image data
`of 8 bits per pixel (256 gradation levels) is divided into 4x4
`pixel blocks. In each of FIGS. 3A, 3B and 3C, part (a)
`indicates an input image, and part (b) indicates Hadamard
`transform coefficients. In cases where a vertical edge exists
`in the input image block as shown in part (a) of FIG. 3A, an
`Hadamard transform coefficient corresponding to the posi
`tion of the Hadamard transform coefficient Y shown in part
`(b) of FIG. 2 takes a large value as shown in part (b) of FIG.
`3A. In cases where a horizontal edge exists in the input
`image block as shown in part (a) of FIG. 3B, an Hadamard
`transform coefficient corresponding to the position of the
`Hadamard transform coefficient Y shown in part (b) of
`FIG. 2 takes a large value as shown in part (b) of FIG.3B.
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 22
`
`

`

`US 2005/0281337 A1
`
`Dec. 22, 2005
`
`In cases where the input image block contains a flat image
`as shown in part (a) of FIG. 3C, an Hadamard transform
`coefficient corresponding to the position of the Hadamard
`transform coefficient Y shown in part (b) of FIG. 2 takes
`a large value as shown in part (b) of FIG. 3C, and the other
`coefficients take a value of 0. In general, in cases where a
`Vertical edge exists in the input image block, only an
`Hadamard transform coefficient indicating that the Spatial
`frequency in the vertical direction is low takes a large value,
`as shown with Hadamard transform coefficients encircled by
`an ellipse in part (b) of FIG. 3A. In cases where a horizontal
`edge exists in the input image block, only an Hadamard
`transform coefficient indicating that the Spatial frequency in
`the horizontal direction is low takes a large value, as shown
`with Hadamard transform coefficients encircled by an ellipse
`in part (b) of FIG.3B. In cases where the input image block
`contains a flat image, Hadamard transform coefficients
`indicative of AC components of the input image take a Small
`value.
`0055. Using the above-described method enables an
`image pattern of the input image, Such as a Vertical edge, a
`horizontal edge or flatness, to be determined based on
`Hadamard transform coefficients obtained by performing
`Hadamard transform on the input image.
`0056. The intra prediction is described next with refer
`ence to FIGS. 4A, 4B and 4.C. FIGS. 4A, 4B and 4C
`illustrate examples of the respective intra prediction modes.
`0057. In FIGS. 4A, 4B and 4C, a to p represent pixel
`values of an input image block to be predicted. A to M
`represent pixel values belonging to the adjacent blockS. The
`pixel values a top and Ato M are located in the same frame.
`The intra prediction generates predicted pixel values a' to p'
`using the pixel values A to M. The predicted pixel values a'
`to p' are lumped together to form a predicted image.
`0.058. The various intra prediction modes are described
`next in detail.
`0059. In the vertical intra prediction mode shown in FIG.
`4A, prediction is performed in the vertical direction. The
`predicted pixel values a' to p' are generated by predicting
`that the pixel values a, e, i and m each equal the pixel value
`A, the pixel values b, f, and in each equal the pixel value
`B, the pixel values c, g, k and o each equal the pixel value
`C, and the pixel values d, h, 1 and p each equal the pixel
`value D. Thus, the following pixel values are generated:
`
`0062) c'=g'=k'-o'-C
`0063) d'=h'=1'=p'=D
`0064.
`In the horizontal intra prediction mode shown in
`FIG. 4B, prediction is performed in the horizontal direction.
`The predicted pixel values a' to p' are generated by predict
`ing that the pixel values a, b, c and d each equal the pixel
`value I, the pixel values e, f, g, and h each equal the pixel
`value J, the pixel values i, j, k and 1 each equal the pixel
`value K, and the pixel values m, n, o and p each equal the
`pixel value L. Thus, the following pixel values are gener
`ated:
`
`0069. In the DC intra prediction mode shown in FIG. 4C,
`prediction is performed Such that all of the pixels have the
`Same value. The predicted pixel values a' to p' are generated
`by predicting that all of the pixel values a to p are equal.
`Thus, the following pixel values are generated:
`
`0071. The smaller the difference between the predicted
`pixel value and the actual pixel value of an input image, the
`higher the precision of prediction becomes, thus enabling
`efficient image compression.
`0072 The method for designating an intra prediction
`mode according to an image pattern is described next.
`0073. The intra prediction mode designation unit 102
`designates an optimal intra prediction mode, from among
`the above-described intra prediction modes, based on an
`image pattern determined by the image pattern determina
`tion unit 101 as described above. For example, if it is
`determined that the input image block contains a vertical
`edge, the intra prediction mode designation unit 102 desig
`nates the vertical intra prediction mode. If it is determined
`that the input image block contains a horizontal edge, the
`intra prediction mode designation unit 102 designates the
`horizontal intra prediction mode. If it is determined that the
`input image block contains a flat image, the intra prediction
`mode designation unit 102 designates the DC intra predic
`tion mode.
`0074 More specifically, in cases where a vertical edge
`exists in the input image block as shown in FIG. 3A, pixel
`values having the following relationship are obtained:
`0075)
`0.076 b=f==n
`0.077
`0078
`0079. This input image block is subjected to Hadamard
`transform to obtain Hadamard transform coefficients. If the
`image pattern determination unit 101 determines that the
`input image block has an image pattern containing a vertical
`edge, the intra prediction mode designation unit 102 desig
`nates the vertical intra prediction mode, in which the pre
`cision of prediction is highest for the input image block.
`0080. In cases where a horizontal edge exists in the input
`image block as shown in FIG. 3B, pixel values having the
`following relationship are obtained:
`
`0085. This input image block is subjected to Hadamard
`transform to obtain Hadamard transform coefficients. If the
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1023, p. 23
`
`

`

`US 2005/0281337 A1
`
`Dec. 22, 2005
`
`image pattern determination unit 101 determines that the
`input image block has an image pattern containing a hori
`Zontal edge, the intra prediction mode designation unit 102
`designates the horizontal intra prediction mode, in which the
`precision of prediction is highest for the input image block.
`0.086. In cases where the input image block contains a flat
`image as shown in FIG. 3C, pixel values having the
`following relationship are obtained:
`
`0088. This input image block is subjected to Hadamard
`transform to obtain Hadamard transform coefficients. If the
`image pattern determination unit 101 determines that the
`input image block has an image pattern containing a flat
`image, the intra prediction mode designation unit 102 des
`ignates the DC intra prediction mode, in which the precision
`of prediction is highest for the input image block.
`0089. Then, the selectors 103 and 107 select one of the
`intra prediction units 104 to 106 corresponding to the intra
`prediction mode designated by the intra prediction mode
`designation unit 102, thus causing the Selected prediction
`unit to perform intra prediction processing on the input
`image. As a result, predicted pixel values can be generated
`according to an optimal intra prediction mode. In addition,
`in cases where the image pattern determination unit 101
`cannot certainly determine an image pattern of the input
`image and the intra prediction mode designation unit 102
`cannot designate an optimal intra prediction mode, or in
`cases where none of the intra prediction units 104 to 107 is
`selected by the selectors 103 and 107 for any other reason,
`the intra prediction unit 609 outputs the input image as it
`Stands.
`0090. If the time required for Hadamard transform in
`determining an image pattern is taken into consideration, a
`timing adjustment unit, Such as a memory, for providing a
`predetermined delay time may be added to the Stage before
`the selector 103, or the intra prediction unit itself may
`perform timing adjustment, So that an image determined for
`an image pattern can be matched in timing with an image to
`be Subjected t