(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2007/0035641 A1
`(43) Pub. Date:
`Feb. 15, 2007
`Yamada et al.
`
`US 20070035641A1
`
`(54) IMAGE PROCESSINGAPPARATUS, IMAGE
`PROCESSING METHOD, AND COMPUTER
`PROGRAM
`
`(76) Inventors: Rui Yamada, Kanagawa (JP);
`Mitsuharu Ohki, Tokyo (JP)
`Correspondence Address:
`FINNEGAN, HENDERSON, FARABOW,
`GARRETT & DUNNER
`LLP
`901 NEW YORK AVENUE, NW
`WASHINGTON, DC 20001-4413 (US)
`(21) Appl. No.:
`10/565,597
`
`(22) PCT Filed:
`(86). PCT No.:
`
`Apr. 25, 2005
`PCT/UP05/07854
`
`S 371(c)(1),
`(2), (4) Date: Aug. 11, 2006
`Foreign Application Priority Data
`
`(30)
`
`May 27, 2004 (JP)...................................... 2004-157098
`Oct. 19, 2004 (JP)...................................... 2004-3O3795
`
`Publication Classification
`
`(51) Int. Cl.
`(2006.01)
`HO)4N 5/21 7
`(52) U.S. Cl. .............................................................. 348/241
`
`(57)
`
`ABSTRACT
`
`An apparatus and a method for efficiently executing correc
`tion of false color, such as purple fringe, caused by chro
`matic aberration and for generating and outputting high
`quality image data are provided. A white-saturated pixel is
`detected from image data, a false-color-pixel detection area
`is set around the detected white-saturated pixel, and pixels
`having color corresponding to false color Such as purple
`fringe are detected from the set area. The detected pixels are
`determined as false-color pixels and correction processing
`based on the values of the Surrounding pixels is performed
`on the determined false-color pixels. With this configuration,
`an area of false color Such as purple fringe generated in the
`neighborhood of a white-saturated pixel can be efficiently
`detected, pixel values can be partially corrected, and high
`quality image data can be generated and output without
`affecting the entire image.
`
`O2
`
`
`
`O
`
`
`
`
`
`1 O9
`
`11 O
`
`VIDEO
`ENCODER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SOLD-STATE
`IMAGE
`CAPTURE
`DECE
`
`CORRELATED
`DOUBLE
`SAMPLING
`CIRCUIT
`(CDS)
`
`DGITAL
`SIGNAL
`ROCESSOR
`(DSP)
`
`TIMING
`GENERATOR
`
`
`
`
`
`
`
`FLASH
`CONTROL
`UNIT
`
`Align Ex. 1008
`U.S. Patent No. 9,962,244
`
`0001
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 1 of 22
`
`US 2007/0035641 A1
`
`
`
`O 1 160 180 ||| '9)|–|
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TOHINOO
`
`LIN?)
`
`0002
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 2 of 22
`
`US 2007/0035641 A1
`
`FIG 2
`
`COLOR-SPACE CONVERSION PROCESSING
`
`S101
`
`
`
`S102
`WHITE-SATURATEDAREADETECTION PROCESSING
`(WHITE-SATURATION MASKDETERMINATION)
`
`FALSE-COLOR-PIXELDETECTION-AREA
`G (DiLATE PROCESSING)
`M
`S A
`
`w
`
`FALSE-COLOR PIXELDETECTION PROCESSING
`
`S104
`
`FALSE-COLOR PIXEL CORRECTION PROCESSING
`
`S105
`
`COLOR BLURPROCESSING
`(FILTERING)
`
`COLOR-SPACE INVERSE-CONVERSION
`PROCESSING
`
`S106
`
`S107
`
`END
`
`0003
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 3 of 22
`
`US 2007/0035641 A1
`
`0004
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 4 of 22
`
`US 2007/0035641 A1
`
`FIG. 4
`
`
`
`
`
`S2O4.
`
`S2O5
`S(x, y) = false
`
`
`
`Yes
`
`0005
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 5 of 22
`
`US 2007/0035641 A1
`
`FIG. 5
`
`START
`
`INTIALIZATION OFFALSECOLORAREAMASKPS3Ol
`
`y R O
`
`S3O4 No
`No
`Yes
`S305
`DETERMINATION OFAREA(x0, x,y0,y)
`
`y O
`
`S3O8 No
`S(i, j) = false ?
`Yes.
`P(i, j) = true
`
`S309
`
`e -- 1
`s
`
`Yes
`j=j- 1
`
`S31 O
`
`S3 1
`
`S32
`
`Yes
`xm. x - 1
`
`sw-
`Yes
`yety
`
`S34
`
`S3 15
`
`S316
`
`No
`
`No
`
`Ng5-2ds
`
`Yes
`END
`
`0006
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 6 of 22
`
`US 2007/0035641 A1
`
`ZOZ
`
`
`
`0007
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 7 of 22
`
`US 2007/0035641 A1
`
`FIG. 7
`
`
`
`
`
`
`
`P(x, y) true?
`
`S4O4.
`/ C (x, y) IS COLOR Y.
`CORRESPONDINGTOFALSE)
`Y COLOR (PURPLE)?
`
`si
`
`0008
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 8 of 22
`
`US 2007/0035641 A1
`
`
`
`g
`
`0009
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 9 of 22
`
`US 2007/0035641 A1
`
`
`
`O
`O
`
`y
`
`CN
`
`0010
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 10 of 22
`
`US 2007/0035641 A1
`
`
`
`
`
`TEXld (EÐNIHH ETd8nd) HOTOO ESTWH ; zos
`
`
`
`
`
`%
`
`
`
`
`
`
`
`TEXld (EÐNIN- EldHnd) (JOTOO ESTW+;zoe[…]
`
`
`
`
`
`
`
`
`
`TE}X|d èJEH10: coc
`
`0011
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 11 of 22
`
`US 2007/0035641 A1
`
`FIG. 11
`
`START
`
`p P
`
`S505
`
`S506 No
`P(x, y)=true?
`
`
`
`IS THERE PIXELTHAT
`/ SATISFIESP'(i,j) = FALSE
`K AND S(, )= FALSE INEIGHT
`PXELS (i,j) OF(x, y)?
`Yes
`(x,y) INTERPOLATION PROCESSINGS508
`
`-
`
`NEIGHBORHOOD
`
`W
`
`P" (x, y)=false
`
`x Ex
`
`S5O9
`S51O.
`
`Yes
`y=y+ 1
`
`sh-
`Yes
`
`S512
`
`S513.
`S514
`
`No
`
`No
`
`S5 15
`T > timax OR
`THE NUMBER OF PIXELS THAT
`YN SATISFYP" = TRUE ISO
`Yes
`S516
`
`0012
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 12 of 22
`
`US 2007/0035641 A1
`
`
`
`58
`P" (x, y) = true?
`
`No
`
`Yes
`
`SS 19
`C(x,y)COLORSATURATION
`REDUCTION PROCESSING
`
`S52O
`
`x = x - 1
`
`No
`
`S52
`
`Yes
`
`S522
`
`No
`
`S523
`
`Yes
`
`0013
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 13 of 22
`
`US 2007/0035641 A1
`
`FIG. 13
`
`
`
`Yes
`
`No.
`
`siz S61 O
`Yes
`
`S6
`
`No
`
`- S612
`
`Yes
`END
`
`0014
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 14 of 22
`
`US 2007/0035641 A1
`
`||Nf]
`
`NOIS?-JEANOO
`
`LINTI NOILOBIEC]
`
`
`
`(E?NIHH ETd8 md)
`
`
`
`>JOTOO EST\/-|
`
`
`
`
`
`
`
`
`
`
`0015
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 15 of 22
`
`US 2007/0035641 A1
`
`
`
`• • • • • • • • • • • • •O L• • • • • • • • • • ** * * *O |
`
`G92
`
`(e)(HONWN|WnTTW?LOW)
`
`0016
`
`

`

`mD1
`
`n.m
`
`m
`
`2
`
`w,WIImax:0wmmNFAmVmmm
`
`.mmmN
`
`PIoom
`
`
`m92223::53
`
`
`
`1mE-883.mmmzzozm558Amoz<z__>_3.2:5$.mmwszoEmSwagmrO_n_
`
`nf061
`
`US 2007/0035641 A1
`
`fidxi08m.
`
`Amfixio:<
`
`\L|J
`
`w._m_x_n_
`
`“mamEUmmu
`
`o
`
`>>
`
`Ifllj
`
`7maxim0mix:2
`
`
`«\hvm._mx_n_omm5._<>moz<2=>5._
`
`wflag62223moédmzaéooszmv
`
`on
`
`0017
`
`0017
`
`
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 17 of 22
`
`US 2007/0035641 A1
`
`
`
`
`
`992
`
`STEX?d
`
`992
`
`STEXldO
`
`9
`
`0018
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 18 of 22
`
`US 2007/0035641 A1
`
`
`
`(TEXld QBIVADIWS BLIHM
`
`
`1OB?80S -JO E ONWN|WQT TWO LOW 000G009
`JO BONWNIWn i EnHI)
`
`
`
`BONV8 BTGV-JENEO (dd) (JOTOO-ESTWA
`
`
`
`
`STBXld JO HEGWN EH1)
`
`NO||W}}[\.|\/S E 1?HM WOH-]
`
`0019
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 19 of 22
`
`US 2007/0035641 A1
`
`°N vogs
`
`eg?ogs
`
`se Å
`
`2, en u?.= º§§
`
`C|GOSS
`
`
`
`
`
`
`
`
`
`0020
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 20 of 22
`
`US 2007/0035641 A1
`
`
`
`0021
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 21 of 22
`
`US 2007/0035641 A1
`
`
`
`
`
`E5)\/W|| E.ONE|}}}-{E}}
`
`
`
`
`
`QEHdWH9OLOHd BOWWI HO BOWINI NOILOTOSEH-MOT)
`
`ol v
`
`| || 17
`
`Z || ?
`
`0022
`
`

`

`Patent Application Publication Feb. 15, 2007 Sheet 22 of 22
`
`US 2007/0035641 A1
`
`9 Otz
`
`
`
`E0WW|| E.ONERHE-E'}}
`
`HOE@WWINOLTTOSEH-MOT)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`0023
`
`

`

`US 2007/0035641 A1
`
`Feb. 15, 2007
`
`IMAGE PROCESSINGAPPARATUS, IMAGE
`PROCESSING METHOD, AND COMPUTER
`PROGRAM
`
`TECHNICAL FIELD
`0001. The present invention relates to image processing
`apparatuses, image processing methods, and computer pro
`grams. Moe specifically, the present invention relates to an
`image processing apparatus, an image processing method,
`and a computer program for generating high-quality image
`data by correcting image data in which false color is
`generated, for example, due to lens aberration.
`
`BACKGROUND ART
`0002 For photographing with a camera, various prob
`lems are caused by lens aberration. Examples of typical
`chromatic aberration include Seidel's five aberrations,
`which are monochromatic aberrations. This is five aberra
`tions analyzed by Seidel in Germany and based on a lens
`sphere and is a collective term of spherical aberration, coma
`aberration, astigmatism aberration, distortion aberration,
`and field curvature. Apart from those aberrations, it is known
`that chromatic aberration can also cause a serious problem.
`The chromatic aberration results from the fact that the
`refractive index of light with respect to lens material varies
`due to its wavelength, and produces false color at an image
`capture Surface.
`0003) As typical examples of chromatic aberration, axial
`chromatic aberration, which causes color blur since a focal
`point position on an optical axis varies depending on the
`wavelength, and magnification chromatic aberration, which
`causes color shift since the image magnification varies
`depending on the wavelength, are well known. In general, a
`phenomenon called "purple fringe (purple tringe) in
`English is also an important cause of image deterioration.
`This is a phenomenon in which false color is generated at an
`edge portion in an image because of a difference in point
`spread due to a light wavelength. Even when it is not so
`visible among typical pixels, the presence of a high-contrast
`edge portion where white Saturation, i.e., a state in which the
`luminance level is Saturated, occurs causes purple false color
`to be generated around the edge portion, thus forming an
`unnatural image. In general, false color generated in the
`neighborhood of a white-saturated portion is called “purple
`fringe' since a large amount of purple-based color is gen
`erated. However, the false color may be any colors, such as
`green-tined color, depending on a lens and photography
`conditions. Hereinafter, the “purple fringe” refers to a false
`color generation phenomenon that occurs at a high-contrast
`edge portion where white Saturation occurs, regardless of
`color generated.
`0004 As a technology for reducing color aberration,
`lenses using glass made of special material. Such as fluorite,
`are available. However, since high cost is required to manu
`facture such lenses, the price is also high. Such lenses are
`used for some high-class cameras, such as lens-replaceable
`cameras, but cannot be said to be widely used.
`0005 Patent Document 1 describes a method for reduc
`ing false color generated due to chromatic aberration by
`performing image processing. In this processing, processing
`for Suppressing color, i.e., reducing color Saturation, is
`performed on a portion where green-channel high-frequency
`
`components are high. As measures for a case in which white
`saturation occurs, Patent Document 1 also discloses a con
`figuration for performing processing for reducing the color
`saturation of a white-saturated portion by photographing two
`images with varied exposure and estimating the original
`luminance of the white-saturated portion.
`0006. However, in the processing described in Patent
`Document 1, since the false color is reduced by reducing the
`color Saturation, the color Saturation of original Subject color
`is also reduced to thereby make it impossible to faithfully
`reproduce the original color of the Subject. As a result, there
`is a problem in that an image that looks unnatural is output.
`Additionally, in order to estimate the luminance of a white
`saturated portion, photography needs to be performed twice.
`When hand movement or subject movement occurs during
`the two photography operations, there is a problem in that it
`is difficult to obtain a correct result.
`0007
`Patent Document 1 Japanese Unexamined Patent
`Application Publication No. 2003-60983
`
`DISCLOSURE OF INVENTION
`0008. The present invention has been made in view of the
`problems described above, and an object of the present
`invention is to provide an image processing apparatus, an
`image processing method, and a computer program which
`allow high-quality image data to be generated and output
`without affecting the entire image, by paying attention to
`false color Such as purple fringe generated around a white
`saturated portion, efficiently detecting the false color area,
`and executing partial correction.
`0009. A first aspect of the present invention provides an
`image processing apparatus.
`0010 The image processing apparatus has:
`0011 a white-saturation detection unit for detecting a
`white Saturated pixel from image data;
`0012 a false-color-pixel detection-area setting unit for
`setting a false-color-pixel detection area around the white
`saturated pixel detected by the white-saturation detection
`unit;
`0013 a false-color detection unit for identifying pixels
`having color corresponding to false color as false-color
`pixels in the area set by the false-color-pixel detection-area
`setting unit; and
`0014) a pixel-value correction unit for executing pixel
`value correction processing, based on values of Surrounding
`pixels, on the false-color pixels detected by the false-color
`detection unit.
`0015. In addition, according to one embodiment of the
`image rocessing apparatus of the present invention, the
`pixel-value correction unit has a compensation interpolation
`unit for executing compensation interpolation processing,
`based on the Surrounding pixel values, on the false-color
`pixels, and a color blur processing unit for executing color
`blur processing on the false-color pixels.
`0016.
`Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`pixel-value correction unit selects pixels other than false
`color pixels and white-saturated pixels from pixels that exist
`
`0024
`
`

`

`US 2007/0035641 A1
`
`Feb. 15, 2007
`
`around the false-color pixels and executes compensation
`interpolation processing based on the selected pixels.
`0017 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color detection unit executes processing for identify
`ing, as false-color pixels, pixels having preset specific color
`in the area set by the false-color-pixel detection-area setting
`unit.
`0018. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false color is purple fringe, and the false-color detection unit
`executes processing for identifying, as false-color pixels,
`purple pixels in the area set by the false-color-pixel detec
`tion-area setting unit.
`0019. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area setting unit executes pro
`cessing for determining the false-color-pixel detection area
`set around the white-saturated pixel detected by the white
`saturation detection unit, in accordance with at least one
`piece of data of a diaphragm during photography of image
`data to be processed, focal distance information, and a
`distance from an optical center to a white-saturated pixel (X,
`y).
`0020 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`white-saturation detection unit executes processing for
`selecting a white-saturated pixel having a luminance higher
`than or equal to a predetermined threshold.
`0021 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`image processing apparatus further has a color conversion
`unit for executing data conversion processing for separating
`input image data into luminance-component image data and
`color-component image data, and executes pixel-value cor
`rection processing based on the converted data generated by
`the color conversion unit.
`0022. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area setting unit has a reference
`image analyzing unit for receiving a reference image in
`which a pixel at a position corresponding to a white
`saturated pixel in an image to be processed does not have a
`saturation luminance value and obtaining a luminance value
`of the reference-image pixel corresponding to the white
`saturated pixel in the image to be processed. The reference
`image is a photographed image of the same Subject as the
`image to be processed. The false-color-pixel detection-area
`setting unit further has a false-color-pixel detection-area
`determining unit for setting a false-color-pixel detection area
`in accordance with the luminance value of the corresponding
`reference-image pixel, the luminance value being obtained
`by the reference-image analyzing unit.
`0023. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area determining unit sets a false
`color-pixel detection area by using a lookup table in which
`the luminance value of the reference image and a false-color
`generable pixel range or the false-color-pixel detection area
`are associated with each other.
`
`0024. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area determining unit executes
`processing for setting a wider false-color-pixel detection
`area as the luminance value of a corresponding pixel in the
`reference image increases.
`0025 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, by
`using, as the reference image, an image photographed by an
`image capture device having a low-sensitivity sensor or an
`image photographed under a condition of a low exposure
`less than an adequate exposure, the false-color-pixel detec
`tion-area determining unit obtains a reference-image lumi
`nance -value of a portion corresponding to a white-saturated
`pixel in an image to be processed.
`0026.
`In addition, a second aspect of the present inven
`tion provides an image processing method. The image
`processing method includes:
`0027 a white-saturation detecting step of detecting a
`white Saturated pixel from image data;
`0028 a false-color-pixel detection-area setting step of
`setting a false-color-pixel detection area around the white
`saturated pixel detected in the white-saturation detecting
`step;
`0029 a false-color detecting step of identifying pixels
`having color corresponding to false color as false-color
`pixels in the area set in the false-color-pixel detection-area
`setting step; and
`0030) a pixel-value correcting step of executing pixel
`value correction processing, based on values of Surrounding
`pixels, on the false-color pixels detected in the false-color
`detecting step.
`0031 Additionally, according to one embodiment of the
`present invention, the pixel-value correcting step has: a
`compensation interpolation step of executing compensation
`interpolation processing, based on the Surrounding pixel
`values, on the false-color pixels; and a color blur processing
`step of executing color blur processing on the false-color
`pixels.
`0032. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`pixel-value correcting step is a step of selecting pixels other
`than false-color pixels and white-saturated pixels from pix
`els that exist around the false-color pixels and executing
`compensation interpolation processing based on the selected
`pixels.
`0033 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color detecting step is a step of executing processing
`for identifying, as false-color pixels, pixels having preset
`specific color in the area set in the false-color-pixel detec
`tion-area setting step.
`0034 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false color is purple fringe, and the false-color detecting step
`is a step of executing processing for identifying, as false
`color pixels, purple pixels in the area set in the false-color
`pixel detection-area setting step.
`
`0025
`
`

`

`US 2007/0035641 A1
`
`Feb. 15, 2007
`
`0035). Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area setting step is a step of
`executing processing for determining the false-color-pixel
`detection area set around the white-saturated pixel detected
`in the white-saturation detecting step, in accordance with at
`least one piece of data of a diaphragm during photography
`of image data to be processed, focal distance information,
`and a distance from an optical center to a white-saturated
`pixel (x, y).
`0036) Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`white-saturation detecting step is a step of executing pro
`cessing for selecting a white-saturated pixel having a lumi
`nance higher than or equal to a predetermined threshold.
`0037 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`image processing method further has a color converting step
`of executing data conversion processing for separating input
`image data into luminance-component image data and color
`component image data and of executing pixel-value correc
`tion processing based on the converted data generated by the
`color conversion unit.
`0038. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area setting step having: a refer
`ence-image analyzing step of inputting a reference image in
`which a pixel at a position corresponding to a white
`saturated pixel in an image to be processed does not have a
`saturation luminance value and obtaining a luminance value
`of the reference-image pixel corresponding to the white
`saturated pixel in the image to be processed, the reference
`image being a photographed image of the same Subject as
`the image to be processed; and a false-color-pixel detection
`area determining step of setting a false-color-pixel detection
`area in accordance with the luminance value of the corre
`sponding reference-image pixel, the luminance value being
`obtained in the reference-image analyzing step.
`0039. Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area determining step is a step of
`setting a false-color-pixel detection area by using a lookup
`table in which the luminance value of the reference image
`and a false-color generable pixel range or the false-color
`pixel detection area are associated with each other.
`0040 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, the
`false-color-pixel detection-area determining step is a step of
`executing processing for setting a wider false-color-pixel
`detection area as the luminance value of a corresponding
`pixel in the reference image increases.
`0041 Additionally, according to one embodiment of the
`image processing apparatus of the present invention, in the
`false-color-pixel detection-area determining step, an image
`photographed by an image capture device having a low
`sensitivity sensor or an image photographed under a condi
`tion of a low exposure less than an adequate exposure is used
`as the reference image, and a reference-image luminance
`value of a position corresponding to a white-saturated pixel
`in an image to be processed is obtained.
`
`0042. In addition, a third aspect of the present invention
`provides a computer program for causing a computer to
`execute image processing. The program has:
`0043 a white-saturation detecting step of detecting a
`white Saturated pixel from image data;
`0044) a false-color-pixel detection-area setting step of
`setting a false-color-pixel detection area around the white
`saturated pixel detected in the white-saturation detecting
`step;
`0045 a false-color detecting step of identifying pixels
`having color corresponding to false color as false-color
`pixels in the area set in the false-color-pixel detection-area
`setting step; and
`0046) a pixel-value correcting step of executing pixel
`value correction processing, based on values of Surrounding
`pixels, on the false-color pixels detected in the false-color
`detecting step.
`0047. Additionally, according to one embodiment of the
`present invention, the false-color-pixel detection-area set
`ting step includes: a reference-image analyzing step of
`inputting a reference image in which a pixel at a position
`corresponding to a white-saturated pixel in an image to be
`processed does not have a saturation luminance value and
`obtaining a luminance value of the reference-image pixel
`corresponding to the white-saturated pixel in the image to be
`processed, the reference image being a photographed image
`of the same Subject as the image to be processed; and a
`false-color-pixel detection-area determining step of setting a
`false-color-pixel detection area in accordance with the lumi
`nance value of the corresponding reference-image pixel, the
`luminance value being obtained in the reference-image
`analyzing step.
`0048. The computer program of the present invention can
`be supplied to, for example, a general-purpose computer
`system that can execute various program codes via a storage
`media, such as a CD, FD, or MO in a computer-readable
`format, or through a communication medium, Such as a
`network. Supplying Such a program in a computer-readable
`format can cause a computer to achieve processing accord
`ing to the program.
`0049 Further objects, features, and advantages of the
`present invention will become apparent from more detailed
`description based on embodiments described below accord
`ing to the present invention and the accompanying drawings.
`The term "system' herein refers to a logical combination of
`a plurality of apparatuses and is not limited to a system in
`which individual apparatus are included in the same hous
`1ng.
`0050. According to the configuration of the present
`invention, a white-saturated pixel is detected from image
`data, a false-color-pixel detection area is set around the
`detected white-saturated pixel, pixels having color corre
`sponding to false color Such as purple fringe are detected
`from the set area, the detected pixels are determined as
`false-color pixels, and correction processing based on the
`values of the surrounding pixels is executed on the identified
`false-color pixels. Thus, it is possible to efficiently detect the
`area of false color, Such as purple fringe, generated in the
`neighborhood of a white-saturated pixel and to partially
`
`0026
`
`

`

`US 2007/0035641 A1
`
`Feb. 15, 2007
`
`correct the values of pixels. It is also possible to generate and
`output high-quality image data without affecting the entire
`image.
`0051. According to the configuration of the present
`invention, it is possible to perform appropriate correction on
`false color, such as purple fringe, caused by chromatic
`aberration occurred to an image photographed by a camera,
`and it is possible to achieve the generation and output of
`high-quality image data. With a conventional camera, there
`is a possibility that an unnatural image is photographed
`unless settings, such as a lens diaphragm, are adjusted in a
`situation in which purple fringe is generated. However,
`when the present invention is applied, it is possible to
`efficiently extract and correct false color, Such as purple
`fringe, contained in a photographed image and it is possible
`to generate and output a high-quality image. Thus, there is
`no need to pay attention to the lens diaphragm and the focal
`distance Such that purple fringing does not occur during
`photography, thus making it possible to perform photogra
`phy with a higher degree of freedom.
`0.052
`In addition, according to the configuration of the
`present invention, a subject actual luminance corresponding
`a white-saturated pixel is estimated based on a reference
`image, a false color (purple fringe) detection area corre
`sponding to the Subject actual luminance is set, and false
`color (purple fringe) pixels in the set area are detected and
`corrected. Thus, a false color (purple fringe) detection area
`corresponding to an area in which false color (purple fringe)
`is likely to be generated can be set, without the set false color
`(purple fringe) detection area becoming too large or too
`small. False color (purple fringe) pixels can be reliably and
`efficiently detected, the accuracy of image correction is
`increased, and the efficiency is increased.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0053 FIG. 1 is a diagram showing the configuration of an
`image processing apparatus of the present invention.
`0054 FIG. 2 is a flow chart illustrating the entire pro
`cessing sequence of an image processing method of the
`present invention.
`0.055
`FIG. 3 is a diagram illustrating the configuration of
`image data to be corrected by the image processing appa
`ratus of the present invention.
`0056 FIG. 4 is a detailed sequence of a white-saturated
`area detection processing executed in the image processing
`of the present invention.
`0057 FIG. 5 is a flow chart illustrating a detailed
`sequence of a false-color-pixel detection-area setting pro
`cessing executed in the image processing of the present
`invention.
`0.058
`FIG. 6 is a diagram illustrating a processing
`example of the false-color-pixel detection-area setting pro
`cessing executed in the image processing of the present
`invention.
`0059 FIG. 7 is a flow chart illustrating a detailed
`sequence of a false-color-pixel detection processing
`executed in the image processing of the present invention.
`0060 FIG. 8 is a diagram illustrating a processing
`example (a case in which CIE L*a*b* is used) of the
`
`false-color-pixel detection processing executed in the image
`processing of the present invention.
`0061
`FIG. 9 is a diagram illustrating a processing
`example (a case in which YCbCr is used) of the false-color
`pixel detection processing executed in the image processing
`of the present invention.
`0062 FIG. 10 includes diagrams illustrating a processing
`example of a false-color-pixel correction processing
`executed in the image processing of the present invention.
`0063 FIG. 11 is a flow chart illustrating a detailed
`sequence of a false-color-pixel correction processing
`executed in the image processing of the present invention.
`0064 FIG. 12 is a flow chart illustrating a detailed
`sequence of the false-color-pixel correction processing
`executed in the image processing of the present invention.
`0065 FIG. 13 is a flow chart illustrating a detailed
`sequence of color blur processing executed in the image
`processing of the present invention.
`0066 FIG. 14 is a block diagram illustrating the func
`tional configuration of a digital signal processing for execut
`ing false-color correction according to a first embodiment, in
`the image processing apparatus of the present invention.
`0067 FIG. 15 includes graphs illustrating the relationship
`a Subject actual luminancefan edge configuration and the
`generation of false color (purple fringe).
`0068 FIG. 16 includes graphs illustrating a subject actual
`luminance, an edge configuration, a photographed-image
`luminance, and the generation of false color (purple fringe).
`0069 FIG. 17 includes graphs illustrating the luminance
`information of a reference image obtained using, for
`example, a low-sensitivity sensor to estimate a subject actual
`luminance.
`0070 FIG. 18 includes graphs illustrating the relationship
`between a subject actual luminance and the generation range
`of false color (purple fringe).
`0071
`FIG. 19 is a flow chart illustrating a processing
`procedure for setting a false-color detection area in a second
`embodiment.
`0072 FIG. 20 is a table showing an example of the
`structure of a lookup table (LUT) used in the processing for
`setting the false-color detection area in the second embodi
`ment.
`0073 FIG. 21 is a block diagram illustrating the func
`tional configuration of a digital signal processing for execut
`ing false-color correction according to the second embodi
`ment, in the image processing apparatus of the present
`invention.
`0074 FIG. 22 is a block diagram illustrating the configu
`ration of a false-color-pixel detection-area setting unit in the
`digital signal processing for executing false-color correction
`according to the second embodiment, in the image process
`ing apparatus of the present invention.
`
`BEST MODE FOR CARRYING OUT THE
`INVENTION
`0075 Details of an image processing apparatus, an image
`processing method, and a computer program according to
`
`0027
`
`

`

`US 2007/0035641 A1
`
`Feb. 15, 2007
`
`the present invention will be described below with reference
`to the accompanying drawings.
`0076)
`First Embodiment
`0.077
`First, an example of the configuration of an image
`processing apparatus will be described with reference to
`FIG.1.