111111
`
`1111111111111111111111111111111111111111111111111111111111111111111111111111
`US 20150029367Al
`
`(19) United States
`c12) Patent Application Publication
`TANAKA
`
`(10) Pub. No.: US 2015/0029367 A1
`Jan. 29, 2015
`(43) Pub. Date:
`
`(54) COLOR IMAGING APPARATUS
`
`Publication Classification
`
`(71) Applicant: FUJIFILM Corporation, TOKYO (JP)
`
`(72)
`
`Inventor: Seiji TANAKA, Saitama-shi (JP)
`
`(73) Assignee: FUJIFILM CORPORATION, TOKYO
`(JP)
`
`(21) Appl. No.: 14/501,949
`
`(22) Filed:
`
`Sep.30,2014
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 13/560,458, filed on
`Jul. 27, 2012, now Pat. No. 8,878,970, which is a
`continuation of application No. PCT/JP2011/067420,
`filed on Jul. 29, 2011.
`
`(30)
`
`Foreign Application Priority Data
`
`Feb.28,2011
`Jul. 25, 2011
`
`(JP) ................................. 2011-042831
`(JP) ................................. 2011-162413
`
`(51)
`
`(2006.01)
`
`Int. Cl.
`H04N9/04
`(52) U.S. Cl.
`CPC .......... H04N9/045 (2013.01); H04N 2209/045
`(2013.01)
`USPC .. .. ... ... ... ... ... .. ... ... ... ... .. ... ... ... ... ... .. ... ... 348/280
`ABSTRACT
`(57)
`A color imaging apparatus comprising: a color imaging ele(cid:173)
`ment comprising a plurality of pixels and color filters of a
`color filter array arranged on the plurality of pixels, the color
`filter array including first filters corresponding to a first color
`that most contributes to obtaining luminance signals and sec(cid:173)
`ond filters corresponding to two or more second colors, and
`the first filters including two or more sections adjacent each
`other in horizontal, vertical, and oblique directions; a direc(cid:173)
`tion determination unit acquiring pixel values of pixels of the
`two or more sections of the first filters near a target pixel of
`demosaicking processing and determining a correlation
`direction of luminance; a demosaicking processing unit that
`calculates a pixel value of another color at a pixel position of
`the target pixel and that uses one or more pixels of another
`color in the correlation direction to calculate the pixel value.
`
`18
`~
`
`DRIVE
`UNIT
`
`l \2
`
`1~4
`
`20
`~
`
`CONTROL
`UNIT
`
`IMAGE
`IMAGE
`COLOR
`PROCESSING
`f---+ (SYNCHRONIZATION
`IMAGING ~PROCESSING
`ELEMENT
`UNIT
`
`l \6
`PROCESSING) -o
`
`OUTP UT
`E I MAG
`
`
`1~0
`
`IMAGING
`r--
`OPTICAL
`SYSTEM
`
`Align Ex. 1010
`U.S. Patent No. 9,962,244
`
`0001
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 1 of 9
`
`US 2015/0029367 A1
`
`FIG.1
`
`1)0
`
`18
`~
`
`DRIVE
`UNIT
`
`_L \2
`
`1)4
`
`20
`~
`
`CONTROL
`UNIT
`
`IMAGE
`IMAGE
`COLOR
`IMAGING
`PROCESSING
`r---. (SYNCHRONIZATION
`OPTICAL f-tJ IMAGING f-tJ PROCESSING
`SYSTEM ELEMENT
`UNIT
`
`l 116
`PROCESSING) -o
`
`OUTP UT
`E
`I MAG
`
`FIG.2
`
`HORIZONTAL
`DIRECTION
`
`0002
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 2 of 9
`
`US 2015/0029367 A1
`
`FIG.3
`
`A ARRAY
`
`BARRAY
`
`FIG.4
`
`0003
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 3 of 9
`
`US 2015/0029367 A1
`
`FIG.5
`
`FIG.6
`
`FIG.7A
`
`0004
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 4 of 9
`
`US 2015/0029367 A1
`
`FIG.78
`
`FIG.8
`
`A ARRAY
`
`BARRAY
`
`0005
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 5 of 9
`
`US 2015/0029367 A1
`
`FIG.9
`
`FIG.10
`
`0006
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 6 of 9
`
`US 2015/0029367 A1
`
`FIG.11
`
`0007
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 7 of 9
`
`US 2015/0029367 A1
`
`FIG.12
`
`0008
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 8 of 9
`
`US 2015/0029367 A1
`
`FIG.13
`
`0009
`
`

`

`Patent Application Publication
`
`Jan. 29, 2015 Sheet 9 of 9
`
`US 2015/0029367 A1
`
`.
`(9
`LL
`
`0010
`
`

`

`US 2015/0029367 AI
`
`Jan.29,2015
`
`1
`
`COLOR IMAGING APPARATUS
`
`[0001] This application is a Continuation of copending
`application Ser. No. 13/560,458, filed on Jul. 27,2012, which
`is a Continuation application and claims the priority benefit
`under 35 U.S.C. §120 ofPCT Application No. PCT/JP2011/
`067420 filed on Jul. 29, 2011 which application designates
`the U.S., and also claims the priority benefit under 35 U.S.C.
`§119 ofJapanese Patent Applications No. 2011-042831 filed
`on Feb. 28, 2011 and No. 2011-162413 filed on Jul. 25, 2011,
`which applications are all hereby incorporated in their entire(cid:173)
`ties by reference.
`
`TECHNICAL FIELD
`
`[0002] The present invention relates to a color imaging
`apparatus, and particularly, to a color imaging apparatus that
`can suppress generation of color moire.
`
`BACKGROUND ART
`
`In a color imaging apparatus including a single(cid:173)
`[0003]
`plate color imaging element, an output image from the color
`imaging element is a RAW image (mosaic image). Therefore,
`a multi-channel image is obtained by a process of interpolat(cid:173)
`ing a pixel of a missing color from a surrounding pixel (demo(cid:173)
`saicing processing). In this case, there is a problem in repro(cid:173)
`duction characteristics of a high-frequency image signal.
`[0004] A primary-color Bayer array as a color array most
`widely used in the single-plate color imaging element
`includes green (G) pixels arranged in a check pattern and red
`(R) and blue (B) arranged line-sequentially. Therefore, there
`is a problem oflow-frequency coloring (color moire) caused
`by folding of high frequency signals exceeding reproduction
`bands of the colors and caused by deviation of phases of the
`colors.
`[0005] For example, a black and white oblique high fre(cid:173)
`quency image as shown in FIG. 14(A) enters an imaging
`element in a Bayer array shown in FIG. 14(B), and the image
`is sorted into Bayer color arrays to compare the colors. As
`shown in FIGS. 14(C) to 14(E), Rand B form light and flat
`color images, while G forms a dark and flat color image.
`Assuming that the value ofblack is 0 and the value of white is
`255, the black and white oblique high frequency image turns
`green, because only G is 255. In this way, the oblique high
`frequency image cannot be correctly reproduced in the Bayer
`array.
`In the color imaging apparatus using the single-plate
`[0006]
`color imaging element, an optical low-pass filter formed by
`an anisotropic substance such as crystal is generally arranged
`on the front side of the color imaging element to prevent
`optically reducing the high frequency wave. However,
`although the coloring caused by folding of the high frequency
`signal can be reduced in the method, there is a problem that
`the resolution is reduced accordingly.
`[0007] To solve the problem, a color imaging element is
`proposed, wherein a color filter array of the color imaging
`element is a three-color random array satisfYing array restric(cid:173)
`tions in which an arbitrary target pixel is adjacent to three
`colors including the color of the target pixel on four sides of
`the target pixel (PTL 1 ).
`[0008] An image sensor of a color filter array is also pro(cid:173)
`posed, wherein the image sensor includes a plurality of filters
`with different spectral sensitivity, and first and second filters
`among the plurality of filters are alternately arranged in a first
`predetermined period in one of the diagonal directions of a
`
`pixel grid of the image sensor and are alternately arranged in
`a second predetermined period in the other diagonal direction
`(PTL 2).
`[0009] Meanwhile, PTL 3 describes a technique of using
`surrounding pixels of a target pixel of a mosaic image in a
`Bayer array to calculate correlations in horizontal, vertical,
`and oblique (NE, NW) directions (four directions), and
`weights are applied according to ratios of the calculated cor(cid:173)
`relations to interpolate the pixels.
`[0010] An image apparatus including a color imaging ele(cid:173)
`ment is also proposed, wherein R and B among the three
`primary colors of RGB are arranged every three pixels in
`horizontal and vertical directions, and G is arranged between
`Rand B (PTL 4). In the color imaging element described in
`PTL 4, G pixels that most contribute to obtaining luminance
`signals are arranged much more than RB pixels, on the
`ground that the resolution of color difference signals can be
`lower than the resolution of the luminance signals. This can
`increase the resolution in the horizontal and vertical direc(cid:173)
`tions.
`
`CITATION LIST
`
`Patent Literature
`
`{PTL 1}
`Japanese Patent Application Laid-Open No. 2000-
`
`[0011]
`[0012]
`308080
`[0013]
`[0014]
`136766
`[0015]
`[0016]
`104019
`[0017] {PTL 4}
`[0018]
`Japanese
`8-23543
`
`{PTL 2}
`Japanese Patent Application Laid-Open No. 2005-
`
`{PTL 3}
`Japanese Patent Application Laid-Open No. 2010-
`
`Patent Application Laid-Open No.
`
`SUMMARY OF INVENTION
`
`Technical Problem
`
`[0019] The three-color random array described in PTL 1 is
`effective for low-frequency color moire, but is not effective
`for a false color of a high frequency section.
`In the color filter array of the image sensor described
`[0020]
`in PTL 2, the R, G, and B filters are periodically arranged in
`the lines in the horizontal and vertical directions of the color
`filter array. In demosaicing processing of a mosaic image
`output from the image sensor including the color filter array in
`the invention described in PTL 2, a local area in a predeter(cid:173)
`mined image size is extracted around the target pixel, statis(cid:173)
`tics related to a color distribution shape of the color of the
`target pixel in the local area and a color distribution shape of
`another color to be estimated are calculated, and the color
`distribution shapes are linearly regressed based on the inten(cid:173)
`sity of the colors at the target pixel position and the statistics
`of the color distribution shapes to thereby calculate an esti(cid:173)
`mation value of the another color at the target pixel position.
`The calculation of the statistics (covariance values) related to
`the color distribution shapes and the regression calculation
`process are necessary in the invention described in PTL 2, and
`there is a problem that the image processing is complicated.
`[0021] Meanwhile,
`the pixel
`interpolation method
`described in PTL 3 is applied to a mosaic image in a Bayer
`array. However, G pixels are not consecutive in the horizontal
`
`0011
`
`

`

`US 2015/0029367 AI
`
`Jan.29,2015
`
`2
`
`and vertical directions in the Bayer array, and the correlations
`in the horizontal and vertical directions cannot be calculated
`at minimum pixel intervals. For example, the correlations are
`falsely determined when a vertical-striped or horizontal(cid:173)
`striped high frequency wave is input, and there is a problem
`that the pixels cannot be accurately interpolated.
`[0022] The ratio of the G pixels relative to the RB pixels in
`the color imaging element described in PTL 4 is higher than
`the ratio in the Bayer array, and the resolution in the horizon(cid:173)
`tal and vertical directions can be increased. However, the
`RGB signals output from the color imaging element are uni(cid:173)
`formly synchronized by interpolation filters in the imaging
`apparatus described in PTL 4, and there is a problem that a
`false color is easily generated.
`[0023] The present invention has been made in view of the
`circumstances, and an object of the present invention is to
`provide a color imaging apparatus that can suppress genera(cid:173)
`tion of a false color of a high frequency section by simple
`image processing.
`
`Solution to Problem
`
`[0024] To attain the object, an invention according to an
`aspect of the present invention includes: a single-plate color
`imaging element including: a plurality of pixels including
`photoelectric conversion elements arranged in horizontal and
`vertical directions; and color filters of a predetermined color
`filter array arranged on the plurality of pixels, wherein the
`color filter array includes first filters corresponding to a first
`color that most contributes to obtaining luminance signals
`and second filters corresponding to two or more second colors
`other than the first color, the first and second filters being
`periodically arranged, and the first filters include two or more
`sections adjacent to each other in horizontal, vertical, and
`oblique (NE, NW) directions; an image acquisition unit that
`acquires a mosaic image corresponding to the color filter
`array from the color imaging element; a direction determina(cid:173)
`tion unit that acquires, for a target pixel of demosaicing pro(cid:173)
`cessing extracted from the mosaic image, pixel values of
`pixels corresponding to the first filters near the target pixel
`and adjacent to each other in the horizontal, vertical, and
`oblique (NE, NW) directions and that determines which one
`of the horizontal, vertical, and oblique (NE, NW) directions is
`a correlation direction ofluminance based on the pixel values
`of the adjacent pixels; a demosaicing processing unit that
`calculates a pixel value of another color at a pixel position of
`the target pixel of the demosaicing processing extracted from
`the mosaic image and that uses one or more pixel values of
`one or more pixels of another color in the correlation direction
`determined by the direction determination unit to calculate
`the pixel value; and a control unit that repeatedly operates the
`direction determination unit and the demosaicing processing
`unit while shifting the target pixel of the demosaicing pro(cid:173)
`cessing extracted from the mosaic image by a target pixel unit
`of the demosaicing processing.
`[0025] The color filter array of the color imaging element
`includes sections where two or more first filters that most
`contribute to obtaining the luminance signals are adjacent to
`each other in the horizontal, vertical, and oblique (NE, NW)
`directions (four directions). Therefore, which one of the four
`directions is the correlation direction of luminance can be
`determined at minimum pixel intervals based on the pixel
`values of the pixels adjacent to each other in the directions. In
`the calculation of the pixel value of another color at the pixel
`position of the target pixel of the demosaicing processing
`
`extracted from the mosaic image, one or more pixel values of
`the one or more pixels of another color in the determined
`correlation direction can be used to accurately estimate the
`pixel value of the pixel of another color, and the generation of
`the false color at the high frequency section can be sup(cid:173)
`pressed.
`In the color imaging apparatus according another
`[0026]
`aspect of the present invention, the predetermined color filter
`array of the color imaging element includes a basic array
`pattern including the first and second filters, the basic array
`pattern is repeatedly arranged in the horizontal and vertical
`directions, and one or more of the first filters and one or more
`of the second filters are arranged in each line in the horizontal
`and vertical directions of the basic array pattern. One or more
`of the first filters and one or more of the second filters are
`arranged in each line in the horizontal and vertical directions
`of the basic array pattern. Therefore, the generation of color
`moire (false color) in the horizontal and vertical directions
`can be suppressed to improve the resolution. The basic array
`pattern is repeated in the horizontal and vertical directions in
`the color filter array. Therefore, demosaicing (interpolation)
`processing in a later stage can be executed according to the
`repeated pattern.
`[0027] Preferably, in the color filter array of the color imag(cid:173)
`ing apparatus according to another aspect of the present
`invention, the first filters are arranged at a center and four
`corners of a 3x3 pixel group, and the 3x3 pixel group is
`repeatedly arranged in the horizontal and vertical directions.
`The first filters are arranged at four comers of the 3x3 pixel
`group. Therefore, if the 3x3 pixel group is repeatedly
`arranged in the horizontal and vertical directions, the color
`filter array includes square arrays corresponding to 2x2 pixels
`including the first filters. The pixel values of the 2x2 pixels
`can be used to determine the direction with high correlation
`among the horizontal, vertical, and oblique (NE, NW) direc(cid:173)
`tions.
`[0028] Preferably, in the color filter array of the color imag(cid:173)
`ing apparatus according to another aspect of the present
`invention, the first filters are vertically and horizontally
`arranged across a filter at a center of a 3x3 pixel group, and the
`3x3 pixel group is repeatedly arranged in the horizontal and
`vertical directions. The first filters are vertically and horizon(cid:173)
`tally arranged across the filter at the center of the 3x3 pixel
`group. Therefore, if the 3x3 pixel group is repeatedly
`arranged in the horizontal and vertical directions, the first
`filters are adjacent to each other (in two pixels) in the hori(cid:173)
`zontal and vertical directions across the filter at the center of
`the 3x3 pixel group in the color filter array. The pixel values
`of the pixels (eight pixels in total) corresponding to the first
`filters can be used to determine the correlation direction of the
`four directions.
`In the color imaging apparatus according to another
`[0029]
`aspect of the present invention, the direction determination
`unit calculates difference absolute values of pixel values of
`adjacent pixels in each of the horizontal, vertical, and oblique
`(NE, NW) directions and determines a direction with a mini(cid:173)
`mum difference absolute value among the difference absolute
`values in the directions as the correlation direction.
`In the color imaging apparatus according to another
`[0030]
`aspect of the present invention, the direction determination
`unit calculates ratios of pixel values of adjacent pixels in each
`of the horizontal, vertical, and oblique (NE, NW) directions
`and determines a direction with the ratio closest to 1 among
`the ratios in the directions as the correlation direction.
`
`0012
`
`

`

`US 2015/0029367 AI
`
`Jan.29,2015
`
`3
`
`[0031] A plurality of difference absolute values or ratios
`may be calculated for each of the directions, and a sum or an
`average value of the plurality of difference absolute values or
`an average value of the ratios may be calculated for each of
`the directions. In this case, the correlation direction can be
`determined more accurately.
`In the color imaging apparatus according to another
`[0032]
`aspect of the present invention, the demosaicing processing
`unit sets the pixel value of a pixel of another color in the
`correlation direction determined by the direction determina(cid:173)
`tion unit as the pixel value of the another color at the pixel
`position of the target pixel, or sets a value obtained by inter(cid:173)
`polating the pixel values of a plurality of pixels of another
`color in the correlation direction determined by the direction
`determination unit as the pixel value of the another color at the
`pixel position of the target pixel.
`In the color imaging apparatus according to another
`[0033]
`aspect of the present invention, if there is no pixel of the
`another color in the correlation direction determined by the
`direction determination unit, the demosaicing processing unit
`interpolates the pixel value of the target pixel based on a color
`difference or a color ratio at the pixel positions of the pixels of
`the another color near the target pixel to calculate the pixel
`value of the another color. The color difference and the color
`ratio at the pixel positions of the pixels of the another color
`denote a difference (color difference) and a ratio (color ratio)
`between the pixel value of the pixel at the pixel position and
`the pixel value of the another color already estimated by the
`direction determination in the correlation direction.
`In the color imaging apparatus according to another
`[0034]
`aspect of the present invention, the color filters include R
`filters, G filters, and B filters corresponding to red (R), green
`(G), and blue (B) colors, wherein when the target pixel of the
`demosaicing processing is a G pixel, the pixel value is G, and
`there are no Rand B pixels in the correlation direction deter(cid:173)
`mined by the direction determination unit, if the pixel values
`of R and B pixels near the G pixel are Rand B, and the pixel
`values of Gat the pixel positions of the pixels are G R and G B'
`the demosaicing processing unit calculates pixel values RG
`and BG of the Rand B pixels at the position of the target pixel
`by following formulas
`
`[0035] when the target pixel of the demosaicing processing
`is an R pixel, the pixel value is R, and there are no G and B
`pixels in the correlation direction determined by the direction
`determination unit, if the pixel values of G and B pixels near
`the R pixel are G and B, and the pixel values ofR at the pixel
`positions of the pixels are RG and RB, the demosaicing pro(cid:173)
`cessing unit calculates pixel values G R and B R of the G and B
`pixels at the position of the target pixel by following formulas
`
`[0036] when the target pixel of the demosaicing processing
`is a B pixel, the pixel value is B, and there are no G and R
`pixels in the correlation direction determined by the direction
`determination unit, if the pixel values of G and R pixels near
`the B pixel are G and R, and the pixel values ofB at the pixel
`positions of the pixels are BG and BR, the demosaicing pro(cid:173)
`cessing unit calculates pixel values G Band RB of the G and R
`pixels at the position of the target pixel by following formulas
`
`GB~B+(G-BG) and RB~B+(R-BR)·
`
`In the color imaging apparatus according to another
`[0037]
`aspect of the present invention, the color filters include R
`
`filters, G filters, and B filters corresponding to red (R), green
`(G), and blue (B) colors, wherein when the target pixel of the
`demosaicing processing is a G pixel, the pixel value is G, and
`there are no Rand B pixels in the correlation direction deter(cid:173)
`mined by the direction determination unit, if the pixel values
`of Rand B pixels near the G pixel are R and B, and the pixel
`values of Gat the pixel positions of the pixels are G R and G B'
`the demosaicing processing unit calculates pixel values RG
`and BG of the Rand B pixels at the position of the target pixel
`by following formulas
`
`RG~Gx(R/GR) and BG~Gx(B/GB), wherein
`
`[0038] when the target pixel of the demosaicing processing
`is an R pixel, the pixel value is R, and there are no G and B
`pixels in the correlation direction determined by the direction
`determination unit, if the pixel values of G and B pixels near
`the R pixel are G and B, and the pixel values ofR at the pixel
`positions of the pixels are RG and RB, the demosaicing pro(cid:173)
`cessing unit calculates pixel values G R and B R of the G and B
`pixels at the position of the target pixel by following formulas
`
`GR~Rx(GIRG) andBR~Rx(B/RB), and wherein
`
`[0039] when the target pixel of the demosaicing processing
`is a B pixel, the pixel value is B, and there are no G and R
`pixels in the correlation direction determined by the direction
`determination unit, if the pixel values of G and R pixels near
`the B pixel are G and R, and the pixel values ofB at the pixel
`positions of the pixels are BG and BR, the demosaicing pro(cid:173)
`cessing unit calculates pixel values G Band RB of the G and R
`pixels at the position of the target pixel by following formulas
`
`GB~Bx(GIBG) and RB~Bx(R/BR)·
`
`In the color imaging apparatus according to another
`[0040]
`aspect of the present invention, the direction determination
`unit determines that there is no correlation direction when
`difference values of the pixel values of the pixels adjacent to
`each other in the horizontal, vertical, and oblique (NE, NW)
`directions are equal, and if the direction determination unit
`determines that there is no correlation direction, the demo sa(cid:173)
`icing processing unit uses one or more pixel values of one or
`more pixels of another color near the pixel position of the
`target pixel of the demosaicing processing to calculate the
`pixel value of the another color at the pixel position.
`In the color imaging apparatus according to another
`[0041]
`aspect of the present invention, the color filters include R
`filters, G filters, and B filters corresponding to red (R), green
`(G), and blue (B) colors, and the filter array includes: a first
`array corresponding to 3x3 pixels, the first array including G
`filters arranged at a center and four corners, B filters vertically
`arranged across the G filter at the center, and R filters hori(cid:173)
`zontally arranged across the G filter at the center; and a
`second array corresponding to 3x3 pixels, the second array
`including G filters arranged at a center and four corners, R
`filters vertically arranged across the G filter at the center, and
`B filters horizontally arranged across the G filter at the center,
`wherein the first and second arrays are alternately arranged in
`the horizontal and vertical directions.
`[0042] According to the color filter array with the configu(cid:173)
`ration, when 5x5 pixels (local area of mosaic image) are
`extracted around the first or second array, there are 2x2 G
`pixels at four corners of the 5x5 pixels. The pixel values of the
`2x2 G pixels can be used to determine the correlation direc(cid:173)
`tion of the four directions.
`In the color imaging apparatus according to another
`[0043]
`aspect of the present invention, the color filters include R
`
`0013
`
`

`

`US 2015/0029367 AI
`
`Jan.29,2015
`
`4
`
`filters, G filters, and B filters corresponding to red (R), green
`(G), and blue (B) colors, and the filter array includes: a first
`array corresponding to 3x3 pixels, the first array including an
`R filter arranged at a center, B filters arranged at four corners,
`and G filters vertically and horizontally arranged across the R
`filter at the center; and a second array corresponding to 3x3
`pixels, the second array including a B filter arranged at a
`center, R filters arranged at four corners, and G filters verti(cid:173)
`cally and horizontally arranged across the B filter at the cen(cid:173)
`ter, wherein the first and second arrays are alternately
`arranged in the horizontal and vertical directions.
`[0044] According to the color filter array with the configu(cid:173)
`ration, when 5x5 pixels (local area of mosaic image) are
`extracted around the first or second array, there are G pixels
`adjacent to each other in the horizontal and vertical directions
`across the pixel (R pixel orB pixel) at the center of the 5x5
`pixels. The pixel values of the G pixels (eight pixels in total)
`can be used to determine the correlation direction of the four
`directions.
`
`Advantageous Effects oflnvention
`
`[0045] According to the present invention, a color imaging
`element including color filters including sections where two
`or more first filters that most contribute to obtaining lumi(cid:173)
`nance signals are adjacent to each other in horizontal, vertical,
`and oblique (NE, NW) directions (four directions) is used.
`The color imaging element includes the first filters and sec(cid:173)
`ond filters corresponding to two or more second colors other
`than a first color that are periodically arranged in lines in the
`horizontal and vertical directions. A correlation direction of
`luminance is determined based on difference values of pixel
`values of pixels adjacent to each other in the directions.
`Therefore, the pixel values at minimum pixel intervals can be
`used to determine the correlation direction. In calculation of
`a pixel value of another color at a pixel position of a target
`pixel of demosaicing processing extracted from a mosaic
`image, the pixel value of a pixel of the another color in the
`determined correlation direction is used to calculate the pixel
`value of the another color at the pixel position of the target
`pixel. Therefore, the pixel value of the pixel of the another
`color can be accurately estimated, and generation of a false
`color at a high frequency section can be suppressed.
`
`BRIEF DESCRIPTION OF DRAWINGS
`
`[0046] FIG. 1 is a block diagram showing embodiments of
`a color imaging apparatus according to the present invention.
`[0047] FIG. 2 is a diagram showing a color filter array of
`color filters arranged on a color imaging element of a first
`embodiment.
`[0048] FIG. 3 is a diagram showing a basic array pattern
`included in the color filter array of the color imaging element
`of the first embodiment.
`[0049] FIG. 4 is a diagram showing a state in which the
`basic array pattern of 6x6 pixels included in the color filter
`array of the color imaging element of the first embodiment is
`divided into A arrays and B arrays of3x3 pixels.
`[0050] FIG. 5 is a diagram showing a state in which the
`basic array pattern of 6x6 pixels included in the color filter
`array of the color imaging element of the first embodiment are
`divided into A arrays and the B arrays of 3x3 pixels, and the
`A arrays and the B arrays are arranged.
`
`[0051] FIG. 6 is a diagram used to explain a determination
`method of a correlation direction of luminance and a pixel
`interpolation method in demosaicing processing.
`[0052] FIG. 7A is a diagram showing an oblique high fre(cid:173)
`quency image incident on the color imaging element.
`[0053] FIG. 7B is a diagram used to explain a determination
`method of the correlation direction when the oblique high
`frequency image is incident on the color imaging element.
`[0054] FIG. 8 is a diagram showing a second embodiment
`of the color imaging element applied to the present invention.
`[0055] FIG. 9 is a diagram showing a third embodiment of
`the color imaging element applied to the present invention.
`[0056] FIG. 10 is a diagram showing a fourth embodiment
`of the color imaging element applied to the present invention.
`[0057] FIG. 11 is a diagram showing a fifth embodiment of
`the color imaging element applied to the present invention.
`[0058] FIG. 12 is a diagram showing a sixth embodiment of
`the color imaging element applied to the present invention.
`[0059] FIG. 13 is a diagram showing a seventh embodiment
`of the color imaging element applied to the present invention.
`[0060] FIG. 14 is a diagram used to explain a problem of a
`conventional color imaging element including color filters in
`a Bayer array.
`
`DESCRIPTION OF EMBODIMENTS
`
`[0061] Hereinafter, preferred embodiments of a color
`imaging apparatus according to the present invention will be
`described in detail with reference to the attached drawings.
`[0062] {Overall Configuration of Color Imaging Appara(cid:173)
`tus}
`[0063] FIG. 1 is a block diagram showing embodiments of
`the color imaging apparatus according to the present inven(cid:173)
`tion.
`[0064] An imaging optical system 10 images a subject, and
`an optical image indicating a subject image is formed on a
`light receiving surface of a color imaging element 12 (color
`imaging element of a first embodiment).
`[0065] The color imaging element 12 is a single-plate color
`imaging element including: a plurality of pixels (not shown)
`including photoelectric conversion elements arranged inhori(cid:173)
`zontal and vertical directions (two-dimensional array); and
`color filters in a predetermined color filter array arranged on
`light receiving surfaces of the pixels. The color filter array of
`the color imaging element 12 is characterized by including:
`filters of all red (R), green (G), and blue (B) colors periodi(cid:173)
`cally arranged in lines in the horizontal and vertical direc(cid:173)
`tions; and sections where two or more G filters corresponding
`toGs that most contribute to obtaining luminance signals are
`adjacent to each other in horizontal, vertical, and oblique
`(NE, NW) directions. Details of the color imaging element 12
`will be described later.
`[0066] The photoelectric conversion elements convert the
`subject image formed on the color imaging element 12 to
`signal charges corresponding to amounts of incident light.
`The signal charges accumulated on the photoelectric conver(cid:173)
`sion elements are sequentially read out from the color imag(cid:173)
`ing element 12 as voltage signals (image signals) correspond(cid:173)
`ing to the signal charges based on drive pulses provided from
`a drive unit 18 according to an instruction of a control unit 20.
`The image signals read out from the color imaging element 12
`are R, G, and B signals indicating a mosaic image ofR, G, and
`B corresponding to the color filter array of the color imaging
`element 12. The color imaging element 12 is not limited to a
`CCD (Charge Coupled Device) color imaging element and
`
`0014
`
`

`

`US 2015/0029367 AI
`
`Jan.29,2015
`
`5
`
`may be another type of imaging element such as a CMOS
`(Comple