US007245319B1
`
`a2) United States Patent
`US 7,245,319 Bl
`(10) Patent No.:
`Jul. 17, 2007
`(45) Date of Patent:
`Enomoto
`
`
`(54) DIGITAL IMAGE SHOOTING DEVICE WITH
`LENS CHARACTERISTIC CORRECTION
`UNIT
`
`Jun Enomoto, Kanagawa(JP)
`Inventor:
`(75)
`(73) Assignee: Fujifilm Corporation, Tokyo (JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/330,096
`
`(22)
`
`Filed:
`
`Jun. 11, 1999
`
`Foreign Application Priority Data
`(30)
`Jun. 11,1998
`(IP)
`eecceccsecsecseessessseeesteees 10-163194
`
`(51)
`
`Int. CL.
`(2006.01)
`HOAN 5/228
`(2006.01)
`HOAN 5/217
`(52) US. CD. ie eecceeeeecneneeeeeees 348/222.1; 348/241
`(58) Field of Classification Search ................ 348/360,
`348/355. 96. 97. 105. 222.1. 241. 335. 231.99
`348/231.6 931.3: 382/298. 300: 235/462.05.
`,
`,
`535/462.11
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,185,667 A *
`2/1993 Zimmermann.............. 358/209
`
`5,276,519 A *
`............ 358/209
`1/1994 Richardset al.
`5,818,527 A * 10/1998 Yamaguchiet al.
`........ 348/335
`......... 348/240.99
`5,905,530 A *
`5/1999 Yokotaet al.
`
`.......... 348/218.1
`6,097,430 A *
`8/2000 Komiya et al.
`4/2001 Kiriki et al. 382/167
`6,219,446 BL*
`
`11/2001 Enomoto ........ 355/40
`6,323,934 B1L*
`
`1/2002 Tsuruoka et al.
`........... 382/163
`6,343,146 B1*
`
`6,476,869 B1* 11/2002 Sekine et al. wc. 348/335
`6,538,691 BL*
`3/2003 Macy etal. 0.0... 348/222.1
`6,587,224 B1*
`7/2003 Nabeshimaet al.
`......... 358/19
`6,618,081 B1*
`9/2003 Harada et al... 348/231.6
`6,775,419 B2*
`8/2004 Maedaet al. oo. 382/275
`FOREIGN PATENT DOCUMENTS
`6-98170 A
`4/1994
`6-178197 A
`6/1994
`6-181530 A
`6/1994
`6-205273 A
`7/1994
`6-319106 A
`11/1994
`7-67028 A
`3/1995
`9-74514 A
`3/1997
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`
`.
`.
`* cited by examiner
`Primary Examiner—Vivek Srivastava
`Assistant Examiner—Justin Misleh
`(74) Attorney, Agent, or Firm—Birch, Stewart, Kolasch &
`Birch, LLP
`
`(57)
`
`ABSTRACT
`
`The digital image shooting device is composed of an image
`forming lens, an image sensor element, a data processing
`unit for processing an output signal from the image sensor
`element
`into digital
`image data, an image memory for
`storing the digital
`image data; and a lens characteristic
`correction unit for performing a process of correcting a
`deterioration of an image quality derived from the image
`forming lens upon the digital image data by using a lens
`characteristic of the image forming lens and a position of a
`frame image photographed. The digital
`image shooting
`device is capable of obtaining the image quality which is
`high enough for the utilization even by using the compara-
`tively low-price lens which does not exhibit a high perfor-
`mance.
`
`12 Claims, 3 Drawing Sheets
`
`
`INTERPOLATION
`COORDINATE
`PROCESSING
`
`TRANSFORMATION
`UNIT
`
`
`PROCESSING
`(ELECTRONIC MAGNIFICATION
`
`UNIT Ir=ir+ A r+D
`VARYING PROCESS)
`ADDRESS: Ir/N
`INTERPOLATION
`
`
`COORDINATE
`PROCESSING
`
`
` > 18,22
`UNIT
`TRANSFORMATION
`
`
`(ELECTRONIC MAGNIFICATION
`PROCESSING _
`
`
`
`UNIT
`Ig=ig+D
`VARYING PROCESS)
`
`
`
`ADDRESS: Ig/N
`
`
`34
`
`INTERPOLATION
`
`
`COORDINATE
`PROCESSING
`
`
`TRANSFORMATION
`UNIT
`PROCESSING
`(ELECTRONIC MAGNIFICATION
`UNIT Ib=ib+ A b+D
`VARYING PROCESS)
`
`ADDRESS : Ib/N
`
`
`36b
`
`PETITIONERS EX1013
`Page 1
`
`PETITIONERS EX1013
`Page 1
`
`

`

`U.S. Patent
`
`Jul. 17, 2007
`
`Sheet 1 of 3
`
`US 7,245,319 Bl
`
`NOISSSudWOD3d
`
`/NOISSSYdWOD
`
`LINN
`
`AMOWNIIN|oeeceoalesafeccoeeesONIGWIY
`
`3LINNTOYLNOD
`
`
`
`[T°OLld
`
`INISSIDOUd
`
`LUNN
`
`VivaA
`
`PETITIONERS EX1013
`Page 2
`
`PETITIONERS EX1013
`Page 2
`
`
`
`
`

`

`U.S. Patent
`
`Jul. 17, 2007
`
`Sheet 2 of 3
`
`US 7,245,319 Bl
`
`ecSl
`
`NOLLVOISINSWWSINOWLOI13)ONISSADONd
`
`
`
`
`(SS300¥dDNIANVAG+4V+41=4)LINN
`
`
`
`N/4|-SSHYCOV
`
`NOLLV10d44LNI
`
`
`
`QNISSIOONdJLVNIGYOOD
`
`
`
`~LINNNOLLVWYOASNVYL
`
`C*Old
`
`NOLLW1OduALNI
`
`
`
`ONISSIIOUdILVNIGNOOD
`
`
`
`JINNNOILWWYOSSNVUL
`
`NOLLVIIINDVWDINOWLDITa)NISSAVONd
`
`
`
`
`
`
`
`(SSH90UdONIAYVAq+61=6|LINN
`
`N/6|:SSaYGaV
`
`
`
`NOLLV10du4LNI
`
`
`
`SNISSSOONdALVNIGUOOD
`
`
`
`LINNNOILLVWYOASNVYL
`
`
`
`
`
`NOLLVOISINSVWDINOYL33714)INISSAIONd
`
`(SS30OUdONIANVA
`
`
`g+qV+q!=q]LINN
`
`N/QI:SSaYdgV
`
`
`
`q9€EOE
`
`SNISSADOUd
`
`JOVI
`
`LINN
`
`ve
`
`PETITIONERS EX1013
`Page 3
`
`PETITIONERS EX1013
`Page 3
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jul. 17, 2007
`
`Sheet 3 of 3
`
`US 7,245,319 B1
`
`FIG.3A
`
`FIG.3B
`
`Y
`
`
`
`
`
`
`yintelEELLAP|ee
`
`Ss
`tNSESE
`ee|t=’
`eS
`
`
`
`
`SN
`
`FIG.3C
`
`PETITIONERS EX1013
`Page 4
`
`PETITIONERS EX1013
`Page 4
`
`

`

`US 7,245,319 Bl
`
`1
`DIGITAL IMAGE SHOOTING DEVICE WITH
`LENS CHARACTERISTIC CORRECTION
`UNIT
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to a technicalfiled
`of a digital image shooting device such as a digital camera,
`and more particularly to a digital image shooting device
`capable of photographing (outputting) a high-quality image
`of which aberrations are corrected even in a case of incor-
`porating a comparatively low-price lens which performance
`is not so good.
`A digital camera, which hadinitially a small number of
`pixels and of which representation was in monochrome,has
`therefore been limited its use to a special application. With
`a remarkable decrease in costs for the electronic parts of
`CCDsensors, etc., however, a color representation becomes
`possible easily, and the application of image data processing
`expands with a spread of a personal computer. Under such
`circumstances, the digital camera sharply decreases in price
`and seems to be popularized.
`Recently, a high quality digital camera of which pixel
`number exceeds 1,000,000 have been commercialized one
`after another, and also with regard to a memory for storing
`(recording) the image, there is a tendency of the digital
`camera to shift from a type of being built in the bodyofit
`to a type of using a removable storage medium having a
`larger capacity.
`In the latter type of the digital camera using the removable
`storage medium,a processof taking the recorded image data
`into a personal computer becomes highly simple.
`As explained above, an enhancementof the image quality
`in the digital camera is largely influenced by, in addition to
`an increase in the numberof recordable pixels, a perfor-
`mance improvement of an image forming lens, and an
`improvement of an image processing technology of the
`photographed image data.
`For instance, some of the digital cameras (DS-300 made
`by Fuji Photo Film Co.,Ltd.) use a 3-powered zoom lens as
`an image forming lens, of which a lens configuration is
`based on 3 lens units containing 9 lens elements, and this
`type of the digital camera becomessubstantially equal to a
`compact camera in terms of the image quality.
`However, by using the high-accuracy and high-perfor-
`mancelens, the digital camera which can output the high-
`quality image can be realized and, while on the other hand,
`it might be a factor for raising the costs of the digital camera.
`conversely, by using the comparatively low-price lens, it
`is possible to decrease the costs of the digital camera.
`However it can not be denied that the low-price lens is
`inferior in performance so that for example, there might
`occur a chromatic aberration of magnification, a distortion
`aberration and a decrease in marginal lumination, and qual-
`ity of the image falls down such that a distortion and a
`density gradient occur with the outputted image.
`
`SUMMARY OF THE INVENTION
`
`The present invention has been accomplished under these
`circumstances and has as an object providing a low cost
`digital image shooting device capable of obtaining the image
`quality which is high enough for the utilization even by
`using the comparatively low-price lens which performance
`is not so high.
`In orderto attain the object described above, the invention
`provides a digital image shooting device comprising: an
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`image forming lens; an image sensor element; a data pro-
`cessing unit for processing an output signal from the image
`sensor elementinto digital image data; an image memory for
`storing the digital
`image data; and a lens characteristic
`correction unit for performing, by using a lens characteristic
`of the image forming lens and a position of a frame image
`photographed, a process of correcting a deterioration of an
`image quality derived from the image forming lens upon the
`digital image data.
`It is preferable that the image quality deterioration cor-
`rected by the lens characteristic correction unitis at least one
`of a chromatic aberration of magnification, a distortion
`aberration, defocusing and a decrease in marginal lumina-
`tion.
`
`It is also preferable that the lens characteristic correction
`unit corrects the deterioration of the image quality before a
`process of compressing the digital image data.
`It is further preferable that the lens characteristic correc-
`tion unit performs the correction before the photographing
`of a next frame or during the photographing of the next
`frame onward, andthe digital image data of the frame which
`is performed the correction by the lens characteristic cor-
`rection unit is stored in the image memory.
`It is also further preferable that the image memory is a
`built-in image recording medium or a removable image
`recording medium.
`It is still further preferable that the digital image shooting
`device of the invention further comprises an image display
`unit for displaying the photographed image, wherein an
`image based on the digital image data which is or is not
`performed the correction process in the lens characteristic
`correction unit, is displayed on the image display unit, and
`the digital image data performed the correction process in
`the lens characteristic correction unit, is stored in a memory.
`It is also still further preferable that an image of a region
`larger than a photographic region confirmed by a photogra-
`pher is formed on the image sensor element in accordance
`with missing of pixels which is caused as a result of the
`correction by the lens characteristic correction unit.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`In the accompanying drawings:
`FIG. 1 is a block diagram schematically showing one
`example of the digital camera according to the present
`invention;
`FIG. 2 is a block diagram schematically showing a data
`processing unit of the digital camera shown in FIG. 1; and
`FIGS. 3A, 3B and 3C are conceptual diagrams each
`showing an image deformation by aberration correction.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The digital image shooting device of the present invention
`will hereinafter be described in details based on a preferred
`embodiment which will be illustrated in the accompanying
`drawings.
`FIG. 1 is a block diagram showing one example of the
`digital camera according to the present invention.
`Referring to FIG. 1, a digital camera 10 includes an image
`shooting unit 12, an A/D (analog-to-digital) converter 14, a
`data processing unit 16, a data compressing unit (which is
`hereinafter simply termed a compression unit) 18, an image
`memory 20, a display unit 22, an I/F (interface) unit 24, a
`reading control unit 26, and a CPU 28 for managing and
`controlling these units and the whole digital camera 10.
`PETITIONERS EX1013
`Page 5
`
`PETITIONERS EX1013
`Page 5
`
`

`

`US 7,245,319 Bl
`
`3
`Note that solid lines indicate a flow of image signals
`(image data), and dotted lines indicate a flow of control
`signals respectively in FIG. 1.
`The image shooting unit 12, which shoots a photographic
`subject (a scene) and outputs signals after photoelectric
`conversion, has a color CCD sensor (hereinafter simply
`referred to as a sensor) 30 for photoelectrically reading the
`image by separating the image into three primary colors red
`(R), green (G) and blue (B), an image forming lens 32 for
`forming the subject image on the sensor 30 and, in addition,
`though not
`illustrated, a variable stop for controlling a
`quantity of light beams incident in the sensor 30, and an
`amplifier for amplifying output signals from the sensor 30.
`The output signals (the image signals) from the imaging
`unit 12 are converted to digital image signals by the A/D
`converter 14, and thereafter transmitted to the data process-
`ing unit 16.
`Note that the reading control unit 26 controls such as an
`image reading (sampling) timing of the sensor 30, an elec-
`tronic shutter speed (a storage time), and a processing timing
`of the image signal in the data processing unit 16, corre-
`sponding thereto.
`The data processing unit 16, which makes a predeter-
`mined data processing on the digital image signals converted
`by the A/D converter 14 and outputs it as an output image
`data (hereinafter simply referred to as image data) to the
`compression unit 18 and the display unit 22, as illustrated in
`FIG. 2, includes an image processing unit 34 and a lens
`characteristic correction unit 36 that is particular to the
`present invention.It is to be noted that a processing condi-
`tion in the data processing unit 16 is set by, e.g., the CPU 28.
`The image processing unit 34 makes a variety of image
`processes such as a conversion of the image signal into the
`image (density) data, white balance control of the image,
`gray balance control thereof, density control thereof, color
`balance control thereof and a sharpness (sharpening) pro-
`cess.
`
`On the other hand, the lens characteristic correction unit
`(hereinafter be simply called a characteristic correction unit)
`36 corrects deterioration of image quality origination in lens
`characteristics of the image forming lens 32 by using the
`lens characteristics of the image forming lens 32 an image
`position. The characteristic correction unit 36,
`in the
`embodimentillustrated herein, corrects a chromatic aberra-
`tion of magnification and a distortion aberration originating
`in the lens characteristic and, more preferably, corrects
`aberrations such as a decrease in marginal lumination and
`defocusing.
`As known well, a color image is formed of the three
`primary colors of R, G and B, however, a refractive index
`(an image forming magnification) of the lens is subtly
`different depending on a wavelength, and therefore the
`image forming magnifications of light of R, G and B is
`different, viz.,
`the chromatic aberration of magnification
`occurs. Hence, if the image formed on the sensor 30 is just
`reproduced as a visible image, a color deviation occurs on
`the obtained image.
`Further, it is required for obtaining a high-quality photo-
`graphic imagethat a plane perpendicular to the optical axis
`is formed on the image forming plane correspondingthereto.
`In a lens which performance is not so good, however, a
`deviation of the image forming position occurs in the
`optical-axis direction and a distortion i.e. distortion aberra-
`tion occurs on the formed image. Consequently, if the image
`formed on the sensor 30 is just reproduced as the visible
`image, the distortion might occur on the obtained image.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`60
`
`65
`
`4
`Asdescribed above, if the image containing the chromatic
`aberration of magnification and the distortion aberration is
`shown on a display, and just reproducedas the visible image
`in the form of a hardcopy or etc., on outputted image
`becomes an image whichhas a color deviation, a peripheral
`distortion, a decrease in the marginal lumination and defo-
`cusing and so on.
`Therefore, the digital camera 10 according to the present
`invention includes the characteristic correction unit 36 to
`
`correct the chromatic aberration of magnification and the
`distortion aberration originating in the lens characteristics so
`that even when using a low-price lens, the digital camera 10
`is able to output the high-quality image with a stability that
`there is neither the distortion nor the color deviation, and a
`cost for the digital camera can be remarkably reduced.
`The characteristic correction unit 36 memorizes the lens
`characteristics of the image forming lens 32 incorporated
`into the digital camera 10, and corrects the chromatic
`aberration of magnification and the distortion aberration, etc.
`of the image on the basis of the lens characteristics.
`The lens characteristics is not limited particularly. How-
`ever, generally the characteristics of the chromatic aberra-
`tion of magnification and the distortion aberration ofthe lens
`and the characteristics of the aberrations such as the
`
`decrease in the marginal lumination and the defocusing, can
`be approximated to some extent by a cubic function which
`uses a distance (which is expressed by, e.g., x-y) from the
`optical axis of thelens, i.e., from the center of the image as
`a parameter. Hence, corresponding to the image forming
`lens 32 mounted on the camera, a function representing the
`characteristic of the chromatic aberration of magnification
`and a function representing the distortion aberration corre-
`sponding thereto, may be stored as lens characteristics.
`The characteristic correction unit 36 performs the process
`of correcting the chromatic aberration of magnification and
`the distortion aberration by use of the above-mentionedlens
`characteristics and the imageposition 1.e., an information of
`the position of the image data (pixels), e.g., a coordinate
`position (pixel number from the center) from the center of
`the image.
`Note that the coordinates in this case may be the x-y
`coordinates or the polar coordinates. Further, the informa-
`tion of the position of the image data is not restricted to the
`center of the imageas a basis, but various kind of things can
`be used. For example, a corner portion (such as a left upper
`corner, etc.) of the image and a certain pixel (e.g., a pixel of
`a pixel number No.1) may be set as a basis. Namely,
`according to the present
`invention,
`the various kind of
`position information can be used, which detects the position
`of the image (pixels) relatively.
`Herein, however, if the chromatic aberration of magnifi-
`cation and the distortion aberration are separately corrected
`by using the lens characteristics and the position information
`(hereinafter referred to as pixel position), the calculation
`takes a long time andaninterpolating calculation needsto be
`done a plurality of times so that there arises a problem that
`the image quality might be deteriorated.
`Therefore, in the embodimentillustrated herein, a pref-
`erable mode is that image magnifications of R and B are
`converted with “G” being normally a basic color among the
`three primary colors of R, G, and B, the chromatic aberration
`of magnification is corrected by matching the R and B
`images with the G image, and thereafter the distortion
`aberration is corrected. A proper position of each pixel is
`thereby calculated, and the image data of each pixel
`is
`subjected to the interpolating calculation, thereby making it
`feasible to obtain the image data in which the chromatic
`PETITIONERS EX1013
`Page 6
`
`PETITIONERS EX1013
`Page 6
`
`

`

`US 7,245,319 Bl
`
`5
`aberration of magnification and the distortion aberration of
`the photographed image are corrected.
`Accordingly, the calculation for the G image may suffice
`for the distortion aberration, and hence the chromatic aber-
`ration of magnification and the distortion aberration can be
`corrected under a more preferable condition because of a
`reduction in quantity of normal and interpolating calcula-
`tions.
`The characteristic correction unit 36 for carrying out the
`correcting method described above, as shown in the con-
`ceptual diagram in FIG. 2, includes a coordinate transfor-
`mation processing unit 36a and an interpolation processing
`unit 365.
`Incidentally, referring to FIG.2, Ir, Ig and Ib designate the
`pixel positions (addresses) of the image data (input image
`data) supplied from the image processing unit 34, ir, ib and
`ib represent the pixel positions of the image data of which
`the chromatic aberration of magnification and the distortion
`aberration are corrected, Ar and Ab denote the deviation
`quantities (1.e.,
`the correction quantities) of the R- and
`B-pixel positions from the G-pixel position due to the
`chromatic aberration of magnification, and D designates the
`deviation quantity of the G-pixel position due to the distor-
`tion aberration.
`
`The characteristic correction unit 36, when supplied with
`the image data from the image processing unit 34, makes the
`coordinate transformation processing unit 36a calculate the
`deviation quantities Ar and Ab of the pixel values ir and ib
`of the R- and B-imagedata from the G-image data ig due to
`the chromatic aberration of magnification by use of the lens
`characteristics described above, and further calculate the
`deviation quantity D of the input G-image data ig due to the
`distortion aberration.
`Next, as shown in FIG. 2, the coordinate transformation
`processing unit 36a calculates the pixel position Ir of the
`R-image data of which the chromatic aberration of magni-
`fication and the distortion aberration are corrected by adding
`Ar and D to each pixel position ir of the input R-image data.
`Then,
`the coordinate transformation processing unit 36a
`calculates the pixel position Ib of the B-image data of which
`the chromatic aberration of magnification and the distortion
`are corrected by adding Ab and D to each pixel position ib
`of the input B-image data. Further, the coordinate transfor-
`mation processing unit 36a calculates the pixel position Ig of
`the G-image data of which the distortion is corrected by
`adding D to each pixel position ig of the input G-imagedata.
`That is, the calculation proceeds such as correcting the
`chromatic aberrations of magnification of the R- and B-im-
`ages with the G-image as a basis, making the positional
`alignment of the whole images with the G-image,correcting
`the distortion aberration of the whole by use of the deviation
`quantity D of the G-image due to the distortion aberration,
`and thus calculating the pixel positions of the R- G- and
`B-images of which the chromatic aberrations of magnifica-
`tion and the distortion aberration are corrected.
`The interpolation processing unit 36d calculates the image
`data of which the chromatic aberration of magnification and
`the distortion aberration are corrected by executing a data
`interpolating
`process
`(N-times
`interpolation), which
`involves the use ofthe pixel positionsIr, Ig and Ib with the
`chromatic aberration of magnification and the distortion
`aberration being corrected in the coordinate transformation
`processing unit 36a, and supplies the thus calculated image
`data to a decompression unit 18 and/or a display unit 22.
`Namely, according to this method, the deviation quantity of
`the pixel position which is attributed to the chromatic
`aberration of magnification and the distortion aberration, is
`
`20
`
`35
`
`40
`
`45
`
`55
`
`6
`calculated, thereby recognizing which position each pixel
`(the image data concerned) should normally exist in. Then,
`the interpolating (calculating) process of the image data is
`executed in accordance with that proper position, thereby
`obtaining the image data of which the chromatic aberration
`of magnification and the distortion aberration are corrected.
`The interpolation processing method is not particularly
`limited, and a variety of known methods such as, e.g., a
`bilinear interpolation and a spline interpolation, etc. can be
`utilized.
`There have been in recent years many types of digital
`cameras 10, to which a zooming function is added, capable
`of the photography to form an image by changing the
`magnification through a zoom lens.
`The zooming photographyis optically conducted by prin-
`cipally changing the image forming magnification of the
`image forming lens 32. If the optical variable powerof the
`image forming lens 32 is not sufficient enough to cover a
`photographic magnification variable range, however, an
`electronic magnification varying process based on the image
`data is executed, therehy enlarging or reducing the image
`Note that 11 photographed by use ofthe zoom lens, the
`aberrations described above may be corrected preferably
`based on the lens characteristics corresponding to each focal
`length. In this case, it is preferable to store the lens charac-
`teristics at a plurality of focal lengths of the zoom lens,
`convert the lens characteristics at
`the plurality of focal
`lengths with the focal length when photographed, and obtain
`the lens characteristic at the focal length when photographed
`through the lens. Herein, as for the lens characteristics, each
`of the lens characteristics at switchable focal lengths may be
`held (stored), or the lens characteristic at each focal length
`exhibiting a high photographic probability, may respectively
`be stored, or the lens characteristics on the order of three
`stages such as T (Telephoto), M (Middle, or Standard) and
`W (Wide-angle) may be stored. Note that a lens character-
`istic at a focal length which is not stored can be obtained by
`effecting the interpolating calculation from the lens charac-
`teristics at focal lengths anterior and posterior to that non-
`stored focal length.
`On the other hand, the digital cameras 10 each having an
`interchangeable lens has been recently put on the market,
`and the interchangeable lens of the conventional camera
`using the photographic film is usable in some of those
`cameras. In the case of taking a photo using such an
`interchangeable lens, there are required the lens character-
`istics of the interchangeable lens. Therefore, the lens char-
`acteristics corresponding to lens data are stored for every
`usable interchangeable lens, the interchangeable lens data
`are received from the interchangeable lens on the occasion
`of the photography, andthe aberrations described above may
`be corrected based on the lens characteristics.
`
`Incidentally, the distortion aberration normally takes a
`bobbin winder shape shown in FIG.3B at the telephoto, and
`a barrel shape in contrast with the bobbin winderat the wide
`angle, and may be corrected in both cases. The chromatic
`aberration of magnification is not, however, conspicuousat
`the wide angle, and therefore the correcting process may be
`omitted.
`Thus, in the case of performing the photography using the
`zoom lens and the interchangeable lens,it is preferable that
`the image aberrations derived from the lens through which
`the subject image is photographedbe corrected by using the
`data on the focal length of the lens when photographing the
`subject image, the lens characteristics at that focal length,
`and the positional data of the image. It is also preferable on
`the occasion of correcting the aberrations described above
`PETITIONERS EX1013
`Page 7
`
`PETITIONERS EX1013
`Page 7
`
`

`

`US 7,245,319 Bl
`
`7
`that the lens characteristics at the plurality of focal lengths
`stored are converted with the focal length when photograph-
`ing, the lens characteristic at the focal length when photo-
`graphing through this lens are obtained, or the lens charac-
`teristic at the focal length of the interchangeable lens used
`are obtained,
`there is calculated using the obtained lens
`characteristic the deviation quantities, attributed to the chro-
`matic aberration of magnification, of the image positions of
`the colors other than the color whichis fiducial for the three
`
`primary colors, a proper position of each image of which the
`distortion aberration and the chromatic aberration of mag-
`nification are corrected is calculated by use of the deviation
`quantity due to the chromatic aberration of magnification
`and the deviation quantity of the fiducial color image
`position which is attributed to the distortion aberration, and
`the distortion aberration and the chromatic aberration of
`magnification are corrected using this properposition, or the
`electronic magnification varying process is executed using
`this proper position.
`Such processes being thus implemented, in the case of
`even the subject image photographed by a comparatively
`low-price digital camera using the lens which does not
`exhibit a high lens performance,it is feasible to correct the
`deterioration of the image quality by executing the image
`processing, whichis derived from the image aberration, viz.,
`the image quality deterioration induced from the aberration
`of the lens, and output stably the high-quality image with no
`distortion and no color deviation, depending on neither the
`photographic magnification nor the focal length.
`As known well,
`the electronic magnification varying
`process is executed by effecting the interpolating process on
`the image data, however, the interpolating process is fol-
`lowed by the deterioration of the image quality, and there-
`fore it is preferable that the numberof interpolating pro-
`cesses be minimized.
`
`Accordingly, the interpolation processing unit 36d in the
`embodimentillustrated herein, by way of a preferable mode
`on the occasion of executing the electronic magnification
`varying process, controls a magnification (N-poweredinter-
`polation) in the interpolating calculation in accordance with
`an electronic magnification varying rate when effecting the
`interpolating process for obtaining the image data of which
`the above-mentioned aberrations are corrected, and simul-
`taneously executes the electronic magnification varying pro-
`cess. It is feasible in this embodiment to correct the chro-
`
`matic aberration of magnification and the distortion
`aberration, and execute the electronic magnification varying
`process by performing the interpolating process once.
`Incidentally, as discussed above, the image exhibiting
`neither the color deviation nor the distortion can be obtained
`
`after correcting the chromatic aberration of magnification
`and the distortion aberration. When these aberrations are
`
`corrected, however, there might be a case where a configu-
`ration and a size of the image change.
`For instance,if the image before being corrected presents
`the bobbin winder shaped distortion aberration as shown in
`FIG.3A,the imageafterthe correction,as illustrated in FIG.
`3B, takes a configuration that the central portions of face-
`to-face sides of the rectangular shape largely shrink from
`both side ends toward the center in X- and Y-directions,
`wherein there might occur so-called vignetting which cor-
`respondsto oblique-line regions where the image can not be
`reproduced.
`The digital camera 10 performs the photographing while
`confirming the photographic region on the display device
`22. In this case, however, when displaying on the display
`device 22 the image whoseaberrations have been corrected,
`
`8
`the image containing the vignetting as shown in FIG. 3B is
`displayed thereon, and it follows that
`the image to be
`outputted outside similarly becomes the vignetted image.
`Therefore, in the digital camera 10 of the present inven-
`tion, assuming previously the vignetting caused asa result of
`correcting the aberrations, as illustrated in FIG. 3C, it is
`preferable that a region 1 of the image data to be taken in
`after being actually photographed (after forming the image
`on the sensor 30) is set larger than an image region a (a
`visualfield of a viewfinder) to be reproduced onthe display
`device 22, and that the image region to be outputted outside
`is coincident with the image region a to be reproduced on the
`display device 22. Alternatively,
`the image region to be
`reproduced on the display device 22 may be set smaller than
`the regionto be actually photographed, and the same outside
`outputting may be carried out. Further, the image after being
`corrected is displayed, and the image vignetted region and a
`cut-out region may be framed.
`Moreover, the interpolating magnification in the interpo-
`lation processing unit 36d is set corresponding to a degree of
`the vignetting, and the slightly-enlarged image may be
`displayed and/or outputted outside.
`According to the device in the embodiment illustrated
`herein, the characteristic correction unit 36 corrects both of
`the chromatic aberration of magnification and the distortion
`aberration by way of the preferable mode, however, the
`present invention is not limited to this mode. The charac-
`teristics of any one of the chromatic aberration of magnifi-
`cation and the distortion aberration are stored as the lens
`
`characteristics, and any one of these aberrations may be
`corrected by use of the lens characteristics and the image
`position.
`Furthermore, there are stored the information (data) about
`not only at least one of the chromatic aberration of magni-
`fication and the distortion aberration but also the defocusing
`(PSF: Point Spread Function), the decrease in the marginal
`lumination, etc. attributed to the lens characteristics, and at
`least one of the chromatic aberration of magnification and
`the distortion aberration is corrected. In addition to or
`instead of this correction,
`the decrease in the marginal
`lumination and the defocusing may also be corrected.
`Another option is that whether or not
`the correction
`pertaining to those lens characteristics should be conducted
`is set selectable, and the correction to be carried out may also
`be set selectable.
`
`30
`
`35
`
`40
`
`45
`
`The image data processed by the interpolation processing
`unit 365 of the