Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 1 of 20 PageID #: 153
`Case 5:23-cv-00152-RWS Document1-5 Filed 12/29/23 Page 1 of 20 PagelD #: 153
`
`EXHIBIT 4
`EXHIBIT 4
`
`

`

`US007 199821B2
`
`(12) United States Patent
`Miyao et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,199,821 B2
`Apr. 3, 2007
`
`(54) IMAGING APPARATUS AND METHOD FOR
`CONTROLLING WHITE BALANCE
`(75) Inventors: Haruhiko Miyao, Mito (JP): Takahiro
`Nakano, Hitachinaka (JP)
`
`(73) Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 736 days.
`(21) Appl. No.: 10/321,842
`
`5,550,587 A * 8/1996 Miyadera ................. 348,223.1
`6,693,673 B1* 2/2004 Tanaka et al. .............. 348,371
`FOREIGN PATENT DOCUMENTS
`4-10887
`1, 1992
`JP
`5-344530
`12/1993
`JP
`9-9136
`1, 1997
`JP
`11-205806
`7, 1999
`JP
`* cited by examiner
`Primary Examiner Ngoc-Yen Vu
`Assistant Examiner Richard M. Bembern
`tattorney Agent, or Firm McDermott Will & Emery
`
`(22) Filed:
`
`Dec. 18, 2002
`
`(65)
`
`Prior Publication Data
`US 2003/O160876 A1
`Aug. 28, 2003
`
`ABSTRACT
`(57)
`An imaging apparatus has: a white balance control circuit
`for detecting an achromatic portion of an image of an object
`and controlling gains of the chrominance; an object distance
`detecting circuit for detecting a distance to the object; and a
`Foreign Application Priority Data
`(30)
`Zoom value detecting circuit for detecting a Zoom value of
`Feb. 27, 2002 (JP)
`............................. 2002-051841
`the optical system. The imaging apparatus further has: an
`object brightness detecting circuit for detecting brightness of
`(51) Int. Cl.
`the object; and a white balance control amount adjustment
`(2006.01)
`HO)4N 9/73
`value setting circuit for forming a white balance control
`(52) U.S. Cl. .................................... 348,223.1: 348/348
`(58) Field of Classification Search ............. 1821 amount adjustment value to adjust a control amount in the
`348/224.1, 227.1, 228.1, 348: 358,516
`white balance control circuit on the basis of object bright
`See application file for complete search history
`ness information, object distance detection information, and
`Zoom value information, wherein the white balance control
`References Cited
`amount is adjusted on the basis of the white balance control
`U.S. PATENT DOCUMENTS
`amount adjustment value.
`
`(56)
`
`5,283,632 A * 2/1994 Suzuki et al. ............ 348,223.1
`
`7 Claims, 6 Drawing Sheets
`
`6
`
`7
`
`ENCODER
`
`2
`
`AGCCIRCUIT
`
`
`
`SIGNAL
`PROCESSING
`CIRCUT
`
`5
`
`WHITEBALANCE -8
`CONROCRC
`
`OBJECTD STANCE
`THRESHOLD
`VALUESETTING
`CIRCU
`
`G
`
`O
`
`WHEBALANCE
`CONTROL
`AMOUNT
`ADIUSVEN
`WALUESETTING
`CRCU
`
`9
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 2 of 20 PageID #: 154
`
`11
`
`
`
`CBECDSTANCE
`DETECTING
`CIRCUIT
`
`AOCMWALUE
`DETECTING
`CIRCUIT
`
`OBEC
`BRIGHTNESS
`DETECTING
`CRCUT
`
`

`

`U.S. Patent
`
`Apr. 3, 2007
`
`Sheet 1 of 6
`
`US 7,199,821 B2
`
`FIG.
`
`
`
`1
`
`2
`
`3
`
`CCD
`
`AGCCIRCUIT
`
`SIGNAL
`PROCESSING
`CIRCUT
`
`
`
`OBECDSTANCE
`DETECTING
`CIRCUIT
`
`2:5
`CIRCUIT
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 3 of 20 PageID #: 155
`
`OBEC
`BRIGHTNESS
`DETECTING
`CIRCUT
`
`WHTEBALANCE
`CONTROLCERCLT
`
`8
`
`OBJECTD STANCE
`THRESHOLD
`VALUESETTING
`CIRCU
`
`WHEBALANCE
`CONTROL
`AMOUNT
`ADJUSTMEN
`VALUESETTING
`CRCU
`
`

`

`U.S. Patent
`
`Apr. 3, 2007
`
`Sheet 2 of 6
`
`US 7,199,821 B2
`
`20a
`
`2Ob
`
`
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 4 of 20 PageID #: 156
`
`

`

`U.S. Patent
`
`Apr. 3, 2007
`
`Sheet 3 of 6
`
`US 7,199,821 B2
`
`FIG. 3
`
`OBJECTDSTANCE
`THRESHOLD WALUE G
`LARGE
`
`
`
`OBJEC
`LUMINANCEF
`DARK
`
`L1
`
`L2
`
`UGHT
`
`SMALL
`
`LOW
`
`ZOOM
`MAGNIFICATIONE
`
`HIGH
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 5 of 20 PageID #: 157
`
`

`

`U.S. Patent
`
`Apr. 3, 2007
`
`Sheet 4 of 6
`
`US 7,199,821 B2
`
`WHTEBALANCE CONTROL
`AMOUNTADUSTMENT VALUE C
`
`OBJECT DISTANCE
`DISCRIMINATION
`COUNTERVALUEN
`
`OBJECT DISTANCE
`DISCRIMINATION
`COUNTERVALUEN
`
`WHTEBALANCE CONTROL
`AMOUNTADJUSTMENT VALUE C
`
`WHITEBALANCE CONTROL
`AMOUNTADJUSTMENT VALUE C
`
`OBJECT DISTANCE
`DISCRIMINATION
`COUNERVALUEN
`
`No No NH
`LARGE
`
`N.
`SMA
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 6 of 20 PageID #: 158
`
`WHTEBALANCE CONTROL
`AMOUNTADJUSTMENT VALUE C
`
`
`
`OBJECT DISTANCE
`DISCRIMINATION
`COUNTERVALUEN
`
`Nu No N No N. NH
`SMALL
`LARGE
`
`

`

`U.S. Patent
`
`Apr.
`
`3, 2007
`
`Sheet 5 of 6
`
`US 7,199,821 B2
`
`00¢cLS
`
`
`
`
`
`
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 7 of 20 PageID #: 159
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 7 of 20 PagelD #: 159
`
`
`
`
`
`
`
`
`
`
`
`
`TOWLNOOJONVTVESLIHM||TOKLNODSONV1VESLIKM||TOKLNODSONVTVSLIHM||TOHLNODSONYTVSLIHM||TOXLNODSONVIVESLIHM||JOWLNODZONVIVESLIHM
`
`
`LNNO9NOLWNINIEOSIONOUNSESO”danaKCNHSHO193780INGONOLINSta
`
`
`COCLS|aaims<aneHGRHheLsiS:teclScel
`
`
`NanyaCTORSSG<—NOUWNINOSIOSONWISI5a“SMCORSETNIDNOUWNINOSIO3ONWLSI5INNOO
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LNAWLSAGYLNNOWYINAWLSAPAYLNAOWYLNAWLSAPGYLNNOWYINAWLSAPCYLNNOAYLNAWLSNPOVINNOWYANSWLSNPOYINNOWY
`
`
`
`SONVISIOLOSrEO<CSTWDNHOUMSINNO9NOLVWNINIEOSIC
`nseDSNiWACOHSSEH]NOOSANSaTWAGIOHSSEHLQQDNSLOST"
`
`
`MNOLLYNINIHDSIOSONWLSIG.>a|NOLYNINIHDSIO3ONKLSIG>NANTWA
`
`
`
`
`
`
`NOLLYNINIZOSIC103780——FiouwisicZONHSI0LOHHO>NOLWNIEDSICSONS105
`
`
`
`
`
`Asi103CIOHSEHLINNONOLWNINOSIQFONVISIGLOSraO=
`LINDHaddnNANTINTENON
`OSMAoOSNWAOSTAOANAOANA
`
`[=(NN)CNN}=Cn-'N)/ON-N)=Q=
`SICLDNNTOHSSMSCONVTAEARL
`‘acaNSTOACEHSHALNOKOLAR
`SNIHOLIMSCIOHSSYHLSNIHOLIMS(IOHSSHHL
`
`
`
`LNNOSNOLWNINIYOSICINNO9NOLLWNIWIYOSIC
`
`
`
`
`OZELSOLELS
`
`19780<198° OccLSOleLSOcbLS—Sonyisiasanvisialoso>«=OFELS
`
`
`
`
`
`SONVLSICLOArEO530ASONVLSICLO3PEO
`
`CSANOS103080cSwaios3SVSHONI193ra0as¥3d0a0
`
`
`
`L=0=SANIASINTWA
`OOLLSCyuws)AvSis
`
`
`OOvVLSOns
`
`INSWISOPCYINNOWYTOHLNCO.
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Apr. 3, 2007
`Sheet 6 of 6
`F.G. 5A
`
`US 7,199,821 B2
`
`WHTEBALANCE
`CONTROLSIGNALH
`
`8
`r
`
`3O
`
`WHTEBALANCE CONTROL
`AMOUNTADJUSTMENT
`VALUE C
`
`FIG. 5B
`
`
`
`WHTEBALANCE
`CONTROLSIGNAL1
`
`WHTEBALANCE
`CONTROLSIGNAL2
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 8 of 20 PageID #: 160
`
`
`
`WHEBALANCE
`CONTROLSIGNALn
`
`
`
`
`
`31
`
`WHTEBALANCE CONTROL
`AMOUNTADJUSTMENT
`WAUE C
`
`

`

`US 7,199,821 B2
`
`1.
`IMAGING APPARATUS AND METHOD FOR
`CONTROLLING WHITE BALANCE
`
`BACKGROUND OF THE INVENTION
`
`10
`
`15
`
`30
`
`35
`
`40
`
`25
`
`The invention relates to an imaging apparatus Such as
`Video camera, digital still camera, or the like using an image
`pickup device and, more particularly, to an imaging appa
`ratus having a white balance function and its method.
`In recent years, video cameras have been spread and a
`Video camera by which an image of higher picture quality
`can be easily photographed and which can be readily
`handled is demanded. As one of important requirements for
`making the image to have the high picture quality, it is
`necessary to prevent color deviation (hereinafter, referred to
`as white balance deviation) of an object to be photographed
`in the image. For example, it is important to avoid the
`occurrence of the color deviation (what is called coloring)
`such that a white portion of the image is colored. For this
`purpose, the video camera is equipped with an automatic
`white balance function for automatically correcting the color
`deviation (white balance deviation).
`As an automatic white balance function which is used in
`the video camera or the like, white balance control can be
`mainly classified into two types: what is called an external
`photometric type such that control for directly detecting a
`color temperature of an illumination light Source by a color
`temperature sensor or the like and correcting the white
`balance deviation on the basis of a detection result (herein
`after, such control is referred to as white balance control) is
`made; and what is called an internal photometric type Such
`that the white balance deviation of an object to be photo
`graphed is detected from a video signal or the like and the
`white balance control is made on the basis of a detection
`result. In recent years, the internal photometric type is the
`mainstream in view of simplicity of a construction, costs,
`and the like.
`As a method for the white balance control of the internal
`photometric type, there is a general method of correcting the
`white balance deviation by constructing a feedback loop
`Such that chrominance signals such as color difference
`signals are formed from a signal (that is, image pickup
`signal) which is obtained by photographing an object, a
`white portion is discriminated from the chrominance signals,
`a white portion range of an image is set, and the chromi
`nance signals included in the white portion range are
`extracted and integrated, thereby detecting the white balance
`deviation, and gains of the chrominance signals such as
`primary color signal R (red), primary color signal B (blue),
`and the like are controlled on the basis of a result of the
`detection.
`As an example of Such a method, for instance, there is a
`technique disclosed in Japanese Patent No. 3 193456.
`According to white balance control in Such a technique, two
`color difference signals (R-Y) and (B-Y) and a luminance
`55
`signal Y are formed from primary color signals of R, G
`(green), and B which are obtained by color-separating an
`image pickup signal that is outputted from an image pickup
`device. Further, chrominance signals (R-B) and (R+B-2Y)
`obtained by base-converting the color difference signals by
`executing adding/subtracting processes to the color differ
`ence signals (R-Y) and (B-Y) are formed. A level of a
`threshold value provided in accordance with a level of the
`luminance signal Y is compared with those of the chromi
`nance signals (R-B) and (R+B-2Y), respectively, thereby
`discriminating an achromatic portion (white portion) of the
`object. An extracting region (white portion extracting range)
`
`45
`
`50
`
`60
`
`65
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 9 of 20 PageID #: 161
`
`2
`of the chrominance signals (R-B) and (R+B-2Y) is set. The
`chrominance signals (R-B) and (R+B-2Y) in the extracting
`region are extracted and integrated, respectively. The white
`balance deviation of the achromatic portion (white portion)
`of the object is detected from the signals obtained by the
`integration. By making the white balance control for chang
`ing the gains of the primary color signals R and B in
`accordance with the detected white balance deviation, the
`white balance deviation is corrected.
`According to Such white balance control, the chrominance
`signals included in the specific white portion extracting
`range on color temperature signal coordinates are regarded
`as signals according to the white balance deviation of the
`achromatic portion of the object, and the feedback control is
`made so as to correct the white balance deviation on the
`basis of those signals. Therefore, even if a signal of a
`chromatic portion in which the object is colored is included
`in the white portion extracting range (there is a case where
`if a photographing environment change and a color tem
`perature of the light source changes, for example, even in the
`chromatic portion which is slightly colored, the levels of the
`chrominance signals (R-B) and (R+B-2Y) are equal to or
`less than the threshold value), there is a case of occurrence
`of an erroneous operation Such that such a chromatic portion
`is regarded as an achromatic portion and the correcting
`operation of the white balance deviation is executed.
`To reduce Such an erroneous operation, in the white
`balance control according to the technique disclosed in the
`Japanese Patent No. 3 193456, for example, there is used a
`method whereby the white portion extracting range is finely
`set to a plurality of regions every temperature, and by
`sequentially switching those regions, it is avoided that the
`signal of the chromatic portion of the object is included in
`the white portion extracting range.
`However, as mentioned above, there is actually a case
`where it is difficult to accurately separate the white balance
`deviation of the achromatic portion of the object from the
`chromatic portion which is slightly colored. For example, in
`the case where an object is photographed so that a human
`face (object of a light skin color) occupies the whole picture
`plane, a portion of the human face is erroneously discrimi
`nated as an achromatic (white) portion and the white balance
`deviation is corrected. Thus, the erroneous operation of the
`correction of the white balance deviation Such that an image
`in which the human face is faded into blue is obtained
`OCCU.S.
`As mentioned above, although the conventional white
`balance control is effective to the object having various
`saturation in the picture plane, there is a case where the
`proper correction of the white balance deviation cannot be
`performed to the picture plane in which the object having
`unbalanced Saturation, particularly, the chromatic object in a
`light color occupies at a large ratio.
`As a method of reducing the erroneous operation of the
`correction of the white balance deviation to the object
`having the unbalanced Saturation, for example, a technique
`disclosed in Japanese Patent No. 2532968 has been known.
`In white balance control according to Such a technique, an
`angle of view of a camera lens and a distance to the object
`are obtained, an effective area of an image pickup picture
`plane is obtained, and time constants of gain control of
`chrominance signals are changed in accordance with the
`effective area. If the effective area is small, a method of
`limiting or stopping the gain control of the chrominance
`signals is used. Therefore, in the case of photographing by
`setting the human face so as to be displayed on almost the
`whole image pickup picture plane, either an angle of view is
`
`

`

`US 7,199,821 B2
`
`3
`narrowed due to the Zooming operation or the distance to the
`object is close in Such photographing. Therefore, the effec
`tive area in this case becomes Small and, at this time, the
`control of the chrominance signals is limited or stopped.
`Thus, it is possible to prevent the human face from being 5
`faded into light blue.
`However, in the conventional technique disclosed in
`Japanese Patent No. 2532968 mentioned above, in the case
`of photographing so that the white object or the object of the
`light source occupies almost the whole picture plane, since 10
`the effective area is small, the correcting operation of the
`white balance deviation is limited or stopped. That is, in
`spite of the fact that the white object exists, the white
`balance control is not made. There is, consequently, a
`problem such that the white balance deviation occurs in the 15
`white portion and picture quality of an image deteriorates.
`In the case where the distance to the object is far and the
`object having various saturation exists in the picture plane,
`color components are not always equivalent in the picture
`plane. Therefore, if the white balance control is made in such 20
`a state, although the slight white balance deviation occurs,
`since the white portion is Small on the picture plane, it is
`hardly conspicuous. On the other hand, in a photographing
`state where almost the whole picture plane is occupied by
`the white object by Zooming or approaching the object, 25
`unless the white balance control is made, even if the white
`balance deviation is Small, since the object occupies the
`screen at a large ratio, the white balance deviation is
`conspicuous.
`
`4
`information; and white balance control amount adjustment
`value setting means for comparing the object distance detec
`tion information with the object distance threshold value
`information and obtaining the white balance control amount
`adjustment value according to a result of the comparison.
`Thus, even if the object photographed onto the whole picture
`plane is the white object or the light source, the white
`balance deviation can be optimally corrected.
`The white balance control amount is a change amount of
`a speed or a gain at the time of controlling the gains of the
`chrominance signals. Thus, a control speed or a control
`amount at the time of controlling the gains of the chromi
`nance signals can be varied.
`The larger the Zoom value is, the larger the object distance
`threshold value formed by the object distance threshold
`value setting means is. The darker the object brightness is,
`the larger the object distance threshold value is. Thus, even
`if the object distance is close (short), in the case of a bright
`object, the control amount of the white balance control can
`be adjusted.
`The white balance control amount adjustment value
`changes in a binary or multivalue manner. Thus, the control
`amount of the white balance control can be finely varied.
`The white balance control amount adjustment value
`changes like hysteresis characteristics in accordance with a
`change in object distance. Thus, the distance to the object,
`the Zoom value, and the brightness change and even when
`the object distance detection information and the object
`distance threshold value information have close values, the
`fine switching operation of the control amount of the white
`balance control is unnecessary.
`As mentioned above, even in the case where the object
`which occupies almost the whole image pickup picture plane
`is the white object or the light Source, by changing the
`adjustment value for adjusting the white balance control
`amount on the basis of the information of the object bright
`ness, the control amount of the white balance deviation is
`changed and the white balance deviation can be optimally
`corrected.
`Other objects, features and advantages of the invention
`will become apparent from the following description of the
`embodiments of the invention taken in conjunction with the
`accompanying drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a constructional diagram showing an embodi
`ment of an imaging apparatus according to the invention;
`FIGS. 2A, 2B, and 2C are diagrams each showing an
`example of an image pickup picture plane in the case where
`an object is far, respectively;
`FIG. 3 is a diagram showing a specific example of a
`setting method of an object distance threshold value accord
`ing to the embodiment shown in FIG. 1;
`FIGS. 4A, 4B, 4C, 4D, and 4E are diagrams each showing
`a specific example of a white balance control amount
`adjustment value in the embodiment shown in FIG. 1 and a
`flowchart for the operation; and
`FIGS. 5A and 5B are diagrams each showing a specific
`example of an adjusting method of a white balance control
`amount in the embodiment shown in FIG. 1.
`
`DESCRIPTION OF THE EMBODIMENTS
`
`Embodiments of the invention will be described herein
`below with reference to the drawings.
`
`SUMMARY OF THE INVENTION
`
`30
`
`It is an object of the invention to solve the foregoing
`problems and to provide an imaging apparatus which can
`make optimum correction of white balance deviation even in 35
`the case where an object which is photographed so as to
`occupy almost the whole picture plane is a white object or
`a light source.
`To accomplish the above object, according to the inven
`tion, there is provided an imaging apparatus which has white 40
`balance control means for detecting an achromatic portion of
`an image of an object on the basis of chrominance signals
`formed from an output signal of an image pickup device
`derived via an optical system and controlling gains of the
`chrominance signals in accordance with a deviation amount 45
`of white balance detected from the achromatic portion,
`object distance detecting means for detecting a distance to
`the object, and Zoom value detecting means for detecting a
`Zoom value of the optical system and executes a correcting
`operation of the white balance deviation, comprising: object 50
`brightness detecting means for detecting brightness of the
`object; and white balance control amount adjustment value
`forming means for forming a white balance control amount
`adjustment value to adjust a white balance control amount of
`the white balance control means on the basis of object 55
`brightness information detected by the object brightness
`detecting means, object distance detection information
`detected by the object distance detecting means, and Zoom
`value information detected by the Zoom value detecting
`means, wherein the control amount of the white balance 60
`control means is adjusted on the basis of the white balance
`control amount adjustment value.
`The white balance control amount adjustment value form
`ing means has: object distance threshold value setting means
`for forming an object distance threshold value for discrimi- 65
`nating a magnitude of the distance to the object on the basis
`of the object brightness information and the Zoom value
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 10 of 20 PageID #: 162
`
`

`

`6
`analog image pickup signal in accordance with the object
`image and outputting it. The analog image pickup signal is
`amplified to the predetermined signal level by the AGC
`circuit 3, thereafter, converted into the digital image pickup
`signal by the A/D converting circuit 4, and Supplied to the
`signal processing circuit 5.
`In the signal processing circuit 5, digital primary color
`signals R, G, and B are formed from the inputted digital
`image pickup signal. The gains of the digital primary color
`signals R and B among them are controlled and the white
`balance deviation is corrected. Further, the digital primary
`color signals R, G, and B in which the white balance
`deviation has been corrected are subjected to processes Such
`as gamma correction and the like. The digital luminance
`signal and digital color difference signals (R-Y) and (B-Y)
`are formed from the digital primary color signals R, G, and
`B to which the foregoing correcting process has been
`performed as mentioned above and outputted to the encoder
`6. By using the digital color difference signals (R-Y) and
`(B-Y), by a method similar to the conventional technique,
`an achromatic (white) portion of the object is extracted, and
`white balance deviation of the achromatic portion is detected
`and Supplied as white balance deviation information A to the
`white balance control circuit 8.
`In the white balance control circuit 8, a control signal
`(white balance control signal in the block 8) indicative of the
`white balance control amount is formed on the basis of the
`white balance deviation information A which is supplied
`from the signal processing circuit 5. The object distance
`detection value D indicative of the distance to the object
`which is at present being photographed is Supplied from the
`object distance detecting circuit 11 to the white balance
`control amount adjustment value setting circuit. The object
`distance threshold value G is supplied from the object
`distance threshold value setting circuit 10 to the white
`balance control amount adjustment value setting circuit. The
`white balance control amount adjustment value C is formed
`on the basis of the object distance detection value D and the
`object distance threshold value G. The white balance control
`amount of a white balance control signal B is adjusted on the
`basis of the white balance control amount adjustment value
`C. The gain control signal B of the primary color signals R
`and B according to the adjusted white balance control signal
`is formed. The gain control signal B is Supplied to the signal
`processing circuit 5. As mentioned above, the gain control of
`the primary color signals R and B for the correction of the
`white balance deviation is made by the signal processing
`circuit 5.
`As mentioned above, in a manner similar to the technique
`disclosed in Japanese Patent No. 2532968, a feedback loop
`is constructed in a manner Such that the information of the
`white balance deviation is detected on the basis of the color
`difference signals which are obtained from the primary color
`signals R, G, and B in which the white balance deviation has
`been corrected by controlling the gains of the primary color
`signals R and B, the gains of the primary color signals R and
`B are controlled on the basis of this information, and the
`white balance deviation is corrected. The white balance
`deviation is corrected by such a feedback loop.
`Further, according to the embodiment, in the white bal
`ance control circuit 8, when the gain control signal B of the
`primary color signals R and B is formed on the basis of the
`white balance deviation information A which is supplied
`from the signal processing circuit 5, by controlling the white
`balance control signal by using the white balance control
`amount adjustment value C, the white balance control
`amount is adjusted, thereby adjusting the gain control
`
`Case 5:23-cv-00152-RWS Document 1-5 Filed 12/29/23 Page 11 of 20 PageID #: 163
`
`45
`
`50
`
`60
`
`65
`
`US 7,199,821 B2
`
`10
`
`15
`
`30
`
`35
`
`40
`
`5
`FIG. 1 is a constructional diagram showing an embodi
`ment of an imaging apparatus according to the invention.
`Reference numeral 1 denotes a camera lens; 2 an image
`pickup device; 3 an AGC circuit (automatic gain control
`circuit); 4 an A/D converting circuit, 5 a signal processing
`circuit; 6 an encoder; 7 a D/A converting circuit; 8 a white
`balance control circuit; 9 a white balance control amount
`adjustment value setting circuit; 10 an object distance
`threshold value setting circuit; 11 an object distance detect
`ing circuit; 12 a Zoom value detecting circuit; and 13 an
`object brightness detecting circuit.
`In the diagram, the camera lens 1 comprises a focusing
`lens and a Zoom lens. The image pickup device 2 is a CCD
`image pickup device or the like. The image pickup device 2
`photoelectrically converts an object image formed on an
`image pickup Surface by the camera lens 1 and outputs an
`image pickup signal in an analog form (analog image pickup
`signal). The AGC circuit 3 amplifies the analog image
`pickup signal to a predetermined signal level. The A/D
`converting circuit 4 converts the analog image pickup signal
`amplified by the AGC circuit 3 into an image pickup signal
`in a digital form (digital image pickup signal). The signal
`processing circuit 5 forms a digital luminance signal and
`digital chrominance signals from the digital image pickup
`signal and executes various signal processes such as gamma
`25
`correction and the like. The encoder 6 forms a digital
`television signal according to the standard television system
`such as an NTSC system or the like from the digital
`luminance signal and the digital chrominance signals out
`putted from the signal processing circuit 5. The D/A con
`verting circuit 7 converts the digital television signal into a
`standard television signal in the analog form. The white
`balance control circuit 8 forms a gain control signal for
`controlling gains of primary color signals in the signal
`processing circuit 5 in accordance with information of white
`balance deviation. The object distance detecting circuit 11
`detects a distance to the object to which a focal point has
`been set by the focusing lens of the camera lens 1 and forms
`distance information (assumed to be an object distance
`detection value D here) from a result of the detection. The
`Zoom value detecting circuit 12 detects a Zoom value
`(magnification, focal distance, angle of view, etc.) according
`to the Zoom lens of the camera lens 1 and forms Zoom value
`information (assumed to be a Zoom magnification E here).
`The object brightness detecting circuit 13 detects brightness
`(illuminance, luminance, etc.) of the object and forms
`brightness information (assumed to be object illuminance F
`here). On the basis of the Zoom magnification and the object
`illuminance, the object distance threshold value setting
`circuit 10 forms threshold value information (assumed to be
`an object distance threshold value Ghere) for discriminating
`a magnitude of the object distance detection value from the
`object distance detecting circuit 11 and sets it. The white
`balance control amount adjustment value setting circuit 9
`compares the object distance detection value D with the
`object distance threshold value G mentioned above, forms
`information (hereinafter, referred to as a white balance
`control amount adjustment value C) for adjusting a control
`amount of the white balance control (a change in speed or
`gain at the time of controlling the gains of the primary color
`signals, or the like: hereinafter, referred to as a white balance
`control amount) on the basis of a result of the comparison,
`and sets it.
`The operation of the embodiment will now be described.
`The image pickup device 2 photoelectrically converts an
`object image formed on the image pickup surface by the
`camera lens 1 into an electric signal, thereby forming an
`
`55
`
`

`

`7
`amounts of the primary color signals R and B. Thus, even to
`an object as a single body which occupies almost the whole
`picture plane, the correction of the white balance deviation
`can be precisely made.
`Subsequently, the adjustment of the white balance control
`amount will be described.
`The information E of the Zoom magnification of the Zoom
`lens of the camera lens 1 detected by the Zoom value
`detecting circuit 12 and the information F of the object
`illuminance detected by the object brightness detecting
`circuit 13 are supplied to the object distance threshold value
`setting circuit 10. On the basis of those information, the
`object distance threshold value G is formed, set, and Sup
`plied to the white balance control amount adjustment value
`15
`setting circuit 9. The object distance detection value D
`detected by the object distance detecting circuit 11 is also
`Supplied to the white balance control amount adjustment
`value setting circuit 9 and compared with the object distance
`threshold value G. On the basis of a result of the comparison,
`the white balance control amount adjustment value C to
`adjust the white balance control amount is formed and
`supplied to the white balance control circuit 8. In the white
`balance control circuit 8, on the basis of the white balance
`control amount adjustment value C, the white balance con
`trol amount of the white balance control signal is adjusted.
`On the basis of it, the gain control signal B to control the
`gains of the primary color signals R and B is formed in the
`signal processing circuit 5.
`Examples of the actual white balance deviation and a
`30
`specific example of the object distance threshold value G
`corresponding to them will now be described with reference
`to FIGS. 2A, 2B, 2C, and 3.
`FIG. 2A shows an image pickup picture plane 20a in the
`case where the object distance is far (long). Since objects 21,
`22, and the like having various saturation exist in the image
`pickup picture plane 20a, this picture plane is in an effective
`state to make the white balance control as mentioned above.
`On the other hand, FIG. 2B shows an image pickup
`picture plane 20b in the case where the distance of the object
`21 is close and illustrates a case where the face (object of a
`light skin color) of the human 21 occupies the image pickup
`picture plane 20b at a large ratio.
`In this case, if the white balance control according to the
`conventional technique disclosed in Japanese Patent No.
`3193456 mentioned above is made, the face of the human 21
`is displayed in faded pale blue. If the white balance control
`for Suppressing the erroneous operation to the chromatic
`object with a light color according to the technique disclosed
`in Japanese Patent No. 2532968 mentioned above is made,
`if the distance of the object 21 is close (reliability of color
`information in the picture plane is relatively low), the white
`balance control is stopped, thereby preventing the face of the
`human 21 from being displayed in faded pale blue.
`FIG. 2C shows an image pickup picture plane 20c in the
`case where the distance of the object 22 is close. In this case,
`the white dog (white object) 22 occupies the image pickup
`picture plane 20c at a large ratio. In Such a state, according
`to the conventional technique disclosed in Japanese Patent
`No. 2532968 mentioned above, the effective area is dis
`criminated to be small and the white balance control is not
`made. Therefore, even if slight white balance deviation
`occurs, white balance deviation such that the white dog 22
`is conspicuous from the white color is caused.
`According to the embodiment, in the case of the i