United States Patent (19)
`Usami
`
`US005696840A
`Patent Number:
`11
`45 Date of Patent:
`
`5,696,840
`Dec. 9, 1997
`
`54 IMAGE PROCESSINGAPPARATUS
`75) Inventor: Akihiro Usami, Yokohama, Japan
`73) Assignee: Canon Kabushiki Kaisha, Tokyo,
`Japan
`
`3/1990 Udagawa ................................. 35818O
`4,908,701
`4,945,406 7/1990 Cok ...........
`358/516
`4,962,418 10/1990 Kamaga
`. 358/29
`5,049,985 9/1991 Outa ...
`... 358/80
`5,073,818 12/1991 Iida ........................................... 358/80
`FOREIGN PATENT DOCUMENTS
`258740 3/1988 European Pat. Off..
`21 Appl. No.: 340,881
`266.186 5/1988 European Pat. Off..
`22 Filed:
`Nov. 15, 1994
`Primary Examiner-Joseph Mancuso
`Attorney, Agent, or Firm-Fitzpatric, Cella, Harper &
`Related U.S. Application Data
`Scinto
`63 Continuation of Ser. No. 76,707, Jun. 15, 1993, abandoned,
`ABSTRACT
`57
`which is a continuation of Ser. No. 514,317, Apr. 25, 1990,
`abandoned.
`There is provided an image processing apparatus for execut
`ing a process to an input image in accordance with the
`Foreign Application Priority Data
`30
`characteristics thereof, comprising: an input device to input
`Apr. 28, 1989
`JP
`Japan ................................... 1-109511
`image data; a detector to detect the characteristics of the
`May 10, 1989
`JP
`Japan .................................... -70 input image; and a processor to execute first and second
`[51] Int. Cl. ... G06K9/00
`processes such as brightness correcting processes or color
`52) U.S. Cl. ......................................... 382167.358520
`balance correcting processes for the input image data on the
`58 Field of Search ............................ assississo, basis of weights corresponding to the characteristics of the
`358/526, 500; 356/402, 405, 406; 382ió2.
`input image. When the weight of the first process is large, the
`167
`weight of the second process is set to a small value. As the
`characteristics of the input image are deviated from a
`predetermined condition, the processor sets the weight of the
`References Cited
`first process to a small value and sets the weight of the
`U.S. PATENT DOCUMENTS
`second process to a large value. With the apparatus, the
`brightness and the color balance of the input image are
`: preferably corrected and a natural image can be output
`: E.
`e al. ..
`... 358/80
`irrespective of the characteristics of the image reading
`4,683,492 7/1987 Sugiura ..
`... 358/76
`apparatus.
`4,736,245 4/1988 Seto et al.
`4,853,768 8/1989 Suzuki ..................................... 358/80
`4,883,360 1/1989 Kawada et al. ......................... 356,402
`
`56
`
`9 Claims, 16 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`START
`
`SAMPLE N IMAGE DATA WITHIN MEMORY
`
`S3
`
`Ri, Gi, Bi : i-TH R, G B DATA
`OBTAIN RMAX, GMAX, BMAX TO
`S32
`MAXIMIZE. min (Ri, 6i, Bi)
`IN UNSATURATED Ri, Gil, Bi
`OF i = MN
`
`S33
`
`S34
`
`DMAX=max (Rmax, Gmax, Brnax)
`DMIN=min (Rmax, Gmax, Brax)
`DSADMAX-DMIN
`
`OBTAIN AVERAGE OF N DATA
`AND (MAX-MIN) OF AVERAGE
`N
`AVER=2, Ri/N
`AVEG= Gi/N
`AVEB= 2. Bi/N
`s
`AVEMAX=max (AVER, AVEG, AVEB)
`AVEMIN=min(AVER, AVEG, AVEB)
`AVESA=AVEMAX-AVENIN
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`Sheet 1 of 16
`
`5,696,840
`
`80 LINOW
`
`
`
`
`
`
`
`
`
`
`ŒTI NOI1038800 DT9||
`
`}}-8
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`OUTPUT
`
`Dec. 9, 1997
`Sheet 2 of 16
`FIG 2
`(6) (5)OG) (2) CD
`
`5,696,840
`
`INPUT
`
`FIG 3
`
`OUTPUT
`
`
`
`(6)(SQG)(2)(DINPUT
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 3 of 16
`
`5,696,840
`
`FIG 4A
`
`STAR
`
`SAMPLE N IMAGE DATA WITHIN MEMORY
`1
`Ri, Gi, Bi i-TH R,G,B DATA
`3
`OBTAIN Rmax, Gmax, Bmax T0
`MAXIMIZE min (Ri, Gi, Bi)
`IN UNSATURATED Ri, Gi, Bi
`OF i = 1 MN
`IOMAX=1 WHILE IDMAX=0
`WHEN ALL SATURATED
`
`
`
`DMAX=max (Rmax, Gmax, Bmax)
`DMIN=min (Rmax, Gmax, Bmax)
`DSAEDMAX-DMIN
`
`OBTAIN AWERAGE OF N DATA
`AND (MAX-MIN) OF AVERAGE
`N
`AVER=2, Ri/N
`AVEGE i. Gi/N
`N
`AWEB=
`Bi/N
`AVEMAX=max (AVER, AVEG, AVEB)
`AVEMIN=min (AVER, AVEG, AVEB)
`SAEAVEMAX-AVEMIN
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 4 of 16
`
`5,696,840
`
`
`
`(x.eu/H/XWWO) (XWW.HAW/09)
`
`ON
`
`
`
`
`
`
`
`
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`US. Patent
`
`Dec. 9, 1997
`Dec. 9, 1997
`
`Sheet 5 of 16
`Sheet 5 of 16
`
`5,696,840
`5,696,840
`
`
`
`
`
`<\Axmaw\x<=o.m+wm><\x<=m><vAx<zm><\wevmm>_nx<=oH
`
` @V\Axmsm\x<=o.m+mm><\x<=m><vAx<zm><\wwvvwv<momumo
`
`q\Axmsm\x<zg.m+mw><\x<zm><vAx<zm><\muvNamux<2m><vma
`
`
`
`mm><\x§m><
`
`mm><\x<zm><
`
`cm><\x<zm><
`
`
`
`
`
`mm;
`
`mm;
`
`19>WS
`.mv<m
`
`NeAx<=m><
`
`aonx<on
`
`oz
`
`_wv<m
`
`Nuvx<=m><
`
`onx<on
`
`HTC, Exhibit 1019
`
`ON
`oz
`
`:
`
`19»WS
`.ov<m
`
`
`
`
`
`
`
`
`
`(9)
`
`e«.3mt,
`
`z-g7 99
`
`HTC, Exhibit 1019
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 6 of 16
`
`5,696,840
`
`80|INOW
`
`807100
`
`| -0?š? | } }}}}
`
`ÅHOWBW
`
`6 8
`
`0
`
`l
`
`0
`
`Å TWN9IS O AS
`
`HTC, Exhibit 1019
`
`

`

`US. Patent
`
`m
`
`cm7
`
`04on,69695
`
`mSs32
`
`moczczI$805IIBe:IE38Iwas:IHISEESIEOE:
`
`7m?a
`
`mEul‘SI
`
`mmmm5mmmm5
`
`9,I..I355mEgg“@222:'zoEwmma”5
`
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`Sheet 8 of 16
`FIG 7
`OUTPUT (8)OG6)(5)(DG)(2)O
`
`5,696,840
`
`INPUT
`
`
`
`(8)CDC6)(S)GDG)(2)() INPUT
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 9 of 16
`
`5,696,840
`
`FIG 9A
`
`Ri, Gi, Bi : i-TH R, G, B DATA
`OBTAIN RMAX, GMAX, BMAX T0
`S32
`MAXIMIZE min (Ri, Gi, Bi)
`IN UNSATURATED Ri, Gi, Bi
`OF i = 1 MN
`
`S33
`
`S34
`
`
`
`DMAX=max (Rmax, Gmax, Bmax)
`DMIN=min (Rmax, Gmax, Bmax)
`DSA=DMAX-DMIN
`
`OBTAIN AVERAGE OF N DATA
`AND (MAX-MIN) OF AVERAGE
`AVER= y Ri/N
`
`AWEG ic Gi/N
`
`Bi/N
`AWEB-
`AVEMAX=max (AVER, AVEG, AVEB)
`AVEMIN=min (AVER, AVEG, AVEB)
`AVESA=AVEMAX-AVEMIN
`
`HTC, Exhibit 1019
`
`

`

`US. Patent
`
`Dec. 9, 1997
`
`Sheet 10 of 16
`
`x<2a
`
`Di
`
`mi
`
`53a;
`
`mmm
`
`ooI581%
`
`ONES;Allvass
`
`
`
`x5520m“.V229;
`
`oHm><z
`
`
`
`Tl:covxémz
`
`
`
`V25::2:mi;22:5
`
`zH<._.mo
`WES;AlEaw5
`
`
`”ES;.llisvxéowoo
`
`WEE;
`
`SIV252
`.515“mas
` Al'vaémzw5TI5vx§m><w§
`
`
`
`x<zw><Immm
`Sims"ms;
`
`Tl5%in
`
`mm6E
`
`7mm.oE
`
`«1%6E
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`
`
`

`

`US. Patent
`
`m
`
`04006.,9,0,
`
`Eu_<o<
`
`s5cm<89m,T1svécwommoi<8;
`£V<$><wo<mE5;Pug;flo<V<mw><n«we;SE5;2:50awmmI
`<mm><o<wm><WE
`_n5;Til
`
`
`wa
`
`
`
`<829E<3;22.58
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`

`

`US. Patent
`
`Dec. 9, 1997
`
`Sheet 12 of 16
`
`5,696,840
`
`DDx53x<zm><
`
`<83:—<8;
`
`mmm
`
`ovm
`
`NIDQD\Kmém6E
`
`mmmm
`
`Tom.3“.
`
`Alswwzoommmz._<m.oz.#<m.mz._<mn5mzo>25
`
` sz._<m\m._<mHmz._<m
`
`
`2cm;SEEzozommmoo625%”3.5055:35
`
`
`Egofiézotommmoo325%$38madame:
`
`Ilia:*28;:+iix<=o*<m§|S<m
`ixémi*28;:+|.lix§g383:3;
`
`E:;Em..
`
`m3<>me<mm><o».E>m_._BE;“502km
`I5:5I*2mm;5+x<2o*<wo;lm._<m
`
`22oEémmomzz:052$sz
`
`om><
`
`x<zm><
`
`mm><
`
`
`
`-x55I
`
`33.33SiEeusié
`
`3223;?ms;
`
`
`
`sz._<.m\912mH$2.2m
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`
`

`

`US. Patent
`
`f0
`
`96w,5
`
`0m
`
`
`
`mzoEmES”322.8%
`
`35;3%;551
`
`
`
`
`
`
`
`3—mme;ov2.53;V2.3;_+m_>§*:Imsnus,
`1wo9,5;_+$3*:-m5<eu3%;a.15:17:45???D_153*
`hDDV252IEL5mi;:3m6x53m2V25:ll.513%5+3NI423”m2
`
`
`:-mzéeu5§
` :52:ZSEES5255ass55:22zofiommzoo823$”.38258 Nugm6E
`5:223v25;:8”.5:2220:85.8wmmzzsmm25%
`
`
`
`
`..(m;5E
`ma5we;92
`
`
`
`
`
`
`
`
`
`
`
`mmwzgsmme:m2%232.228558825%E3822m2250
`
`
`
`»5:22zobommmoo
`
`6.,ea
`
`
`
`SE33,n3nmpomxmoowmE23.22305Nm».6%i»5<m;*m<;l|xx.
`
`353%;um»
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`US. Patent
`
`5,696,840
`048.,6
`
`083..$55m$5.3$5.
`
`
`
`
`9Forowo5m>.lln|.|u1w>l||ul+l.«>41
`
`.3_
`,._,..____
`
`6............._.114.__11r...........I:
`
` ____m823818.1ch822853.110”gills”a,$8$5Em$23%$558$5“Em$5$8$589m2%8_o8<m\,o$<m;«<88Nc$<_0x588_n“IIAII
`
`
`_x<z$><&,__
`.....23$238.885.m5%$5:8$5:8ouoaom3838<m3j$535%$<3
`
`“mom.®+hz4mmwmflm
`
`7"54m$5.58939new$5.3<3m$><33$><3
`
`
`
`........+1»ill}--+i{I}:P.I-“:11!3
`
`llllllllll.H-134“uuuuuuuI:P.lllllllifH11111113_.
`
`<83<83<833<83x553a;x383
`
`Q
`<3
`VN
`
`N 8
`
`S
`
`
`
`HTC, Exhibit 1019
`
`HTC, Exhibit 1019
`
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 15 of 16
`
`5,696,840
`
`WDMAX
`1
`
`FIG 1 1A
`
`0
`
`Co
`
`C-
`
`DMAX
`
`255
`
`WAVE
`1
`
`1-b
`
`?
`
`FIG 1 1B
`
`FIG 11C
`
`e
`0 Do D D
`
`255
`
`AVEMAX
`
`WDSA
`
`1- c."
`
`O Bo
`
`r
`B1
`
`DSA
`
`255
`
`WSA
`
`1-d'
`
`1
`
`FIG 1 1D
`
`0 Ao A1
`
`255
`
`AVESA
`
`HTC, Exhibit 1019
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 16 of 16
`
`5,696,840
`
`HOLINOW DI 83000NE DE
`0OTWNW I
`
`
`
`
`
`-0EHH00
`
`NOI!
`
`
`
`HTC, Exhibit 1019
`
`

`

`5,696,840
`
`1.
`IMAGE PROCESSINGAPPARATUS
`
`This application is a continuation of application Ser. No.
`08/076,707 filed Jun. 5, 1993, which was a continuation of
`application Ser, No. 07/514317 filed Apr. 25, 1990, both
`now abandoned.
`
`5
`
`2
`FIG. 6 is a circuit block diagram of the third embodiment
`of the invention;
`m
`FIGS. 7 and 8 are diagrams showing contents of correc
`tion tables written in an ROM 43:
`FIGS. 9A to 9C are flowcharts for the third embodiment
`of the invention;
`FIG. 10 is a schematic diagram showing processes in the
`third embodiment of the invention;
`FIGS. 11A to 11D are diagrams showing contents of
`O
`correction tables based on other membershipfunctions in the
`third embodiment of the invention; and
`FIG. 12 is a circuit block diagram of the fourth embodi
`ment of the invention.
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention relates to an image processing
`apparatus for executing a process according to characteris
`tics of an input image.
`2. Related Background Art
`A color image signal from an image reading apparatus
`having an image pickup device, for instance, from an SV
`(still video) apparatus is digitized and subjected to signal
`processes such as a masking process and the like and,
`thereafter, it is color printed. Or, before the color image
`signal is digitized, the level for digitization is adjusted by an
`automatic gain controller (AGC) or a white balance con
`troller.
`However, for the color image signal from the foregoing
`image reading apparatus, a method of setting a color balance
`and a recording level (brightness) differ depending on the
`manufacturing maker of such an apparatus or the type of
`apparatus. Therefore, if a color image signalis simply stored
`into a memory and displayed by a color monitor or is
`processed and color printed, there are problems such that the
`color balance is lost and the resultant color image is slightly
`blue or slightly red or is dark.
`SUMMARY OF THE INVENTION
`It is an object of the present invention to preferably
`process an input image.
`Another object of the invention is to preferably correct an
`input image.
`Still another object of the invention is to preferably
`correct the color balance.
`Further another object of the invention is to preferably
`correct the brightness.
`Further another object of the invention is to prevent that
`an improper image process is executed.
`Further another object of the invention is to prevent that
`an input image is excessively corrected.
`Further another object of the invention is to prevent that
`an extreme process is executed to an input image.
`Further another object of the invention is to output a
`natural image irrespective of the characteristics of an image
`from reading apparatus.
`The above and other objects and features of the present
`invention will become apparent from the following detailed
`description and the appended claims with reference to the
`accompanying drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram showing a circuit construction
`of the first embodiment of the present invention.
`FIGS. 2 and 3 are diagrams showing contents of correc
`tion tables which are previously written into an ROM 13;
`FIGS. 4A and 4B are flowcharts for the first embodiment
`of the invention;
`FIG. 5 is a block diagram showing a circuit construction
`of the second embodiment of the invention;
`
`15
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`DETALED DESCRIPTION OF THE
`PREFERRED EMBODEMENTS
`First embodiment
`According to the embodiment, the minimum value of
`each component signal of a given color image signal is
`obtained every pixel and the color image signal is corrected
`on the basis of a ratio of the component signals of the pixels
`in which the minimum value satisfies a predetermined
`condition.
`According to the embodiment, there is disclosed an appa
`ratus using a method whereby an average value and the
`maximum RGB values which are considered to be a white
`level are obtained from RGB image signals which were
`fetched into a memory from an SW camera or an SV floppy
`and image data is corrected by using the average value and
`the maximum RGB values. However, the invention is not
`limited to such an embodiment but can be also applied to
`other various methods.
`FIG. 1 is a circuit block diagram showing a construction
`of the first embodiment of the invention. Image signals from
`an SV floppy or an SV camera which are input by, for
`instance, a Y signal and a C signal (color difference signal)
`are input to an analog decoder1. In the analog decoder 1, the
`conversion into analog RGB signals and the correction of
`the levels of the analog signals by an AGC are executed.
`Then, the analog signals are digitized by an A/D converter
`2. The digital signals are stored into a memory 3 or 8 each
`having a memory capacity of one picture plane. A CPU 7
`extracts N points (1sNsthe number of all of the pixels)
`from the image data in the memory and processes the data.
`The CPU 7 selects or makes the optimum correction table
`from an ROM 13 and sets into a correction table 4 or 9. In
`the case of selecting and setting the optimum correction
`table, the CPU 7 selects one of curves CD to (6) in FIG. 3
`and sets into the correction table 4. The CPU 7 selects one
`of the curves CD to G)in FIG. 2 and sets into the correction
`table 9.
`A method of selecting the correction table by using the
`image data in the memory will now be described. FIGS. 4A
`and 4B are flowcharts showing procedures for such pro
`cesses. First, the CPU 7 sequentially reads out Nimage data
`from the memory (step 1 in FIG. 4A). Then, the CPU 7
`extracts the pixel data whose signal values are not saturated
`(for instance, in the case of 8bits, the signal value is not 255)
`from among signal values R, G, and B, (ith pixel data;
`1sisN) of RGB. The CPU 7 then sets the pixel whose
`min(R,G,B) (that is, the minimum value among R, G,
`and B) is the largest among the pixel data into R, G,
`and B of the maximum RGB values and sets such that
`DMAX=1. On the other hand, when all of the signal values
`have been saturated, it is set such that IDMAX=0 (step 3 in
`
`HTC, Exhibit 1019
`
`

`

`5,696,840
`
`10
`
`15
`
`20
`
`25
`
`30
`
`3
`FIG. 4A). In the case of the pixel which is slightly red,
`min(R,G,B)=R. In the case of the pixel which is slightly
`green, min(R,G,B)=G. In the case of the pixel which is
`slightly blue, min(R, G, B}=B.
`The maximum values R, G, and B obtained are
`considered to be the pixel data of the portion indicative of
`white in one picture plane which was stored in the memory
`3 or 8. Therefore, assuming that the difference between the
`maximum value and the minimum value of R, G, and
`B is set to DSA, if DSA=0, this means that the color
`balance (white balance) is obtained. However, when
`DSAé0, it is necessary to correct so as to obtain DSA=0. For
`R. G. and B,
`the min(R,G,B) is not necessarily
`set to the maximum pixel but may be setto, for instance, the
`second or third pixel from the maximum pixel. On the other
`hand, for example, it is also possible to calculate the average
`value of at most ten pixels.
`In step 7 in FIG. 4A, the CPU 7 calculates average values
`AVER, AVEG, and AVEB of Nimage data, that is, all of the
`pixel data of one picture plane and calculates the difference
`between the maximum average value and the minimum
`average value and sets the difference into SA. If SA=0, the
`average density in one picture plane corresponds to achro
`matic color and this means that the color balance of the
`original image is obtained to a certain degree. This corre
`sponds to the case where the theorem of Evans which is used
`when silver saltprinting a transmission film is applied to the
`image signal from the SV. When SA70, this means that the
`color balance is deviated, so that it is necessary to correct so
`as to obtain SA=0.
`A condition section to select the correction table will now
`be described by using processing steps 14 to 19 in FIG. 4B.
`In FIG. 4B, G, G, G, G, and Gs are preset constants. For
`instance, G=55, G=127, G=30, G-50, and G=50. Even
`if the above values are slightly deviated, the picture quality
`of the output image does not largely deteriorate and those
`values can be also set by shifting by about +20.
`It is the fundamental idea that an SW image in which the
`difference SA of the average values and the difference DSA
`between the maximum and minimum values of the maxi
`mum RGB values are respectively smaller than certain
`values G and G is used as an object for the color balance
`correction, while an image in which the maximum value
`AVEMAX of the average values is darker than a certain
`value G and is brighter than a certain value G is also
`subjected to the correction so as to brighten such an image.
`Further, the brightness is corrected by the average value and
`the color balance is corrected by more significantly consid
`ering the maximum RGB values rather than the average
`50
`value.
`In step 14 in FIG. 4B, an image in which the maximum
`value AVEMAX of the average value is larger and brighter
`than G in the case where the deviations of the balances of
`both of SA and DSA lie within a range of certain values G.
`and G and in which the maximum RGB values are not
`saturated is used for the color balance correction. The CPU
`7 corrects the color balance of such an image by using the
`maximum RGB values.
`That is,
`
`35
`
`45
`
`55
`
`Y=DMAX/G,
`Y=DMAX/B
`where, Y, Y, and Y denote gamma values and indicate
`inclinations in FIG. 2.
`In step 15 in FIG. 4B, the condition of the maximum value
`AVEMAX of the average value differs from that in step 14
`
`65
`
`4
`in FIG. 4B. Step 15 relates to the case where the AVEMAX
`is smaller than G and larger than Gs, that is, the case where
`the image is dark to a certain degree. In this case, the CPU
`7 uses the average value as a correction to brighten and
`corrects the color balance in a manner similar to the process
`in step 14 in FIG. 4B. That is,
`Y=(G/AVEMAX) (DMAX(R)
`Y=(G/AVEMAX) (DMAX/G)
`Y=(G/AVEMAX) (DMAX/B)
`Steps 16 and 17 in FIG. 4B relate to the processing steps
`to discriminate the case where the difference DSA of the
`maximum RGB values is slightly large. Since there is a
`possibility such that a pixel in which the maximum RGB
`values are slightly deviated from the white portion in one
`picture plane was picked up, the CPU 7 corrects the color
`balance by using both of the maximum RGB values and the
`average value.
`When the maximum value AVEMAX of the average value
`is larger than G and the image is bright, in step 16 in FIG.
`4B, the following processes are executed.
`Y=(AVEMAX/AVER+3DMAX/R).f4
`Y=(AVEMAXIAVEG+3DMAX/G)/4
`Y=(AVEMAX/AVEB+3DMAX/B)/4
`On the other hand, when the maximum value AVEMAX
`of the average value is Smaller than G and larger than G,
`that is, when the image is dark to a certain degree, in step 17
`in FIG. 4B, the following processes are executed.
`Y=(GJAVEMAX) (AVEMAXIAVER+3DMAX/R)/4
`Y=(G/AVEMAX) (AVEMAXIAVEG+3DMAX/G)
`f4
`Y=(G/AVEMAX) (AVEMAX/AVEB+3DMAX/B)/4
`Steps 18 and 19 in FIG. 4B relate to the case where the
`value of the difference DSA of the maximum RGB values is
`further large. In this case, further, under the condition such
`that the difference SA of the average value is small and the
`maximum RGB values are saturated, the color balance is
`corrected by using the average value.
`In the case where the AVEMAX is larger than G and the
`image is bright, the following processes are executed.
`Y=AVEMAX/AVER
`Y=AVEMAX/AVEG
`Y=AVEMAX/AVEB
`On the other hand, in the case where the AVEMAX is
`smaller than G and the image is dark, the following
`processes are executed.
`Y=GAVER
`Y=G/AVEG
`y=GAVEB
`In the case of the image which does not satisfy the
`conditions in steps 14 to 19, the CPU 7 sets Y=y-Y=1 so
`as not to correct anything. As mentioned above, Y, Y, and
`Y denote the inclinations in FIG. 2. The CPU 7 selects the
`inclinations near the Y obtained among CD to G) in FIG.2
`for RGB, respectively, and sets into the correction table 9.
`On the other hand, in correspondence to either one of the
`gamma correction curves Oto (6), the CPU 7 also selects
`the table in FIG. 3 and sets into the correction table 4. As
`mentioned above, the correction tables 4 and 9 are set.
`The processes after the correction tables were set will now
`be described.
`When the correction table 4 is set, the SV image in the
`memory 3 is subjected to the degamma process (the given
`image signal is raised to 2.2nd power) and the density
`converting (logarithmic conversion) process by the correc
`
`HTC, Exhibit 1019
`
`

`

`5
`tion table 4 together with the corrections of the color balance
`and brightness. That is, the correction table 4 converts each
`of the RGB signals of the SVimage which was read out of
`the memory 3 in accordance with the tables which were
`independently set and outputs as Y. H, and C signals. In the
`next masking porcess, the YHC signals are subjected to the
`processes such as elimination of impurity of the color,
`undercolor removal, and the like, so that the signals of Y, M,
`C, Bk (yellow, magenta, cyan, black) are obtained and
`output to a printer 6. As a printer 6, any one of the
`electrophotographic type, inkjet type, and thermal copy
`transfer type can be used.
`When the correction table 9 is set, the color balance and
`brightness of the image data of the memory 8 are corrected
`by the correction table 9. The corrected image data is
`converted into the analog signals by a D/A converter 10 and
`converted into the luminance (Y) signal and color difference
`signal for monitor by an analog encoder 11 and displayed by
`a color monitor 12.
`In the above description, one of the gamma correction
`curves to be written into the tables 4 and 9 has been selected
`from among the correction tables CD to (6) in FIGS. 2 and
`3. However, such a gamma correction curve may be also
`formed from the values of Yi, Y, and Y by the CPU 7. On
`the other hand, the number of tables to be selected can be
`25
`also increased or decreased as necessary. On the other hand,
`although the brightness has been corrected by using the
`average value of the whole picture plane, it is also consid
`ered to use the maximum RGB values or to combine both of
`the average value and the maximum RGB values. In
`addition, although the maximum RGB values have prefer
`entially been used to correct the color balance, it is also
`possible to change the order of priorities.
`Further, although the embodiment has been described
`with respect to the case of processing by the RGB, it is also
`possible to input yellow, magenta, and cyan signals and to
`process.
`Second embodiment
`On the other hand, as shown in FIG. 5, a correction table
`20 can be also provided before the memory 3. That is, the
`correction table 20 is once set such that YY Y-1, the
`image signals are stored into the memory 3, and the color
`balance and brightness are reset into the correction table 20
`from the values of Y, Y, and Y which were examined and
`obtained by the processes of FIG. 4. Furthermore, the image
`is again stored into the memory 3 or 8 from the SV camera
`or SV floppy and the image whose color balance and
`brightness were corrected by the correction table 20 is
`written into the memory.
`In this case, in the correction table 4, the degamma
`process and the density conversion are executed. In the
`correction table 9, the conversion is not performed. It is also
`possible to construct in a manner such that the degamma
`process is executed in the correction table 20, the density
`conversion is performed in the correction table 4, and the
`gamma process is performed in the correction table 9.
`On the other hand, in the embodiment, although the
`process to make the image bright is executed for only the
`dark image, it is also possible to add a method of correction
`such as to return the image which is too bright to the image
`of the normal brightness.
`As described above, according to the embodiment, by
`sampling the image signal and by selecting or making a
`correction table so as to correct the color balance and
`brightness, the deviation of the color balance or the differ
`ence of the recording level (brightness) which is caused due
`to the difference of the makers of the still video camera or
`
`35
`
`45
`
`SO
`
`55
`
`65
`
`5,696,840
`
`O
`
`15
`
`20
`
`30
`
`6
`due to the difference of the apparatuses can be corrected and
`there is an effect such that a natural preferable image is
`obtained.
`Although the explanation has been made with respect to
`the example in which the image which was photographed by
`the still video camera is corrected, the invention is not
`limited to such an example but can be also applied to other
`image signal such as a color image signal which was
`obtained by photoelectrically converting an image of a film
`by a film scanner.
`As mentioned above, according to the embodiment, the
`color balance can be also preferably corrected.
`Third embodiment
`According to the third embodiment, the white level and
`the average level of the input color image signals are
`calcualted, further, the dependencies on the white level and
`average level of the color balance correction and brightness
`correction are calculated, and thereby executing the correc
`tion based on the dependencies calculated and thereby
`executing the optimum correction to the color image signals
`on the basis of the white level and average level.
`FIG. 6 is a circuit block diagram of the embodiment. First,
`the operation of FIG. 6 will be described. The image signal
`is extracted from the SV (still video) camera of from the SV
`floppy in which the image which was photographed by the
`SV camera was stored. The extracted image signal is input
`to an analog decoder 31. The conversion from the R (red),
`G (green), and B (blue) digital signals into the analog signals
`is executed and the levels of the analog signals are corrected
`by the AGC (automatic gain controller). The input signals
`are not limited to the RGB signals but aY (luminance) signal
`and a C (chrominance) signal can be also used as shown in
`FIG. 1.
`The level correction of the analog signals is executed as
`a pre-processing for making it easy to execute the correction
`of the digital signals which is executed subsequently.
`The analog signals are then converted into the digital
`signals by an AD (analog/digital) converter 32 and the
`digital siganls are stored into a memory 33 or 38.
`A CPU 37 extracts N points (1sNs the number of all of
`the pixels) from the image data in the memory and processes
`the data. Then, the CPU 37 selects the optimum correction
`table from a plurality of tables which have previously been
`registered in an ROM 43 and sets into a correction table 34
`or 39. One of correction curves CD to (6) in FIG. 8 is
`selected and set into the correction table 34. One of correc
`tion curves CD to (6) in FIG. 7 is selected and set into the
`correction table 39.
`The selection of the correction table will now be
`described. FIG. 10 is a diagram to obtain the inclination of
`FIG. 7 of the correction table. FIGS. 9A to 9C are flowcharts
`to obtain the inclination of the correction table.
`First, the CPU 37 sequentially reads out N images data
`from the memory 33 (step S31) and extracts the pixel data
`in which the signal value is not saturated (for instance, when
`the image data is constructed by eight bits, assuming that
`pure white is set to 255, the case where the signal value is
`not saturated corresponds to a value other than 255) among
`signal values R, G, and B (ith pixel data; 1sisN) of R, G,
`and B. This is because if even one of R, G, and B is
`saturated, the data is deviated from the original image, so
`that the color balance of the original image cannot be
`accurately judged.
`The CPU 37 obtains the pixel in which the minimum
`value among the extracted signal values R, G, and B, is the
`largest among the N pixels. Then, the CPU 37 sets the R, G,
`and B components of such a pixel into RMAX, GMAX, and
`
`HTC, Exhibit 1019
`
`

`

`7
`BMAX (step S32). The minimum value among R, G, and
`B, shows the feature of the pixel data. That is, in the case of
`a pixel which is slightly red, the minimum value of R, G,
`and B is set to R. The pixel in which the minimum value
`among R,G, and B, is largest denotes the pixel in which the
`components indicative of the feature of the pixel data are
`smallest. Therefore, RMAX, GMAX, and BMAX are con
`sidered to be the R, G, and B components of the pixel
`indicative of the white portion in the SVimage. The CPU 37
`sets the maximum value among RMAX, GMAX, and
`BMAX into DMAX and also sets the difference between the
`maximum value and the minimum value into DSA (step
`S33). DMAX corresponds to the smallest one of the R, G,
`and B components of the pixel which is considered to be
`white. When DSA=0, this means that the color balance
`(white balance) of the pixel which is considered to be white
`is obtained. However, when DSA70, it is necessary to
`correct such as to obtain DSA=0.
`Then, the CPU 37 obtains the average values AVER,
`AVEG, and AVEB of the Nimage data R, G, and B. The
`maximum value among AVER, AVEG, and AVEB is set into
`AVEMAX and the difference between the maximum value
`and the minimum value is set into AVESA (step S34).
`AVEMAX corresponds to the smallest one of the R, G, and
`B components of the average pixel data. If AVESA=0, the
`average density of the SV image denotes an achromatic
`color and shows that the color balance of the SV image is
`obtained to a certain degree. Such a case corresponds to that
`the theorem of Evans which is used when a transmission film
`is silver saltprinted is applied to the SVimage. If AVESA70,
`this means that the color balance is deviated. Therefore, it is
`necessary to correct such as to obtain AVESA=0.
`In FIG. 10, reference numerals 1-a, 1-b, 1-c, and i-d
`denote membership functions with respect to the maximum
`value DMAX among the components of the white level, the
`35
`maximum value AVEMAX among the average values of R,
`G, and B, of all of the image data, a chromatic color
`chromaticity DSA of the white level, and the chromatic color
`chromaticity AVESA of the average value of all of the image
`data. The membership functions 1-a, 1-b, 1-c, and 1-d have
`previously been registered as tables in the ROM 43, respec
`tively.
`The CPU 37 obtains the grade of a white level depen
`dency WDMAX for brightness correction from the maxi
`mum value DMAX of the white level by the membership
`45
`function 1-a (step S35). The membership function 1-a is set
`in a manner such that when the maximum value DMAX of
`the white level is large, a significance is paid to the white
`level than the average value of all of the image data when the
`brightness is corrected. That is, the fact that the value of
`DMAX is large denotes that the pixel which was determined
`to show white is a pixel which is actually close to white. On
`the contrary, the fact that the value of DMAX is small means
`that the pixel which was decided to show white is a pixel
`which is actually far from white and, therefore, an impor
`55
`tance cannot be paid to the white level extracted from N
`image data. In more detail, the membership function 1-a
`specifies the relation between the maximum value DMAX of
`the whiteleveland the white level dependency WDMAX for
`the brightness correction in the following manner. That is,
`when DMAX is larger than a certain value C, WDMAX=1.
`When DMAX is smaller than a certain value Co (CCC),
`WDMAX-0. When DMAX has a value within a range from
`Co. to C, WDMAX gradually changes from 0 to 1.
`For example, Co-60 and C=127. Even if those values are
`slightly deviated, the picture quality of the output image
`does not largely deteriorate. Those values can be also set by
`
`50
`
`25
`
`30
`
`65
`
`5,696,840
`
`10
`
`15
`
`8
`deviating by about +20. On the other hand, for instance, the
`membership function 1-a can be also set as shown in FIG.
`11A.
`Then, the CPU 37 obtains the grade of a brightness
`correction ratio WAVE from the average maximum value
`AVEMAX by the membership function 1-b (step S36). The
`membership function 1-b is set in a manner such that the
`degree to correct the brightness decreases as the average
`maximum value AVEMAX is smaller or larger than the
`central value. That is, when the input image is inherently
`dark, if the correction to brighten the image is executed, the
`characteristics of the input image will have been changed
`than they