EXHIBIT 1061
`
`U.S. PATENT NO. 7,304,670 TO HUSSEY et al.
`
`(“the ‘670 Patent”)
`
`TRW Automotive U.S. LLC: EXHIBIT 1061
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NUMBER 8,599,001
`IPR2015-00436
`
`

`
`US007304670B1
`
`(12) Ulllted States Patent
`Hussey et a].
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,304,670 B1
`Dec. 4, 2007
`
`(54) METHOD AND APPARATUS FOR
`COMPENSATING FOR FIXED PATTERN
`NOISE IN AN IMAGING SYSTEM
`
`(75) Inventors: Robert M. Hussey, Liverpool, NY
`(US)' William H Havens Skaneateles
`’
`'
`’
`’
`NY (Us)
`
`(73) Assignee: Hand Held Products, Inc., Skaneateles
`Falls, NY (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 08/828,340
`
`(22) F1led:
`
`Mar. 28, 1997
`
`(51) Int- Cl-
`(2006.01)
`H04N 5/217
`(200601)
`G06K 7/10
`(52) US. Cl. ................................ .. 348/241; 235/462.11
`(58) Field of Classi?cation Search .............. .. 348/241,
`348/243,247, 250, 251,254; 382/261,373;
`358/406, 504; 235/462.11,462.24
`-
`.
`.
`S
`?l f
`l t
`h h t
`.
`t
`l
`ee app 10a Ion e or Comp e e Seam 15 Dry
`References Cited
`
`(56)
`
`6/1992 Willsie
`5,120,940 A
`5,153,418 A 10/1992 Batterman et a1.
`5,189,292 A
`2/ 1993 Batterman et a1.
`5,212,777 A
`5/1993 Gove et a1.
`5’223’70l A
`6/1993 Batterman et 31'
`5’243’655 A
`9/1993 Wang
`5,260,554 A 11/1993 Grodevant
`5,262,623 A 11/1993 Batterman et a1.
`5,286,960 A
`2/1994 Longacre, Jr. et a1.
`
`EP
`EP
`EP
`EP
`GB
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`978990 A2
`2/2000
`1050793 A2 11/2000
`978990 A3
`4/2001
`1050793 A3
`9/2004
`2 357 209 A
`6/2001
`(Continued)
`
`OTHER PUBLICATIONS
`
`VVL 1070 Engineering Evaluation Kit Speci?cation, Sep. 27, 1994,
`V11’ PP' 1'5'
`
`(Continued)
`,d
`_
`_
`Pr’mry ExammerfDewl Ome‘Z
`Assistant ExamineriLuong T. Nguyen
`(74) Attorney, Agent, or FirmiMarjama Muldoon Blasiak
`& Sullivan LLP
`
`US. PATENT DOCUMENTS
`
`(57)
`
`ABSTRACT
`
`_
`_
`_
`_
`There 15 descnbed a Plxe1 Value adlustmem method and
`apparatus. In one embodiment, pixel values can be adjusted
`by execution of an algorithm for adjusting pixel values. In
`one embodiment, an apparatus capable of executing an
`algorithm for adjusting pixel values is capable of operating
`in a decoding operating application. In one embodiment, an
`apparatus capable of executing an algorithm for adjusting
`p1xel tvalues
`capable of operat1ng 1n a v1deo d1splay
`Operamg aPP 1C2‘ 1011'
`
`-
`
`.
`
`.
`
`.
`
`~
`
`-
`
`8/1972 Christie et a1.
`3,684,868 A
`7/1986 Temes ...................... .. 348/243
`4 602 291 A *
`2/I9g9 Kley
`4,806,776 A
`2/ 1989 Yamamoto et a1, _______ __ 348/251
`4,806,780 A *
`4,873,426 A 10/1989 Sarna et a1.
`4,874,936 A 10/1989 Chandler et a1~
`4,896,029 A *
`1/1990 Chandler et a1~
`i
`et :11‘
`""""""" " 348/246
`4,998,010 A
`W199 1 Chandler et a1‘
`5,038,391 A
`8/1991 Yamaguchi
`5,047,861 A *
`9/1991 Houchin et a1. .......... .. 348/247
`5,091,975 A
`2/ 1992 Berger et a1.
`
`,
`
`,
`
`lIl'lO 0 e
`
`.
`
`73 Claims, 7 Drawing Sheets
`
`DSYNOH 1
`
`EXP!) RE
`
`1061-001
`
`

`
`US 7,304,670 B1
`Page 2
`
`Us. PATENT DOCUMENTS
`
`3/1994 Hammond, Jr. et a1.
`5,294,783 A
`3/1994 Chandler et a1.
`5,296,690 A
`4/1994 Johnson et a1. ........... .. 358/406
`5,303,064 A
`4/1994 Wang
`5,304,787 A
`4/1994 Haysshi et a1.
`5,305,122 A
`5/1994 Joseph et a1.
`5,311,001 A
`8/1994 Figarella et a1.
`5,343,028 A
`9/1994 Denyer
`5,345,266 A
`1/1995 Adachi
`5,378,881 A
`1/1995 Batterman et a1.
`5,378,883 A
`2/1995 Schlack et a1.
`5,392,447 A
`3/ 1995 Krichever er 91-
`5,396,054 A
`4/1995 Koenck et a1.
`5,410,141 A
`5/1995 Smith
`5,412,197 A
`5/1995 Durbin
`5,414,251 A
`5/1995 Zheng eta1~
`5,418,862 A
`5/ 1995 Longacre, Jr et 91-
`5420409 A
`5/1995 Mak
`5420943 A
`6/1995 Kubo
`5422470 A
`6/1995 Zhéng et 91
`5428211 A
`6/1995 Tam er a1~
`5,428,212 A
`7/1995 Hammond, Jr. et a1.
`5,430,286 A
`5471515 A 11/1995 Possum er a1~
`5477042 A 12/1995 Wang
`5,478,999 A 12/1995 Figarella et a1.
`5,487,115 A
`V1996 Surka
`5489769 A
`2/1996 Kubo
`5504524 A
`4/1996 Lu eta1~
`5,513,264 A
`4/1996 Wang et 91-
`5,519,441 A
`5/1996 Gusmano et a1. ......... .. 358/504
`5,524,068 A
`6/1996 Kacandes er 91-
`5,534,684 A
`7/1996 Danielson
`5,537,431 A
`7/1996 Chen eta1~
`5,545,886 A
`8/1996 Metlitsky er 91-
`5550366 A
`8/1996 Rommel
`5565669 A 10/1996 Liu
`5,572,006 A 11/1996 Wang et a1.
`5,591,955 A
`V1997 Laser
`5,591,956 A
`1/1997 Longacre, Jr. et a1.
`5,612,524 A
`3/1997 Sant’Anselmo et a1.
`5,637,849 A
`6/1997 Wang et a1.
`5,638,465 A
`6/1997 $11110 er a1~
`5642442 A
`6/1997 Morton er 91-
`5,646,390 A
`7/1997 Wang et a1.
`5,663,549 A
`9/1997 KHZ eta1~
`5,666,167 A
`9/1997 Tults
`5,698,833 A 12/1997 Skmger
`5,702,059 A 12/1997 Chu et a1.
`5,703,349 A 12/1997 Meyerson et a1.
`57101417 A
`V1998 Joseph er 91-
`5714745 A
`2/1998 Ju er 91-
`5,723,853 A
`3/1998 Longacre, Jr et 91-
`5,723,868 A
`3/1998 Hammond, Jr et 91-
`5726435 A
`3/1998 Hm eta1~
`5739518 A
`4/1998 Wang
`5756981 A
`5/1998 Roustaei er 91-
`5774357 A
`6/1998 Ho?herg eta1~
`5,780,834 A
`7/1998 Havens et 91
`5,784,102 A
`7/1998 Hussey et a1. ............ .. 348/296
`5,798,847 A
`8/1998 Aerts .......... ..
`348/251
`5,801,962 A
`9/1998 Sheu et a1. ............... .. 358/406
`5,814,801 A
`9/1998 Wang et 91-
`5,818,028 A 10/1998 Meyerson er 91-
`5,821,518 A 10/1998 Sussmeier et a1.
`
`
`
`5,821,523 A 10/1998 Bunte et 31. 5,825,006 A 10/1998 Longacre, Jr. et a1.
`
`5,834,754 A 11/1998 Feng et a1.
`5,857,029 A
`1/1999 Patel
`5,867,594 A
`2/1999 Cymbalski
`5,867,595 A
`2/1999 Cymbalski
`5,875,108 A
`2/1999 Hof?aerg et a1.
`
`3/1999 Tani et a1.
`5,877,487 A
`6/1999 Cymbalski
`5,917,945 A
`7/1999 Feng
`5,920,061 A
`7/1999 H01?)erg et a1‘
`5,920,477 A
`7/1999 Denyer et a1‘
`5926214 A
`8/1999 Hussey et a1.
`5,932,862 A
`9/1999 Karpen et a1‘
`5,949,054 A
`9/1999 White
`5,949,056 A
`5,979,763 A 11/1999 Wang et a1‘
`5,992,753 A 11/1999 Xu
`6,003,008 A 12/1999 Postrel et a1.
`6,017,496 A
`1/2000 Nova et a1‘
`6,019,286 A
`2/2000 Li et 31‘
`6,034,379 A
`3/2000 Bunte et a1. .............. .. 382/313
`6,053,407 A
`4/2000 Wang et a1‘
`6,062,475 A
`5/2()()() Feng
`6,064,763 A
`5/2()()() Maltsev
`6,070,800 A
`6/2000 Fujita et a1.
`6,075,240 A
`6/2000 Watanabe et al.
`6,082,619 A
`7/2000 Ma et a1.
`6,094,509 A
`7/2000 Zheng et a1.
`6,119,179 A
`9/2000 Whitridge et a1.
`6,123,261 A
`9/2000 Roustaei
`6,129,278 A 10/2000 Wang et al‘
`6,157,027 A 12/2000 Watanabe et a1.
`6,170,749 B1
`1/2001 Goren et a1.
`6,176,428 B1
`1/2001 Joseph et 31‘
`6,215,992 B1
`4/2001 Howel et a1.
`6,232,973 B1
`5/2001 Dow et al.
`6,264,015 B1
`7/2001 De Kock
`6,298,176 B2 10/2001 Longacre, Jr. et a1.
`6,304,313 B1
`l0/200l Honma
`6,329,139 B1
`12/2001 Nova et a1.
`6,347,163 B2
`2/2002 Roustael
`6,375,075 B1
`4/2002 Ackley et a1.
`6,486,911 B1
`11/2002 Denyer et al.
`6,491,223 B1
`12/2002 Longacre, Jr. et a1.
`6,493,029 B1
`12/2002 Denyer et a1.
`6,508,404 B2
`1/2003 Hecht
`6,547,139 B1
`4/2003 Havens et a1.
`6,547,142 B1
`4/2003 Goren et a1‘
`6,561,428 B2
`5/2003 Meier et 31‘
`6,565,003 B1
`5/2003 Ma
`6,585,159 B1
`7/2003 Meier et a1.
`6,606,171 B1
`8/2003 Renk et a1.
`6,621,598 B1
`9/2003 Oda
`6,655,595 B1
`12/2003 Longacre, Jr. et a1.
`6,655,597 B1
`12/2003 Swartz et a1.
`6,678,412 B1
`1/2004 Shigekusa et a1.
`6,688,523 B1
`2/2004 Koenck
`6,722,569 B2
`4/2004 Ehrhart et a1‘
`6,732,929 B2
`5/2004 Good et a1.
`6,736,321 B2
`5/2004 Tsikos et a1.
`6,739,511 B2
`5/2004 Tsikos et a1.
`6,742,707 B1
`6/2004 Tsikos et a1.
`6,834,807 B2 12/2004 Ehrhart et a1.
`6,854,649 B2
`2/2005 Worner et a1.
`6,857,570 B2
`2/2005 Tsikos et a1.
`6,863,216 B2
`3/2005 Tsikos et a1.
`6,889,904 B2
`5/2005 Bianculli et a1.
`2001/0009428 A1
`7/2001 DOW et 31‘
`2001/0046036 A1 11/2001 Honma
`2002/0000470 A1
`1/2002 Lanzaro et 31‘
`2004/0155110 A1
`8/2004 Ehrhart et a1.
`2005/0167504 A1
`8/2005 Meier et a1.
`
`GB
`JP
`JP
`WO
`WO
`
`6/2001
`2357209 A
`8/1999
`11232378 A
`2/2000
`2000-050028 A2
`w090/16033 A2 12/1990
`w091/10207 A1
`7/1991
`
`1061-002
`
`

`
`US 7,304,670 B1
`Page 3
`
`WO94/10652
`WO
`WO95/34043
`WO
`WO96/39676
`WO
`WO97/08647
`WO
`W0 03/001435
`W0
`W0 03/081520
`W0
`W0 03/081521
`W0
`W0 WO 2004/064382
`
`5/1994
`12/1995
`12/1996
`3/1997
`1/2003
`10/2003
`10/2003
`7/2004
`
`OTHER PUBLICATIONS
`
`Marshall Electronics, Optical Systems Division, Monochrome
`Monolithic Image Sensor With Analogue and Digital Outputs
`VVL1070 Speci?cation, Believed to be published in 1994, pp. 1-24.
`
`VLSI Vision Ltd., High Resolution EINCCIR Monochrome Mono
`lithic Camera Speci?cation, VVL-1060, Apr. 1994, pp. 1-23.
`VLSI Vision Ltd., Serial Interface Speci?cation, VVL-1060, Apr.
`1994, pp. 1-9.
`Sony Corporation, ICX084AL, Technical Speci?cation, Believed to
`be published prior to Jan. 22, 2000.
`Intermec Corporation, 17010 Hand-Held Imager User’s Manual,
`1995, 60 pages.
`Text String Extraction from Images of Colour-Printed Documents
`IEEE Proc. -Vis. Image Signal Process, vol. 143, Suen H. M., et al.,
`No. 4, Aug. 1996, pp. 210-216.
`
`* cited by examiner
`
`1061-003
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 1 of 7
`
`US 7,304,670 B1
`
`GAIN
`CONTROL
`
`I
`
`INT/EXT
`EXPOSURE
`CONTROL
`
`AND
`CONTROL
`
`F|G.1
`
`1061-O04
`
`1061-004
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 2 0f 7
`
`US 7,304,670 B1
`
`PIXEL ROW
`
`FIG. 2
`
`FIG. 3
`
`1061-005
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 3 0f 7
`
`US 7,304,670 B1
`
`OUT
`
`\
`
`FIG 4
`
`1061-006
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 4 0f 7
`
`US 7,304,670 B1
`
`COUNT = 0
`
`I
`CAPTuRE FRAME
`(UNIFORM SCENE) \_ 102
`
`I
`
`COUNT = COUNT +1
`
`DETERMINE AVERAGE
`WHITE VALUE
`FOR EACH PIXEL
`
`’\_ 104
`
`I
`COMPARE EACH PIXEL‘S AVERAGE
`WHITE VALUE TO A REFERENCE \_
`VALUE TO DETERMINE A
`106
`CORRECTION VALUE FoR EACH PIXEL
`
`T
`
`STORE INTO MEMORY
`
`FIG. 5A
`
`1061-007
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 5 0f 7
`
`US 7,304,670 B1
`
`COUNT = 0
`
`CAPTURE FRAME
`(FIRST SCENE) \- 10s
`
`I
`
`COUNT = COUNT +1
`
`COUNT = N
`?
`
`YES
`
`COUNT = 0
`
`f 110
`
`'
`I
`CAPTURE FRAME
`(SECOND SCENE)
`
`I
`
`COUNT = COUNT +1
`
`NO
`
`COUNT = N
`
`YES
`
`FIG. 5B
`
`109
`f
`I
`DETERMINE AVERAGE WHITE
`VALUE FOR EACH PIXEL DURING
`EXPOSURE TO SCENE 1
`
`111
`/
`I
`DETERMINE AVERAGE WHITE
`VALUE FOR EACH PIXEL DURING
`EXPOSURE TO SCENE 2
`
`112
`f
`I
`SOLVE FOR SYSTEM OF
`EQUATIONS TO‘ DETERMINE M
`AND A FOR EACH PIXEL
`
`v
`
`STORE INTO MEMORY
`
`1061-008
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 6 0f 7
`
`US 7,304,670 B1
`
`COUNT = 0
`
`I
`
`v
`
`II
`CAPTURE FRAME
`(UNIFORM SCENE) \" 120
`
`I
`DETERMINE
`COLUMN WHITE VALUES
`FOR EACH FRAME
`
`122
`
`II
`
`COUNT = COUNT +1
`
`DETERMINE AVERAGE
`COLUMN WHITE VALUES
`FOR EACH COLUMN
`
`124
`
`II
`
`COMPARE EACH AVERAGE COLUMN
`WHITE VALUE TO A REFERENCE VALUE /\__ 126
`TO DETERMINE A CORRECTION VALUE
`FOR EACH COLUMN
`
`Ir
`
`STORE INTO MEMORY
`
`FIG. 6
`
`1061-009
`
`

`
`U.S. Patent
`
`Dec. 4, 2007
`
`Sheet 7 0f 7
`
`US 7,304,670 B1
`
`|.|||||||||||.||||.||| .
`
`r130
`
`AVERAGE
`COLUMN
`WHITE
`VALUE
`
`136
`134
`COLUMN LOCATION
`
`140
`
`FIG. 7
`
`1061-010
`
`

`
`US 7,304,670 B1
`
`1
`METHOD AND APPARATUS FOR
`COMPENSATING FOR FIXED PATTERN
`NOISE IN AN IMAGING SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The invention relates generally to image signal processing
`and in particular to a method and apparatus for adjusting
`pixel values.
`2. Background of the Prior Art
`When pixel values, representative of pixel voltages are
`read from an image sensor pixel array, slight inconsistencies
`are observable betWeen the voltages of the various pixels
`even When the array is exposed to a scene of uniform
`radiance. The inconsistency in the pixel values read from the
`various pixels is commonly referred to as “noise” either of
`a ?xed pattern noise type or a random noise type.
`Fixed pattern noise refers to inconsistencies in the respon
`siveness of an image sensor output that results from sensor
`characteristics or properties affecting pixels of the sensor in
`the same Way each time data is captured from those pixels.
`Random, or stochastic noise refers to inconsistencies Which
`vary each time data is captured from the pixels.
`While random, or stochastic noise cannot be compen
`sated, it is useful to adapt an imaging system in video signal
`generating or decoding applications to automatically com
`pensate pixel values that deviate from patterns that might be
`expected based on the array’s radiance pattern.
`Various problems have been noted in prior art imaging
`systems con?gured for ?xed pattern noise compensation,
`both in set up routine methods for con?guring an imaging
`system for ?xed pattern noise compensation, and in methods
`for compensating pixel values read from a pixel array.
`In one knoWn setup routine for con?guring an imaging
`system to compensate for ?xed pattern noise, a pixel array
`is exposed to uniform “dark frame” by shielding the pixel
`array from light. The pixel values read from the array during
`exposure to the dark frame are then used to determine pixel
`offset terms. This method can be employed to estimate an
`additive component of ?xed pattern noise, but cannot be
`used to determine a multiplicative component of ?xed
`pattern noise. Furthermore, the setup method involving
`exposure to a dark frame is susceptible to measurement
`clipping. If a change in incident radiance does not result in
`a change in pixel value, then clipping has occurred.
`In a knoWn analog hardWare method for compensating
`voltages read from a pixel array, hardWare components are
`provided and selectively activated to compensate pixel volt
`ages shifted out of a pixel array. A major problem With this
`scheme is that values read from the array can be corrected
`only to a small number of discrete voltages, thereby limiting
`the precision of the correction. Also, this scheme requires
`additional hardWare.
`Failure to compensate adequately for ?xed pattern noise
`has signi?cant consequences, for example, When an imaging
`system is employed in indicia decoding applications. In such
`applications, it is common to de?ne an area of interest in a
`captured image before decoding algorithms commence.
`Fixed pattern noise has been observed to cause a symbology
`decoding apparatus to misplace edge positions, and to result
`in erroneous identi?cation of edge positions. The likelihood
`of an erroneous edge position identi?cation increases When
`a decoding apparatus is employed to captured symbols
`having higher densities.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`SUMMARY OF THE INVENTION
`
`There is described a pixel value adjustment method and
`apparatus. In one embodiment, pixel values can be adjusted
`by execution of an algorithm for adjusting pixel pixel values
`is capable of operating in a decoding operating application.
`In one embodiment, an apparatus capable of executing an
`algorithm for adjusting pixel values is capable of operating
`in a video display operating application.
`The features discussed above and other features of the
`invention Will become clear to persons skilled in the art from
`a reading of a Detailed Description of the Preferred Embodi
`ments combined With the referenced draWings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`In the draWings, Wherein like numerals are used to indi
`cate the same elements throughout the vieWs,
`FIG. 1 is a block diagram of an image processing system
`in Which the present invention may be implemented;
`FIG. 2 is a simpli?ed graphical representation of a digital
`output of an image sensor exposed to a scene of uniform
`radiance corresponding to a single roW of pixels;
`FIG. 3 shoWs a pixel array and an associated lens attached
`thereto;
`FIG. 4 shoWs an electrical diagram of an image sensor
`illustrating schematic representations of electrical compo
`nents of the image sensor Whose variance in physical
`properties gives rise to variance in emitted signal output
`betWeen the several pixels;
`FIG. 5a is a How diagram of a con?guration method for
`con?guring an imaging system to compensate for an addi
`tive component of pixel-speci?c ?xed pattern noise;
`FIG. 5b is a How diagram of a con?guration method for
`con?guring an imaging system to compensate for an addi
`tive and multiplicative component of pixel-speci?c ?xed
`pattern noise.
`FIG. 6 is a How diagram of a con?guration method for
`con?guring an image system to compensate for column
`dominant ?xed pattern noise;
`FIG. 7 is a table for use in describing a column dominant
`noise compensation con?guration method Which summa
`rizes average column White values for a group of columns.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`There is described a pixel value adjustment method and
`apparatus. In one embodiment, pixel values can be adjusted
`by execution of an algorithm for adjusting pixel values. In
`one embodiment, an apparatus capable of executing an
`algorithm for adjusting pixel values is capable of operating
`in a decoding operating application. In one embodiment, an
`apparatus capable of executing an algorithm for adjusting
`pixel value is capable of operating in a video display
`operating application.
`The invention relates to a method and apparatus for
`con?guring an imaging system to compensate for ?xed
`pattern noise, variations in pixel values captured from an
`image sensor that vary according to a ?xed pattern. In a
`method for con?guring an imaging system for compensating
`an additive term component of ?xed pattern noise, a pixel
`array is exposed to a scene of knoWn radiance and an
`average White value is determined for each pixel of an array.
`Each average White value is compared to a predetermined
`reference value to determine a correction value for each
`pixel. In a method for con?guring an imaging system for
`
`1061-011
`
`

`
`US 7,304,670 B1
`
`3
`compensating both an additive and multiplicative compo
`nent of ?xed pattern noise, a pixel array is exposed to a ?rst
`scene having a ?rst knoWn radiance, and frames of the scene
`are captured, and then the array is exposed to a second scene
`having a second knoWn radiance and frames of the second
`scene are captured. After ?rst and second average White
`values are determined for each pixel additive and multipli
`cative term correction values for each pixel are then deter
`mined by solving for a system of equations relating the ?rst
`and second average White values and ?rst and second scene
`radiances. Correction values can be utiliZed to correct pixel
`values of a frame of image data, and a decoding algorithm
`for attempting to decode a frame of image data including
`pixel values can be activated.
`According to its major aspects and broadly stated the
`present invention relates to a method and apparatus for
`con?guring an imaging system to compensate image signals
`for the effects of ?xed pattern noise, Which affects image
`sensor element outputs in the same Way each time data is
`captured from those elements. An imaging system according
`to the invention can include a setup mode of operation for
`determining parameters for use in compensating ?xed pat
`tern noise, and a run mode of operation Wherein the param
`eters determined in a setup mode are used to compensate
`pixel values or values captured from the pixel elements.
`Fixed pattern noise can result from a variety of sources.
`For example ?xed pattern noise can result from inconsis
`tencies in the physical properties of electrical components of
`the image sensor such as sWitching components associated
`With each pixel, or in certain types of image sensors With
`column ampli?ers associated With each column. Fixed pat
`tern noise can also result from certain illumination apparatus
`con?gurations. If the illumination is characteristically non
`uniform, but its characteristics are ?xed each time data is
`captured, then ?xed pattern noise results. Another common
`source of ?xed pattern noise is an optical phenomena knoWn
`as the cosine4 (cosine) effect Which results in outer pixels of
`an array receiving loWer intensity ligh rays than center
`pixels on exposure to a scene of uniform radiance.
`Fixed pattern noise associated With a pixel can be
`expressed mathematically by
`
`PMIMPREFA-A Eq. 1
`Where PM is the observed or measured pixel value, PREP
`is a reference value normally selected to correspond to an
`actual irradiance of light received at a pixel, M is a multi
`plicative or gain term, and A is an additive, or offset term
`in?uencing ?xed pattern noise. Typically, M is a value near
`unit and A is a small fraction of PM, the measured pixel
`Value. For some sources of ?xed pattern noise, such as the
`cosine effect, the multiplicative term is dominant While for
`other sources of ?xed pattern noise, such as ampli?er offset
`?xed pattern noise, the additive term is dominant. Other
`sources of ?xed pattern noise have both signi?cant multi
`plicative and signi?cant additive terms contributing to ?xed
`pattern noise.
`In certain setup routines that can be implemented accord
`ing to the invention Which are best used in the case Where the
`additive term of ifxed pattern noise is expected to be
`dominant, the multiplicative term of ?xed pattern noise may
`be considered to be unitary. When the multiplicative term is
`assumed t be unitary, Eq. 1 reduces to
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`PMIPREFA-A Eq. 2
`
`In a setup routine that can be implemented for determin
`ing only an additive term of ?xed pattern noise, an image
`sensor is exposed to a scene of uniform radiance, and a
`
`65
`
`4
`predetermined number of frames of the scene are captured in
`the system’s memory. The gray scale value, or White value
`associated With each pixel of the pixel array image sensor is
`then calculated and also stored into memory. As the number
`of predetermined frames increases the contribution of Zero
`mean random noise to the average White value diminishes
`since, by de?nition, Zero means random noise is as likely to
`result in an increase in a pixel value as it is to result in a
`decrease in pixel value. That is, positive and negative
`random noise terms tend to cancel each other as the number
`of frames increases.
`The scene of uniform radiance to Which a pixel array is
`exposed is preferably a gray scale uniform radiance scene,
`that is, a scene Which does not result either in a maximun
`clipped or minimum clipped pixel value being read from the
`pixels in the pixel array. By providing a uniform radiance
`scene yielding gray scale pixel values betWeen these tWo
`limits, the problem of measurement clipping is avoided.
`Once an average White value is determined for each pixel,
`a reference value is subtracted from an average White value
`to determine a correction value for each pixel. The reference
`value, substituted for PREP in Eq. 2 may be a predetermined
`gray scale value or may be determined based on the value of
`all or some of the remaining average White values. The
`reference value normally corresponds to the actual light
`irradiance of light falling on the pixel. While this value
`cannot be measured directly, it can be assumed for the
`purposes of determining a correction value, in one simpli?ed
`embodiment, that each pixel in an array that is exposed to a
`scene of uniform radiance has equal irradiance of light
`falling thereon, or that pixels in a prede?ned region of an
`array have the same levels of irradiance received thereon.
`This is used to calculate the correction values. Once deter
`mined, the correction value associated With each pixel is
`stored into memory. When the imaging system is sWitched
`from a setup mode to a run mode of operation, the correction
`values are retrieved from memory and used to correct pixel
`values of captured frames. Because the correction values are
`determined before the imaging system is sWitched from a
`setup mode of operation to a run mode of operation, the
`correction values are independent of any image represented
`in the captured frames captured by the imaging system When
`in a run mode of operation.
`In a setup method that can be implemented accord accord
`ing to the invention for determining both multiplicative term
`and additive terms in?uencing ?xed pattern noise a pixel
`array is exposed to at least tWo scenes having uniform but
`differing radiances. It can be seen With reference to Eq. 1
`that if the relationship betWeen the differing radiances of the
`?rst and second scenes is knoWn, the multiplicative term,
`along With the additive term of ?xed pattern noise can be
`determined.
`In accordance With a method that can be implemented for
`determining both multiplicative and additive ?xed pattern
`noise terms, a plurality of frames of a pixel array are
`captured When the array is exposed to a scene of a ?rst
`radiance, and a plurality of additional frames are captured
`When an array is exposed to a scene having a second
`radiance. In order to determine the multiplicative and addi
`tive terms for a given pixel, the average White value of the
`pixel folloWing exposure to the ?rst scene is substituted for
`PM in equation 1 in the development of a ?rst equation, and
`the average White value of the pixel folloWing exposure to
`the second scene is substituted for PM in the development of
`a second equation. PREP for a given pixel in the ?rst and
`second equations Will normally correspond to the expected
`actual irradiance of the pixel during exposure to the ?rst and
`
`1061-012
`
`

`
`US 7,304,670 B1
`
`5
`second scenes, respectively. Solving for the set of tWo
`equations having tWo unknowns using Well knoWn compu
`tational methods solves for both the multiplicative and
`additive components of ?xed pattern noise.
`Further aspects of the invention relate to methods and
`apparatuses for compensating pixel values read from a pixel
`array, once the additive term and/ or multiplicative term
`correction values have been determined. As discussed in the
`background, prior art compensation schemes include hard
`Ware components Which are selectively activated so that
`voltages read from a pixel array are automatically corrected
`in accordance With previously determined correction terms.
`In th present invention, gray scale correction values can
`be stored in predetermined memory locations 0 the imaging
`system. Preferably, so that processing time is minimized, the
`correction terms are read from memory and used to correct
`pixel values in the case of decoding applications only When
`stored pixel values are read from memory during execution
`of a decoding algorithm. An advantage of the compensation
`method of the present invention is that the method of the
`present invention alloWs correction values to be selected
`betWeen a large number of discrete values, Without requiring
`the providing of costly and space consuming physical ele
`ments as in the prior art design. The correction method of the
`present invention may be used in combination With a hard
`Ware implemented compensation scheme.
`An imaging system 10 having an image sensor 12 and
`control processor 14 in Which the present invention may be
`incorporated is shoWn in FIG. 1.
`Image sensor 12 includes, on a single substrate a tWo
`dimensional array 16 of light sensitive picture elements, or
`pixels upon Which an optical system is arranged to focus an
`image of indicia and further includes a timing and control
`circuit 18 for electrically addressing pixels P via horiZontal
`and vertical address lines 20 and 22, thereby causing elec
`trical signals representing the image to be read out of array
`16 via output 24 thereof. After ampli?cation by ampli?er 26,
`and conversion by analog-to-digital converter 28 the latter
`signals are output from image sensor 12 as serial digital
`parallel image data signals on lines D0:D7 or as a digital
`serial image data signal on data lines D0-D1.
`Digital image data is supplied from sensor 12 to processor
`14 With the aid of a ?rst signal knoWn as a frame start pulse
`and a second signal knoWn as data valid pulse. Because the
`use of such signals is Well knoWn to those skilled in the art,
`their use Will not be described in detail herein.
`Image sensor control processor 14 preferably comprises a
`microprocessor 30 Which may be of either the single chip or
`multi-chip types, and Which may include on chip and/or off
`chip program and data (Working) memories 32 and 34,
`depending upon the complexity of the program used to
`control image sensor 12 and to process image data produced
`thereby. Microprocessor 30 may be of the type Which is
`formed on a single integrated circuit chip and Which
`includes an on-chip read only programmable program
`memory 32 and an on-chip read-Write Working memory 34.
`Microprocessor 30 may also be of the type Which includes
`four eight-bit (I/O) ports, 40 through 46, each bit of each
`port being independently controlled by the stored program.
`Because microprocessor systems of this type, and their
`multi-chip equivalents are Well knoWn to those skilled in the
`art, microprocessor 30 Will not be described in detail herein.
`With reference noW to speci?c attributes of the present
`invention, FIG. 2 is a simpli?ed representation of the digital
`output of image sensor 12 When pixel array 16 is exposed to
`a scene of uniform radiance. For purposes of simplifying the
`description of the invention, the digital output of FIG. 2
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`illustrates only 20 representative column values of the pixel
`columns in a commercially available pixel array, Which are
`available in many different dimensions including 160 roWs
`by 160 columns (in a VVL 1070 image sensor available from
`VLSI Vision Limited), and Which are expected to become
`available in increasing densities. The above referenced VVL
`1070 image sensor is a monochrome image sensor.
`In an ideal image sensor, output 50 might be expected to
`yield an identical pixel value for each pixel in a roW on
`exposure to a scene of uniform radiance. From roW output
`50 it is seen that an actual image sensor does not exhibit such
`behavior. Instead, pixel values vary from pixel to pixel in the
`roW. The variation, or inconsistency in the pixel values
`generated by the pixels in an array on exposure of the sensor
`to a uniform radiance scene is referred collectively to as
`“noise.”
`Fixed pattern noise refers to a variation among pixel
`values resulting from factors affecting pixel values in the
`same Way and to the same extent each time data is read from
`the pixels.
`One source of ?xed pattern noise results from differences
`in the siZe and other physical characteristics of miniature
`integrated circuit components. With reference to the pixel
`array schematic diagram of FIG. 4 each pixel location 72, 74
`has associated thereWith sWitching circuit component (not
`shoWn) and a separate column ampli?er 80 is associated
`With each column of array 16. The voltages exhibited by
`pixels of the array Will vary from one another on exposure
`to a uniform radiance scene as a result in part of slight
`differences in the physical properties of sWitching circuit
`components and ampli?ers 80, and other electrical compo
`nents Which may be associated With array 16. Column
`ampli?ers are the dominant source of ?xed pattern noise in
`several types of available image sensors. Differences in the
`physical characteristics of optical components, namely of
`pixels 72, 74 of pixel array 16 can also give rise to noise that
`varies according to a ?xed pattern.
`Fixed pattern noise can also result from certain illumina
`tion apparatus con?gurations. If the illumination is charac
`teristically nonuniform, but its characteristics are ?xed every
`time an image is captured, then ?xed pattern noise results.
`An illumination apparatus can result in ?xed pattern noise if,
`for example, the illumination apparatus includes a bank of
`illuminators Which are not precisely matched, or if one or
`more illuminators are sWept over a target scene. By correct
`ing the effects of ?xed pattern noise, a truer representation
`of a given scene is produced by an imaging system.
`Another source of ?xed pattern noise is an optical phe
`nomena knoWn as the cosine“, or cosine effect, Which is
`described With reference to FIG. 3. According to the cosine
`effect, the irradiance at a given pixel 64 is proportional to the
`cosine to the fourth poWer of the angle 66 de?ned betWeen
`optical axis 67 and pixel 64 as seen by lens 68. Therefore
`outer pixels, eg. 69 tend to have loWer irradiances associated
`thereWith than center pixels 70. A detailed discussion of the
`cosine effect appears in Modern Optical Engineering, Smith,
`Warren J. (McGraW-Hill, 1966), pp. 132-133.
`Referring again to the block diagram of FIG. 1 aspects of
`imaging system enabling system 10 to compensate for ?xed
`pattern noise Will be described in detail. According to the
`invention, imaging system 10 has tWo modes of operation:
`A ?rst, set up mode for con?guring th