EXHIBIT 1309
`
`PCT PUB. NO. WO 93/11631, PUBLISHED JUNE 10, 1993
`
`("DENYER")
`
`TRW Automotive U.S. LLC: EXHIBIT 1309
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NUMBER 8,599,001
`
`
`
`PCT PUB. NO. WO 93/11631, PUBLISHED JUNE 10, 1993
`
`("DENYER")
`
`TRW Automotive U.S. LLC: EXHIBIT 1309
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NUMBER 8,599,001
`
`
`
`PCI‘
`
`International Bureau
`WORLD INTELLECTUAL PROPERTY ORGANIZATION
`
`
`
`INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`(51) International Patent Classification 5 =
`
`(11) International Publication Number:
`
`WO 93/11631
`
`H04N 5/335, 9,097: 5/225
`
`(43) International Publication Date:
`
`10 June 1993 (10.06.93)
`
`(21) International Application Number:
`
`PCT/GB92/02260
`
`(22) International Filing Date:
`
`4 December 1992 (04.12.92)
`
`(81) Designated States: GB, JP, US, European patent (AT, BE,
`CH, DE, DK, ES, FR, GB, GR, IE, IT, LU, MC, NL,
`PT, SE).
`
`Published
`With international search report.
`Before the expiration of the time limit for amending the
`claims and to be republished in the event of the receipt of
`amendments.
`
`(30) Priority data:
`91259549
`
`6 December 1991 (06.12.91) GB
`
`(71) Applicant (for all designated States except US): VLSI VI-
`SION LIMITED [GB/GB]; Technology Transfer Cen-
`tre, King’s Buildings, Mayfield Road, Edinburgh EH9
`3JL (GB).
`
`(72) Inventor; and
`(75) Inventor/Applicant (for US only) : DENYER, Peter, Brian
`[GB/GB]; 91 Colinton Road, Edinburgh EH10 SDF
`(GB).
`
`(74) Agents: McCALLUM, William, Potter et al.; Cruikshank
`and Fairweather, 19 Royal Exchange Square, Glasgow
`G1 3AE (GB).
`'
`
`
`
`
`
`(54) Title: SOLID STATE SENSOR ARRANGEMENT FOR VIDEO CAMERA
`
`(57) Abstract
`
`The present invention relates to an image capture system suitable for use in an electronic camera system C and comprising
`a solid state image capture device (1) comprising an integrated circuit (5) having at least two sensor arrays (4), each said array
`having an image sensing surface (3) and a respective lens system (8) associated therewith.
`
`1309-001
`
`1309-001
`
`
`
`(l
`
`{V
`
`FOR THE PURPOSES OF INFORMATION ONLY
`
`Codes used to identify States party to the PCT on the front pages of pamphlets publishing international
`applications under the PCT.
`
`Viet Nam
`
`France
`Gabon
`United Kingdom
`Guinea
`Greece
`Hungary
`Ireland
`Italy
`Japan
`Democratic People‘a Republic
`of Korea
`Republic of Korea
`szikhslan
`Liechtenstein
`Sri lanka
`Luxembourg
`Monaco
`Madagascar
`Mali
`Mongolia
`
`AT
`AU
`BB
`BE
`BF
`BC
`8.]
`BR
`CA
`CF
`CG
`CH
`CI
`CM
`C5
`CZ
`DE
`DK
`PS
`Fl
`
`Amtria
`Australia
`Barbados
`Belgium
`Burkina Faso
`Bulgaria
`Benin
`Brazil
`Canada
`Central African Republic
`Congo
`Swit'lcrland
`('51:: d‘lvoire
`'ameroon
`(In-choslovakia
`Cluck Republic
`Germany
`Denmark
`Spain
`Finland
`
`Mauritania
`Malawi
`Netherlands
`Norway
`New Zealand
`Poland
`Portugal
`Romania
`Russian Federation
`Sudan
`Sweden
`Slovak Republic
`Senegal
`Soviet Union
`Chad
`Togo
`Ukraine
`United States of America
`
`1309-002
`
`1309-002
`
`
`
`WO93/ll63l
`
`_ 1
`
`_
`
`PCT/GB92/02260
`
`VSOLID STATE SENSOR ARRANGEMENT FOR VIDEO CAMERA
`
`The present invention relates to electronic cameras
`
`including electronic colour cameras.
`
`It is well known that colour sensors can be produced by
`
`discriminating three images of the primary colours
`
`(blue, green, red) of the scene. All colours can be
`
`analysed and synthesised via these primaries (or other
`
`complementary triples like cyan, magenta, yellow).
`
`Conventional electronic cameras classically use one of
`
`two approaches for forming the separate colour images.
`
`3-tube cameras use a single lens followed by a prism
`
`which forms three separate r.g.b images. Three sensors
`
`are used simultaneously to detect these three images.
`
`If the sensors are accurately aligned the resulting
`picture is of very high quality. However the sensors
`
`are separated in space and orientation and their
`
`assembly and alignment with the prism and lens is
`
`difficult for a volume manufacturing process. This
`
`technique is therefore used exclusively for expensive
`
`broadcast-quality equipment. Colour-Mosaic Cameras use
`
`a single lens and sensor, but the sensor surface is
`
`covered with a high-resolution mosaic or grid of colour
`
`filters, with the pattern dimension equal to the
`
`pixel-pitch for a semiconductor CCD or MOS sensor
`
`10
`
`15
`
`20
`
`25
`
`array. Pixels of different colours are demultiplexed at
`the sensor output and interpolated to form synchronous
`
`parallel colour signals. This is well-suited to volume
`
`production as the surface colour mosaic can be
`
`fabricated as an extension of the semiconductor wafer
`
`fabrication process.
`
`The techniques for mosaic
`
`3O
`
`fabrication are restricted to relatively few companies
`
`worldwide who supply the colour sensor market and thus
`
`they are not commonly available. Furthermore,
`
`associated with this technique there are technical
`
`problems concerned with resolution and aliasing. Much
`
`1309-003
`
`1309-003
`
`
`
`W0 93/1 1631
`
`- 2
`
`-
`
`PCT/(2892/02260
`
`work has been done to correct these effects, but usually
`
`at some cost in image-processing hardware.
`
`It is an object of the present invention to avoid or
`minimise one or more of the above disadvantages.
`
`5
`
`the present invention
`In one of its broadest aspects,
`provides an image capture system comprising a solid,
`state image capture device which device comprises an
`
`integrated circuit having at least two sensor arrays,
`each said array having an image sensing surface and a
`respective lens system associated therewith.
`
`10
`
`Thus in effect the present invention provides two or
`more cameras on one chip each with its own lens system
`
`and sensor array. With such an arrangement the problem
`of alignment is greatly reduced by the fabrication of
`the various sensors required one one chip. This ensures
`
`that the sensors all lie in the same plane and have the
`
`same rotational orientation, and this is an important
`advantage. Assuming lenses can be accurately assembled
`in a parallel plane (see below),
`the only alignment
`errors which are likely to occur are simple orthogonal
`translations in the form of vertical and horizontal
`errors in the centres of the optical axes.
`It is
`
`relatively easy though to calibrate these cameras after
`assembly and electronically to correct for these
`translations. Whilst the inevitable lateral off-set
`between the cameras at even the closest dispositions of
`the cameras on the chip, will of course give rise to a
`
`degree of parallax error, it has now been found that
`with a preferred system of the present invention with
`generally adjacent sensor arrays, the degree of error in
`producing a single composite image (i.e. a single image
`produced by the more or less accurately aligned super
`imposition of two or more corresponding images e.g. at
`
`15
`
`20
`
`25
`
`30
`
`1309-004
`
`1309-004
`
`
`
`W0 93/1163]
`
`- 3
`
`-
`
`PCT/GB92/02260
`
`different wavelengths, of the same scene) can be
`
`acceptably small for small camera geometries for low to
`
`medium resolution applications.
`
`Thus in said one
`
`preferred aspect the present invention provides a
`
`composite image camera of particularly simple and
`
`economic construction.
`
`The present invention also provides in another aspect a
`
`stereoscopic image capture system where larger sensor
`
`spacings are used to provide a greater parallax
`
`differential for producing different images with a more
`
`or less accurately defined parallax differential for use
`
`in producing stereoscopic image pairs. Again the use of
`
`two or more cameras mounted on a single chip helps
`
`substantially to minimise alignment problems in
`
`producing an accurate stereoscopic view.
`
`Advantageously the lens systems are mounted
`
`substantially directly on the image sensing surfaces.
`
`Preferably there is used a lens system in accordance
`
`with our earlier British Patent Application No.
`
`9103846.3 dated 23rd February 1991 (published in
`
`International Publication No. WO92/15036) which lens
`
`comprises a lens and a spacer in substantially direct
`
`contact with each other, said spacer preferably having a
`
`refractive index not less than that of said lens, said
`
`lens and spacer having refractive indices and being
`
`dimensioned so as to form an image in a plane at or in
`
`direct proximity to a rear face of said spacer element
`
`remote from said lens element,
`
`from an object, whereby
`
`in use of the lens system with said lens system mounted
`
`substantially directly on the image sensing surface of
`
`the image capture device an optical image may be
`
`captured thereby.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`1309-005
`
`1309-005
`
`
`
`WO 93/11631
`
`- 4
`
`"
`
`PCT/GB92/02260
`
`These lens systems have the advantage of physical
`dimensions which can be made similar to those of the
`sensor array itself, so that sensors and lenses may be
`immediately adjacent to each other. Camera separations
`
`The flat surfaces of the
`minimise the parallax error.
`cylindrical lens spacer also help to maintain accurate
`planarity for groups of lenses attached to the same chip
`substrate.
`
`It is also possible though to use more conventional,
`albeit similarly small,
`lens systems which are mounted
`on a suitable support so as to be spaced from the sensor
`surface with an air gap therebetween. One advantage of
`such systems is that they allow the use of more
`conventional and cheaper lens materials without the need
`for special materials having particular refractive
`indices.
`
`In general at least one of the individual cameras
`constituted by respective lens system and sensor arrays,
`is provided with a filter means for passing a desired
`wavelength (or wavelength range) of the electromagnetic
`radiation spectrum, whereby there may be captured a
`composite image comprised of two or more (depending on
`the number of individual cameras used)
`images of the
`same object differing substantially only in the
`
`wavelength thereof.
`
`In one preferred form of the invention the integrated
`circuit has three sensor arrays provided with respective
`lens systems and filters for three different wavelengths
`e.g. red, green, and blue, or cyan, magenta, and yellow
`for providing a desired composite image e.g. a
`
`full-colour image.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`1309-006
`
`1309-006
`
`
`
`\V()93/11631
`
`'
`
`— 5
`
`—
`
`PC17(}B92/02260
`
`Where three or more individual cameras are used, it will
`
`be appreciated that various different layouts of the
`
`sensor arrays relative to each other may be employed
`
`including e.g.
`
`linear arrangements or generally
`
`"circular" or other close-packed arrangements.
`
`In cameras using the 3 primary colours, green is
`
`dominant in providing image acuity since it generally
`
`dominates the derived luminance.
`
`In all cases
`
`therefore,
`
`the green camera is desirably made as central
`
`10
`
`as possible, and the red and blue cameras are referenced
`
`to it.
`
`The parallax errors will therefore show up on
`
`red and blue only.
`
`With reference to lens systems of our earlier
`
`application No. 9103846.3,
`
`the expression "substantially
`
`15
`
`direct contact" is used to mean that there should not be
`
`any significant interspace containing low refractive
`
`index material such as air i.e. no interspace having a
`
`thickness resulting in a significant optical effect.
`
`In
`
`the case of an air gap this should normally be not more
`
`20
`
`than 500 um, preferably not more than 100 um,
`
`thick,
`
`In
`
`the case where resin or like material is used between
`
`the components adhesively to secure them together and
`
`has a refractive index comparable to that of the lens or
`
`spacer, it may be considered as an extension of the lens
`
`25
`
`or spacer and thus need not be so restricted in
`
`thickness though preferably the thickness thereof should
`
`not be excessive and should be more or less similarly
`
`restricted.
`
`Advantageously there is used a plane-convex (or possibly
`
`30
`
`plano-concave - see below)
`
`lens with a substantially
`
`plane spacer for manufacturing convenience and economy
`
`but other combinations e.g. a bi-convex lens and a
`
`plane-concave spacer, may also be used°
`
`1309-007
`
`1309-007
`
`
`
`“093/1163:
`
`_ 6' _
`
`PCT/GB92/02260
`
`Preferably the lens system is secured to said image
`sensing surface by an optical grade adhesive i.e. a
`substantially transparent optically uniform adhesive.
`Desirably there is used between the lens and spacer an
`adhesive having the same refractive index as the lens
`(or if preferred, as the spacer) and between the spacer
`and the sensing surface, an adhesive having the same
`
`refractive index as the spacer.
`
`10
`
`Preferably the spacer has a higher, most preferably a
`substantially higher, refractive index than the lens.
`Where the same refractive index is acceptable for both
`
`then it will be appreciated that the spacer could be
`
`formed integrally with the lens.
`
`15
`
`It will be appreciated that the radius (or radii) of
`curvature of the lens element and its refractive index
`may be varied through a wide range of values depending
`on the required performance in terms of depth of field,
`image size,
`freedom from aberrations etc.
`In general
`there will desirably be used solid state image capture
`
`20
`
`devices in the form of photoelectric sensor arrays
`
`(wherein photons are used to generate electric current
`and/or voltage or change electrical properties such as
`resistance etc.) which have relatively small size image
`
`in the range from 0.1 to 5 cms
`sensing surfaces e.g.
`across. Thus the lens system should in such cases
`desirably be formed and arranged to provide a similarly
`small-sized image. Where a wide angle field of view is
`also required (e.g.
`in surveillance applications),
`then
`a lens of relatively short focal length should be used
`
`e.g. for a field of View angle of 80 degrees the
`(maximum) focal length will not normally exceed 1.19
`times the image height and for 60 degrees will not
`normally exceed 1.73 times the (maximum)
`image height,
`
`25
`
`3O
`
`J‘a
`
`q!
`
`1309-008
`
`1309-008
`
`
`
`'wo 93/11631
`
`_ 7
`
`_
`
`PCT/GB92/02260
`
`the (maximum)
`
`image height corresponding to half the
`
`sensing surface diameter.
`
`The use of a high refractive
`
`index spacer and the exclusion of any low refractive
`
`index material from the optical path significantly
`
`decreases aberration due to Petzval Curvature (otherwise
`
`known a curvature of field aberration) and limits
`
`spherical aberration.
`
`The lens system is therefore
`
`particularly advantageous in wide field and/or large
`
`aperture applications required for low light
`
`conditions.
`
`In general there is desirably used, for
`
`such wide angle applications, a lens element having a
`
`refractive index and in the range from 1.45 to 1.65, and
`
`a spacer element with a higher, refractive index and in
`
`the range from 1.45 to 1.85.
`
`Various optical grade materials having suitable
`
`refractive indices are widely available. Low—dispersion
`
`glass such as type BK7 (available from various sources
`
`e.g. Schott Glaswerke)
`
`is particularly suitable for the
`
`lens element.
`
`The spacer element may be made of LaKlO
`
`glass also readily available.
`
`other materials that may
`
`be used for the lens and/or spacer elements comprise
`
`plastics materials, although these are generally less
`
`preferred in view of their lower resistance to
`
`scratching and other damage and the lower refractive
`
`indices available. Nevertheless they may be acceptable
`
`for certain applications requiring low cost such as
`
`consumer door-entry and security cameras.
`
`10
`
`15
`
`20
`
`25
`
`Suitable adhesive materials for use between the spacer
`
`and lens elements and between the spacer element and the
`
`30
`
`solid state image capture device include optical grade
`
`epoxy resins.
`
`In a preferred image capture system of the present
`
`invention the solid state image capture device comprises
`
`1309-009
`
`1309-009
`
`
`
`W0 93/ 1 I631
`
`— 8
`
`—
`
`PCTIGB92/02260
`
`an integrated circuit image array sensor such as that
`disclosed in our earlier International patent
`application No. PCT/GB90/01452 (publication No.
`W09l/O4633 the contents of which are hereby incorporated
`wherein by reference thereto) which has on-board signal
`processing means formed and arranged for directly
`providing a video signal output. Naturally though other
`image capture devices such as CCD, MOS and CCD sensors
`may also be used. Also the image capture device may
`comprise simply a sensor chip on which are only provided
`the sensor arrays with all the electronic circuitry
`required to detect the response of individual sensor
`cells to incident radiation and further processing of
`
`the detected response provided externally of the sensor
`chip, and of course other arrangements with a greater or
`lesser part of this electronic circuitry provided on the
`chip bearing the sensor arrays, are also possible.
`Accordingly references to "cameras" herein includes
`references to apparatus in which substantially the whole
`of the electronic circuitry required to produce a video
`output signal is provided on the same chip as the sensor
`arrays, as well as apparatus in which a greater or
`lesser part is provided separately. Thus references to
`camera alignment relate only to alignment of the lenses
`and sensor arrays (and not to any other components that
`may be required to produce a video signal output.
`
`Thus using miniature, chip-mounted lenses, it is
`possible to fabricate multiple independent cameras on
`single VLSI chips. These cameras accurately lie in the
`same plane and are rotationally in substantially perfect
`alignment.. Any remaining alignment errors are primarily
`translational and can be easily corrected by retiming
`
`the readout control sequences.
`
`Us):
`
`‘1
`
`10
`
`15
`
`20
`
`25
`
`30
`
`1309-010
`
`1309-010
`
`
`
`W0 93/1163]
`
`— 9
`
`-
`
`PCT/GB92/0226U
`
`Further preferred features and advantages of the present
`
`invention will appear from the following detailed
`
`description by way of example of a preferred embodiment
`
`illustrated with reference to the accompanying drawings
`
`in which:
`
`Fig.
`
`1 is a schematic perspective view of a composite
`
`image colour video camera of the invention with three
`
`individual camera units;
`
`Figs 2(a)
`
`to (d) are schematic views showing 4 different
`
`2-D arrangements of the three camera elements relative
`
`to each other;
`
`Fig.
`
`3 is a schematic illustration of the optical
`
`performance of a camera of the invention;
`
`, Fig. 4 is a block circuit diagram of one possible
`
`electronic architecture for a camera of the invention;
`
`Fig. 5 is a sectional elevation of another camera of the
`
`invention; and
`
`'
`
`Fig.
`
`6 is a schematic perspective View of a spacer
`
`support element suitable for use in the camera of Fig. 5.
`
`Fig.1 shows a miniature colour video camera system 9
`
`having three cameras 1 each comprising a lens system 2
`
`mounted directly onto the image sensing surface 3 of a
`
`respective solid state image capture device in the form
`
`of an integrated circuit image array sensor 4.
`
`The
`
`sensors 4 are formed as separate sections of a single
`
`monolithic VLSI microchip 5 mounted in a suitable
`
`housing SE containing a power supply 6 and provided with
`
`a video signal output interface 7.
`
`In more detail the lens system 2 comprises a generally
`
`hemispherical lens 8 having a radius of curvature of the
`
`order of 0.85 mm, and a cylindrical spacer element 9 of
`
`substantially larger diameter
`
`(ca. 1.7mm) and a length
`
`of 1.59 mm, with an aperture stop 10 therebetween.
`
`The
`
`aperture stop 10 is of metal e.g. steel alloy with a
`
`10
`
`15
`
`20
`
`25
`
`30
`
`1309-011
`
`1309-011
`
`
`
`wo 93/11631
`
`— 10 —
`
`PCT/GB92/02260
`
`thickness of 0.15 mm and an iris diameter of 0.8 mm
`
`The
`providing an effective lens aperture of f2.0.
`aperture opening is filled with clear epoxy resin 11
`which has a refractive index substantially similar to
`
`that of the lens 8 and secures the lens 8 and spacer 9
`
`to each other and to the aperture stop 10.
`
`Alternatively an aperture stop of metal or other
`
`material could simply be printed onto the spacer or lens
`
`10
`
`The R, G, B
`e.g. using a photolithographic technique.
`(red, green and blue) filters 12 for the three
`respective lenses 8 can also be disposed between the
`
`lenses 8 and spacers 9.
`
`0
`
`The lens 8 is of low dispersion glass (Bk?) having a
`
`refractive index nd of 1.568 and the spacer is of LaKlO
`
`15
`
`glass which has a higher refractive index nd of 1.7200.
`This combination produces low image blur and large image
`
`size (ca. 1.4mm image height from central axis). The
`
`spacer 9 has a length of around 1.59mm. This lens
`
`system has an effective depth of field of from 2cms
`to
`CX> with a field of View angle of 90° and has an
`
`20
`
`rms blur of around 5um which is within the unit sensor
`
`pixel dimensions thereby providing a reasonably good
`video signal image output from the video signal output
`
`connection 7.'
`
`25
`
`It will be appreciated that various modifications may be
`
`made to the above described embodiment without departing
`from.the scope of the present invention. Thus for
`example the spacer element could be a composite element
`
`made up of a plurality of plane components.
`
`The lens
`
`30
`
`element could also be composite though this would
`
`normally be less preferred due to the significantly
`
`increased complexity.
`
`The various surfaces of the lens
`
`system could moreover be provided with diverse coatings.
`for e.g. reducing undesirable reflectidns and selective
`
`1309-012
`
`1309-012
`
`
`
`W0 93/1163!
`
`- 11 -
`
`PCT/GB92/02260
`
`filtration of the incident light rays in generally known
`
`manner. Also the R, G, B filters could be mounted on a
`
`suitable support in front of the lenses 8 as further
`
`described hereinbelow.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`Fig. 2 shows some possible alternative layouts for the
`
`individual cameras on the single chip.
`
`In layout
`
`(a)
`
`the absolute red-green and blue-green distances are
`
`minimised. All of the parallax error is vertical. This
`
`may not be optimum for TV applications, as in this case
`
`the colour signal is greatly averaged horizontally, but
`
`not at all vertically. Layout
`
`(b) forces all of the
`
`parallax error into the horizontal dimension to take
`
`advantage of this. Layout
`
`(c) has slightly worse
`
`red-green and blue-green parallax than (a), but the
`
`red-blue distance is greatly less.
`
`Intuitively, this
`
`configuration minimises the total parallax error.
`
`Layout
`
`(3)
`
`is as for (c), but pushes most of the r-g,
`
`b-g errors into the horizontal axis.
`
`Apart from its substantial simplicity,
`
`the colour camera
`
`system of the present invention provides two further
`
`potential technical advantages over the alternative
`
`known approaches. Axial colour aberration causes the
`
`focal plane for blue to be slightly closer to the lens,
`
`and for red slightly further from the lens,
`
`than green.
`
`This aberration may actually be an advantage if we
`
`design the lens for green light and the blue and red
`images become slightly defocussed,
`thereby also blurring
`
`the effect of parallax errors.
`
`For ordinary glasses,
`
`the blue and green images are blurred by around 1 pixel
`
`at the geometries used above.
`
`If we wish,
`
`the 3—lens
`
`approach could be adapted to accommodate this aberration
`
`by fine-tuning the focal length of each lens. This is
`
`impossible with either of the existing single-lens
`
`approaches. Transversal colour aberration is a change
`
`1309-013
`
`1309-013
`
`
`
`WO 93/11631
`
`- 12 - '
`
`PCT/GB92/02260
`
`in magnification factors at different wavelengths.
`Again it may be possible to correct for this by
`modifying slightly the red and blue lens geometries.
`
`3 shows how the parallax errors between closely
`Fig.
`adjacent identical cameras can be maintained at or below
`one pixel for cameras with useful resolutions of several
`hundred pixels.
`If the cameras are calibrated to
`then an
`provide image alignment for objects at infinity,
`object at distance 0 (on the optical axis of one camera
`for simplicity) is imaged at an offset of e pixels in
`the second camera.
`It is obvious that the parallax
`
`error is greatest for objects which are closest to the
`
`cameras.
`To help in generalising the result, suppose we
`wish to image objects at minimum range 0mm with a field
`of View of 2c degrees and sensor resolution of P
`pixels.
`Then by trigonometry,
`the parallax error e,
`
`in
`
`10
`
`15
`
`pixels, is:-
`
`e = P.s / 0.2 tan (o) pixels
`
`(1)
`
`where s is the camera separation. This important result
`
`20
`
`is independent of the focal length of the lens and shows
`that the error is reduced by lowering s,
`lengthening O
`
`or increasing o.
`
`25
`
`is the parallax error for a
`Actually, e in equation (1)
`close object with reference to objects at infinity.
`Advantageously we might calibrate the cameras to provide
`alignment for objects at some mid range and achieve a
`balance of errors between close and far objects.
`It is
`
`easy to show that if we calibrate on objects at range
`2x0 then the worst case parallax error for near and far
`
`It
`
`30
`
`objects is:-
`
`e’ % P.s / 0.4 tan (o) pixels
`
`(2)
`
`1309-014
`
`1309-014
`
`
`
`\V()93/ll631
`
`_13—
`
`PC17(}B92/02260
`
`Thus in the case of a colour camera for computer vision
`
`using a typical standard resolution of e.g. 240 x 320
`
`pixels,
`
`then if we require a lens with 52° angle~of-view
`
`(o = 26°), horizontal resolution, P, of 320 pixels,
`
`minimum range 0, of 50cm and We achieve a separation, s,
`of 3mm, e' will be slightly less than one pixel.
`In the
`
`case of a low resolution colour camera for video
`
`telephones using the QCIF standard (144 X 178 pixels),
`
`this device must work at close range, say down to 25cm.
`
`Say we use the same angle of view, 52°, and achieve a
`
`lens separation of 1.5mm (allowing for the smaller
`
`array),
`
`then the error in this case will be
`
`approximately 0.5 pixels.
`
`In both cases,
`
`the parallax
`
`error is less than one pixel and therefore of the same
`
`order as aliasing and interpolation errors in
`
`single-chip colour mosaic cameras and accordingly
`
`reasonably acceptable.
`
`Conversely it will be appreciated that where it is
`
`desired to capture stereo images the parallax error and
`
`should be greater than 1.
`
`It may be seen from the above
`
`equations that parallax error increases for small object
`
`ranges or distances 0 and for larger camera separations
`
`5. With a sensor size P of 240 pixels, camera pitch
`separation s of 3.1.mm, and field of View of 45°
`(2o),
`
`rstereo image capture is feasible at ranges up to 1.8
`
`metres. This range can be increased simply by
`
`corresponding increases in the camera separation s.
`
`The electronic architecture of the camera is
`
`substantially independent of the optical and sensor
`
`arrangement described above. Nevertheless,
`
`the
`
`availability of synchronous, continuous RGB colour
`
`signals minimises the required image processing. This
`
`results in a simpler and lower-cost electronic
`
`10
`
`15
`
`20
`
`25
`
`30
`
`1309-015
`
`1309-015
`
`
`
`WO 93/11631
`
`- l4 -
`
`PCT/G 892/02260
`
`implementation than for colour—mosaic cameras.
`
`the electronic requirements may be
`Furthermore,
`implemented feasibly on the same chip as the sensors
`where CMOS sensor technology such as that described in
`
`our earlier Patent Publication No. W091/04498,
`
`is used.
`
`Figure 4 gives an overview of one possible electronic
`architecture. Three colour arrays are driven in similar
`
`style to a monochrome array, except that the timing of
`
`vertical and horizontal control on the red and blue
`
`10
`
`arrays is altered by offset values loaded into the
`
`controller from an off-chip PROM (Programmable Read Only
`
`Memory) Chip which has been programmed with the offset
`
`calibration data.
`
`is provided as for
`(ABC)
`Automatic Exposure Control
`monochrome arrays.
`The same exposure value is used for
`
`15
`
`all three arrays. Exposure is monitored via the green
`
`output alone, or possibly by deriving luminance from a
`
`combination of RGB signals.
`
`Automatic Gain Control
`
`(AGC)
`
`is also provided as for
`
`20
`
`monochrome arrays, but in this case,
`
`independent gains
`
`can be set for each colour.
`
`AGC provides three
`
`functions:—
`
`(i)
`
`automatic peak level calibration using a saturated
`
`reference line in the Green array.
`
`In normal exposure
`
`25
`
`circumstances,
`
`the gain of the green channel is fixed at
`
`this value and this forms a nominal gain for the red and
`
`blue channels also.
`
`‘11
`
`K9
`
`1309-016
`
`1309-016
`
`
`
`“K)93H163l
`
`— 15 —
`
`PCTVGB9ZNH2MD
`
`(ii)
`
`dynamic colour balancing to correct for variations
`
`in the colour of ambient light by adjusting the blue and
`
`red gains according to colour analysis of the three
`channels.
`
`(iii)
`
`automatic gain of weak images in low light
`
`conditions when ABC has reached maximum exposure. At
`
`first dynamic colour balancing can continue, but at the
`
`lowest light levels (maximum gain), flexibility may be
`
`lost. This is characteristic of many colour cameras.
`
`10
`
`15
`
`The resulting balanced colour signals are passed through
`
`a colour correction matrix, which performs weighted
`
`mixing of the three colours (see e.g. D'Luna and
`
`Parulski,
`
`IEEE JSSC Vol. 26, No. 5, pp. 727-737)
`
`followed by gamma correction on each colour. Both these
`
`functions are standard requirements for colour cameras
`
`intended for TV displays as they correct for known
`
`colour and amplitude nonlinearities in the display tubes.
`
`The matrix may be implemented in analogue CMOS, either
`
`by using switched capacitors or switched
`
`20
`
`current-sources. Either of these can accommodate
`
`alterable coefficients in digital form, or they could be
`
`fixed in layout.
`
`In the former case the coefficients
`
`,may be stored in the PROM already provided for offset
`
`25
`
`calibration and this may afford a useful degree of
`flexibility.
`
`The gamma corrected RGB signals are then passed to an
`
`appropriate encoder for whatever standard is required.
`
`Suitable encoders for e.g. NTS/PAL are readily available.
`
`In Figs.
`
`5 and 6 like parts corresponding to those in
`
`30
`
`Fig.
`
`1 are indicated by like reference numerals.
`
`The
`
`Figs.
`
`5 and 6 illustrate an alternative embodiment
`
`in
`
`which is used a lens system 20 supported at its edges 21
`
`1309-017
`
`1309-017
`
`
`
`1)
`
`WO 93/11631
`
`_ 16 _
`
`PCT/GB92/02260
`
`on spacer elements 22 with a substantial free air space
`23 between the three lenses 8 of the lens system and the
`respective sensors 4.
`In more detail,
`the lens system
`20 is a one-piece moulding from a suitable optical grade
`plastics material and incorporating an array of three
`lens portions 8 joined edge-to—edge.
`It will be
`appreciated that this affords a particularly economic
`and convenient form of production whilst at the same
`time simplyfying assembly of the camera insofar as the
`three lenses for the R, G, B components of the image,
`are automatically aligned with each other. Nevertheless
`
`it will be understood that, if desired,
`
`the lenses 8
`
`could be manufactured from other materials e.g. glass or
`any other suitable optical material and/or as discrete
`individual components. Moreover each lens could
`comprise more than one element e.g. a doublet. The
`lenses 8 may conveniently be aspheric or spherical or
`
`planar at either surface thereof.
`
`As in the first embodiment,
`the sensors 4 are formed as
`separate sections of a single monolithic VLSI microchip
`5, mounted in a housing pg (only part shown).
`An
`optical support sub-housing 24 has various shoulder
`portions 25-28, for respectively securing the monolithic
`chip 5 to the housing gfl, supporting the lens system 20
`on the spacer elements 22 above the chip 5, supporting
`R, G and B filters 29-31 above the lenses 8, and
`supporting a protective outer Infra Red filter 32
`(conveniently of doped glass e.g. Schott KG3 or BG39).
`Additional support to the lens system 21 and the filters
`
`10
`
`15
`
`20
`
`25
`
`30
`
`29-31 is conveniently provided by spacer walls 33, 34
`
`which have the further advantage of acting as light
`
`C)
`
`baffles between the three, R, G, B, cameras 1 to prevent
`
`cross-imaging between each lens 8 and the other sensors
`
`4. As shown in Fig.
`
`6 the lower spacer walls 33 may
`
`35
`
`conveniently be formed integrally with the support
`
`sub-housing 24.
`
`1309-018
`
`1309-018
`
`
`
`WO93/11631
`
`- 17 -
`
`PCT/GB92/02260
`
`It will be understood that various modifications may
`
`readily be made to the above described embodiment.
`
`Thus, for example,
`
`the order of the filters 29-31 and 32
`
`may be changed.
`
`(I
`
`1309-019
`
`1309-019
`
`
`
`\V()93/1163l
`
`- 18 -
`
`CLAIMS
`
`PCT/GB92/02260
`
`1. An image capture system comprising a solid state
`
`image capture device which device comprises an
`
`integrated circuit having at least two sensor arrays,
`
`each said array having an image sensing surface and a
`
`respective lens system associated therewith.
`
`'0
`
`2.
`
`A system according to claim 1 in which system at
`
`least one of the lens systems is mounted spaced apart
`
`from the sensor means with a fluid medium between the
`
`lens and the sensor.
`
`10
`
`A system according to claim 1 in which system at
`3.
`least one of the lens systems, comprises a lens in
`
`substantially direct contact with a transparent spacer
`
`in substantially direct contact with.the sensor and
`
`extending between said lens and sensor, said lens and
`
`15
`
`spacer having refractive indices and being dimensioned
`
`so as to form an image in a plane at or in direct
`
`proximity to a rear face of said spacer element remote
`from said lens element,
`from an object, whereby in use
`
`of the lens system with said lens system.mounted
`
`20
`
`substantially directly on the image sensing surface of
`
`the image capture device an optical image may be
`
`captured thereby.
`
`4.
`
`A system according to claim 3 Wherein sai
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
