`Schofield et al.
`
`USOO5796094A
`Patent Number:
`11
`45 Date of Patent:
`
`5,796,094
`Aug. 18, 1998
`
`54) VEHICLE HEADLIGHT CONTROL USING
`MAGING SENSOR
`
`75 Inventors: Kenneth Schofield, Holland; Mark L.
`Larson, Grand Haven; Keith J. Vadas,
`Coopersville, all of Mich.
`73 Assignee: Donnelly Corporation, Holland, Mich.
`
`OTHER PUBLICATIONS
`Article entitled “Generation of Vision Technology," pub
`lished by VLSI Vision Limited, publication date unknown.
`Article entitled "On-Chip CMOS Sensors for VLSI Imaging
`Systems" published by VLSI Vision Limited, 1991. (no
`month).
`Commonly assigned United States patent application Serial
`No. 08/023.918 filed on Feb. 26, 1993, by Kenneth (NMI)
`(21) Appl. No.: 621,863
`Schofield and Mark L. Larson for an Automatic Rearview
`Mirror System. Using a Photosensor Array.
`22 Filed:
`Mar 25, 1996
`Commonly assigned Unites States patent application Serial
`No. 08/478,093 filed Jun. 7, 1995, by Kenneth (NMI)
`Related U.S. Application Data
`Schofield and Mark L. Larson for an Automatic Rearview
`Mirror. Vehicle Lighting Control and Vehicle Interior Moni
`63) Continuation-in-part of Ser. No. 23,918, Feb. 26, 1993, Pat.
`toring System Using Photosensor Array.
`No. 5550,677.
`6
`Primary Examiner-Edward P. Westin
`HO5B 37.2
`(51) int. Cl. .............................. B60Q t
`Assistant Examiner-John R. Lee
`52 U.S. C. ................................. 520s. 2592 AL;
`Attorney; Agent, or Firm-Van Dyke. Gardner, Linn &
`250/226:362/61; 315/82
`Burkhart, LLP
`58) Field of Search ..................................... 250/226, 205,
`250/208.1, 208.2, 214 D, 214 AL, 216;
`ABSTRACT
`57
`356/218, 221, 222, 225; 315/82, 83; 362/61,
`64, 65, 71 A vehicle headlamp control method and apparatus includes
`providing an imaging sensor that senses light in spatially
`separated regions of a field of view forward of the vehicle.
`Light levels sensed in individual regions of the field of view
`are evaluated in order to identify light sources of interest,
`such as oncoming headlights and leading taillights. The
`vehicle's headlights are controlled in response to identifying
`such particular light sources or absence of such light
`sources. Spectral signatures of light sources may be exam
`ined in order to determine if the spectral signature matches
`that of particular light sources such as the spectral signatures
`of headlights or taillights. Sensed light levels may also be
`Eaccipial distribution in order to identify
`3.
`o
`
`56
`(56)
`
`References Cited
`eere
`e
`U.S. PATENT DOCUMENTS
`--
`3. As Risium
`4357.558 11/1982 Massoni et al. .
`4,727.290 2/1988 Smith et al..
`4,862.037 8/1989 Farber et al. .
`4,891.559
`1/1990 Matsumoto et al....................... 315/82
`4.967,319 10/1990 Seko .......................................... 362/61
`3. gE.
`awe .
`5,124.549
`6/1992 Michaels et al. .
`5,182,502
`1/1993 Slotkowski et al. .
`5.426.294 6/1995 Kobayashi et al. .
`5.537,003 7/1996 Bechtel et al. ............................ 315/82
`
`53 Claims, 11 Drawing Sheets
`
`- - - - - - - -m - - - - 4 - - - - - - - - -
`
`-- 13
`
`---,
`E. OTECTION
`
`38
`/
`
`|
`
`
`
`iSEK. ALS FOUND
`8O
`82
`
`3.
`
`68
`
`DETECTION/
`HRESHOLDING
`(WHITE)
`
`LIGHTING
`CONROL
`
`HG-BEAM
`ENABLE
`
`94.
`
`AMBENT
`SENSE
`LOGIC
`
`
`
`HEADLIGHT
`NHBT
`
`
`
`
`
`
`
`
`
`92
`
`:
`
`/
`
`
`
`
`
`
`
`WEHICLE
`LIGHTING
`CONTROL
`OGIC
`
`WEHICLE TELEMETRY
`STEERING, SPEED,
`ETc... 3
`
`VWGoA EX1009
`U.S. Patent No. 9,955,551
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet of 11
`
`5,796,094
`
`
`
`
`
`FORWARD
`FIELD OF WEW
`
`
`
`VEHICLE
`LIGHTING
`CONTROL
`LOGIC
`
`4.
`
`36
`
`/1
`
`Z
`
`/
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 2 of 11
`
`5,796,094
`
`96
`
`
`
`{’‘‘O 13|-————————————————————–
`
`| |
`
`| | |
`
`|CINTO: STIWL|/NOILOBIBO
`
`ºNiCJTOHS3|}|H_i
`
`NO|103130]
`
`TIO?H_1. NOO
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`U.S. Patent
`
`Aug. 18, 1998
`Aug. 18, 1998
`
`Sheet 3 of 11
`Sheet 3 of 11
`
`5,796,094
`5,796,094
`
`52
`D2
`
`54
`
`44
`44
`
`Cee
`ibvia [CHOYCH
`TL_POPOHoroooro
`fron(CL—foraroror
`P—Ororoojn
`
`6
`
`48
`48
`
`O
`o
`
`38
`@m
`
`
`Ch CHCH
`TOCLi 20
`
`
`CD
`Li–
`<t
`
`
`
`HOHCHOCHCHCHCH
`'''
`
`PHOHH
`
`PHCH
`
`
`
`
`
`
`
`
`
`
`
`
`
`48
`
`FIG.5
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 4 of 11
`
`5,796,094
`
`
`
`
`
`
`
`NM, WN
`7,
`2do soooo. 466 520 solo '7ooledo odo
`- "t.
`- uv
`NIR
`WAVELENGTH, NM
`FIG.8c
`
`
`
`36b
`
`
`
`
`
`1S 55 N 7
`V
`V
`7N A I/N
`A
`7 N
`
`
`
`38
`
`
`
`FG.6
`
`2O
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 5 of 11
`
`5,796,094
`
`START
`
`GRAB
`FRANETO2
`
`zero white" 194
`w
`ASEP
`
`OO
`
`NO
`
`
`
`
`
`OS
`
`
`
`
`
`O8
`
`O
`
`
`
`S
`PROCESS
`TALS"
`EQUAL
`YES."
`p
`YES
`SET SHUTTER TO
`HEADAMP"
`SETING
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FOR is 2 TO
`# PXELS
`
`2
`
`NO
`
`IS
`THIS
`A RED"
`
`P:E
`
`INCREMENT
`'RED COUNT
`C
`
`O9
`
`SET SHUTTER
`TO TAILLAMP"
`SETTING
`
`FOR = 2 TO
`Hit PXELS
`
`PROCESS TALS"
`"YES"
`
`
`
`24
`
`TO FIG. 7c
`
`6
`
`PROCESS TALS"
`"NO"
`
`TO FIG.7 G) GB)rg FG.
`7
`FG.7d.
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 6 of 11
`
`5,796,094
`
`8
`
`ARE
`WHTE"
`AND RED"
`COUNTS k
`THRESH
`p
`
`INDEXHIGH BEAMCOUNT
`RESET LOW BEAM'COUNT
`
`
`
`
`
`
`
`
`
`
`
`
`
`32
`
`S
`"LOW BEAM'
`COUNT)
`THRESH
`
`SWITCH TO"LOW
`BEAM MODE.
`SE "LOW BEAM
`COUNT TO THRESHOLD
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`S
`'WHITE
`COUNT" OR
`RED COUNT
`D THRESHOLD
`?
`
`
`
`RESET HIGH BEAM"COUNT
`INDEXLOW BEAM" COUNT
`
`RESETHGH BEAM"COUNT
`RESET LOW BEAM"COUNT
`
`
`
`
`
`
`
`HGH BEAM
`COUNT >
`THRESH
`?
`
`SWITCH TO HGH
`BEAM MODE.
`SETHIGHBEAM"
`COUNT TO THRESHOLD
`
`
`
`RETURN
`
`FG.7C
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 7 of 11
`
`5,796,094
`
`
`
`
`
`
`
`O 8
`
`
`
`A EPA
`- I
`Y
`FIA IV Y
`EIN IN
`N
`W T
`V
`N T
`7, (NT, \, .N
`
`2OO 3OO 4OO 46O 52O 58O G4.O 7OO 8OO 900
`UV
`WAVELENGTH, NM
`VSABLE
`FG.8
`
`NIR
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`NR
`
`
`
`
`
`200 3OO 4OO 46O 52O 58O 64O 7OO 8OO 900
`UV
`WAVELENGTH, NM
`WSABLE
`FG. 8d.
`a.a.Eart. A
`EY
`YY INI
`A
`ANI
`A
`V
`IV A.
`EAV VA
`F/NIXAXA
`6AN/NXVN
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 8 of 11
`
`5,796,094
`
`gt
`
`
`GNNO3SSTIVL
`SNIDTIOHSSYHL/NOILISL3G
`
`JIOIHSAa1avN3
`
`LHOIGW3H
`
`auvMGavH
`
`LHSIMN0V3H
`
`LIGIHNI
`
`LNSIGNV
`
`3SN3S
`
`
`
`500ohJO¥LNODBeSZILI9IGbeto]VUSNVS
`ONILHONMSNILHOM/34INDDWona’
`‘ONINSILS.{(91333aJTOIH3SA6Qs
`“"OL3‘Q33dS
`
`
`
`
`NV'38-H9IHSNIDTIOHS3dHL(wute\0)OWN)/NOILD3LAC
`
`sevna(3.LIHM)
`
`2|s
`
`
`
`
`
`ONIWIL/TONLNODQNINIL/INAS
`
`NOILI3L30
`
`TWYLNOSD
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 9 of 11
`
`5,796,094
`
`O
`
`28
`
`32
`
`3O
`
`
`
`34.
`
`7
`
`S.,
`I-DOS &
`
`3.
`
`2.
`38
`404
`36
`1-1-
`
`(O5 - N
`
`FORWARD
`LIGHT
`
`2O
`
`
`
`6
`
`8
`
`VEHICLE
`LIGHTING CONTROL
`LOGIC
`
`34
`
`FGO
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 10 of 11
`
`5,796,094
`
`FG.
`
`42
`
`
`
`98d.
`
`98.
`
`
`
`
`
`FG. Ilc
`
`98c
`
`
`
`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 11 of 11
`
`5,796,094
`
`NORMAL (DRY) CONDITIONS:
`
`
`
`
`
`5,796.094
`
`1
`VEHICLE HEADLIGHT CONTROL USING
`IMAGING SENSOR
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`This application is a continuation-in-part of application
`Ser. No. 08/023.918 filed Feb. 26, 1993, by Kenneth
`Schofield and Mark Larson now U.S. Pat. No. 5,550,677.
`
`2
`the ability to identify the headlights of oncoming vehicles
`and the taillights of leading vehicles, the state of the head
`lights of the controlled vehicle may be adjusted in response
`to the presence or absence of either of these light sources or
`the intensity of these light sources.
`This is accomplished according to an aspect of the inven
`tion by providing an imaging sensor which divides the scene
`forward of the vehicle into a plurality of spatially separated
`sensing regions. A control circuit is provided that is respon
`sive to the photosensors in order to determine if individual
`regions include light levels having a particular intensity. The
`control circuit thereby identifies particular light sources and
`provides a control output to the vehicle that is a function of
`the light source identified. The control output may control
`the dimmed state of the vehicle's headlamps.
`In order to more robustly respond to the different char
`acteristics of headlights and taillights, a different exposure
`period is provided for the array in order to detect each light
`source. In particular, the exposure period may be longer for
`detecting leading taillights and significantly shorter for
`detecting oncoming headlights.
`According to another aspect of the invention, a solid-state
`light imaging array is provided that is made up of a plurality
`of sensors arranged in a matrix on at least one semiconductor
`substrate. The light-imaging array includes at least one
`spectral separation device, wherein each of the sensors
`responds to light in a particular spectral region. The control
`circuit responds to the plurality of sensors in order to
`determine if spatially adjacent regions of the field of view
`forward of the vehicle include light of a particular spectral
`signature above a particular intensity level. In this manner,
`the control identifies light sources that are either oncoming
`headlights or leading taillights by identifying such light
`sources according to their spectral makeup.
`According to another aspect of the invention, a solid-state
`light-imaging array is provided that is made up of a plurality
`of sensors that divide the scene forward of the vehicle into
`spatially separated regions, and light sources are identified,
`at least in part, according to their spatial distribution across
`the regions. This aspect of the invention is based upon a
`recognition that headlights of oncoming vehicles and tail
`lights of leading vehicles are of interest to the control,
`irrespective of separation distance from the controlled
`vehicle, if the source is on the central axis of travel of the
`vehicle. Oncoming headlights and leading taillights may
`also be of interest away from this axis, or off axis, but only
`if the source has a higher intensity level and is spatially
`larger. These characteristics of headlights and taillights of
`interest may be taken into consideration by increasing the
`resolution of the imaging array along this central axis or by
`increasing the detection threshold off axis, or both. Such
`spatial evaluation may be implemented by selecting char
`acteristics of an optical device provided with the imaging
`sensor, such as providing increased magnification central of
`the forward scene, or providing a wide horizontal view and
`narrow vertical view, or the like, or by arrangement of the
`sensing circuitry, or a combination of these.
`The present invention provides a vehicle headlight control
`which is exceptionally discriminating in identifying oncom
`ing headlights and leading taillights in a commercially
`viable system which ignores other sources of light including
`streetlights and reflections of the controlled vehicle's head
`lights off signs, road markers, and the like. The present
`invention further provides a sensor having the ability to
`preselect data from the scene forward of the vehicle in order
`to reduce the input data set to optimize subsequent data
`
`BACKGROUND OF THE INVENTION
`This invention relates generally to vehicle control systems
`and, in particular, to a system and method for controlling the
`headlights of the vehicles. The invention is particularly
`adapted to controlling the vehicle's headlamps in response
`to sensing the headlights of oncoming vehicles and taillights
`of leading vehicles.
`It has long been a goal to automatically control the state
`of a vehicle's headlights in order to accomplish automati
`cally that which is manually performed by the driver. In
`particular, the driver of a vehicle whose headlights are in a
`high-beam state will dim the headlights upon conscious
`realization that the headlights are a distraction to the driver
`of an oncoming vehicle or a leading vehicle. It is desirable
`to relieve the driver of such duties and thereby allow the
`driver to concentrate on the driving task at hand. The ideal
`automatic control would also facilitate the use of high beams
`in conditions which allow their use, increasing the safety for
`the controlled vehicle as well as reducing the hazard caused
`by the occasional failure of the driver to dim the headlights
`when such headlights are distracting another driver.
`Prior attempts at vehicle headlight dimming controls have
`included a single light sensor which integrates light in the
`scene forward of the vehicle. When the integrated light
`exceeds a threshold, the vehicle headlights are dimmed.
`Such approaches have been ineffective. The headlights of
`oncoming vehicles are, at least from a distance, point
`sources of light. In order to detect such light sources in an
`integrated scene, it is necessary to set a sufficiently low
`threshold of detection that many non-point-sources at lower
`intensities are interpreted as headlights or taillights. Such
`prior art vehicle headlight dimming controls have also been
`ineffective at reliably detecting the taillights of leading
`vehicles. The apparent reason is that the characteristics of
`these two light sources; for example, intensity, are so dif
`ferent that detecting both has been impractical. In order to
`overcome such deficiencies, additional solutions have been
`attempted, such as the use of infrared filtering, baffling of the
`optic sensor, and the like. While such modifications may
`have improved performance somewhat, the long-felt need
`for a commercially useful vehicle headlight dimming control
`has gone unmet.
`
`15
`
`25
`
`30
`
`35
`
`45
`
`50
`
`SUMMARY OF THE INVENTION
`The present invention provides a vehicle control which is
`capable of identifying unique characteristics of light sources
`based upon a precise evaluation of light source characteris
`tics made in each portion of the scene forward of the vehicle.
`in the vicinity of each light source, by separating each light
`source from the remainder of the scene and analyzing that
`Source to determine its characteristics. One characteristic
`used in identifying a light source is the spectral character
`istics of that source which is compared with spectral signa
`tures of known light sources, such as those of headlights and
`taillights. Another characteristic used in identifying a light
`source is the spatial layout of the light source. By providing
`
`55
`
`65
`
`
`
`5.796,094
`
`O
`
`15
`
`25
`
`30
`
`35
`
`3
`processing. The invention is especially adapted for use with.
`but not limited to, photoarray imaging sensors. Such as
`CMOS and CCD arrays.
`These and other objects. advantages, and features of this
`invention will become apparent upon review of the follow
`ing specification in conjunction with the drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a side elevation of a portion of a vehicle
`embodying the invention:
`FIG. 2 is a partial side elevation view and block diagram
`of a vehicle headlight dimming control system according to
`the invention:
`FIG. 3 is a block diagram of the control system in FIG. 2;
`FIG. 4 is a layout of a light-sensing array useful with the
`invention;
`FIG. 5 is a block diagram of an imaging sensor;
`FIG. 6 is an alternative embodiment of an imaging sensor;
`FIGS. 7a-7d are a flowchart of a control program;
`FIGS. 8a-3c are spectral charts illustrating spectra
`regions useful with the invention;
`FIG. 9 is the same view as FIG. 3 of another alternative
`embodiment;
`FIG. 10 is the same view as FIG. 2 of an alternative
`mounting arrangement;
`FIGS. 11a-11c are views forward of a vehicle illustrating
`different forms of spatial filtering; and
`FIGS. 12a and 12b are illustrations of use of the invention
`to detect particular atmospheric conditions.
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`Referring now specifically to the drawings and the illus
`trative embodiments depicted therein, a vehicle 10 includes
`a vehicle headlight dimming control 12 made up of an
`imaging sensor module 14 which senses light from a scene
`forward of vehicle 10, an imaging control circuit 13 which
`receives data from sensor 14, and a vehicle lighting control
`logic module 16 which exchanges data with control circuit
`13 and controls headlamps 18 for the purpose of modifying
`the headlight beam (FIGS. 1 and 2). Such control may be a
`binary control of the aim of the beam, such as by switching
`between lamps or lamp filaments, or may be a continuous
`variation of the aim of a single lamp more or less forward of
`the vehicle. The control may also control the intensity or
`pattern of the beam. Additionally, the lights of a vehicle
`equipped with daytime running lights may be switched
`between a daytime running light condition and a low-beam
`condition. Vehicle headlight dimming control 12 can per
`form a wide range of additional control operations on the
`vehicle, including turning the headlights ON and OFF,
`modifying the light intensity of the instrument panel, and
`providing an input to an electro-optic mirror system.
`Vehicle lighting control logic module 16 receives an input
`20 from imaging control circuit 13. In particular
`embodiments, such as ones which adjust the state of the
`headlights between continuously variable states, module 16
`may supply data to imaging control circuit 13. Such as the
`speed of the vehicle, which may be combined with the data
`sensed by imaging sensor 14 in establishing the state of
`headlights 18. In the illustrated embodiment, imaging sensor
`module 14 may be fixedly mounted in a housing 28 by a
`bracket 34 mounted to, or near, the vehicle's windshield 32.
`Bracket 34 also mounts an interior rearview mirror 30. This
`
`4
`is a preferred mounting for imaging sensor module 14
`because the location within the interior of the vehicle
`substantially eliminates environmental dirt and moisture
`from fouling the light sensor module. Additionally, the
`position behind windshield 32, which typically is kept
`relatively clear through the use of washers and wipers and
`the like, ensures a relatively clear view of the scene forward
`of vehicle 10. Alternatively, imaging sensor module 14 may
`be mounted within a housing 29 of interior rearview mirror
`30 facing forward with respect to vehicle 10 (FIG. 10). In
`such embodiment, control circuit 13 may be combined with
`the circuit which controls the partial reflectance level of
`mirror 30 if mirror 30 is an electro-optic mirror such as an
`electrochromic mirror. Other mounting techniques for sen
`sor module 14 will be apparent to the skilled artisan.
`Imaging sensor module 14 includes an optical device 36,
`such as a lens, an array 38 of photon-accumulating light
`sensors, and a spectral separation device for separating light
`from the scene forward of vehicle 10 into a plurality of
`spectral bands, such as a filter array 40 disposed between
`optical device 36 and light-sensing array 38. Light-sensing
`array 38 is described in detail in U.S. Pat. No. 5550,677
`issued to Kenneth Schofield and Mark Larson for an AUTO
`MATC REARVIEW MIRROR SYSTEM USING A PHO
`TOSENSOR ARRAY, the disclosure of which is hereby
`incorporated herein by reference. Light-sensing array 36
`includes a plurality of photosensor elements 42 arranged in
`a matrix of columns and rows (FIG. 4). In the illustrated
`embodiment. an array of 512 rows and 512 columns of
`light-sensing pixels, each made up of a photosensor element
`42 is utilized. However, a greater or lesser number of
`photosensor elements may be utilized and may be arranged
`in matrix that is laid out in other than columns and rows.
`Each photosensor element 42 is connected to a common
`word-line 44. To access the photosensor array, a vertical
`shift register 46 generates word-line signals to each word
`line 44 to enable each row of photosensor elements 42. Each
`column of photosensor elements is also connected to a
`bit-line 48 which is connected to an amplifier 50. As each
`word-line 44 is accessed, a horizontal shift register 52 uses
`a line 54 to output the bit-line signals on consecutive bit
`lines 48 to an output line 56. In this manner, each photo
`sensor element 42 may be individually accessed by appro
`priate manipulation of shift registers 46 and 52. Output 56
`is supplied to a digital signal processor 13 which is supplied
`on an output 62 as input to control circuit 13 (FIGS. 3-5).
`Digital signal processor 13 includes an analog-to-digital
`converter 58 which receives the output 56 of array 36 and
`converts the analog pixel values to digital values. A digital
`output 68 of AlD converter 58 is supplied to a taillight
`detection circuit 76, a headlight detection circuit 78, and to
`ambient sense logic circuit 84. A detection control circuit 72
`supplies control and timing signals on a line 74 which is
`supplied to array 38, A/D converter 58 taillight detection
`circuit 76, headlight detection circuit 78, and ambient sense
`logic 84. Such signals coordinate the activities of these
`modules and provide any data, from look-up tables provided
`in control circuit 72, needed by each circuit to perform its
`function. For example, control circuit 72 may provide inten
`sity threshold levels to taillight detection circuit 76 and
`headlight detection circuit 78.
`Taillight detection circuit 76 detects a red light source
`having an intensity above a particular threshold as follows.
`For each pixel that is "red" a comparison is made with
`adjacent "green” pixels and "blue" pixels. If the intensity of
`a red pixel is more than a particular number of times the
`intensity of the adjacent green pixel and adjacent blue pixel.
`
`45
`
`50
`
`55
`
`65
`
`
`
`5,796,094
`
`5
`then it is determined that the light source is red. If the
`intensity of the "red" light source is greater than aparticular
`threshold, an indication is provided at 80.
`Headlight detection circuit 78 detects a white light source
`having an intensity above a particular threshold as follows.
`A white light is a combination of red, green. and blue
`components. If adjacent "red." "green." and "blue" pixels all
`exceed a particular threshold, a ratio comparison is made of
`the pixels. If the ratio of the intensity of the adjacent "red."
`"green.” and "blue pixels is within a particular range, such
`as 20 percent by way of example, then a white light source
`is detected.
`Vehicle headlight dimming control 12 additionally
`includes an ambient light-sensing circuit 84 which receives
`an input from digital output signal 68. Ambient detection
`circuit 84 samples a subset of photosensor elements and
`detects light levels sensed by the subset over a long period
`of time in order to produce significant time filtration.
`Preferably, the photosensor elements in the sensed subset
`include sensors that detect portions of the forward-looking
`scene that are just above the earth's horizon which is more
`indicative of the ambient light condition. Ambient detection
`circuit 84 produces an indication 88 of ambient light levels
`which is supplied as an input to a lighting control module 90.
`A high ambient light level may be used by a module 90 to
`inhibit headlight actuation or to switch headlights 18 to a
`daytime running light mode. Ambient detection circuit 84
`can, optionally, perform other functions, such as switching
`the daytime running lights of the vehicle between daytime
`and nighttime modes, controlling the intensity of the vehi
`cle's instrument panel and providing an input to an electro
`optic rearview mirror system.
`Indications 80 and 82 from the light detection units and
`indication 88 from ambient detection circuit 84 are supplied
`to a lighting control circuit 90 which produces a first
`indication 92 that headlights 18 are to be switched on, or
`Switched from a daytime running condition to a night mode.
`and a high-beam enable indication 94 that the headlights
`may be switched to a high-beam state. Vehicle lighting
`control logic module 16 responds to indications 92 and 94 by
`switching headlights 18 to an appropriate mode. An output
`96 from module 16 may be provided to supply lighting
`control circuit 90 with information with respect to vehicle
`telemetry, steering, speed, and any other parameter that may
`be incorporated into the algorithm to determine the state of
`45
`the headlights of the vehicle. Digital signal processor 13
`may be implemented using discrete digital circuit modules
`or with a suitably programmed micro-processor with input
`and output buffers.
`In one embodiment, an imaging sensor module 14a
`includes a single photosensor array 38a, one spectral filter
`array 40a, and one optical device 36a (FIG. 5). In this
`illustrated embodiment, spectral filter array 4.0a includes
`alternating spectrum filter elements for exposing adjacent
`pixels to different regions of the electromagnetic spectrum in
`the red band or green band or blue band. This may be
`accomplished by arranging such filter elements in stripes or
`by alternating filter spectral regions in a manner known in
`the art. Digital signal processor 13a captures a frame of data
`by enabling photosensor array38a for a particular exposure
`period during which each photosensor element 42 accumu
`lates photons. In order to detect oncoming headlights, digital
`signal processor 13a enables photosensor array 38a for a
`first exposure period. In order to detect leading taillights.
`digital signal processor 13a enables photosensor array 38a
`for a second exposure period. Because oncoming headlights
`have an intensity level that is substantially greater than that
`
`65
`
`30
`
`35
`
`50
`
`55
`
`15
`
`20
`
`25
`
`6
`of leading taillights, the exposure period of the frame in
`which leading taillights is detected is at least approximately
`ten times the length of the exposure period during which
`oncoming headlights are detected. Most preferably, the
`exposure period for detecting leading tail lights is approxi
`mately 40 times the exposure period for detecting oncoming
`headlights. In the illustrated embodiment, an exposure
`period of 0.004 seconds is utilized for detecting taillamps
`and 0.0001 seconds for detecting oncoming headlamps. The
`exposure period is the time during which each photosensor
`element 42 integrates photons before being read and reset by
`digital signal processor 13.a. Establishing a different expo
`sure period for detecting headlights verses taillights facili
`tates the use of existing and anticipated sensor technology
`by accommodating the dynamic range of such sensor tech
`nology. Exposure may also be adaptively established on a
`priority basis. In one such embodiment. exposure is set to a
`shorter headlight setting. If headlights are detected, the
`headlights 18 of vehicle 10 are dimmed and the exposure
`period is kept short. If no headlights are detected, the next
`frame is set to a longer exposure period. This has the
`advantage of shorter system cycle time as well as a reduction
`in sensitivity to sensor saturation and blooming. In another
`such embodiment, the exposure period is initially set to a
`long period. If an oncoming headlight is tentatively detected,
`the exposure period could then be switched to a short period
`to confirm the observation.
`Vehicle headlight dimming control 12 carries out a control
`routine 100 (FIGS. 7a-7d). At the beginning of each pass
`through the routine, which occurs for every frame captured
`by the imaging sensor, a frame is grabbed at 102 and all of
`the pixels in the frame are processed as follows. Counters
`used for detecting white headlight sources and red taillight
`sources are zeroed at 104. It is then determined at 106
`whether the previously processed frame was for detecting
`headlights or tail lights. This is determined by looking at a
`variable "process.tails” which will be set to "yes" if the
`previous frame was processed to detect headlights and will
`be set to "no" if the previous frame was processed to detect
`taillights. If it is determined at 106 that the variable "pro
`cess.tails" is set to "yes," the control proceeds to 108 in
`order to process the next frame to detect taillights. If it is
`determined at 106 that the variable process.tails is set to
`"no," then control passes to 109 in order to process the next
`frame as a headlight detecting frame.
`The taillight detecting frame process begins at 108 by
`setting the exposure period for the imaging sensor module to
`grab the next frame according to a headlamp exposure level.
`In the illustrated embodiment, the exposure period for
`detecting headlights is set at 0.0001 seconds. Processing of
`the taillight frame proceeds at 110 by examining, for each
`"red" pixel, whether the intensity of light sensed by that
`pixel is greater than a threshold and whether the intensity of
`light sensed by that pixel is greater than a selected number
`of multiples of the intensity of light sensed by an adjacent
`"blue” pixel and a selected number of multiples of the
`intensity of light sensed by an adjacent "green” pixel. If so,
`then a "red" counter is incremented at 114. Preferably, the
`ratio of red pixel intensity to green or blue pixel intensity is
`selected as a power of 2 (2, 4, 8, 16 . . .) in order to ease
`digital processing. However, other ratios may be used and
`different ratios can be used between red/green and red/blue
`pixels. In the illustrated embodiment, a ratio of 4 is selected
`based upon ratios established from CIE illuminant charts
`known to skilled artisans. Based upon these charts, a ratio
`greater than 4 would provide greater discrimination. Such
`greater discrimination may not be desirable because it could
`
`
`
`5,796,094
`
`10
`
`15
`
`25
`
`30
`
`35
`
`7
`result in failure to identify a leading taillight and, thereby, a
`failure to dim the headlights of the controlled vehicle. After
`all pixels have been processed, the parameter "process.tails'
`is set to "no" at 116 and control proceeds to 118 (FIG. 7c).
`In a similar fashion, processing of a headlight frame
`begins at 109 by setting the exposure period for the imaging
`sensor module to grab the next frame as a red taillight
`detecting frame. This is accomplished by setting the expo
`sure period of the imaging sensor module to 0.004 seconds.
`It is then determined at 120 for each pixel whether an
`adjacent set of "red,” “green," and "blue" pixels each
`exceeds a particular threshold and whether the pixel inten
`sity levels all fall within a particular range, such as within 20
`percent of each other. If all of the red, green, and blue pixels
`exceed a threshold and pass the ratio test, then it is deter
`mined that a white light source is being sensed and a "white"
`counter is incremented at 122. After all of the pixels in the
`frame have been processed, the process.tails flag is set to a
`“yes” state at 124. Control then passes to 118.
`It is determined at 118 whether both the 'white” and the
`"red" counters are below respective high-beam thresholds. If
`so, a high-beam frame counter is incremented and a low
`beam frame counter is set to zero at 120. If it is determined
`at 118 that both the "white" and the "red" counters are not
`less than a threshold, it is then determined at 126 whether
`either the "red" counter or the “white" counter is greater than
`a respective low-beam threshold. If so, the high-beam frame
`counter is set to zero and the low-beam frame counter is
`incremented at 128. If it is determined at 126that neither the
`"red" counter or the “white" counter is greater than the
`respective low-beam threshold, then both the high-beam
`frame counters and the low-beam frame counters are set to
`Zero at 130.
`Control then passes to 132 where it is determined if the
`low-beam frame counter is greater than a particular thresh
`old. If so, high-beam enable signal 94 is set to a "low-beam"
`state at 134. Additionally, the low-beam frame counter is set
`to the threshold level. If it is determined at 132 that the
`low-beam frame counter is not greater than its threshold, it
`is determined at 136 whether the high-beam frame counter
`is greater than its threshold. If so, high-beam enable signal
`94 is set to “high-beam” state at 138 and the high-beam
`frame counter is reset to its threshold level.
`Control routine 100 provides hysteresis by requiring that
`a headlight spectral signature or a taillight spectral signature
`be detected for a number of frames prior to switching the
`headlights to a low-beam state. Likewise, the absence of a
`detection of an oncoming headlight or a leading taillight
`must be made for multiple frames in order to switch from a
`low-beam to a high-beam state. This hysteresis guards
`against erroneous detection due to noise in a given frame and
`eliminates headlamp toggling when sources are at the fringe
`of detection range. In the illustrated embodiment, it is
`expected that a vehicle
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ ChargeStill Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site