PROCEEDINGS
`~SPIE-The International Society for Optical Engineering
`
`Collision A voidance
`and Automated Traffic
`Management Sensors
`
`Alan C. Chachich
`Marten J. de Vries
`Chairs/Editors
`
`25-26 October 1995
`Philadelphia, Pennsylvania
`
`Sponsored and Published by
`SPIE-The International Society for Optical Engineering
`
`Volume 2592
`
`SPIE is an international technical society dedicated to advancing engineering and scientific applications
`of optical, photonic, imaging, electronic, and optoelectronic technologies.
`
`Magna 2049
`TRW v. Magna
`IPR2015-00436
`
`0001
`
`

`
`The papers appearing in this book comprise the proceedings of the meeting mentioned on
`the cover and title page. They reflect the authors' opinions and are published as presented
`and without change, in the interests of timely dissemination. Their inclusion in this
`publication does not necessarily constitute endorsement by the editors or by SPIE.
`
`Please use the following format to cite material from this book:
`Author(s), •ritfe of paper, • in Collision Avoidance and Automated Traffic Management
`Sensors, Alan C. Chachich, Marten J. de Vries, Editors, Proc. SPIE 2592, page numbers
`(1995).
`
`Library of Congress catalog Card No. 95-69910
`ISBN 0-8194-1956-7
`
`Published by
`SPIE-The International Society for Optical Engineering
`P.O. Box tO, Bellingham, Washington 98227..0010 USA
`Telephone 360/676-3290 (Pacific Time) • Fax 360/647-1445
`
`Copyright D1995, The Society of Photo-Optical Instrumentation Engineers.
`
`Copying of material in this book for internal or personal use, or for the internal or personal
`use of specific clients, beyond the fair use provisions granted by the U.S. Copyright law is
`authorized by SPIE subject to payment of copying fees. The Transactional Reporting Service
`base fee for this volume is $6.00 per article (or portion thereof), which should be paid
`directly to the Copyright Clearance Center {CCO, 222 Rosewood Drive, Danvers, MA
`01923. Payment may also be made electronically
`through CCC Online at
`http://www.directory.net/copyright/. Other copying for republication, resale, advertising or
`promotion, or any form of systematic or multiple reproduction of any material in this book
`is prohibited except with permission in writing from the publisher. The CCC fee code is
`~194-1956-7/951$6.00.
`
`Printed in the United States of America.
`
`0002
`
`

`
`Assessment of driver vision enhancement technologies
`
`Louis J. Denes, MEMBER SPIE
`Richard Gtace
`David A. Purta
`Alberto M. Guzman
`
`Carnegie Mellon Research Institute
`700 Technology Drive,
`Pittsburgh. PA 15219-3126
`Telephone: (412) 268-3469,
`Fax: (412)268-7759
`E-mail: ld2s@andrew .cm!ledu
`
`Subject terms:
`Automotive, driver vision enhancement,
`infrared imagers, uncooled thermal arrays,
`rndar systems.
`
`ABSTRACT
`
`Driver vision enhancement systems provide augmented information to improve
`the driver's perceptual ability when visibility is reduced. Vision enhancement is
`a tecbnologically-<:hallenging missioll We surveyed two classes of tecbnologtes:
`imaging systems (visible and inftared) and rndars (~ter-wave and -~
`rndars). Night (!R) vision and rndar-based systems pronuse meamngful VISion
`enhancement functionality to the driver. Available field test data give thennal
`imagets operating in the range of 8 to 12 1J.D1 an edge. This spectral regime bas
`a long (miles) clear night range, adequate object discrimination and. handles
`inclement weather conditions better than other sbnrter wavelength llllllgets.
`UDCOoled thermal imagets, because of their potentially low-<:ast, are emeiging as
`a front runner technology. All weather penetration of a radar based system is
`attractive for certain driving scenarios. They are not particularly adept in high
`resolution imaging. This combination makes them more of interest as automated
`warning devices. Icons replace the actual objects imaged to indicate the hazanl
`ahead. True all-weather high-resolution vision enhancement systems are beyond
`near-tenn capabilities. Overall, vision enhancement systems under development
`today will have good utility with the challenge that they become 'affordable'.
`
`1 INTRODUCTION
`
`~nmental and physiological factors limit driver visioll
`Driving a car under conditions of reduced vision often creates
`considerable discomfort that contributes to ttaffic accidents.
`Statistics show that approximately 42 percent of crashes and
`58 percent of fatal crashes occur at night or during some
`degraded-visibility conditions caused by inclement weather.
`Dlumination characteristic of the automobile's headlights
`limits driver vision at night. Advetse weather conditions such
`as fog, rainfall. heavy mist, sleet, snowfall or dust, finther
`reduce acuity and range ofvisio!llmaging technologies offer a
`way to extend night vision both in range and field of view
`(FOV), improve visibility in inclement weather and provide
`better identification of objects within the FOV (better acuity).
`These featnres are particularly relevant to drivelS over the age
`of 50 who generally have diminished night vision acuity.
`
`Vision is the primaiy source of information for the driver at all
`levels of driving functions such as vehicle control arid
`navigatioll During driving, the driver pelforms a nnmber a
`visual activities including perception of vehicles ahead,
`awareness of pedestrians, recognition of ttaffic ~igns and
`lights, road limits and marl<ings, and perception of optical
`signals such as brake lights, rear lights, turn indicatoiS and
`hazanl warning signals. During nighttime, the driver's eyes
`are operating at a 'mesopic' level where contrast sensitivity is
`the daylight conditions (photopic
`redoced compared to
`vision). Confusion can result from fluctoations
`in the
`brightness distribution within the drivel's field of visioll
`Such fluctoations can come from changes in road lighting,
`illumination from oncoming vehicles or even great changes in
`
`the brightness of the scenery. Glare resulting from these
`sources bas a stronger detrimental effect on older drivers.
`
`Atmospheric effects also impact driver visual pelfonnance.
`Rain or spray from passing vehicles reduces contrast between
`objects and background. Likewise, snowfall creates similar
`effects to night driving in that the vehicle's illumination is
`reflected by the snow. Dense fog gives the most negative
`effect on visnal perceptio!l All of these detrimental effects
`impact the pelfonnance of the visnal task, limiting the ability
`to judge distances to objects, or to estimate one's speed or the
`speed of other vehicles. High incidence of accidents from dusk
`to dawn is attributable to reduced visibility.
`
`This study assesses imaging systems in the visible and
`radars
`inftared regions of the electromagnetic spectrum,
`operating in various bauds from 30 to 300 GHz and laser
`rndars. Maoy technologies have potemial for developing
`automotive vision enhancement systems. The U.S., Japan
`and various European countries are actively worldng on the
`development of systems for the intelligent vehicles. The
`technology base existing in the U.S., as well as other
`countries, flnds its roots in the military establishme~
`Although defense and civilian applications' scenarios are qmte
`different, the automobile indnstry benefits from this wealth a
`technologies. The challenge is adapting these military
`technologies to an automotive industry that requires high
`pelfonnan::e at low cost. This transi_tion _is now. uude~lll:'.
`Examples include: uncooled thermal nnagmg devices, milli(cid:173)
`meter wave radaiS based on monolithic integrated circuits and
`head up displays that project vinnal images of the night
`scenery captured by IR cameras. These fledgling technologies
`
`0-8194-1956-7/951$6.00
`
`SPJE Vol: 2592117
`
`0003
`
`

`
`are moving into production with limited runs and finding
`their way into demonsttation progi3lliS.
`
`2 IMAGERS (CAMERAS)
`
`Imagers or cameras define the scene by detecting radiation
`reflected from or emitted by the various objects within the
`field of view. Imager sensor materials span the entire electro(cid:173)
`magnetic spectrum from the UV through the IR. Yet, the
`"ideal" sensor still needs to be discovered. Our goal is to seek
`out those that can provide high petfOIIII3lre at a low cost.
`Terrestrial environmeniS are dominated by visible radiation
`and thermal radiation. Figure I shows the terrestrial spectta1
`rndiance attributable to solar reflectance, thermal earth shine,
`along with emission from blackbodies at tempemtures ri
`200K, 300K and 700K. Solar refleclan:e is contained mostly
`in the visible diminishing to a small value beyond several
`micrometers. Thermal earthshine has effectively no visible
`Its spectta1
`conte~ but_peaks in the 8 to 14 IJlll region.
`shape 1s qwte close to the thermal emission from a 300K
`bla:kbody. Of particular note, is the rapid falloff in spectta1
`ennttance below 5 IJlll. As a consequence, optical detection
`below 5 IJlll genernlly requires some form of externalligllting
`source such as from sunlight or headlights.
`
`Black body
`raciance
`
`Wavelength, pm
`
`Fig. 1 Spectral characteristics of terrestrial radiance.
`
`The atmosphere is mostly ttansparent in this spectta1 regime
`the spectral atmospheric
`of interest Figure 2 shows
`transmission over a long 0.3 km path over the Chesapeake
`Bay area. Because of the strong absorption bands, imager
`systems are gene1311y optimized fur one of four transmission
`bands:
`
`1.) 0.4 to 1.0 IJlll (VIS-NIR),
`2.) 1.0 to 2.5 IJlll (SWIR),
`3.) 3 to 5 IJlll (MWIR)
`4.) 8 to 14 IJlll (L WIR).
`
`Imagers of many types and charncteristics have been developed
`for operation in each of these four regimes. Besides spectta1
`response, imagers are categorized by their temporal response,
`
`7 8 I SPIE Vol. 2-592
`
`by their cooling needs, and by their anay size. System
`~~ pamweters include: resolution, noise equivalent
`differential temperature (NEDT), modulation tmusfer function
`resolvable temperature
`(MRT), and
`~· minimum
`muumum detectable temperature (MDT). These pammeters
`are interrelated and need to be measured fur complete
`chamcterization of the imaging system
`
`VIS· Nil\ SW1R I MW1R I LWIR I
`1.0 r--::::::::;;;;;;rr-:lii~:rw----,
`1
`
`0.8 1 o .•
`
`~
`~ 0.4
`
`0.2
`
`o.o -1------~-!.li=L.JLL--....1
`1.0
`10.0
`0.1
`100.0
`Wavelength, p.rn
`
`Fig. 2 Atmospheric transmittance over a 300 meter path
`in the Chesapeake Bay area
`
`3 TECHNICAL ASSESSMENT CRITERIA
`
`PatteiSOn1 in his paper on "Multiple Function Sensors for
`Enhanced Vision Application" defines the need for sensor
`selecti~n criteria to support functional integration. Along
`these lines, the following criteria are used for assessing the
`characteristics of the various imager technologies:
`
`1. Clear uight performance.
`2. Inclement weather performance.
`3. Maturity I projected availability.
`4. Product maintainability I survivability.
`5. Cost I projected cost.
`
`Under criteria I and 2, visual factors' include: acuity, field ri
`view, spectral sensitivity, and depth perception.
`Sensor
`factors include: sensitivity, signal-to-noise ratio, component
`time coustants, spectral response, dynamic range,
`image
`resolution, scene contrast, cooling requirements, allowable
`field of view and array size constraints. These pammeters are
`interrelated, so it may not be poSSible to correlate individual
`control opemtion and effects among the various technological
`approaches. Criteria 3 through 5 are educated projections ri
`where the various technologies will evolve in 3 to 5 years.
`
`4 IMAGER TECHNOLOGY ASSESSMENT
`
`4.1 Imaging Optics
`
`V!Siblelnear-IR optics are conuneiCial and affolllable. Such
`optics can fulfill any foreseeable application related to the
`vision enhancement scenario. MWIR and LWIR optics are
`limited and expensive. The optical elements
`involve
`expensive materials (e.g., germaniwn, zinc selenide, etc.)
`
`0004
`
`

`
`And because of the longer wavelength in the IR, the optical
`elements an: larger in older to achieve shaip (i.e., dilliaction(cid:173)
`limited) images. A major problem in the 1R is caused by the
`detector anay "seeing" radiation other than the desired seem
`rndiation from object space. These efl'ects are analogous to
`strny-light problems often eocounteled in the visible. They
`can be obviated by good optical design, but at significant
`cost.
`
`4.2 Pickup Element (Sensor Array)
`
`Table 1 presents an oveiView of the various detector
`teelmologies that an: available today for optical imaging.
`Mechanical scanner imagers an: excluded for the vision
`enhancement scenario becanse of their inherent complexity and
`high cost 3 The table includes: detector type, material
`spectrnl response and ovetall system spectral response. Holst
`et al4 gives expanded details of photon sensor anays.
`
`4.3. Vidicons
`
`Vidicons comprise a broad class of camern tubes. Each ojlerate
`in a specif'IC spectrnl rnnge that spans the visible, NIR, MWlR
`and L WIR portions of the spectrum. Vidicons an: the
`mainstay of commercial broadcast and indnstrial TV. They
`an: highly developed to achieve high spatial resolution.
`Operntionally, they an: complex, ftagile, and costly. Their
`dominance in the field will diminish as solid-state CCD
`imagers come to the forefront The main variant in vidicon
`types is the material used as a "target". The silicon vidicon
`utilizes a silicon photodiode anay as .a target and has the
`characteristics of high sensitivity, low darl< current, very low
`Jag and non-burn-in. This vidicon can be exposed to direct
`sunlight withont damage and has a spectral response that
`extends out in the lR to about 1.1 J.Lm. The Harpicon' is
`another recently developed tube. This vidicon tube uses a
`HARP <High-gain Avalanche Rushing amorphous f.hoto(cid:173)
`conductor) target This technology promises a 100-fold
`increase in sensitivity with no .loss in picture quality. Low
`sensitivity pyroelectric vidicons used in the MWIR and
`L WlR have been commercially available for over 20 years.
`Pyroelectric vidicons produce images only in response to
`changes in target tempernture. Therefore, some provision
`most be made to provide a time varying tempernture profile
`over the scene. For fixed installations, the incoming thermal
`rndiation is normally chopped or shuttered at a controlled
`interval One of the key obstacles in all vidicons
`is
`"registrntion". Low pelfoiii13IlCe vidicons may run a i:w
`hondred dollars, but a high pelfoiii13IlCe vidicon with
`excellent resolution will run in excess of $SOOO.
`
`4.4 Image Intensifiers
`
`complexity (cooling hardware and control) and a slower
`response. This slower response precludes standard video in
`genernl. The image intensifier technology' has emerged to
`provide both sensitivity and adequate speed of response.
`Today's tbird-generntion image intensifiers are still expensive
`(> $10,000).
`
`Table 1 Imaging teehnologies.
`
`.. , ....
`
`Vidicons
`VISible
`Silicon
`Harpicon
`
`I !""'""
`a2l.
`ARRAYS
`INear:
`Silicon CCD
`. TnC'.,.A<
`
`PbS
`
`PbSe
`
`H~C'11Te
`
`Bolo metric
`
`:Golay
`0 ro- rs
`)j
`
`1-T·.
`Amb.
`
`Atub.
`
`Atub.
`~~·
`~~-
`'#·
`
`:00
`~·
`~-
`Amb.
`
`~-
`
`0.4 - 0.7 IJ.lll
`0.4 - 1 IJ.lll
`0.4 - 0.7+ IJ.lll
`8-12Jtm
`o.s - 0.9 IJ.lll
`
`I 0.4-0.7 urn
`0.9 . 1.7 IJ.lll
`
`.
`
`1 - 3 IJ.lll
`
`I - S .5 IJ.lll
`
`3-7 urn
`8 14 ).1m
`
`8 - l41J.lll
`
`18-14 urn
`
`- sum
`- sum
`-
`Jtm
`
`4.5 Ambient Arrays
`
`Silicon CCD
`Remarl<able improvements in CCD imaging teehnologies has
`occurred in recent times. CCD imaging camerns an: now
`available with resolutions exceeding 4 million pixels per
`fiame, image dynamic rnnge exceeding 12 bits, and data
`trnnsfer rntes of more than 100 MB/s.
`In the consumer-video
`market, CCDs an: smaller and inexpensive. Useful CCD
`cameras'are advertised for less than a hundred dollars.
`
`The image intensifier technology was originally developed as
`an amplifying element in military night vision systems. All
`pickup elements have the cbarncteristic of a lower electronic
`noise floor and consequently a higher sensitivity when cooled,
`relative to the ambient environment The 1ladeoff is more
`
`InGaAs
`InGaAs anays7 is an'emerging teehnology. InGaAs detectors
`have high quantum <;fficiencies (T) > 0.7) in the (1 to 1.7 IJ.lll)
`spectr.il range. Mean detectivities of the FP As, D*"-pk have
`
`SPIE Vol. 2592/79
`
`0005
`
`

`
`a near-term goal of 0.1 °C. In a joint development project
`between Hughes and Texas Instruments, a thennal vision
`system called NightsightTW is now on the market 11 This
`technology currently targets law enforcement agencies with a
`$6000 price tag. Figure 3 pictures a night driving scere
`using a visible camera and the NightsightTM system.
`
`Bolometric Arrays
`
`Bolometric .ArnJys is another emetging teclmology with a
`promising future.12 Currently, bolo metric anays have about
`80,000 elements in a 2-D array designed to operate at ambient
`temperatures. Like the fenoelectric arrays, it has a broad, flat
`spectral response typical of thennal detectors. Response times
`are of the order of milliseconds, so high-speed scanning is not
`an option. Nevertheless, each element of the detector is fs
`enough to respond fully to a typical 30 Hz video scanning
`rate. Microbolometers can operate with or without an optical
`chopper. Bolometric cameras provided to the Army by
`Honeywell, exhibited NEDT values of 0.04°C for room
`temperature targets. This is within a factor of four or so of the
`top-of-the-line commercially available cooled
`scaruring
`thermal imagers. Figure 4 displays a driving scene using a
`
`exceeded 1013 cm-..JHvw at room temperature iD:reasing to-
`3 1014 cm-..JHvw when cooled to 230K. This (1 to 1.7 J.UD.)
`spectral region may impact nighttime viewing. Night-light
`contains a mixture of starlight and natural luminescence. It is
`most intense in 1 to 1.3 J.Ull region and thus represents a
`InGaAs
`good overlap with tbe InGaAs spectral response.
`imagers are becoming available but still expensive (>$10K).
`
`PbS & PbSe
`Lead sulfide (PbS) and lead selenide (PbSe) detectors are thin
`film semiconductor devices (pbotoconductors) that work
`in
`the 1 J.UI1 to 3 1.LJ11 and 1 J.U11 to 7 J.U1l spectral regions. PbS
`and PbSe detectors' date back to World War ll in the United
`States. Militaiy and commercial demands still foster R&D in
`this area Detectors achieve high D* values and operate well
`at near ambient temperatures.
`
`4.6 Uncooled Staring Arrays
`Staring sensors operated in the 8 to 12 J.U1l band, collect so
`much flux that quite good imagery is obtained even with low(cid:173)
`responsivity uncooled focal plane arrays (FPAs).9 Methods
`iD:lude:
`the
`available for sensing
`temperature change
`coefficient of
`resistance),
`the
`bolometer
`(temperature
`capacitance bolometer (temperature coefficient of dielectric
`constant),
`the thermo-couple,
`the Golay cell
`(thennal
`expansion of a gas) and the pyroelectric or feooelectric
`detector. Two uocooled detector teclmologies that lave
`recently emerged from tbe pack: fenoelectric (pyroelectric)
`arrays and bolometric arrays are discussed below.
`
`Ferroelectric (Pyroelectric) Arrays
`Fenoelectric materials10 used for the pyroelectric detection cf
`IR rndiation started in the 1960s. These anays work best in
`the L WIR region of the spectrum. .ArnJy sizes are 328 x 245
`with a 2 mil by 2 mil pixel size. Noise equivalent
`temperature difference ofTI's array technology is< 0.2°C with
`
`Fig. 4 Display of a driving scene using a Horeywell
`microbolometer array.
`
`Honeywell microbolometer array. Spatial resolution is good,
`being less than a milliradian and on par with the best
`commercial imagers available. ARPA has selected WlCOOled
`infiared sensors as a key enabling teclmology for its
`Technology Reinvestment Project (TRP). Unit costs cf
`$1000 are foreseen
`
`Tunneling Golay Arrays
`The tunneling Golay array is an emerging teclmology that has
`potential for favorable sensitivity, size and cooling and low
`cost fabrication The tunneling infrared sensor can operate
`without cooling or thermal stabilizltion of the focal plane
`army, while retaining high penonnanc:e. The tunneling
`infrared sensor is not a de-coupled device, so it must be
`time varying signals, akin
`to
`configured
`to measure
`
`Fig. 3 Direct comparison of the NightsightTM and visible
`camera displays of a night driving scene.
`
`20 I SPIE Vol. 2592
`
`0006
`
`

`
`pyroelectric detectors. Jet Propulsion Laboratory researchers
`project the following potential:
`
`pixel size
`sensitivity
`band width
`cost
`
`1 em to< 100 J..lll1
`NEP < 10-10 WI ..JHz
`I kHz
`< $100 sensor package
`
`4. 7 Cryo-cooled Arrays
`
`PtSi
`
`Schottky barrier PtSi focal plane array (FP A) technology has
`matured under DoD sponsored programs.13 PtSi detectors
`offer the largest IR image sensor fonnats currently available.
`Camera systems have been developed for classical 3 to 5 fJlD.
`(MWIR) thermography applications as well as I to 3 fJlD.
`(SWIR) and 200 nm to 5 fJlD. broadband applications. Staring
`FP A camera systems are being developed with fiame rates up
`to 900 frames per second. Half TV (320 x 244) and full TV
`(640 x 480) resolution camera systems are now available with
`advanced features including subframe imaging and exposure
`control. Hybrid structures have been devised that offer a
`nearly 1000/o fill factor. PtSi FPAs exhibit high response
`uniformity and high linearity, but relatively low quantum
`efficiency. Response uniformity is attn"butable to silicon on(cid:173)
`chip fabrication and processing. Quantum efficiency decreases
`from a high of about 8% at 1 J..lll1 falling to 1% at 4 J..lll1 and to
`10
`0.1%.at 5 fJlD.. D* is in the range of6.5x10 cm..JHz-cm-1 ,
`while the noise equivalent temperature difference (NEDT) is
`O. l°C. The minimum resolvable temperature (MR.l) is in the
`rnnge of o.orc. PtSi FPAs require cooling to 77°K b
`It is now standard practice to use a
`optimal operation.
`miniature Stirling cycle ctyo-cooler that has about a half watt
`cooling capacity. Camera systems are expensive (>$50 K).
`
`InSb
`Photovoltaic InSb has been a very popular detector for the 3
`to 5 J..lll1 (MWIR) spectral band. 1
`• Thermally-generated noise
`characteristics, however, require cryogenic cooling to about
`80° K. It has an exceedingly high-peak quantum efficieocy
`(>90% at 5.5 fJlD.).
`lnSb material is highly uniform.
`Processing now provides yields of good to excellent anay
`response unifonnity. Devices are usually made with a p-n
`diode polarity using diffusion or ion implantation. Staring
`arrays ofbackside-illurninated, direct hybrid InSb detectors up
`to 640 x 480 formats are available with readout suitable fer
`high background f/2 operation. These cameras are vety
`expensive (>$50 K). Figure 5 displays the operation of the
`Radiance™ 1 IR camera with a visible camera under
`conditions of light fog.
`
`BgCdTe
`
`HgCdTe detectorsu are selectable to cover the spectral range
`from 1 to 25 IJ.Ill. The versatility of HgCdTe detector
`material relates to the broad range of alloy compositions that
`optimize the response at a particular wavelength. Recently,
`
`ingenious techniques have advanced the development of high
`perfoi'III30Ce HgCdTe focal plane arrays. Arrays are fabricated
`using a hybrid construction technique in which a mosaic anay
`of photovoltaic HgCdTe diodes is flip-chipped and bump(cid:173)
`mounted on a silicon mosaic multiplexer. Columns of indium
`interconnects are used to make electrical contacts between the
`diodes above and the silicon input circuits below. Dissimilar
`thermal-expansion between HgCdTe and silicon restricts the
`maximum mosaic size to a diagonal of about one-half inch.
`At MWIR., HgCdTe anays can operate with thennoelectric
`(IE) cooling. At L W1R, the arrays are cryo-cooled to 77° K.
`Near-peak spectral response anays have high quantum
`efficieocies (> 65% even without anti-reflection coating) with
`o• values ranging about 1011 cm..JHz·cm-1. By all standards,
`these arrays are quite expensive ($10K- $200K).
`Radiance TM
`
`Visible
`
`Fig. 5 Comparison of the Radiance™ IR camera with a
`visible camera under conditions of light fog.
`
`SPIE Vol. 2592 I 21
`
`0007
`
`

`
`4.8 'Smart' Cameras
`
`Smart camems me a vety active mea of development One
`concept is the "Neurommpbic" focal plane anay. 16 TI!ese
`arrays mimic the fo!Dl and function of the venebtate retina.
`Tile neuromorphic focal plane array has the capability cf
`pelfonning pixel-based sensor fusion. and real-time local
`contrast enlJancemem, much like the human eye.
`In par(cid:173)
`ticular, it can pelfoim real-time local contrast enllancernent
`and spatial filtering. Local contrast enlJancement pemlits the
`system to make nse of the full dynamic lllilge of the focal
`plane, and the nser~n!rollable spatial filtering pemlits the
`the
`nser to "tnne" the focal plane's spatial response to
`dimensions of the objects in the field of view. Tile focal plane
`can be selected to produce images with vatying amounts cf
`enhauced edges with respect to the pure radiometric response
`of the focal plane. Both spatial and tempornl charncteristics
`me accomplished using a switcbed-capacitor resistive netwoik,
`integrated into a CMOS readout architecture to pelfoim a two(cid:173)
`dimensional avernging function, as well as tempornl control ri
`charge admitted into
`the spatial avernging anay.
`A
`neuromo~phic anay flees all off-plane processors to execute
`other more complex algorithms.
`
`4.9 Field Data and Observations
`
`All spectral regions from the UV to the L WIR have received
`in-depth study, analysis and device development by the U.S.
`AIDly. going back to the 1960s. TI1eir interests led to tbe
`development of pulse-gated, image intensifier carnet3S (VIS(cid:173)
`NIR), MWIR imager devices, L WIR imager devices or
`"FL!Rs" and SWIR imaging devices. 17 Tiley conducted
`multiple field pelfoiDlanCe studies of these imager systems.
`Some unanticipated imdings from their field evaluations me:
`Fog comes in severn1 forms, some wet and some dty, even
`though both types appear similar to the eye. Wet fogs are
`usually associated with coastal fog. while dty fogs are often
`associated with radiation effects over land tenain. Dty fog
`would limit the eye to abont 200 meters and the TV to 250
`meters, but it would not significantly aflfct the FLIR out to
`beyond 2 km. 18 On the otber hand, under beavy wet fog
`conditions the FLIR was impacted similar to the eye and the
`Tile army ttials establisbed that the FLIR was
`TV.
`significantly superior to pulse-gated, image intensifier camelllS
`(VIS-NIR) under most environmental conditions encountered
`Likewise the 8 to 12 jllll FLIRs were superior to the 3 to 5
`jllll systems in thick fogs. It SUiprised COIWentioml wisdom
`that poor pelfoiDlanCe was shown by the laser pulse-gated
`system under low visibility conditions. Although backscatter
`was eliminated as expected, the attenuation, low contrnst, and
`loss of resolution badly degrnded the long rnnge pelfonnance.
`Increasing the laser power would not help. Farmer18 gives
`some insights as to why FLIRs have such fuvornble
`pelformance charncteristics. One factor is that small differences
`between objects and background emissivities often give
`relatively large target-background thermal contlllSts even when
`tbeimodynamic temperature differences are small.
`For
`example, Wolfe" shows that genernlly, changes in tbe
`
`emissivity of about 0.01 can correspond to tempernture
`changes of about O.SK for 300K ambient temperntnres.
`
`4.10 Passive Imaging Systems
`Passive imaging systems utilize available reflected light or
`thermal rndiation emanating from the target scene. TI!ese
`include sunlight reflection, grnybody thermal emission,
`environmental "earthshine" and earthshine reflectance. During
`daylight, solar reflection, primarily visible and near 1R,
`dominates spectrnl radiance from any scene. At night, thenual
`earthshine (spectral characteristics shown in Figure 1)
`dominates the spectral radiance. Some night-light is present
`from starlight, natnrnl luminescence and man-made lighting
`This light is most intense in the 1 to 1.3 IJ.IIl. region. Table 2
`compares the detectors that operate in this thermal emission
`regime. Tile pyroelectric (ferroelectric) and HgCdTe sensors
`are commercial. The bolometric and tunnel Golay sensor
`anays are laborntoty demonstrntions.
`
`Table 2 Comparison of infrared sensors (50 jllll pixel size).
`
`Sonsor
`Pyroelectric
`Bolometric
`Tunnel Golay
`HgCdTe
`
`Sensitivity
`0.2K
`O.IK
`O.OJK (est.)
`O.OOSK
`
`Thermal Needs
`Temp. stabilized
`Temp. stabilized
`uncooled~
`77K operation
`
`Est Cost
`$2000
`($1000)
`($500+)
`$5000+
`
`4.11 Weatber Effects
`
`Therntal image quality depends on properties of banlware,
`atmosphere, and thermal contrnst of the viewed scene. Tile
`modulation transfer function (MlF) is the standard measure ri
`image quality. 20 Weather aflfcts both the M1F, and tbe
`tbermal contrast of the scene. Atmospheric effects are chiefly
`aerosol light scatter, that causes blurring as well as reduced
`contrast, and absorption that reduces contrast. Thermal
`contrnst of the scene is affected by wind, which tends to
`equalize tem~s, and by dew, which tends to equalize
`emissivity. Farmer" has analyzed emissivity effects on scere
`evaluation through smoke/obscnrants. NightSight™ system
`driven under conditions of fog and heavy rain exlubited no
`image collapse. A gradual degradation of tbe system
`pelfoiDlanCe occurs wben encountering new fog conditions,
`allowlng for the dtiver to slow down the vehicle without the
`need of a panic stop. Rain is surprisingly less harmful to tbe
`system and in a heavy downpour situation, only a 30"/o
`reduction in visual perception was apparent through blurring
`of the image. BMW Research Group at tbe Euro Conference,
`(Hahn") reveal more negative findings based on their driving
`experiences using a L WIR camera:
`
`4.12 Active Dlumination Systems
`
`Active illwnination systems me akin to rndar imaging.
`Lwnination is provided by a cooperntive lighting sowce such
`as could be provided by high intensity discharge lamps, light
`In an active
`emitting diodes (LEOs) or laser sowces.
`
`22 I SPIE Vol. 2592
`
`0008
`
`

`
`illumination system, lighting is chosen to best take advantage
`of the spectral response of the imaging camem in terms cf
`sensitivity, dynamic r.mge and satwation cbamcteristics. A
`number of active illumination systems have undergone
`feasibility checks. One approach22 uses pulsed lighting of the
`scene together with a remote-gated intensified CCD camera to
`reduce the damping effect of conlrast transmission through the
`reduce
`the efli:ct cf
`atmosphere. This
`technique can
`backscattering due to precipitation. The key is to open the
`shutter of the camera after proper delay with respect to the
`short light pulses. A second approach utilizes NIR high-beam
`headlarnp~ to illuminate the road scene operating in the
`spectral region from 0.7 to 1.2 j1lll. A regular CCD camera,
`mounted inside the vehicle at the level of the reaiView mirror,
`takes images of the road scene that are presented to the driver
`on a HUD with a hologrnphic combiner. The CCD spectrn1
`response is only sensitive in the NIR region by adding a
`blocking filter for the visible spectrum. This system enables
`the driver to see well over 300 m (984 ft) at night" NIR
`illuminators can cause inter-system blinding that is oveiCOme
`by mounting 45° polarizers on both the NIR headlights and
`the CCD camera. Continuous illuminators are also more
`susceplll>le to backscatter (glare) from particles in
`the
`atmosphere. Moreover; special cameras are needed to accoiDJ!
`for the photon penetrntion into the sensor array at NIR
`wavelengths.
`
`5 FUTURE TRENDS IN IMAGING
`
`5.1 Imaging Opties
`Diamond-turning is an emerging technology that allows fer
`the generation of computer genel3!ed shapes not possible by
`conventional optical processing techniques.
`
`5.2 New Detector Materials
`We can expect many advances in current-an high-ternpernture
`superconducting materials at !!J3llY labomtories for use as lR
`detectors. Both bolometric'"' and quantmn26 operation are
`being explored. Conductus (Sunnyvale, CA) is developing
`high-temperature supeiCOnducting YBa2Cu307
`(YBCO)
`bolometers. Westinghouse A'IL
`(Linthicum, MD)
`is
`addressing supeiCOnducting quantmn detectors that may be
`more sensitive than !henna! or bolometric detectors. Another
`area of active materials research
`involves strained-layer
`superlattices (SLSs). Long-wavelength lR detectors based on
`Type ll lnAsSb SLSs are being researched at Sandia National
`Labomtories, Los Alamos National Laboratocy, Lawrence
`Livermore National Labomtocy, Naval