Vision Chips
`or
`Seeing Silicon
`
`Third Revision
`Alireza Moini
`March 1997
`
`CHiPTec
`
`The Centre for High Performance Integrated Technologies and Systems
`
`The Centre for High Performance
` Integrated Technologies and Systems
`The University of Adelaide
`SA 5005, Australia
`Tel: 61 8 8303 3403
`Fax: 61 8 8303 4360
`Email: moini@eleceng.adelaide.edu.au
`
`Department of Electronics Engineering
`The University of Adelaide
`SA 5005, Australia
`
`http://www.eleceng.adelaide.edu.au/Groups/GAAS/Bugeye/visionchips/index.html
`
`Magna 2005
`TRW v. Magna
`IPR2015-00436
`
`0001
`
`

`
`Revision 3.5
`Copyright c(cid:13)1995-1997
`Alireza Moini
`All Rights Reserved
`
`0002
`
`

`
`Vision Chips or Seeing Silicon
`
`Alireza Moini
`
`The Centre for High Performance Integrated Technologies and Systems
`Department of Electrical & Electronics Engineering
`The Univ. of Adelaide, SA 5005, Australia
`Tel: +61 8 8303 3403, Fax: +61 8 8303 4360
`email: moini@eleceng.adelaide.edu.au
`WWW: http://www.eleceng.adelaide.edu.au/Personal/moini/
`
`March 1997
`
`0003
`
`

`
`Vision Chips or Seeing Silicon
`
`1
`
`Contents
`
`5
`1 Introduction
`5
`1.1
`Smart sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`6
`1.2 Advantages and disadvantages of vision chips . . . . . . . . . . . . . . . .
`7
`1.3 Challenges
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`8
`1.4 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`8
`1.4.1 CMOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`9
`1.4.2 BiCMOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`9
`1.4.3 CCD and CMOS/CCD . . . . . . . . . . . . . . . . . . . . . . . . .
`1.4.4 GaAs MESFET and HEMT . . . . . . . . . . . . . . . . . . . . . . 10
`1.5 Major groups working on vision chips
`. . . . . . . . . . . . . . . . . . . . 10
`1.6 How vision chips are presented in this report
`. . . . . . . . . . . . . . . . 12
`
`Acknowledgments
`
`14
`
`16
`2 Spatial Vision Chips
`2.1
`Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
`2.2 Mahowald and Mead’s silicon retina . . . . . . . . . . . . . . . . . . . . . . 18
`2.3 Mead’s adaptive retina . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
`2.4 Mahowald and Delbr¨uck’s stereo matching chips . . . . . . . . . . . . . . . 20
`2.5 Bernard et al.’s Boolean arti(cid:12)cial retina . . . . . . . . . . . . . . . . . . . . 22
`2.6 Andreou and Boahen’s silicon retina
`. . . . . . . . . . . . . . . . . . . . . 23
`2.7 Kobayashi et al.’s image Gaussian (cid:12)lter . . . . . . . . . . . . . . . . . . . . 24
`2.8 PASIC sensor from Link¨oping University . . . . . . . . . . . . . . . . . . . 26
`2.9 MAPP2200 sensor from IVP . . . . . . . . . . . . . . . . . . . . . . . . . . 28
`2.10 Forchheimer-(cid:23)Astr¨om’s NSIP sensor
`. . . . . . . . . . . . . . . . . . . . . . 28
`2.11 Sandini et al.’s foveated CCD chip . . . . . . . . . . . . . . . . . . . . . . 29
`2.12 IMEC-IBIDEM’s foveated CMOS chip . . . . . . . . . . . . . . . . . . . . 31
`2.13 Wodnicki et al.’s foveated CMOS sensor
`. . . . . . . . . . . . . . . . . . . 32
`2.14 Standley’s orientation detection chip . . . . . . . . . . . . . . . . . . . . . 33
`2.15 Harris et al.’s Resistive Fuse Vision Chip . . . . . . . . . . . . . . . . . . . 35
`2.16 DeWeerth’s Localization and Centroid Computation Chip . . . . . . . . . . 38
`2.17 Ward & Syrzycki’s Receptive Field Sensors . . . . . . . . . . . . . . . . . . 40
`2.18 Wu & Chiu’s 2D Silicon Retina . . . . . . . . . . . . . . . . . . . . . . . . 42
`2.19 Nilson et al.’s Shunting Inhibition Vision Chip . . . . . . . . . . . . . . . . 43
`2.20 Keast & Sodini’s CCD/CMOS Imager and Processor
`. . . . . . . . . . . . 44
`2.21 Mitsubishi Electric’s CMOS Arti(cid:12)cial Retina with VSP . . . . . . . . . . . 46
`2.22 Venier et al.’s Solar Illumination Monitoring Chip . . . . . . . . . . . . . . 47
`
`The Univ. of Adelaide
`
`0004
`
`

`
`Vision Chips or Seeing Silicon
`
`2
`
`49
`3 Spatio-Temporal Vision Chips
`3.1
`Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
`3.2 Lyon’s eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
`3.3 Tanner and Mead’s correlating motion detection chip . . . . . . . . . . . . 51
`3.4 Tanner and Mead’s optic flow motion detection chip . . . . . . . . . . . . . 53
`3.5 Moore and Koch’s multiplicative motion detector
`. . . . . . . . . . . . . . 54
`3.6 Bair and Koch’s motion detection chip . . . . . . . . . . . . . . . . . . . . 55
`3.7 Delbr¨uck’s focusing chip . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
`3.8 Delbr¨uck’s velocity tuned motion sensor
`. . . . . . . . . . . . . . . . . . . 57
`3.9 Meitzler et al.’s sampled-data motion chip . . . . . . . . . . . . . . . . . . 59
`3.10 Moini et al.’s insect vision-based motion detection chip . . . . . . . . . . . 60
`3.11 Moini et al.’s second insect vision-based motion detection chip . . . . . . . 62
`3.12 Dron’s multi-scale veto CCD motion sensor . . . . . . . . . . . . . . . . . . 65
`3.13 Horiuchi et al.’s delay line-based motion detection chip . . . . . . . . . . . 66
`3.14 Chong et al.’s change detector . . . . . . . . . . . . . . . . . . . . . . . . . 68
`3.15 Gottardi and Yang’s CCD/CMOS motion sensor . . . . . . . . . . . . . . . 69
`3.16 Kramer et al.’s velocity sensor . . . . . . . . . . . . . . . . . . . . . . . . . 70
`3.17 Indiveri et al.’s time-to-crash sensor . . . . . . . . . . . . . . . . . . . . . . 72
`3.18 Indiveri et al.’s direction-of-heading detector . . . . . . . . . . . . . . . . . 74
`3.19 McQuirk’s CCD focus of expansion estimation chip . . . . . . . . . . . . . 75
`3.20 Gruss et al.’s range (cid:12)nder
`. . . . . . . . . . . . . . . . . . . . . . . . . . . 76
`3.21 Sarpeshkar et al.’s pulse mode motion detector . . . . . . . . . . . . . . . . 77
`3.22 Meitzler et al.’s 2D position and motion detection chip . . . . . . . . . . . 79
`3.23 Aizawa et al.’s Image Sensor with Compression . . . . . . . . . . . . . . . 80
`3.24 Hamamoto et al.’s Image Sensor With Motion Adaptive Storage Time . . . 82
`3.25 Simoni et al.’s Optical Sensor and Analog Memory Chip with Change Detection 83
`3.26 Espejo et al.’s Smart Pixel CNN . . . . . . . . . . . . . . . . . . . . . . . . 84
`3.27 Moini et al.’s Shunting Inhibition Vision Chip . . . . . . . . . . . . . . . . 85
`3.28 Etienne-Cummings et al.’s Motion Detector Chip . . . . . . . . . . . . . . 86
`3.29 CSEM’s Motion Detector Chip for Pointing Devices . . . . . . . . . . . . . 88
`
`90
`4 Chips for Vision
`4.1
`Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
`4.2 Hakkaranien & Lee’s AVD CCD Chip for Stereo Vision . . . . . . . . . . . 91
`4.3 Erten’s CMOS Chip for Stereo Correspondence
`. . . . . . . . . . . . . . . 93
`
`95
`5 Optical Neuro Chips
`5.1 Mitsubishi Electric’s Optical neurochip and retina . . . . . . . . . . . . . . 95
`5.2 Yu et al.’s optical neurochip . . . . . . . . . . . . . . . . . . . . . . . . . . 97
`
`99
`6 Active Pixel Sensors
`6.1
`Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
`6.2 JPL’s active pixel sensors
`. . . . . . . . . . . . . . . . . . . . . . . . . . . 99
`6.3 Technion’s Adaptive Sensitivity CCD Imager . . . . . . . . . . . . . . . . . 100
`6.4 Technion’s TDI CCD sensor . . . . . . . . . . . . . . . . . . . . . . . . . . 101
`6.5 Fowler et al.’s pixel level ADC sensor . . . . . . . . . . . . . . . . . . . . . 102
`
`The Univ. of Adelaide
`
`CONTENTS
`
`0005
`
`

`
`Vision Chips or Seeing Silicon
`
`3
`
`104
`7 Principles & Building Blocks
`7.1
`Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
`7.2 Phototransduction, the Doorway to Vision Chips
`. . . . . . . . . . . . . . 106
`7.2.1 Photodetector Elements
`. . . . . . . . . . . . . . . . . . . . . . . . 106
`7.2.2 Quantum E(cid:14)ciency of a Vertical Junction Diode
`. . . . . . . . . . 108
`7.2.3 Quantum E(cid:14)ciency of a Lateral Junction Diode . . . . . . . . . . . 112
`7.2.4 Quantum E(cid:14)ciency of a Vertical Bipolar transistor . . . . . . . . . 114
`7.2.5 Quantum E(cid:14)ciency of a Lateral Bipolar Photodetector . . . . . . . 116
`7.2.6 Mixed structures
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
`7.2.7 Quantum E(cid:14)ciency of a Photogate . . . . . . . . . . . . . . . . . . 120
`7.2.8 The E(cid:11)ect of Scaling on Photodetecting Elements . . . . . . . . . . 122
`7.2.9 Mismatch in Photodetecting Elements
`. . . . . . . . . . . . . . . . 122
`7.3 Photocircuits
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
`7.3.1 Logarithmic Sensor Using MOS Diodes . . . . . . . . . . . . . . . . 124
`7.3.2 Photocircuit with Bu(cid:11)er-like Pull-up . . . . . . . . . . . . . . . . . 124
`7.3.3 Photocircuit with Ampli(cid:12)er-like Pull-up . . . . . . . . . . . . . . . 125
`7.3.4 Bu(cid:11)ered Logarithmic Photocircuit . . . . . . . . . . . . . . . . . . . 126
`7.3.5 Delbr¨uck’s Adaptive Photocircuit. . . . . . . . . . . . . . . . . . . . 128
`7.3.6 Cascoded Photocircuits . . . . . . . . . . . . . . . . . . . . . . . . . 128
`7.3.7 Current Ampli(cid:12)er Photocircuit
`. . . . . . . . . . . . . . . . . . . . 128
`7.3.8
`Integration Based Photocircuits . . . . . . . . . . . . . . . . . . . . 131
`7.4 Circuits and techniques for active pixel sensors . . . . . . . . . . . . . . . . 133
`7.4.1 Photocircuits in active pixel sensors . . . . . . . . . . . . . . . . . . 133
`7.4.2 Correlated double sampling . . . . . . . . . . . . . . . . . . . . . . 134
`7.5 Spatial Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
`7.5.1 Linear Resistive networks
`. . . . . . . . . . . . . . . . . . . . . . . 136
`7.5.2
`Smoothing networks
`. . . . . . . . . . . . . . . . . . . . . . . . . . 137
`7.5.3 Nonlinear Resistive networks . . . . . . . . . . . . . . . . . . . . . . 142
`7.5.4 Resistive Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
`7.5.5 CCD Circuits for Spatial Processing . . . . . . . . . . . . . . . . . . 145
`7.6 Spatio-Temporal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 146
`7.6.1 Analog Memory Elements
`. . . . . . . . . . . . . . . . . . . . . . . 146
`7.6.2 Continuous Delay Elements
`. . . . . . . . . . . . . . . . . . . . . . 148
`7.7 Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
`7.7.1 Light Adaptive Photodetectors
`. . . . . . . . . . . . . . . . . . . . 149
`7.7.2 Light Adaptive Photocircuits
`. . . . . . . . . . . . . . . . . . . . . 149
`7.7.3 Light Adaptive Architectures
`. . . . . . . . . . . . . . . . . . . . . 151
`7.7.4
`Spatial Adaptation Models . . . . . . . . . . . . . . . . . . . . . . . 152
`7.8 Practical issues in designing vision chips
`. . . . . . . . . . . . . . . . . . . 155
`7.8.1 Mismatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
`7.8.2 Digital noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
`7.9 Testing vision chips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
`7.9.1 Design for Testing
`. . . . . . . . . . . . . . . . . . . . . . . . . . . 157
`7.9.2 Tests and Measurements . . . . . . . . . . . . . . . . . . . . . . . . 159
`7.9.3 Test conditions
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
`7.9.4
`Steady-state tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
`7.9.5
`Spatio-temporal tests . . . . . . . . . . . . . . . . . . . . . . . . . . 162
`
`A Other resources
`
`The Univ. of Adelaide
`
`166
`
`CONTENTS
`
`0006
`
`

`
`Vision Chips or Seeing Silicon
`
`B About this report
`
`References
`
`4
`
`167
`
`168
`
`The Univ. of Adelaide
`
`0007
`
`

`
`Vision Chips or Seeing Silicon
`
`5
`
`Chapter 1
`
`Introduction
`
`Smart vision systems will be an inevitable component of future intelligent systems. Con-
`ventional vision systems, based on the system level integration (or even chip level in-
`tegration) of an imager (usually a CCD) camera and a digital processor, do not have
`the potential for application in general purpose consumer electronic products. This is
`simply due to the cost, size, and complexity of these systems. Because of these factors
`conventional vision systems have mainly been limited to speci(cid:12)c industrial and military
`applications. Vision chips, which include both the photosensors and parallel processing el-
`ements (analog or digital), have been under research for more than a decade and illustrate
`promising capabilities.
`
`1.1
`
`Smart sensors
`
`The integration of photodetecting elements and processing circuits on the same chip,
`for obtaining better performance from sensors, or for making the sensing and processing
`system more compact, is not a new idea, but the concept of smart sensing, i.e. sensor
`information processing without redundant and unnecessary data acquisition, and with at-
`sensor-level processing is relatively new. The word \smart-sensors" sometimes has been
`applied to those sensors which have only tried to integrate the sensors and processing
`modules, without any regard to the low level interaction that can exist between the
`sensors and processors. With this meaning in mind, even large systems with a vidicon
`and a 100kg main frame computer could be called a smart-sensor. The only di(cid:11)erence
`would be the size. Here I would like to use a more fundamental meaning for smart-sensors.
`\Smart-sensors are those devices in which the sensors and circuits co-exist, and
`their relationship with each other and with higher-level processing layers goes
`beyond the meaning of transduction. Smart-sensors are information sensors,
`not transducers and signal processing elements. Smart sensors are not general
`purpose devices. Everything in a smart sensor is speci(cid:12)cly designed for the
`application targeted for." With this meaning in mind we exclude any camera-
`processor combination, even if they are integrated on the same chip. However, sensors
`such as NSIP architecture described in [(cid:23)Astr¨om 93, Forchheimer and (cid:23)Astr¨om 94] and
`column parallel architecture in [Hamamoto et al. 96a, Hamamoto et al. 96c], although do
`not integrate the sensors and processors at the pixel level, still possess a tight relationship
`between the sensors and processors. In fact these architectures suggest that some of the
`drawbacks of vision chips, such as loss of resolution and (cid:12)ll-factor, may be relieved, while
`maintaining a semi-parallel processing (in one dimension).
`
`The Univ. of Adelaide
`
`0008
`
`

`
`Vision Chips or Seeing Silicon
`
`6
`
`Traditional photodetectors could only output an analog signal, which required further
`signal conditioning. Still in most imagers the main focus is on the quality of the imaging
`in terms of noise, resolution, speed, and so on. It is assumed that further signal and image
`processing stages can acquire the imager output and process it. In contrast, in vision chips
`the main focus is on the quality of processing. The implementation of a certain algorithm
`using existing components is given the priority and often some image characteristics, such
`as resolution, are sacri(cid:12)ced.
`
`1.2 Advantages and disadvantages of vision chips
`
`When compared to a vision processing system consisting of a camera and a digital pro-
`cessor, a vision chip provides many system level advantages. These include
`
`(cid:15) Speed: The processing speed achievable using vision chips exceeds that of the
`camera-processor combination. A main reason is the information transfer bottleneck
`between the imager and the processor. In vision chips information between various
`levels of processing is processed and transferred in parallel.
`
`(cid:15) Large dynamic range: Many vision chips use photodetectors and photocircuits
`which have a large dynamic range over at least 7 decades of light intensity. Many also
`have local and global adaptation capabilities which further enhances their dynamic
`range. Conventional cameras are at best able to perform global automatic gain
`control.
`
`(cid:15) Size: Using single chip implementation of vision processing algorithms, very com-
`pact systems can be realized. The only parts of the system that may not be scalable
`are the mechanical parts (like the optical interface).
`
`(cid:15) Power dissipation: Vision chips often use analog circuits which operate in sub-
`threshold region. There is also no energy spent for transferring information from
`one level of processing to another level.
`
`(cid:15) System integration: Vision chips may comprise most modules, such as image
`acquisition, and low level and high level analog/digital image processing, necessary
`for designing a vision system. From a system design perspective this is a great
`advantage over camera-processor option.
`
`Although designing single-chip vision systems is an attractive idea, it faces several
`limitations:
`
`(cid:15) Reliability of processing: Vision chips are designed based on the concept that
`analog VLSI systems with low precision are su(cid:14)cient for implementing many low
`level vision algorithms. The precision in analog VLSI systems is a(cid:11)ected by many
`factors, which are not usually controllable. As a result, if the algorithm does not
`account for these inaccuracies, the processing reliability may be severely a(cid:11)ected.
`Vision chips also use unconventional analog circuits which may not be well charac-
`terized and understood.
`
`The Univ. of Adelaide
`
`Introduction
`
`0009
`
`

`
`Vision Chips or Seeing Silicon
`
`7
`
`(cid:15) Resolution: In vision chips each pixel includes a photocircuit1 which occupies a
`large proportion of the pixel area. Therefore, vision chips have a low (cid:12)ll-factor and
`a low resolution. The largest vision chip reported has only 210(cid:2)230 pixels, for a
`photocircuit consisting of six transistors only [Andreou and Boahen 94a].
`
`(cid:15) Di(cid:14)culty of the design: Vision chips implement a speci(cid:12)c algorithm in a limited
`silicon area. Therefore, often o(cid:11)-the-shelf circuits cannot be used in the implemen-
`tation. This involves designing many new analog circuits. Vision chips are always
`full custom designed, and full custom design is known to be time consuming and
`error-prone.
`
`(cid:15) Programming: None of the vision chips are general purpose.
`In other words,
`many vision chips are not programmable to perform di(cid:11)erent vision tasks. This
`inflexibility is particularly undesirable during the development of a vision system.
`
`1.3 Challenges
`
`Vision chip design is a challenging task. One should consider issues from visual processing
`algorithms to low level circuit design problems, from phototransduction principles to high-
`level VLSI architectural issues, from mismatch and digital noise to "readout" techniques,
`from optics to electronics and optoelectronics, from pure analog to mixed analog/digital to
`pure digital design problems, from biologically inspired vision models to intuitive models
`to computational models, . . . .
`A vision chip requires photodetecting elements, image acquisition circuits, analog con-
`ditioning and processing circuits, digital processing and interfacing, and image readout
`circuitry all on the same chip. Many of these components, such as low level analog pro-
`cessing elements, should exist in number the same as photodetectors. In most cases these
`components should interact with at least their nearest neighbors. The area required for
`implementing the circuits and routing the information across the chip has put upper
`bounds on the realization of reliably functional and high resolution vision chips, and in
`most implementations resolution or functionality has been sacri(cid:12)ced for the other. The
`design of vision chips can obviously bene(cid:12)t from the high level integration in current
`VLSI processes, where more than 10 million transistors can be integrated on a single
`chip. Unfortunately, advanced processes for high level integration are usually tuned and
`characterized for leading edge digital processors and DRAMs, su(cid:11)ering from sub-micron
`e(cid:11)ects, such as short channel e(cid:11)ects, hot-carrier e(cid:11)ects, band-to-band tunneling, gate-
`oxide direct tunneling, gate induced drain leakage (GIDL), drain induced barrier lowering
`(DIBL), and threshold voltage control [Fienga et al. 94]. Many available processes, on
`the other hand, do not have any speci(cid:12)c photodetecting element, and are not well tuned
`for analog circuit design. Device mismatch has also severely a(cid:11)ected the analog circuit
`design community, and almost no fabrication processes have been carefully characterized
`and modeled to account for mismatch.
`Design of vision chips has also been a(cid:11)ected by the lack of VLSI friendly computer
`vision algorithms. Most current computational computer vision algorithms are very com-
`plex and are even hardly implementable using powerful workstations to run in real time.
`
`1I have adopted the term \photocircuit" because of its clear and sharp reference to a circuitry which
`processes the photocurrent or photovoltage. Other terms, such as \photoreceptor", have been inter-
`changeably used both for single photodetectors and the circuitry used for processing photocurrents, and
`in a context full of these references become confusing.
`
`The Univ. of Adelaide
`
`Introduction
`
`0010
`
`

`
`Vision Chips or Seeing Silicon
`
`8
`
`Many computer vision algorithms are still not reliable enough for application in general
`uncontrolled environments. Biologically inspired algorithms, on the other hand, rely on
`the fact that many creatures have developed very e(cid:14)cient visual system. These algo-
`rithms, however, are not mature enough and su(cid:11)er from excessive simpli(cid:12)cation caused
`by insu(cid:14)cient understanding of animals visual system.
`Despite these facts, the design of single chip VLSI vision sensors, or smart vision
`sensors is increasingly progressing and many vision chips based on biological or compu-
`tational algorithms have been developed in the past few years. The complexity of vision
`chips has also signi(cid:12)cantly increased, and 2D vision chips with more than 48,000 detectors
`and processing elements have been designed [Andreou and Boahen 94b].
`
`1.4 Technology
`
`Di(cid:11)erent technologies o(cid:11)er advantages and disadvantages for the design of vision chips.
`The dominant technologies available to date are CMOS, BiCMOS, CCD, and GaAs (MES-
`FET and HEMT). CMOS has been exhaustively used in many designs. The additional
`bipolar transistor in BiCMOS processes, though advantageous in achieving better match-
`ing properties and higher speeds, is not easily justi(cid:12)able when comparing other factors
`in the design. While commercial grade CMOS processes are accessible through fabrica-
`tion brokers, such as MOSIS and CMP, the CCD processes available for prototyping are
`of a low quality. GaAs processes have been used only to a very limited extent because
`there are no readily available photodetector structures in such processes, and more impor-
`tantly, analog circuit design is severely limited by gate leakage in MESFET and HEMT
`transistors. In the following sections advantages and disadvantages of each process are
`highlighted.
`
`1.4.1 CMOS
`
`CMOS has been and will remain the dominant technology is almos all VLSI design areas,
`including vision chips. This is a direct result of the following advantages o(cid:11)ered by CMOS
`processes.
`(cid:15) Mature technology: CMOS processes are well established and continue to become
`more mature. The powerful trust by leading edge digital memory and processors
`has led to continuous improvement and down scaling of CMOS processes.
`
`(cid:15) Design resources: Circuit and system design in CMOS is supported by a vast
`number resources. Many design techniques and design libraries for analog and digital
`design are available.
`
`(cid:15) Availability: CMOS processes are now readily available for prototype designs
`through fabrication brokers, at low prices. This has boosted the design knowledge
`by real implementations, rather than pure theoretical treatments.
`
`(cid:15) Price: CMOS is the cheapest process available, when compared against other
`technologies with the same minimum feature size.
`
`The major disadvantages of CMOS technology for implementing vision chips are:
`
`(cid:15) Analog circuit design: Leading edge processes are not characterized and tuned
`for analog circuit design.
`
`The Univ. of Adelaide
`
`Introduction
`
`0011
`
`

`
`Vision Chips or Seeing Silicon
`
`9
`
`(cid:15) Photodetectors: The photodetector structures are not characterized in any of
`the processes. It is the designer’s responsibility to assure that the photodetectors
`function as desired.
`
`(cid:15) Second order e(cid:11)ects: In the scaling process some second order device charac-
`teristics, such as subthreshold operation, are usually ignored or paid less attention,
`and their cancellation is more desired than their improvement.
`
`(cid:15) Mismatch: Mismatch in CMOS devices is relatively high. This is specially hinder-
`ing the reliability of analog processing in vision chips.
`
`1.4.2 BiCMOS
`
`BiCMOS processes provide an additional bipolar device, which has been the workhorse
`of analog design. The bipolar transistor can be used to increase the speed, reduce the
`mismatch, and obtain better circuit characteristics when exponential I-V relatinship is
`required. However, the use of BiCMOS processes has been limited due to its complexity
`and cost. Also the large area required for each bipolar transistor makes them unattractive
`for large vision chips.
`
`1.4.3 CCD and CMOS/CCD
`
`CCD processes have originally been developed for analog signal processing and imaging
`devices. Although this may have facilitated the design of vision chips, due to their draw-
`backs there has been limited success in achieving functional and reliable vision chips.
`Major drawbacks of CCD and CCD/CMOS with respect to CMOS are:
`
`(cid:15) Clocking: To perform even simple operations large number of clock phases are
`required, these clock phases should be distributed to all cells
`
`(cid:15) Process optimization: Special CCD processes do not have optimized CMOS
`devices and CCD/CMOS processes do not have optimized CCD structures
`
`(cid:15) Special read and write circuit: For transferring signals between CMOS and
`CCD parts in a CCD/CMOS circuit special read/write circuits are necessary
`
`(cid:15) Large area: Occupying large area per cell due to the above items
`
`(cid:15) Digital noise: Massive clock-induced-noise to analog circuits in mixed CCD/CMOS
`approach
`
`(cid:15) Power: Power consumption due to large voltage transients required for clocking
`the gates of CCD structures (large capacitive loads)
`
`Despite these numerous drawbacks, CCDs o(cid:11)er easier solutions for some operations.
`For example, in a smoothing CCD vision chip the smoothing width can be increased by
`only leaving the circuit to operate over more clock cycles.
`In other words, CCDs are
`capable of iterating a function without demands on additional space.
`
`The Univ. of Adelaide
`
`Introduction
`
`0012
`
`

`
`Vision Chips or Seeing Silicon
`
`10
`
`1.4.4 GaAs MESFET and HEMT
`
`GaAs processes are recognized by their high speed operation for digital and analog circuits.
`They have also been used in opto-electronic devices. GaAs processes su(cid:11)er from several
`problems
`
`(cid:15) Maturity of technology: The processes are not mature. It was only recently that
`GaAs processes could achieve high integration (in the order one million transistors).
`
`(cid:15) Analog design: Analog circuit design has been a(cid:11)ected by the schuttky diode at the
`gate. This diode is in a forward bias direction. For an enhancement mode MESFET
`this leaves a gate-source voltage range of 0.1 V to 0.6 V.
`
`(cid:15) Price and availability: GaAs processes are generally expensive and not easily acces-
`sible.
`
`(cid:15) Opto-electronic devices are only available in very specialized processes.
`
`1.5 Major groups working on vision chips
`
`(cid:15) The works of Carver Mead’s group in California Institute of Technology, starting
`with Lyon’s optical mouse designed in 1980(See section 3.2), are major contributions
`to this exotic and fascinating area of VLSI design. The idea of neuromorphic engi-
`neering using VLSI technologies was (cid:12)rst introduced by Carver Mead and bloomed
`into several analog VLSI chips appearing in \the Bible of analog VLSI", Analog
`VLSI and Neural Systems published by Addison-Wesley in 1989 [Mead 89b]. This
`work is still continuing in the Carver-Lab in Caltech. The research emphasis in this
`group is on analog VLSI systems. In the past they have designed many chips using
`analog VLSI based on biological models of vision, cochlea, and other neural systems.
`
`(cid:15) Koch-Lab again in Caltech, led by Christoph Koch, has focused on modeling biolog-
`ical neural systems and also implementing them in analog VLSI.
`
`Research in the laboratory of Professor Christof Koch focuses on several areas:
`
`{ Biophysics of Computation in Single Neurons
`
`{ Cortical Circuits Underlying Motion and Visual Attention
`
`{ Psychophysics of Attention and Awareness
`
`{ The Neuronal Correlate of Visual Awareness and Consciousness
`
`{ Neuromorphic Analog VLSI Vision Systems
`
`(cid:15) Analog VLSI group in Johns Hopkins University led by Professor Andreas Andreou
`has had similar interests in analog VLSI systems as the Carver-Lab. Analog VLSI
`chips mainly based on biological models have been designed in this lab. Some
`system designs in this lab include analog VLSI models of auditory processing, early
`vision and silicon retinas, associative memory, adaptive neural networks, and speech
`recognition.
`
`(cid:15) The VLSI group at Laval University is led by Marc Tremblay. The research is
`principally inspired by computational needs in computer vision. The VLSI projects
`are focused on the design and development of smart sensors. Some of the research
`
`The Univ. of Adelaide
`
`Introduction
`
`0013
`
`

`
`Vision Chips or Seeing Silicon
`
`11
`
`topics include the MAR-Camera systems, motion detection, and linear arrays for
`3-D cameras.
`
`(cid:15) The Image Processing Group at Linkoping University in Sweden has been developing
`vision cameras with processing capabilities. The group led by Andres Astrom has
`developed the commercially available LAPP and MAPP series cameras.
`
`(cid:15) Adelaide Uni. in Australia has been pursuing the insect vision based motion detector
`project since 1991. Inspired by the simplicity of the insect visual system, and using
`a VLSI friendly model for insect vision, the (cid:12)rst insect vision chip was designed
`in 1992. The project led by Abdesselam Bouzerdoum and Kamran Eshraghian is
`having a rapid growth in number of people and aspects of the design. The work is
`being funded by strong industrial partners and the federal government of Australia.
`
`(cid:15) IMEC and IBIDEM consortium, involving several universities in Spain and Italy,
`have focused on the design of space variant sensors, more speci(cid:12)cally the "foveated
`sensors". The log-polar mapping performed by these sensors is very attractive for
`applications requiring rotation and scale invariant processing.
`
`(cid:15) A research group in MIT has concentrated on the implementation of early visual
`processing using CCD and CMOS technologies. In this set of projects they have
`targeted vision tasks and algorithms requiring low precision. The reason in selecting
`CCD as the base technology for the implementation has been stated to be the
`achievable compact size.
`
`Some of the chips designed as a part of this project include
`
`{ CMOS image moment and orientation chip [Standley 91b]
`
`{ CCD/CMOS image smoothing and segmentation chip [Keast and Sodini 92]
`
`{ CCD image feature extraction chip [Chiang et al. 90, Chiang and Chuang 91]
`
`{ CCD/CMOS focus of expansion chip
`
`{ CCD multiscale veto motion sensor [Dron 94]
`
`{ CCD/CMOS stereo chip
`
`(cid:15) The Perception System Lab. in ETCA, France is lead by Thierry Bernard. Activi-
`ies in the lab are concentrated on various aspects of designing intelligent systems,
`including vision chips, and in particular programmable arti(cid:12)cial retinas.
`
`(cid:15) The VLSI Systems group in Southern Illinois University at Carbondale is working
`on VLSI design of vision chips for real-time dynamic tasks encountered in manufac-
`turing and assembly, auto-navigation, and un-manned vehicles and robotics.
`
`(cid:15) The Hatori-Aizawa Lab.
`in Tokyo University, has focused on compression sensors
`and adaptive imagers for on-chip compression and adaptation.
`
`(cid:15) The VLSI group in Technion, Israel, headed by Ran Ginosar has been developing
`adaptive sensitivity smart imagers, and techniques for improving the scanning of
`imagers.
`
`(cid:15) A group in Mitsubishi Electric has been developing optical neurochips using exotic
`III-V compound structures. The main focus of the research has been on optical
`interconnection and neural network architectures.
`
`The Univ. of Adelaide
`
`Introduction
`
`0