US007202776B2
`
`(12) United States Patent
`US 7,202,776 B2
`(10) Patent N0.:
`
` Breed (45) Date of Patent: Apr. 10, 2007
`
`
`(54) METHOD AND SYSTEM FOR DETECTING
`OBJECTS EXTERNAL TO A VEHICLE
`
`(51)
`
`Int. Cl,
`B60Q 1/00
`
`(2006.01)
`
`(75)
`
`Inventor: David s_ Breed, Boonton Township,
`Morris County, NJ (US)
`
`(73) Assignee:
`
`Intelligent Technologies International,
`Inc., Denville, NJ (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 214 days.
`
`(21) Appl. No.: 11/034,325
`
`(22)
`
`Filed:
`
`Jan. 12, 2005
`
`(65)
`
`Prior publication Data
`
`US 2005/0134440 A1
`Jun. 23, 2005
`Related US. Application Data
`
`~
`(63) Continuation-in-part of application No. 10/822,445,
`i116}:SgliAtllelltlalltfoslf-(iflftpangvéfzgiyliocaflagj5111603713711111081
`'
`’
`858, filed on Apr. 9, 2002, now Pat. No. 6,720,920,
`which is a continuation-in—part of application No.
`09/177,041, filed on Oct. 22, 1998, now Pat. No.
`6,370,475, said application No. 10/822,445 applica-
`tion No. 10/216,633, filed on Aug. 9, 2002, now Pat.
`No. 6,768,944, is a continuation-in-part of application
`No. 10/ 118,858, filed on Apr. 9, 2002, now Pat. No.
`6,720,920, said application No. 10/118,858 is a con-
`tinuation-in-part of application No. 09/679,317, filed
`on Oct. 43 2000, now Pat. No. 6,405,132, which is a
`continuation-in-part of application No. 09/523, 5 595
`filed on Mar. 10 2000 now abandoned and a con-
`tinuation-in-partsof application No 09/1377 041 filed
`on Oct 22
`1998 now Pat No.
`6 370’475’ said
`a
`lication £10 10/118 858 is. a continuation-ih- art
`0113121
`lication No 09/969 466 filed on Jul 19 26301
`p}; t N 6 526 352 ’
`’
`’
`’
`now a.
`0‘
`’
`’
`’
`(60) Provisional application No. 60/123,882, filed on Mar.
`11, 1999, provisional application No. 60/062,729,
`filed on Oct. 22, 1997.
`
`’
`
`(52) US. Cl.
`
`........................ 340/435; 340/557; 342/70;
`342/118; 701/223
`(58) Field of Classification Search ................ 340/435,
`340/557, 901, 905, 907, 995.1; 342/70, 118,
`342/357.06; 348/113, 118; 701/200, 207,
`701/213, 223, 51, 58, 59, 65
`See application file for complete search history.
`_
`References Clted
`U.S. PATENT DOCUMENTS
`
`(56)
`
`3,902,803 A
`
`9/1975 Lego, Jr.
`
`(Continued)
`
`EP
`EP
`GB
`
`FOREIGN PATENT DOCUMENTS
`0353200
`1/1990
`0750202
`12/1996
`2141890
`“1984
`OTHER PUBLICATIONS
`
`er
`3V1
`e a .,
`ge g3. e
`ac 1ve Illg
`VlSlOIl SyS em or
`Of D 'd t
`I
`t d
`f
`_ ht
`_
`_
`t
`f
`“Ran
`automobilies”, Applied Optics, v01. 45, N0. 28, Oct. 1, 2006, pp.
`7248-7254.
`.
`.
`Primary ExammeriThomas Mullen
`(74) A’wmeyl Age“ 0r F’rmiBnan ROfle
`
`ABSTRACT
`(57)
`Method and system for obtaining information about objects
`in the environment outside of and around a vehicle and
`preventing collisions involving the vehicle includes direct-
`ing a laser beam from the vehicle into the environment,
`receiving from an object in the path of the laser beam a
`reflection of the laser beam at a location on the vehicle, and
`analyzing the received laser beam reflections to obtain
`information about the object from which the laser beam is
`being reflected. Analysis of the laser beam reflections pref-
`erably entails range gating the received laser beam reflec-
`tions to limit analysis of the received laser beam reflections
`to only those received from an object within a defined
`(distance) range such that objects at distances within the
`range are 150131th from surrounding objects.
`
`26 Claims, 25 Drawing Sheets
`
`102
`
`“’4
`
`106
`
`-
`.
`enwmnment amund
`”"9“ "s" ”3”“ "“°
`
`velriele
`\
`[—11
`lusei beam indiesiive or
`Receive relieeiions of
`presence ofohjeoi in path
`
`of laser beam
`
`_l__
`Rang: gal: reflections in
`nannw dislance lungs
`from which reflections
`are processed
`
`Identify/ascertain the
`identity ofcbjecls in
`range used on reflections
`
`
`los
`
`_J_Assess poiemial for
`/ collision/consequences of
`potentill collision with
`llz
`object
`
`Effecl countermeasure if
`—1_._—_
`collision is likely
`
`”4
`
` \
`
`
`
`HS
`
`\ ”’3 ‘
`
`
`Provide diginl map
`_.!__
`Dererrninelourianof
`’ vehicle on map
`laser beam based on
`Define scanning field of
`\\
`/ vehicle’s pesiiien and
`l22
`mu
`
` v
`
`
`|PR2013-00424 — Ex. 1021
`
`Toyota Motor Corp., Petitioner
`1
`
`IPR2013-00424 – Ex. 1021
`Toyota Motor Corp., Petitioner
`1
`
`

`

`US 7,202,776 B2
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`3,947,119 A
`4991412 A
`4,708,473:
`’
`’
`4,915,498 A
`4,970,628 A
`5,255,163 A
`5,314,037 A *
`5,383,200 A
`5,414,439 A
`5,434,754 A
`5,700,078 A
`5,747,792 A
`5,756,989 A
`5,771,326 A
`5,791,757 A
`
`3/1976 Bambérg et al~
`5/1978 5310111111“
`1
`1313:; if d ff
`etz or eta'
`4/1990 Ma1ek
`.
`11/1990 Bergkvrst
`10/1993 Neumann
`5/1994 Shaw eta1.
`1/1995 Barrett et a1.
`5/1995 Groves et 31.
`7/1995 Li
`12/1997 Fohl
`5/1998 Kintz eta1.
`5/1998 Bear eta1.
`6/1998 Fohl
`8/1998 O’Neil
`
`................. 340/903
`
`5,857,770 A
`_
`1/1999 F0111
`5,890,796 A
`4/1999 Marlnelh
`5,971,578 A
`10/1999 Fohl
`6,008,496 A * 12/1999 Winefordner et al.
`6,014,601 A
`1/2000 Gustafson ................... 340/436
`2,372,772 $1
`$588? 59111 1
`1
`6’370’475 B1
`“002 Bemison eta'
`’
`’
`ree
`6,405,132 B1
`6/2002 Breed
`6,422,713 B1
`7/2002 Fohl
`6,429,429 B1
`8/2002 Fohl
`6,483,094 B1
`11/2002 Yahav
`
`6,526,352 B1
`6,538,820 B2
`6,552,342 B2
`6,576,884 B1
`6,690,017 B2
`6,720,920 B2
`6,725,139 B2
`
`2/2003 Breed et 31.
`3/2003 Fohl
`4/2003 Holz et al.
`6/2003 Ostrornek eta1.
`2/2004 Remillard
`4/2004 Breed
`.
`4/2004 Mrller
`.
`5/2004 Remrllard
`6,730,913 B2
`7/2004 Breed
`6768944 BZ
`”004 Stephan
`6774367 BZ
`10/2004 Abel et 211.
`6,803,574 B2
`10/2004 Fohl
`6,809,870 B2
`12/2004 Stephan et al.
`6,828,544 B2
`6,975,246 B1* 12/2005 Trudeau ..................... 340/435
`2002/0185590 A1
`12/2002 Yahav
`2002/0191388 A1
`12/2002 MatveeV
`2003/0034462 A1
`2/2003 Remillard
`
`2003/0036881 A1
`2003/0155513 A1
`2003/0155514 A1
`2003/0193980 A1
`2003/0230705 A1
`2003/0230715 A1
`2004/0031922 A1
`2005/0107954 A1*
`2005/0269481 A1
`
`2/2003 Remillard
`8/2003 Remillard
`8/2003 Remillard
`10/2003 MatveeV
`”/2003 Stephan
`12/2003 Remillard
`2/2004 Stephan
`5/2005 Nahla ......................... 701/213
`12/2005 David et al.
`
`* cited by examiner
`
`2
`
`

`

`
`
`Prior Art
`
`Fig.
`
`1
`
`3
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 2 0f 25
`
`US 7,202,776 B2
`
`
`
`BASE STATION
`
`Prior Art
`
`Fig. 2
`
`4
`
`

`

`U.S. Patent
`
`Apr. 10,2007
`
`Sheet 3 0f 25
`
`US 7,202,776 B2
`
`A; 2
`IN“?
`I ”2
`5V4
`I
`I
`—I_
`57’! _T_
`#1 2 § 7‘
`
`
`
`463°
`// |‘|‘\
`\‘N
`
`I
`
`M16 32
`
`BASE STATION
`
`
`
`Prior Art
`
`Fig. 3
`
`5
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 4 0f 25
`
`US 7,202,776 B2
`
`__————_—pu—u¢-————I———n—
`
`
`GPS 8: DGPS
`PROCESSING
`
`SYSTEM
`
` GPS SATELLITES
`
`
`
`6
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 5 0f 25
`
`US 7,202,776 B2
`
`\r
`
`‘7
`
`1 V
`
`54
`
`56
`
`U1N
`
`GPS
`Receiver
`
`DGPS
`Receiver
`
`Inter-Vehicle
`Communication
`
`Infrastructure
`Communication
`
`3
`
`60v Cameras
`
`‘32
`
`Radar
`
`6‘
`
`Laser Radar
`
`O)0)
`
`0)CD
`
`‘\| o
`
`Warning
`Light/Sound
`
`3 m '0
`Database
`
`Brake
`Servo
`
`72V
`
`Steering
`Servo
`
`74v
`
`Throttle
`Servo
`
`\l 0')
`
`Velocity
`Sensor
`
`100
`
`96
`
`Accelerometers
`
`8
`
`Gyroscopes
`
`‘ 0
`
`Display
`
`Memory
`
`.0 [\J
`
`.0
`
`MIR, RFID
`
`6
`
`Weather
`
`Vehicle
`Diagnostics
`
`Stoplight
`Sensor
`
`Accurate
`Clock
`
`8
`
`.0
`
`.2
`
`- 4
`
`Sensors
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Central Processor & Circuits:
`
`- GPS Ranging
`- DGPS Corrections
`
`
`
`
`- Image Analysis, 150
`
`- Radar Analysis
`
`- Laser Radar Scanning Control
`
`and Analysis of Received
`
`Information
`
`
`
`
`
`
`
`- Warning Message Generation
`- Map Communication
`- Vehicle Control
`
`-
`
`Inertial Navigation System
`Calibrations and Control
`
`
`
`
`- Display Control
`— Precise Positioning
`Calculations
`- Road Condition Predictions
`And Other Functions.
`
`
`
`
`
`Controls
`
`Fig. 5
`
`7
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 6 0f 25
`
`US 7,202,776 B2
`
`
`
`8
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 7 0f 25
`
`US 7,202,776 B2
`
`60
`
`64
`
`Cameras
`
`Laser
`
`61
`
`Feature Extractor
`
`eura63/-
`
`N
`
`I
`
`Control
`
`B—s\100
`-nterface
`
`11!
`
`Fig. 6
`
`9
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 8 of 25
`
`US 7,202,776 B2
`
`
`
`10
`
`10
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 9 of 25
`
`US 7,202,776 B2
`
`12345678910
`
`139140141
`
`RWAL
`
`Fig. 10
`
`11
`
`11
`
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 10 of 25
`
`US 7,202,776 B2
`
`
`
`12
`
`12
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 11 0f 25
`
`US 7,202,776 B2
`
`130
`
`132
`
`
`
`
`
`Determine Absolute
`
`
`Data on Edges of Roadways
`Position of Vehicle
`Yellow lines and Stoplights
`
`
`
`
`
`
`Compare Absolute Position
`to Edges of Roadway
`
`
`
`134
`
`136
`
`ls
`
`
`Absolute Position of
`
`
`
`
`Vehicle Approaching Close to Edge
`of Roadway
`
`
`
`Fig. 12a
`
`Sound Alarm and/or
`
`
`Giude Vehicle to Shoulder
`
`
`130
`
`132
`
`
`Determine Absolute
`
`
`Data on Edges of Roadways
`Position of Vehicle
`Yellow lines and Stoplights
`
`
`
`
`Compare Absolute Position
`to Position of Yellow Lines
`
`
`
`142
`
`146
`
`ls
`
`
`Absolute Position of
`
`
`Vehicle Approaching Close to
`
`a Yellow Line
`
`
`
`Sound Alarm and/or
`
`Fig. 12b
`
`
`Guide Vehicle away from Yellow Line
`or to Shoulder
`
`13
`
`13
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 12 of 25
`
`US 7,202,776 B2
`
`130
`
`132
`
`
`Data on Edges of Roadways
`Determine Absolute
`
`Yellow lines and Stoplights
`Position of Vehicle
`
`
`
`
`Compare Absolute Position
`to Edges of Roadway And
`154
`Position of Stoplight
`
`Determine Color
`
`
`of Stoplight
`
`ls
`
`
`Absolute Position of
`
`
`Vehicle Approaching Close to
`a Red Stoplight?
`
`
`
`Fig. 12c
`Sound Alarm and/or
`
`Giude Vehicle to Shoulder
`
`
`14
`
`14
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 13 of 25
`
`US 7,202,776 B2
`
`15
`
`25
`
`a 1
`
`74
`
`.
`
`172
`
`18
`\
`
`26
`K
`
`Fig. 13
`
`174]
`
`172
`
`15
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 14 of 25
`
`US 7,202,776 B2
`
`180\
`
`170
`
`180
`
`172
`
`180
`
`
`
`182
`
`
`
`182
`
`26
`
`Fig. 14
`
`16
`
`16
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 15 of 25
`
`US 7,202,776 B2
`
`
`
`26
`
`Fig. 15
`
`17
`
`17
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 16 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 16A
`
`18
`
`18
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 17 0f 25
`
`US 7,202,776 B2
`
`
`
`
`Left “EYE”
`Right “EYE”
`
`
`
`mmwfipfilm»?
`
`
`
`Fig. I 8
`
`19
`
`19
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 18 of 25
`
`US 7,202,776 B2
`
`204
`
`226
`
`
`
`receiver
`
`GPS
`
`
`
`
`
`
`
`
`Scanning
`laser radar
`
`
`210
`
`Linear array
`
`camera
`
`
`
` Circuit
`
`board
`
`224/
`
`Data acquisition module
`
`Fig. 17A
`
`20
`
`20
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 19 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 19
`
`21
`
`21
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 20 of 25
`
`US 7,202,776 B2
`
`
`
`
`
`
`D a t a
`
`o n
`
`Transmitter
`
`Encoding
`
`Transmission
`
`”'0'?”
`
`
`
`
`
`242
`
`244
`
`
`
`
`
`
`D a t a
`
`o n
`
`Transmitter
`
`240
`
`242
`
`244
`
`252
`
`‘
`.
`\
`I
`w"
`s
`,....
`‘4'."- “
`.
`\
`:,..- ‘
`.
`,
`.
`.
`v
`.
`
`,
`l
`
`a
`
`252b
`
`Information
`
`Transmitter Data
`
`
`
` 252a
`
`
`
`248
`
`250
`
`Positioning
`Determining
`Device
`
`Processor
`
`
`
`246
`
`Fig. 20
`
`
`
`22
`
`22
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 21 of 25
`
`US 7,202,776 B2
`
`
`
`23
`
`23
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 22 of 25
`
`US 7,202,776 B2
`
`
`
`24
`
`24
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 23 of 25
`
`US 7,202,776 B2
`
`
`
`
`
`25
`
`25
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 24 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 24
`
`26
`
`26
`
`

`

`U.S. Patent
`
`Apr. 10, 2007
`
`Sheet 25 of 25
`
`US 7,202,776 B2
`
`
`102 Direct laser beam into
`
`
`
`environment around
`
`vehicle
`
`
`
`
`
`Receive reflections of
`104
`laser beam indicative of
`
`
`
`
`
`presence of object in path
`of laser beam
`
`
`106
`Range gate reflections to
`
`narrow distance range
`from which reflections
`
`
`
`
`
`are processed
`
`Identify/ascertain the
`identity of objects in
`range based on reflections
`
`108
`
`
`
`Assess potential for
`
`collision/consequences of
`
`
`potential collision with
`
`
`object
`112
`
`
`
`
`114
`
`collision is likely
`
`Effect countermeasure if
`
`
`
`
`
`
`
`
`
`116
`
`\ 118
`
`122
`
`Provide digital map
`
`
`
`
`
`
`
`
`
`Determine location of
`
`vehicle on map
`
`Define scanning field of
`laser beam based on
`
`
`
`
`
`
`vehicle’s position and
`map
`
`FIG. 25
`
`27
`
`27
`
`

`

`US 7,202,776 B2
`
`1
`METHOD AND SYSTEM FOR DETECTING
`OBJECTS EXTERNAL TO A VEHICLE
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation-in-part of US. patent
`application Ser. No. 10/822,445 filed Apr. 12, 2004, now
`US. Pat. No. 7,085,637, which is a continuation-in-part of:
`1) US. patent application Ser. No. 10/118,858 filed Apr.
`9, 2002, now US. Pat. No. 6,720,920, which is:
`A) a continuation-in-part of US. patent application Ser.
`No. 09/177,041 filed Oct. 22, 1998, now US. Pat. No.
`6,370,475, which claims priority under 35 U.S.C. §119
`(e) of US. provisional patent application Ser. No.
`60/062,729 filed Oct. 22, 1997;
`B) a continuation-in-part of US. patent application Ser.
`No. 09/679,317 filed Oct. 4, 2000, now US. Pat. No.
`6,405,132, which is a continuation-in-part of US.
`patent application Ser. No. 09/523,559 filed Mar. 10,
`2000, now abandoned, which claims priority under 35
`U.S.C. §119(e) of US. provisional patent application
`Ser. No. 60/123,882 filed Mar. 11, 1999, and which is
`a continuation-in-part of US. patent application Ser.
`No. 09/177,041 filed Oct. 22, 1998, now US. Pat. No.
`6,370,475, which claims priority under 35 U.S.C. §119
`(e) of US. provisional patent application Ser. No.
`60/062,729 filed Oct. 22, 1997; and
`C) a continuation-in-part of US. patent application Ser.
`No. 09/909,466 filed Jul. 19, 2001, now US. Pat. No.
`6,526,352; and
`2) US. patent application Ser. No. 10/216,633 filed Aug.
`9, 2002, now US. Pat. No. 6,768,944, which is a continu-
`ation-in-part of US. patent application Ser. No. 10/ 118,858
`filed Apr. 9, 2002, now US. Pat. No. 6,720,920.
`All of the above applications are incorporated by refer-
`ence herein.
`
`FIELD OF THE INVENTION
`
`This invention is in the fields of automobile safety,
`intelligent highway safety systems, accident avoidance,
`accident elimination, collision avoidance, blind spot detec-
`tion, anticipatory sensing, automatic vehicle control, intel-
`ligent cruise control, vehicle navigation, vehicle-to-vehicle
`communication, vehicle-to-non-vehicle communication and
`non-vehicle-to-vehicle communication and other automo-
`
`bile, truck and train safety, navigation, communication and
`control related fields.
`
`The invention relates generally to methods for vehicle-
`to-vehicle communication and communication between a
`
`vehicle and non-vehicles and more particularly to apparatus
`and methods using coded spread spectrum, ultrawideband,
`noise radar or similar technologies. The coding scheme can
`use may be implemented using multiple access communi-
`cation methods analogous to frequency division multiple
`access (FDMA), time division multiple access (TDMA), or
`code division multiple access (CDMA) in a manner to
`permit simultaneous communication with and between mul-
`tiple vehicles generally without the use of a carrier fre-
`quency.
`The invention also relates generally to an apparatus and
`method for precisely determining the location and orienta-
`tion of a host vehicle operating on a roadway and location
`of multiple moving or fixed obstacles that represent potential
`collision hazards with the host vehicle to thereby eliminate
`collisions with such hazards. In the early stages of imple-
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`mentation of the apparatus and method and when collisions
`with such hazards cannot be eliminated, the apparatus and
`method will generate warning signals and possibly initiate
`avoidance maneuvers to minimize the probability of a
`collision and the consequences thereof. More particularly,
`the invention relates to the use of a Global Positioning
`System (“GPS”), differential GPS (“DGPS”), other infra-
`structure-based location aids, cameras, radar,
`laser radar,
`terahertz radar and an inertial navigation system as the
`primary host vehicle and target locating system with centi-
`meter level accuracy. The invention is further supplemented
`by a processor to detect, recognize and track all relevant
`potential obstacles,
`including other vehicles, pedestrians,
`animals, and other objects on or near the roadway. More
`particularly,
`the invention further relates to the use of
`centimeter-accurate maps for determining the location of the
`host vehicle and obstacles on or adjacent the roadway. Even
`more particularly, the invention further relates to an inter-
`vehicle and vehicle-to-infrastructure communication sys-
`tems for transmitting GPS or DGPS position data, velocities,
`headings, as well as relevant target data to other vehicles for
`information and control action. The present invention still
`further relates to the use of Kalman filters, neural networks,
`combination neural networks and neural-fuzzy rule sets or
`algorithms for recognizing and categorizing obstacles and
`generating and developing optimal avoidance maneuvers
`where necessary.
`
`BACKGROUND OF THE INVENTION
`
`All of the patents, patent applications, technical papers
`and other references referenced below are incorporated
`herein by reference in their entirety. Various patents, patent
`applications, patent publications and other published docu-
`ments are discussed below as background of the invention.
`No admission is made that any or all of these references are
`prior art and indeed, it is contemplated that they may not be
`available as prior art when interpreting 35 U.S.C. §102 in
`consideration of the claims of the present application.
`There are numerous components described and disclosed
`herein. Many combinations of these components are
`described but to conserve space,
`the inventors have not
`described all combinations and permutations of these com-
`ponents but the inventors intend that each such combination
`and permutation is an invention to be considered disclosed
`by this disclosure. The inventors further intend to file
`continuation and continuation-in-part applications to cover
`many of these combinations and permutations.
`Automobile accidents are one of the most serious prob-
`lems facing society today, both in terms of deaths and
`injuries, and in financial
`losses suffered as a result of
`accidents. The suffering caused by death or injury from such
`accidents is immense. The costs related to medical treat-
`
`ment, permanent injury to accident victims and the resulting
`loss of employment opportunities, and financial
`losses
`resulting from damage to property involved in such acci-
`dents are staggering. Providing the improved systems and
`methods to eventually eliminate these deaths, injuries and
`other losses deserves the highest priority. The increase in
`population and use of automobiles worldwide with the
`concomitant
`increased congestion on roadways makes
`development of systems for collision avoidance and elimi-
`nation even more urgent. While many advances have been
`made in vehicle safety, including, for example, the use of
`seatbelts, airbags and safer automobile structures, much
`room for improvement exists in automotive safety and
`accident prevention systems.
`
`28
`
`

`

`US 7,202,776 B2
`
`3
`There are two major efforts underway that will signifi-
`cantly affect the design of automobiles and highways. The
`first is involved with preventing deaths and serious injuries
`from automobile accidents. The second involves the attempt
`to reduce the congestion on highways. In the first case, there
`are approximately forty two thousand (42,000) people killed
`each year in the United States by automobile accidents and
`another several hundred thousand are seriously injured. In
`the second case, hundreds of millions of man-hours are
`wasted every year by people stuck in traffic jams on the
`world’s roadways. There have been many attempts to solve
`both of these problems; however, no single solution has been
`able to do so.
`
`When a person begins a trip using an automobile, he or
`she first enters the vehicle and begins to drive, first out of a
`parking space and then typically onto a local or city road and
`then onto a highway. In leaving the parking space, he or she
`may be at risk from an impact of a vehicle traveling on the
`road. The driver must check his or her mirrors to avoid such
`
`an event and several electronic sensing systems have been
`proposed which would warn the driver that a collision is
`possible. Once on the local road, the driver is at risk of being
`impacted from the front, side and rear, and electronic sensors
`are under development to warn the driver of such possibili-
`ties. Similarly, the driver may run into a pedestrian, bicyclist,
`deer or other movable object and various sensors are under
`development that will warn the driver of these potential
`events. These various sensors include radar, optical, tera-
`hertz or other electromagnetic frequencies, infrared, ultra-
`sonic, and a variety of other sensors, each of which attempts
`to solve a particular potential collision event. It is important
`to note that as yet, in none of these cases is there sufficient
`confidence in the decision that the control of the vehicle is
`
`taken away from the driver. Thus, action by the driver is still
`invariably required.
`In some proposed future Intelligent Transportation Sys-
`tem (ITS) designs, hardware of various types is embedded
`into the highway and sensors which sense this hardware are
`placed onto the vehicle so that it can be accurately guided
`along a lane of the highway. In various other systems,
`cameras are used to track lane markings or other visual
`images to keep the vehicle in its lane. However, for suc-
`cessful ITS, additional information is needed by the driver,
`or the vehicle control system, to take into account weather,
`road conditions, congestion etc., which typically involves
`additional electronic hardware located on or associated with
`
`the highway as well as the vehicle. From this discussion, it
`is obvious that a significant number of new electronic
`systems are planned for installation ontovehicles. However,
`to date, no product has been proposed or designed which
`combines all of the requirements into a single electronic
`system. This is one of the intents of some embodiments of
`this invention.
`
`The safe operation of a vehicle can be viewed as a process
`in the engineering sense. To achieve safe operation, first the
`process must be designed and then a vehicle control system
`must be designed to implement the process. The goal of a
`process designer is to design the process so that it does not
`fail. The fact that so many people are being seriously injured
`and killed in traffic accidents and the fact that so much time
`
`is being wasted in traffic congestion is proof that the current
`process is not working and requires a major redesign. To
`design this new process, the information required by the
`process must be identified, the source of that information
`determined and the process designed so that the sources of
`information can communicate effectively with the user of
`the information, which will most often be a vehicle control
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`system. Finally, the process must have feedback that self-
`corrects the process when it is tending toward failure.
`Although it
`is technologically feasible,
`it
`is probably
`socially unacceptable at this time for a vehicle safety system
`to totally control the vehicle. An underlying premise of
`embodiments of this invention, therefore, is that people will
`continue to operate their vehicle and control of the vehicle
`will only be seized by the control system when such an
`action is required to avoid an accident or when such control
`is needed for the orderly movement of vehicles through
`potentially congested areas on a roadway. When this hap-
`pens, the vehicle operator will be notified and given the
`choice of exiting the road at the next opportunity. In some
`cases, especially when this invention is first implemented on
`a trial basis, control will not be taken away from the vehicle
`operator but a warning system will alert the driver of a
`potential collision, road departure or other infraction.
`Let us consider several scenarios and what information is
`
`required for the vehicle control process to prevent accidents.
`In one case, a driver is proceeding down a country road and
`falls asleep and the vehicle begins to leave the road, perhaps
`heading toward a tree. In this case, the control system would
`need to know that the vehicle was about to leave the road and
`
`for that, it must know the position of the vehicle relative to
`the road. One method of accomplishing this would be to
`place a wire down the center of the road and to place sensors
`within the vehicle to sense the position of the wire relative
`to the vehicle, or vice versa. An alternate approach would be
`for the vehicle to know exactly where it is on the surface of
`the earth and to also know exactly where the edge of the road
`is.
`
`These approaches are fundamentally different because in
`the former solution every road in the world would require
`the placement of appropriate hardware as well as the main-
`tenance of this hardware. This is obviously impractical. In
`the second case, the use of the global positioning satellite
`system (GPS), augmented by additional systems to be
`described below, will provide the vehicle control system
`with an accurate knowledge of its location. While it would
`be difficult to install and maintain hardware such as a wire
`
`down the center of the road for every road in the world, it
`is not difficult to survey every road and record the location
`of the edges, and the lanes for that matter, of each road. This
`information must then be made available through one or
`more of a variety of techniques to the vehicle control system.
`Another case might be where a driver is proceeding down
`a road and decides to change lines while another vehicle is
`in the driver’s blind spot. Various companies are developing
`radar, ultrasonic or optical sensors to warn the driver if the
`blind spot is occupied. The driver may or may not heed this
`warning, perhaps due to an excessive false alarm rate, or he
`or she may have become incapacitated, or the system may
`fail to detect a vehicle in the blind spot and thus the system
`will fail.
`
`Consider an alternative technology where again each
`vehicle knows precisely where it is located on the earth
`surface and additionally can communicate this information
`to all other vehicles within a certain potential danger zone
`relative to the vehicle. Now, when the driver begins to
`change lanes, his or her vehicle control system knows that
`there is another vehicle in the blind spot and therefore will
`either warn the driver or else prevent him or her from
`changing lanes thereby avoiding the accident.
`Similarly, if a vehicle is approaching a stop sign, other
`traffic marker or red traffic light and the operator fails to
`bring the vehicle to a stop, if the existence of this traffic light
`and its state (red in this example) or stop sign has been made
`
`29
`
`29
`
`

`

`US 7,202,776 B2
`
`5
`available to the vehicle control system, the system can warn
`the driver or seize control of the vehicle to stop the vehicle
`and prevent a potential accident. Additionally, if an operator
`of the vehicle decides to proceed across an intersection
`without seeing an oncoming vehicle, the control system will
`once again know the existence and location and perhaps
`velocity of the oncoming vehicle and warn or prevent the
`operator from proceeding across the intersection.
`Consider another example where water on the surface of
`a road is beginning to freeze. Probably the best way that a
`vehicle control system can know that the road is about to
`become slippery, and therefore that the maximum vehicle
`speed must be significantly reduced, is to get information
`from some external source. This source can be sensors
`
`located on the highway that are capable of determining this
`condition and transmitting it to the vehicle. Altemately, the
`probability of icing occurring can be determined analytically
`from meteorological data and a historical knowledge of the
`roadway and communicated to the vehicle over a LEO or
`GEO satellite system, the Internet or an FM sub-carrier or
`other means. A combination of these systems can also be
`used.
`
`Studies have shown that a combination of meteorological
`and historic data can accurately predict that a particular
`place on the highway will become covered with ice. This
`information can be provided to properly equipped vehicles
`so that the vehicle knows to anticipate slippery roads. For
`those roads that are treated with salt to eliminate frozen
`
`areas, the meteorological and historical data will not be
`sufficient. Numerous systems are available today that permit
`properly equipped vehicles to measure the coefficient of
`friction between the vehicle’s tires and the road.
`It
`is
`
`contemplated that perhaps police or other public vehicles
`will be equipped with such a friction coefficient measuring
`apparatus and can serve as probes for those roadways that
`have been treated with salt. Information from these probe
`vehicles will be fed into the information system that will
`then be made available to control speed limits in those areas.
`Countless other examples exist; however,
`from those
`provided above, it can be seen that for the vehicle control
`system to function without error, certain types of informa-
`tion must be accurately provided. These include information
`permitting the vehicle to determine its absolute location and
`means for vehicles near each other to communicate this
`
`location information to each other. Additionally, map infor-
`mation that accurately provides boundary and lane informa-
`tion of the road must be available. Also, critical weather or
`road-condition information is necessary. The road location
`information need only be generated once and changed
`whenever the road geometry is altered. This information can
`be provided to the vehicle through a variety of techniques
`including prerecorded media such as CD-ROM or DVD
`disks or through communications from transmitters located
`in proximity to the vehicle, satellites, radio and cellular
`phones.
`Consider now the case of the congested highway. Many
`roads in the world are congested and are located in areas
`where the cost of new road construction is prohibitive or
`such construction is environmentally unacceptable. It has
`been reported that an accident on such a highway typically
`ties up traffic for a period of approximately four times the
`time period required to clear the accident. Thus, by elimi-
`nating accidents, a substantial improvement of the congested
`highway problem is obtained. This of course is insufficient.
`On such highways, each vehicle travels with a different
`spacing, frequently at different speeds and in the wrong
`lanes. If the proper spacing of the vehicles could be main-
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`tained, and if the risk of an accident could be substantially
`eliminated, vehicles under automatic control could travel at
`substantially higher velocities and in a more densely packed
`configuration thereby substantially improving the flow rate
`of vehicles on the highway by as much as a factor of 3 to 4
`times. This not only will reduce congestion but also improve
`air pollution. Once again, if each vehicle knows exactly
`where it is located, can communicate its location to sur-
`rounding vehicles and knows precisely where the road is
`located, then the control system in each vehicle has sufficient
`information to accomplish this goal.
`Again, an intention of the system and process described
`here is to totally eliminate automobile accidents as well as
`reduce highway congestion. This process is to be designed
`to have no defective decisions. The process employs infor-
`mation from a variety of sources and utilizes that informa-
`tion to prevent accidents and to permit the maximum vehicle
`throughput on highways.
`The information listed above is still
`
`insufficient. The
`
`geometry of a road or highway can be determined once and
`for all, until erosion or construction alters the road. Properly
`equipped vehicles can know their location and transmit that
`information to other properly equipped vehicles. There
`remains a variety of objects whose location is not fixed,
`which have no transmitters and which can cause accidents.
`
`These objects include broken down vehicles, animals such
`as deer which wander onto highways, pedestrians, bicycles,
`objects which fall off of trucks, and especially other vehicles
`which are not equipped with location determining systems
`and transmitters for transmitting that information to other
`vehicles. Part of this problem can be solved for congested
`highways by restricting access to these highways to vehicles
`that are properly equipped. Also, these highways are typi-
`cally in urban areas and access by animals can be effectively
`eliminated. Heavy fines can be imposed on vehicles that
`drop objects onto the highway. Finally, since every vehicle
`and vehicle operator becomes part of the process, each such
`vehicle and operator becomes a potential source of infor-
`mation to help prevent catastrophic results. Thus, each
`vehicle should also be equipped with a system of essentially
`stopping the process in an emergency. Such a system could
`be triggered by vehicle sensors detecting a problem or by the
`operator strongly applying the brakes, rapidly turning the
`steering wheel or by activating a manual switch when the
`operator observes a critical situation but