`(10) Patent N0.:
`US 7,202,776 132
`Breed
`(45) Date of Patent:
`Apr. 10, 2007
`
`USOO7202776B2
`
`(54) METHOD AND SYSTEM FOR DETECTING
`OBJECTS EXTERNAL TO A VEHICLE
`
`(75)
`
`Inventor: David S. Breed, Boonton Township,
`Morris County, NJ (US)
`
`(73) Assignees
`
`Intelligent Technologies International,
`Inc., Dcnvillc, NJ (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 214 days.
`
`(21)
`
`Appl. N0.: 11/034,325
`
`(22)
`
`Filed:
`
`Jan. 12, 2005
`
`(65)
`
`Prior Publication Data
`
`US 2005/0134440 A1
`
`Jun. 23, 2005
`
`Related US. Application Data
`
`(63) Continuation-in-part of application No. 10/822,445,
`filed onApr. 12. 2004, now Pat. No. 7,085,637, which
`is a continuation-in-part of application No. 10/ 118,
`858, filed on Apr, 9, 2002, now Pat, No. 6,720,920,
`which is a continuation—in—part of application No.
`09/177,041, filed on Oct. 22, 1998, now Pat. No.
`6,370,475, said application No. 10/822,445 applica—
`tion No. 10/216,633, filed on Aug. 9, 2002, now Pat.
`No. 6,768,944, is a continuation—in—part ol‘application
`No. 10/118,858, filed on Apr. 9, 2002, now Pat. No.
`6,720,920, said application No. 10/118,858 is a con-
`tinuation-in-part of application No. 09/679,317, filed
`on Oct. 4, 2000, now Pat. No. 6.405,132, which is a
`continuation-in-part of application No. 09/523,559,
`filed on Mar. 10, 2000, now abandoned, and a c011-
`tinuation-in-part of application No. 09/177,041, filed
`on Oct. 22, 1998, now Pat. No. 6,370,475, said
`application No. 10/118,858 is a continuation—in—part
`01‘ application No. 09/909,466, filed on Jul. 19, 2001,
`now Pat. No. 6,526,352.
`
`(60)
`
`Provisional application No. 60/123,882, filed on Mar.
`11, 1999, provisional application No. 60/062,729,
`filed on Oct. 22, 1997.
`
`
`
`los
`
`Identifil/Iscensln the
`inches ofubjccls in
`range sued uh reflection}
`
`Asses: potential for
`/ collision/conscqwnca or
`Dmenlhl collision With
`Hz
`1
`claim
`+—
`Effecl counlemlusure ii
`collision is likely
`
`\A.
`”4
`
`
`ehwmmml ummd
`10K Directlnsmbelrn Into
`
`vehicle
`\
`Receive reflections of
`laserbun indltflllve of
`imam:“object in pull
`
`atllser beam
`__.—'—_
`Rang: m: mflcctlon: In
`inflowdimme lanes
`from which reflections
`are processed
`
`10'!
`
`we
`
`(52)
`
`(51)
`
`Int. Cl.
`(2006.01)
`B60Q [/00
`[1.8. Cl.
`........................ 340/435; 340/557; 342/70;
`342/118; 701/223
`(58) Field of Classification Search ................ 340/435,
`340/557, 901, 905. 907, 995.1; 342/70, 118,
`342/357.06; 348/113, 118; 701/200, 207,
`701/213, 223, 51, 58, 59, 65
`See application file for complete search history.
`
`(56)
`
`EP
`EP
`GB
`
`References Cited
` U.S. PATENT DOCUMENTS
`
`
`3,902,803 A
`
`9/1975 Lego, Jr
`(Continued)
`
` FOREIGN PATENT DOCUlVlENTS
`0353200
`1/1990
`0750202
`12/1996
`2141890
`1/1984
`OTHER PUBLICATIONS
`
`Ofer David et 21., "Range gated active night vision system for
`automobilies”, Applied Optics. vol. 45. No. 28, Oct. 1, 2006, pp.
`7248-7254.
`
`Primary Examineri'lhomas Mullen
`(74) Attorney, Agent, or FirmiBrian Rofi'e
`
`
`
`ABS TRACT
`(5 7)
`Method and system for obtaining infonnation about objects
`in the environment outside of and around a vehicle and
`preventing collisions involving the vehicle includes direct-
`ing a laser beam from the vehicle into the environment,
`receiving from an object in the path of the laser beam a
`reflection of the laser beam at a location on the vehicle, and
`analyzing the received laser beam rellections to obtain
`information about the object from which the laser beam is
`being reflected, Analysis of the laser beam reflections pref-
`erably entails range gating the received laser beam reflec—
`tions to limit analysis of the received laser beam reflections
`to only those received from an object within a defined
`(distance) range such that objects at distances within the
`range are isolated from surrounding objects.
`
`26 Claims, 25 Drawing Sheets
`
`1 l6
`
`\ng.—.
`
`
`\1 Provide digitnlmap
`_l_—Determine outlet-l or
`vehiel. on nap
`
` v
`
`laser hum based on
`Define scanning field of
`\\
`/ vuhicle’sposilicn And
`
`
`I22
`nlau
`
`|PR2013-00419 - Ex. 1024
`
`Toyota Motor Corp., Petitioner
`1
`
`IPR2013-00419 - Ex. 1024
`Toyota Motor Corp., Petitioner
`1
`
`
`
`US 7,202,776 B2
`
`Pagc 2
`
`US. PATENT DOCUMENTS
`
`3.947.119 A
`212252;: 1:
`1
`1
`‘
`2’32‘232 ‘2
`’,‘
`‘
`4,9/0,628 A
`5,255,163 A
`5,314,037 A *
`5,383,200 A
`51414439 A
`55434754 A
`5,700,073 A
`5,747,792 A
`5,756,989 A
`5,771,326 A
`5,791,757 A
`5,857,770 A
`5.890.796 A
`57971573 A
`6,008,496 A
`6,014,601 A '1
`2123???? $1
`6*g10‘475 B1
`6’465‘132 B1
`61422713 B1
`6,429,429 131
`6,483,094 B1
`
`361936 Samb'erg et 3],
`:13? Sfll‘m‘mer
`Q
`’
`703’
`111336 $323109 al'
`C
`‘
`e
`.
`11/1990 Sergkwst
`10/1993 Ncumann
`................. 340/903
`5/1994 Shaw et a1,
`1/1995 Barrett et a1.
`5/1995 Groves et 31'
`7/1995
`5i
`121997 :011
`5/1998
`11111129111
`5/1998 3carcta]
`6/1998
`:011
`8/1998 O‘Neil
`1/1999
`‘ml
`4/1999 Maxinelh
`10/1999
`3011
`12/1999 Winefordneret a1.
`’1/2000 (jusmfson ................... 340/436
`$3388?
`‘011 1
`1
`$3002
`fmflm et 3‘
`622002 3:;
`7/2002
`:011
`/2002
`‘011
`11/2002 Yahav
`
`
`
`
`
`6,526,352 B1
`6,538,820 B2
`
`2/2003 Breed et al.
`3/2003 F6111
`
`4/2003 HOIZ eL 31.
`6,552,342 B2
`6/2003 Ostromek et a1.
`6,576,884 B1
`2/2004 Remillard
`6,690,017 B2
`4/2004 Breed
`6,720,920 B2
`4/2004 Miller
`6,725,139 132
`7
`.
`,
`5/7004 Rcm'nmd
`67730791” 32
`7/3004 Ere/ed
`6768944 B2
`3/6004 Stephan
`6774567 132
`10/2004 Abel e1 (11,
`6,803,574 32
`10/2004 Fohl
`6,809,870 B2
`12/2004 Stephan etal.
`6,828,544 132
`6,975,246 B1* 122005 Trudeau ..................... 340/435
`2002/0185590 A1
`12/2002 Yahav
`2002/0191388 A1
`12/2002 Matveev
`2003/0034462 A1
`2/2003 Rcmillard
`2003/0036881 A1
`2/2003 Remillard
`2003/0155513 A1
`8/2003 Remillard
`2003/0155514 A1
`8/2003 Remillard
`2003/0193980 A1
`10/2003 Malveev
`2003/0230705 A1
`12,2003 Stephan
`2003/0230715 A1
`12/2003 Rcmillard
`2004/0031922 A1
`2/2004 Stephan
`2005/0107954 A1*
`5/2005 Nahla ......................... 701/213
`2005/0269481 A1
`122005 DaV1d ct a1.
`
`* cited by examiner
`
`
`
` 2
`
`
`
`
`
`Prior Art
`
`Fig.
`
`1
`
`
`
` 3
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 2 of 25
`
`US 7,202,776 B2
`
`
`
`BASE STATION
`
`Prior Art
`
`Fig. 2
`
`
`
` 4
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 3 of 25
`
`US 7,202,776 B2
`
`2
`
`[if/3%
`| 3V2
`"’4
`—|_
`I "’1 7—
`41 3 E J:
`gjk§
`lfl/so
`7|=IT~~~~2
`
`I
`
`‘ M16 (/32
`
`
`
`BASESTATION
`
`Prior Art
`
`Fig. 3
`
`
`
` 5
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 4 of 25
`
`US 7,202,776 B2
`
`G PS SATELLITES
`
`
`
`DGF’S SATELLITE
`
`
`
`GPS 8: DGPS
`PROCESSING
`
`SYSTEM
`
`
`
` 6
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 5 of 25
`
`US 7,202,776 B2
`
`v
`
`v
`
`4 s
`
`54
`
`56
`
`52
`
`GPS
`Receiver
`
`DGPS
`Receiver
`
`Inter-Vehicle
`Communication
`
`Infrastructure
`Communication
`
`3
`
`Accelerometers
`
`8
`
`Gyroscopes
`
`‘ 0
`
`Display
`
`2
`
`Memory
`
`MIR, RFID
`
`Weather
`Sensors
`
`Vehicle
`Diagnostics
`
`Stoplight
`Sensor
`
`.
`o6
`
`8
`
`.0
`
`.2
`
`Accurate
`Clock
`
`.4
`
`
`Central Processor & Circuits:
`
`- GPS Ranging
`- DGPS Corrections
`
`
`- Image Analysis, 150
`
`
`- Radar Analysis
`
`
`Laser Radar Scanning Control
`and Analysis of Received
`Information
`
`Warning Message Generation
`- Map Communication
`Vehicle Control
`
`Inertial Navigation System
`Calibrations and Control
`
`— Display Control
`— Precise Positioning
`Calculations
`— Road Condition Predictions
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`60v Cameras
`
`62
`
`Radar
`
`6
`
`Laser Radar
`
`66
`
`58
`
`70
`
`Warning
`Light/S ound
`
`Ma
`Databgse
`
`Brake
`Servo
`
`72v
`
`Steering
`Servo
`
`74v
`
`Throttle
`Servo
`
`76
`
`Velocity
`Sensor
`
`100
`
`96
`
`And Other Functions.
`
`
`
`Controls
`
`Fig. 5
`
`
`
` 7
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 6 of 25
`
`US 7,202,776 B2
`
`
`
`
`
` 8
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 7 of 25
`
`US 7,202,776 B2
`
`61
`
`-nterface\
` B—s
`
`
`11
`
`Fig. 6
`
`
`
` 9
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 8 of 25
`
`US 7,202,776 B2
`
`
`
`10
`
`
`
`10
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 9 of 25
`
`US 7,202,776 B2
`
`12345678910
`
`139140141
`
`
`
`OUTPUT O O 0
`
`
`
`Fig. 10
`
`11
`
`
`
`11
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 10 of 25
`
`US 7,202,776 B2
`
`Fig.11
`
`
`
`12
`
`
`
`12
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 11 of 25
`
`US 7,202,776 B2
`
`130
`
`132
`\
`\
`/
`
`
`Determine Absolute
`
`
`Data on Edges of Roadways
`
`
`Position of Vehicle
`Yellow lines and Stoolights
`
`
`
`
`
`
`Compare Absolute Position
`to Edges of Roadway
`
`
`134
`
`
`
`
`
`136
`
`is
`
`Absolute Position of
`
`Vehicle Approaching Close to Edge
`
`of Roadway
`
`
`
`Fig. 12a
`
`Sound Alarm and/or
`
`
`Giude Vehicle to Shoulder
`
`
`130
`
`132
`
`
`
`Determine Absolute
`Data on Edges of Roadways
`
`Position of Vehicle
`Yellow lines and Stoplights
`
`
`
`
`
`
`Compare Absolute Position
`to Position of Yellow Lines
`
`142
`
`
`
`
`
`Is
`
`Absolute Position of
`
`Vehicle Approaching Close to
`a Yellow Line
`
`146
`
`
`
`Fig. 12b
`
`
`Sound Alarm andlor
`Guide Vehicle away from Yellow Line
`or to Shoulder
`
`13
`
`
`
`13
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 12 of 25
`
`US 7,202,776 B2
`
`130
`
`132
`
`
`
`Determine Absolute
`Data on Edges of Roadways
`
`
`Yellow lines and Stoplights
`Position of Vehicle
`
`
`
`
`
`
`
`
`Compare Absolute Position
`to Edges of Roadway And
`
`Position of Stoplight
`
`Determine Color
`
`
`
`of Stoplight
`
`
`ls
`
`
`Absolute Position of
`
`
`Vehicle Approaching Close to
`
`a Red Stoplight?
`
`
`Fig. 12c
`Sound Alarm and/or
`
`Giude Vehicle to Shoulder
`
`
`14
`
`
`
`14
`
`
`
`15
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 13 of 25
`
`US 7,202,776 B2
`
`25
`
`Q 1
`
`74
`
`.
`
`172
`
`13
`
`\
`
`26
`K
`
`Fig. 13
`
`174]
`
`172
`
`
`
`15
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 14 of 25
`
`US 7,202,776 B2
`
`180\
`
`170
`
`180
`
`172
`
`180
`
`
`
`
`182
`
`Fig. 14
`
`16
`
`
`
`16
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 15 of 25
`
`US 7,202,776 B2
`
`
`
`26
`
`Fig. 15
`
`17
`
`
`
`17
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 16 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 16A
`
`18
`
`
`
`18
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 17 of 25
`
`US 7,202,776 B2
`
`
`
` Left “EYE” Right “EYE”
`
`
`
`
`
`
`
`
`2.2;meWM 4mm]?
`
`
`
`
`
`19
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 18 of 25
`
`US 7,202,776 B2
`
`226
`
`204
`
`
`
`
`GPS
`receiver
`
`
`
`
`
`
`
`
`20
`
`210
`
`Linear array
`
`camera
`
`206
`
`Video
`
`camera
`
`Circuit
`
`board
`
`Scanning
`
`laser radar
`
`224/
`
`Data acquisition module
`
`Fig. 17A
`
`
`
`20
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 19 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 19
`
`21
`
`
`
`21
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 20 of 25
`
`US 7,202,776 B2
`
`
`
`o n
`D a t a
`Transmitter
`
`Encoding
`
`J//»24o
`
`242
`
`
`
`Transmission
`
`‘u-o‘
`
`244
`
`
`
`
`
`
`
`
`
`
`
`
`o n
`D a t a
`Transmitter
`
`24o
`
`242
`
`
`
`
`
`/252b
`
`Transmissio
`
`244
`
`252
`
`252a
`
`Information Transmitter Data
`
`
`
`
`
`Positioning
`Determining
`Device
`
`rocessor
`
`248
`
`250
`
`Fig. 20
`
`246
`
`22
`
`
`
`22
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 21 of 25
`
`US 7,202,776 B2
`
`
`
`23
`
`
`
`23
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 22 of 25
`
`US 7,202,776 B2
`
`
`
`24
`
`
`
`24
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 23 of 25
`
`US 7,202,776 B2
`
`
`
`25
`
`
`
`25
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 24 of 25
`
`US 7,202,776 B2
`
`
`
`Fig. 24
`
`26
`
`
`
`26
`
`
`
`US. Patent
`
`Apr. 10, 2007
`
`Sheet 25 of 25
`
`US 7,202,776 B2
`
`102
` Direct laser beam into
`
`
`environment around
`vehicle
`
`Receive reflections of
`
`
`
`laser beam indicative of
`
`104
`
`116
`
`\ 118
`
`presence of object in path
`of laser beam
`
`Provide digital map
`
`
`
`
`
`
`
`
`Determine location of
`
`vehicle on map
`
`
`
`
`Define scanning field of
`laser beam based on
`
`
`vehicle’s position and
`map
`
`122
`
`FIG. 25
`
`27
`
`
`
`
`
`
`
`
`
`
`Range gate reflections to
` 106
`narrow distance range
`from which reflections
`
`
`
`
`
`are processed
`
`Identify/ascertain the
`identity of objects in
`range based on reflections
`
`108
`
`
`
`Assess potential for
`collision/consequences of
`
`
`potential collision with
`
`
`112
`object
`
`
`
`
`Effect countermeasure if
`
`collision is likely
`
`114
`
`
`
`27
`
`
`
`US 7,202,776 B2
`
`1
`METHOD AND SYSTEM FOR DETECTING
`OBJECTS EXTER‘IAL TO A VEHICLE
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation-in-part of U.S. patent
`application Ser. No. 10/822,445 filed Apr. 12, 2004, now
`U.S. Pat. No. 7,085.637, which is a continuation-in—part of:
`1) U.S. patent application Ser. No. 10/118,858 filed Apr.
`9, 2002, now U.S. Pat. No, 6,720,920, which is:
`A) a eontinuation—in-part of U.S. patent application Ser.
`No, 09/177,041 filed Oct. 22, 1998, now US. Pat. No.
`6,370,475, which claims priority under 35 U.S.C. §119
`(e) of U.S. provisional patent application Ser. No.
`60/062,729 filed Oct. 22, 1997;
`B) a continuation-in—part of U.S. patent application Ser.
`No. 09/679,317 filed Oct. 4, 2000, now U.S. Pat. No.
`6,405,132, which is a continuation-in-part of U.S.
`patent application Ser. No. 09/523,559 filed Mar. 10,
`2000, now abandoned, which claims priority under 35
`U.S.C. §119(e) of US. provisional patent application
`Ser. No. (50/123,882 filed Mar. 11, 1999, and which is
`a continuation—in—part of U.S. patent application Ser.
`No, 09/177,041 filed Oct. 22, 1998, now US. Pat. No.
`6,370,475, which claims priority under 35 U.S.C. §119
`(e) of U.S. provisional patent application Ser. No.
`60/062,729 filed Oct. 22, 1997; and
`C) a continuation-in—part of U.S. patent application Ser.
`No. 09/909,466 filed Jul. 19, 2001, now U.S. Pat. No.
`6,526,352; and
`2) U.S. patent application Ser. No. 10/216,633 filed Aug.
`9, 2002, now U.S. Pat. No. 6,768,944, which is a continu-
`ation-in-part of U.S. patent application Ser. No. 10/118,858
`filed Apr. 9, 2002, now US. Pat. No. 6,720,920.
`All of the above applications are incorporated by refer—
`ence herein.
`
`FIELD OF THE INVENTION
`
`This invention is in the fields of automobile safety,
`intelligent highway safety systems, accident avoidance,
`accident elimination, collision avoidance, blind spot detec—
`tion, anticipatory sensing, automatic vehicle control, intel—
`ligent cruise control, vehicle navigation, vehicle—to—vehicle
`communication, vehicle-to-non-vehicle communication and
`non-vehicle-to-vehicle communication and other automo-
`bile, truck and train safety, navigation, communication and
`control related fields.
`The invention relates generally to methods for vehicle-
`to-vehicle communication and communication between a
`vehicle and non-vehicles and more particularly to apparatus
`and methods using coded spread spectrum, ultrawideband,
`noise radar or similar technologies. The coding scheme can
`use may be implemented using multiple access communi-
`cation methods analogous to frequency division multiple
`access (EDMA), time division multiple access (TDMA), or
`code division multiple access (CDMA) in a manner to
`permit simultaneous communication with and between inul—
`tiple vehicles generally without the use of a carrier fre—
`quency.
`The invention also relates generally to an apparatus and
`method for precisely detemiining the location and orienta-
`tion of a host vehicle operating on a roadway and location
`of multiple moving or fixed obstacles that represent potential
`collision hazards with the host vehicle to thereby eliminate
`collisions with such hazards. In the early stages of imple-
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`60
`
`65
`
`2
`mentation of the apparatus and method and when collisions
`with such hazards cannot be eliminated, the apparatus and
`method will generate warning signals and possibly initiate
`avoidance maneuvers to minimize the probability of a
`collision and the consequences thereof. More particularly,
`the invention relates to the use of a Global Positioning
`System (“GPS”), differential GPS (“DGPS”), other infra-
`structure-based location aids, cameras, radar,
`laser radar,
`terahertz radar and an inertial navigation system as the
`primary host vehicle and target locating system with centi-
`meter level accuracy. The invention is further supplemented
`by a processor to detect, recognize and track all relevant
`potential obstacles,
`including other vehicles, pedestrians,
`animals, and other objects on or near the roadway. More
`particularly,
`the invention further relates to the use of
`centimeter-accurate maps for determining the location of the
`
`host vehicle and obstacles on or adjacent the roadway. Even
`more particularly, the invention fiirther relates to an inter-
`vehicle and vehicle-to-infrastructure communication sys-
`tems for transmitting GPS or DGPS position data, velocities,
`headings, as well as relevant target data to other vehicles for
`information and control action, The present invention still
`further relates to the rise of Kalman filters, neural networks,
`combination neural networks and neural—fuzzy rule sets or
`algorithms for recognizing and categorizing obstacles and
`generating and developing optimal avoidance maneuvers
`where necessary.
`BACKGROUND OF THE INVENTION
`
`All of the patents, patent applications, technical papers
`and other references referenced below are incorporated
`herein by reference in their entirety. Various patents, patent
`applications, patent publications and other published docu—
`ments are discussed below as background of the invention.
`No admission is made that any or all of these references are
`prior art and indeed, it is contemplated that they may not be
`available as prior art when interpreting 35 U.S.C. §102 in
`consideration of the claims of the present application.
`There are numerous components described and disclosed
`herein. Many combinations of these components are
`described but to conserve space,
`the inventors have not
`described all combinations and permutations of these com-
`ponents but the inventors intend that each such combination
`and permutation is an invention to be considered disclosed
`by this disclosure. The inventors further intend to file
`continuation and continuation—in—part applications to cover
`many of these combinations and permutations.
`Automobile accidents are one of the most serious prob—
`lems facing society today, both in terms of deaths and
`injuries, and in financial
`losses suffered as a result of
`accidents. The suffering caused by death or injury from such
`accidents is immense. The costs related to medical treat-
`ment, permanent injury to accident victims and the resulting
`loss of employment opportunities, and financial
`losses
`resulting from damage to property involved in such acci-
`dents are staggering. Providing the improved systems and
`methods to eventually eliminate these deaths, injuries and
`other losses deserves the highest priority. The increase in
`population and use of automobiles worldwide with the
`concomitant
`increased congestion on roadways makes
`development of systems for collision avoidance and elimi—
`nation even more urgent. While many advances have been
`made in vehicle safety, including, for example, the use of
`seatbelts, airbags and safer automobile structures, much
`room for improvement exists in automotive safety and
`accident prevention systems.
`
`28
`
`
`
`28
`
`
`
`US 7,202,776 B2
`
`3
`There are two major efforts underway that will signifi—
`cantly affect the design of automobiles and highways. The
`first is involved with preventing deaths and serious injuries
`from automobile accidents. The second involves the attempt
`to reduce the congestion on highways. In the first case, there
`are approximately forty two thousand (42,000) people killed
`each year in the United States by automobile accidents and
`another several hundred thousand are seriously injured. In
`the second case, hundreds of millions of man-hours are
`wasted every year by people stuck in traffic jams on the
`world’s roadways. There have been many attempts to solve
`both of these problems; however, no single solution has been
`able to do so.
`When a person begins a trip using an automobile, he or
`she first enters the vehicle and begins to drive, Iirst out ofa
`parking space and then typically onto a local or city road and
`then onto a highway. In leaving the parking space, he or she
`may be at risk from an impact of a vehicle traveling on the
`road. The driver must check his or her mirrors to avoid such
`an event and several electronic sensing systems have been
`proposed which would warn the driver that a collision is
`possible. Once on the local road, the driver is at risk ofbeing
`impacted from the front, side and rear, and electronic sensors
`are under development to warn the driver of such possibili—
`ties. Similarly, the driver may run into a pedestrian, bicyclist,
`deer or other movable object and various sensors are under
`development that will warn the driver of these potential
`events. These various sensors include radar, optical, tera-
`hertz or other electromagnetic frequencies, infrared, ultra-
`sonic, and a variety of other sensors, each of which attempts
`to solve a particular potential collision event. It is important
`to note that as yet, in none of these cases is there sufficient
`confidence in the decision that the control of the vehicle is
`taken away from the driver. Thus, action by the driver is still
`invariably required.
`In some proposed future Intelligent Transportation Sys—
`tem (ITS) designs, hardware of various types is embedded
`into the highway and sensors which sense this hardware are
`placed onto the vehicle so that it can be accurately guided
`along a lane of the highway. In various other systems,
`cameras are used to track lane markings or other visual
`images to keep the vehicle in its lane. However, for suc-
`cessful ITS, additional information is needed by the driver,
`or the vehicle control system, to take into account weather,
`road conditions, congestion etc,, which typically involves
`additional electronic hardware located on or as sociatcd with
`the highway as well as the vehicle. From this discussion, it
`is obvious that a significant number of new electronic
`systems are planned for installation ontovehicles. However,
`to date, no product has been proposed or designed which
`combines all of the requirements into a single electronic
`system. This is one of the intents of some embodiments of
`this invention.
`The safe operation of a vehicle can be viewed as a process
`in the engineering sense. To achieve safe operation, first the
`process must be designed and then a vehicle control system
`must be designed to implement the process. The goal of a
`process designer is to design the process so that it docs not
`fail. The fact that so many people are being seriously injured
`and killed in traffic accidents and the fact that so much time
`is being wasted in tralfic congestion is proof that the current
`process is not working and requires a major redesign. To
`design this new process, the information required by the
`process must be identified, the source of that infonnation
`determined and the process designed so that the sources of
`information can communicate effectively with the user of
`the information, which will most often be a vehicle control
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`60
`
`65
`
`4
`system. Finally, the process must have feedback that self—
`corrects the process when it is tending toward failure.
`Although it
`is technologically feasible,
`it
`is probably
`socially unacceptable at this time for a vehicle safety system
`to totally control the vehicle. An underlying premise of
`embodiments of this invention, therefore, is that people will
`continue to operate their vehicle and control of the vehicle
`will only be seized by the control system when such an
`action is required to avoid an accident or when such control
`is needed for the orderly movement of vehicles through
`potentially congested areas on a roadway. When this hap-
`pens, the vchiclc operator will be notified and given the
`choice of exiting the road at the next opportunity. In some
`cases, especially when this invention is first implemented on
`a trial basis, control will not be taken away from the vehicle
`operator but a warning system will alert the driver of a
`potential collision, road departure or other infraction.
`Let us consider several scenarios and what information is
`required for the vehicle control process to prevent accidents.
`In one case, a driver is proceeding down a country road and
`falls asleep and the vehicle begins to leave the road, perhaps
`heading toward a tree. In this case, the control system would
`need to know that the vehicle was about to leave the road and
`for that, it must know the position of the vehicle relative to
`the road. One method of accomplishing this would be to
`place a wire down the center of the road and to place sensors
`within the vehicle to sense the position of the wire relative
`to the vehicle, or vice versa. An alternate approach would be
`for the vehicle to know exactly where it is on the surface of
`the earth and to also know exactly where the edge ofthe road
`ls.
`
`These approaches are fundamentally different because in
`the former solution every road in the world would require
`the placement of appropriate hardware as well as the main-
`tenance of this hardware. This is obviously impractical. In
`the second case, the use of the global positioning satellite
`system (GPS), augmented by additional systems to be
`described below, will provide the vehicle control system
`with an accurate knowledge of its location. While it would
`be difficult to install and maintain hardware such as a wire
`down the center of the road for every road in the world, it
`is not difficult to survey every road and record the location
`of the edges, and the lanes for that matter. of each road. This
`information must then be made available through one or
`more of a variety oftechniques to the vehicle control system.
`Another case might be where a driver is proceeding down
`a road and decides to change lines while another vehicle is
`in the driver’s blind spot. Various companies are developing
`radar, ultrasonic or optical sensors to warn the driver if the
`blind spot is occupied. The driver may or may not heed this
`warning, perhaps due to an excessive false alarm rate, or he
`or she may have become incapacitated, or the system may
`fail to detect a vehicle in the blind spot and thus the system
`will fail.
`Consider an alternative technology where again each
`vehicle knows precisely where it is located on the earth
`surface and additionally can communicate this information
`to all other vehicles within a certain potential danger zone
`relative to the vehicle. Now, when the driver begins to
`change lanes, his or her vehicle control system knows that
`there is another vehicle in the blind spot and therefore will
`either warn the driver or else prevent him or her from
`changing lanes thereby avoiding the accident.
`Similarly, if a vehicle is approaching a stop sign, other
`traffic marker or red traific light and the operator fails to
`bring the vehicle to a stop, if the existence of this traffic light
`and its state (red in this example) or stop sign has been made
`
`29
`
`
`
`29
`
`
`
`US 7,202,776 B2
`
`5
`available to the vehicle control system, the system can warn
`the driver or seize control of the vehicle to stop the vehicle
`and prevent a potential accident. Additionally, if an operator
`of the vehicle decides to proceed across an intersection
`Without seeing an oncoming vehicle, the control system will
`once again know the existence and location and perhaps
`velocity of the oncoming vehicle and warn or prevent the
`operator from proceeding across the intersection.
`Consider another example where water on the surface of
`a road is beginning to freeze. Probably the best way that a
`vehicle control system can know that the road is about to
`become slippery, and therefore that the maximum vehicle
`speed must be significantly reduced, is to get information
`from some external source. This source can be sensors
`located on the highway that are capable of determining this
`condition and transmitting it to the vehicle. Alternately, the
`probability of icing occurring can be detennined analytically
`from meteorological data and a historical knowledge of the
`roadway and communicated to the vehicle over a LEO or
`GEO satellite system, the Internet or an FM sub-carrier or
`
`otler means. A combination of these systems can also be
`used.
`Studies have shown that a combination of meteorological
`and historic data can accurately predict that a particular
`place on the highway will become covered with ice. This
`information can be provided to properly equipped vehicles
`so that the vehicle knows to anticipate slippery roads. For
`those roads that are treated with salt to eliminate frozen
`
`
`areas, the meteorological and historical data will not be
`
`su 7icient. Numerous systems are available today that permit
`properly equipped vehicles to measure the coeflicient of
`friction between the vehicle’s tires and the road.
`It
`is
`contemplated that perhaps police or other public vehicles
`will be equipped with such a friction coefficient measuring
`apparatus and can serve as probes for those roadways that
`have been treated with salt. Information from these probe
`vehicles will be fed into the information system that will
`then be made available to control speed limits in those areas.
`Countless other examples exist; however,
`from those
`provided above, it can be seen that for the vehicle control
`system to fiinction without error, certain types of informa-
`tion must be accurately provided. These include information
`permitting the vehicle to determine its absolute location and
`means for vehicles near each other to communicate this
`location information to each other. Additionally, map infor-
`mation that accurately provides boundary and lanc informa-
`tion of the road must be available. Also, critical weather or
`road—condition information is necessary. The road location
`information need only be generated once and changed
`whenever the road geometry is altered. This information can
`be provided to the vehicle through a variety of techniques
`including prerecorded media such as CD-ROM or DVD
`disks or through communications from transmitters located
`in proximity to the vehicle, satellites, radio and cellular
`phones.
`Consider now the case of the congested highway. Many
`roads in the world are congested and are located in areas
`where the cost of new road construction is prohibitive or
`such construction is environmentally unacceptable. It has
`been reported that an accident on such a highway typically
`ties up trallic for a period of approximately four times the
`time period required to clear the accident. Thus, by elirni—
`nating accidents, a substantial improvement of the congested
`highway problem is obtained. This of course is insufficient.
`
`
`On such highways, each vehicle travels with a different
`
`
`spacing, frequently at di erent speeds and in the wrong
`lanes. If the proper spacing of the vehicles could be main-
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`60
`
`65
`
`6
`tained, and if the risk of an accident could be substantially
`eliminated, vehicles under automatic control could travel at
`substantially higher velocities and in a more densely packed
`configuration thereby substantially improving the flow rate
`of vehicles on the highway by as much as a factor of 3 to 4
`times. This not only will reduce congestion but also improve
`air pollution. Once again, if each vehicle knows exactly
`where it is located, can communicate its location to sur-
`
`rounding vehicles and knows precisely where the road is
`
`
`
`located, then the control system in each vehicle has su icient
`information to accomplish this goal.
`Again, an intention of the system and process described
`here is to totally eliminate automobile accidents as well as
`reduce highway congestion. This process is to be designed
`to have no defective decisions. The process employs infor—
`mation from a variety of sources and utilizes that informa-
`tion to prevent accidents and to permit the maximum vehicle
`throughput on highways.
`insuflicient. The
`The information listed above is still
`geometry of a road or highway can be determined once and
`for all, until erosion or construction alters the road. Properly
`equipped vehicles can know their location and transmit that
`information to other properly equipped vehicles. There
`remains a variety of objects whose location is not fixed,
`which have no transmitters and which can cause accidents.
`These objects include broken down vehicles, animals such
`as deer which wander onto highways, pedestrians, bicycles,
`objects which fall off of trucks, and especially other vehicles
`which are not equipped with location determining systems
`and transmitters for transmitting that information to other
`vehicles. Part of this problem can be solved for congested
`highways by restricting access to these highways to vehicles
`that are properly equipped. Also, these highways are typi-
`cally in urban areas and access by animals can be effectively
`eliminated. Heavy fines can be imposed on vehicles that
`drop objects onto the highway. Finally, since every vehicle
`and vehicle operator becomes part of the process, each such
`vehicle and operator becomes a potential source of infor—
`mation to help prevent catastrophic results. Thus, each
`vehicle should also be equipped with a system of essentially
`stopping the process in an emergency. Such a system could
`be triggered by vehicle sensors detecting a problem or by the
`operator strongly applying the brakes, rapidly turning the
`steering wheel or by activating a manual switch when the
`operator observes a critical situation but is not himself in
`immediate danger. An example of the latter case is where a
`driver witnesses a box falling off o