United States Patent [191
`Nishio
`
`mm mm m ||m||1|| IIJIIQLQILISITTIIIS [11111 I111 mm I1 Ill“ 11
`5,541,590
`*Jul. 30, 1996
`
`[11] Patent Number:
`[45] Date of Patent:
`
`[54] VEPHCLE CRASH PREDICTIVE AND
`EVASIVE OPERATION SYSTEM BY NEURAL
`NETWORKS
`
`[75] Inventor: Tomoyuki Nishio, Kawasaki, Japan
`
`[73] Assignee: Takata Corporation, Tokyo, Japan
`
`[*] Notice:
`
`The term of this patent shall not extend
`beyond the expiration date of Pat. No.
`5,377,108.
`
`FOREIGN PATENT DOCUMENTS
`
`0358628
`2554612
`3837054
`4001493
`04008639
`9002985
`
`3/1990
`5/1985
`6/1989
`7/1991
`l/l992
`3/1990
`
`European Pat. Off. .
`
`France .
`
`Germany .
`
`Germany .
`
`Japan .
`
`WIPO .
`
`OTHER PUBLICATIONS
`
`[21] Appl. No.: 375,249
`[22] Filed:
`Jan. 19, 1995
`
`Rumelhart et al “Parallel Distributed Processing”, vol. 1 pp.
`161, 162, copyrighted 1986.
`
`Related U.S. Application Data
`
`Primary Examiner—Brent A. Swarthout
`Attorney, Agent, or Firm—Kanesaka & Takeuchi
`
`[63] Continuation of Ser. No. 97,178, Jul. 27, 1993, abandoned.
`[30]
`Foreign Application Priority Data
`
`Aug. 4, 1992
`
`[JP]
`
`Japan .................................. .. 4-229201
`
`[51] Int. Cl.6 ..................................................... .. G08G 1/16
`[52] U.S. Cl. ........................ .. 340/903; 340/435; 348/148;
`364/424.04; 395/23; 395/905
`[58] Field of Search ................................... .. 340/435, 995,
`340/903, 905; 348/170, 113, 148, 149;
`364/424.0l, 424.04, 424.05; 395/905.22,
`11, 21, 23; 382/104, 157
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`7/1992 Nabet et a1. .......................... .. 364/807
`5,130,563
`5,161,014 11/1992 Pearson et a1.
`.... .. 395/11
`
`5,161,632 11/1992 Asayama . . . . .
`
`5,162,997 ll/l992 Takahashi
`5,189,619
`2/1993 Adachi et al.
`5,200,898
`4/1993 Yuhara et a1. ..... ..
`5,214,744
`5/1993 Schweizer et a1. ..
`5,249,157
`9/1993 Taylor ........... ..
`
`. . . . . . .. 340/435
`
`36414241
`...... .. 340/903
`364/43104
`364/807
`340/903
`
`5,270,708 12/1993 Karnishirna . . . . .
`
`. . . . . . .. 340/905
`
`364/424.0l
`l/1994 Gioutsos et al. .
`5,282,134
`364/424.0l
`2/1994 Takahashi
`5,285,523
`364/42405
`5,377,108 12/1994 Nishio ........ ..
`5,434,927
`7/1995 Brady et a1. .......................... .. 348/148
`
`[57]
`
`ABSTRACT
`
`A system for predicting and evading crash of a vehicle
`includes an image pick-up device mounted on the vehicle for
`picking up images of actual ever-changing views when the
`vehicle is on running to produce actual image data, a crash
`predicting device associated with said image pick-up device,
`said crash predicting device being successively supplied
`with the actual image data for predicting occurrence of crash
`between the vehicle and potentially dangerous objects on the
`roadway to produce an operational signal when there is
`possibility of crash and a safety drive ensuring device
`connected to said crash predicting device for actuating, in
`response to the operational signal, an occupant protecting
`mechanism which is operatively connected thereto and
`equipped in the vehicle. The crash predicting device
`includes a neural network which is previously trained with
`training data to predict the possibility of crash, the training
`data representing ever-changing views previously picked-up
`from said image picking-up device during driving of the
`vehicle for causing actual crash.
`
`4 Claims, 7 Drawing Sheets
`
`TO STEERING WHEEL
`
`TO THROTTLE VALVE
`
`TO BRAKE
`
`STEERING
`ACTUATOR
`
`THROTTLE
`ACTUATOR
`
`21
`
`22
`
`CRASH
`PREDICTING
`
`CIRCUIT ? OUTPUT l/F
`
`1
`
`Mercedes-Benz USA, LLC, Petitioner - Ex. 1005
`
`

`

`US. Patent
`
`Jul. 30, 1996
`
`Sheet 1 of 7
`
`5,541,590
`
`mx<mmOP
`
`
`
`w>4<>MJFFOmTC.O._.
`
`
`
`Ame>>ZEmmlrmOP
`
`KOF<DHO<
`
`mx<mm
`
`0v
`
`«5.55%o<
`
`
`
`
`
`Nd.FOmI._.
`
`mOP<DB<
`
`025$me
`
`NN
`
`
`
`n.)._.Dn_z_
`
`_‘N
`
`nSxon
`
`mos):
`
`Hmd‘moEn._
`
`o_u
`
`momzmm05E
`mOmzmm
`
`
`
`mamaozEmEm
`
`mmmmm
`
`VNmm
`
`2
`
`
`
`
`
`
`
`
`
`

`

`US. Patent
`
`Jul. 30, 1996
`
`Sheet 2 0f 7
`
`5,541,590
`
`INPUT xi
`
`><=Z xi wi
`'
`OUTPUT
`
`F I G. 2
`RR\OR ART
`
`' 0.5 0/
`
`Xv
`f(><)=______1____
`1+exp(-X)
`
`Fl G. 3
`PRIOR ART
`
`3
`
`

`

`US. Patent
`
`Jul. 30, 1996
`
`Sheet 3 of 7
`
`5,541,590
`
`on
`
`
`
`x.mmmm.mhlllllllllllllllllllllllIIllllllllllllII.llIl.—
`
`ozFoBmE
`
`mm<mo
`
`
`
`
`
`n.).Sntbo.EDOEOu)._.Dn_z_
`
`ow
`
`0?mm
`
`
`
`szmEm“IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIL5239‘”5:554~62qu“93%Sioux»
`
`
`
`
`
`
`
`
`
`mxémckm>._<>MEET:Eflux;02.5umOF
`
`n
`
`O—
`
`mo<2_
`
`$4:on
`
`_N
`
`4
`
`
`

`

`U.S. Patent
`
`Jul. 30, 1996
`
`Sheet 4 of 7
`
`5,541,590
`
`TO 50
`
`PICKED UP
`"J IMAGE
`
`5
`
`

`

`US. Patent
`US. Patent
`
`Jul. 30, 1996
`Jul. 30, 1996
`
`Sheet 5 of 7
`Sheet 5 of 7
`
`5,541,590
`5,541,590
`
`
`
`6
`
`

`

`US. Patent
`
`Jul. 30, 1996
`
`Sheet 6 0f 7
`
`5,541,590
`
`69:7 z‘
`{1 2i
`
`if
`
`/ ‘47K /7
`80c
`
`\
`
`F l G . 7
`
`80d
`
`FIG.8
`
`7
`
`

`

`US. Patent
`
`Jul. 30, 1996
`
`Sheet 7 of7 -
`
`5,541,590
`
`TO 50
`
`FROM 22
`
`FlG.
`
`8
`
`

`

`5 ,541 ,590
`
`1
`VEHICLE CRASH PREDICTIVE AND
`EVASIVE OPERATION SYSTEM BY NEURAL
`NETWORKS
`
`This application is a continuation of application Ser. No.
`08/097,178, ?led Sep. 27, 1993, now abandoned.
`
`BACKGROUND OF THE INVENTION
`
`This invention generally relates to a system for predicting
`and evading crash of a vehicle, in case of
`In driving a car, a driver unconsciously senses various
`conditions through the objects in view and, as a case may be,
`he must take an action to evade any possible crash or
`collision. However, drivers will often be panicked at the
`emergency. Such a panicked driver may not properly handle
`the vehicle. Besides, the response delay to stimuli in varying
`degrees is inherent to human beings, so that it is physically
`impossible in some cases to evade crash or danger. With this
`respect, various techniques have been developed to evade
`collision by means of mounting on a vehicle a system for
`determining the possibility of crash in a mechanical or
`electrical manner before it happens. Accidents could be
`reduced if drivers had an automatic system or the like
`warning of potential collision situations.
`An automobile collision avoidance radar is typically used
`as this automatic system. Such an automobile collision
`avoidance radar is disclosed in, for example, M. Kiyoto and
`A. Tachibana, Nissan Technical Review: Automobile Colli
`sion-Avoidance Radar, Vol. 18, Dec. 1982 that is incorpo
`rated by reference herein in its entirety. The radar disclosed
`comprises a small radar radiation element and antennas
`installed at the front end of a vehicle. A transmitter transmits
`microwaves through the radiation element towards the head
`way. The microwave backscatter from a leading vehicle or
`any other objects as echo returns. The echo returns are
`received by a receiver through the antennas and supplied to
`a signal processor. The signal processor carries out signal
`processing operation to calculate a relative velocity and a
`relative distance between the object and the vehicle. The
`relative velocity and the relative distance are compared with
`predetermined values, respectively, to determine if the
`vehicle is going to collide with the object. The high possi
`bility of collision results in activation of a proper safety
`system or systems.
`However, the above mentioned radar system has a disad
`vantage of faulty operation or malfunctions, especially when
`the vehicle implementing this system passes by a sharp
`curve in a road. The radar essentially detects objects in front
`of the vehicle on which it is mounted. The system thus tends
`to incorrectly identify objects alongside the road such as a
`roadside, guard rails or even an automobile correctly run
`ning on the adjacent lane.
`An intelligent vehicle has also been proposed that com
`prises an image processing system for cruise and traction
`controls. The views ahead the vehicle are successively
`picked up as image patterns. These image patterns are
`subjected to pattern matching with predetermined reference
`patterns. The reference patterns are classi?ed into some
`categories associated with possible driving conditions. For
`example, three categories are de?ned for straight running,
`right turn and left turn. When a matching result indicates the
`presence of potentially dangerous objects in the picked up
`image, a steering wheel and a brake system are automati
`cally operated through a particular mechanism to avoid or
`evade crash to that object.
`
`10
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`2
`The image processing system of the type described is
`useful in normal driving conditions where the pattern match—
`ing can be effectively made between the image patterns
`successively picked up and the reference patterns for safety
`driving control. However, image patterns representing vari
`ous conditions on the roadway should be stored previously
`in the intelligent vehicle as the reference patterns. Vehicle
`orientation at initiation of crash varies greatly, so that huge
`numbers of reference patterns are required for the positive
`operation. This means that only a time-consuming calcula—
`tion will result in a correct matching of the patterns, which
`is not suitable for evading an unexpected crash.
`It is, of course, possible to increase operational speed of
`the pattern matching by using a large image processor.
`However, such a processor is generally complex in structure
`and relatively expensive, so that it is difficult to apply the
`same as the on-vehicle equipment. In addition, on-vehicle
`image processors, if achieved, will perform its function
`sufficiently only in the limited applications such as a supple
`mental navigation system during the normal cruising.
`
`SUMMARY OF THE INVENTION
`
`An object of the present invention is to provide a system
`for predicting and evading crash of a vehicle using neural
`networks.
`Another object of the present invention is to provide a
`system capable of training neural networks by means of
`collected image data representing scenes along the moving
`direction of a vehicle until the vehicle collides with some
`thing.
`It is yet another object of the present invention to provide
`a system for predicting crash though matching operation
`between data obtained on driving a vehicle and data learned
`by neural networks. It is still another object of the present
`invention to provide a system for evading crash of a vehicle
`using neural networks to actuate a vehicle safety system for
`protecting an occupant.
`In order to achieve the above mentioned objects, the
`present invention is provided with a system for predicting
`and evading crash of a vehicle comprising: an image pick-up
`device mounted on the vehicle for picking up images of
`ever-changing views when the vehicle is on running to
`produce image data; a crash predicting circuit associated
`with the image pick-up device, the crash predicting circuit
`being successively supplied with the image data for predict
`ing occurrence of crash between the vehicle and potentially
`dangerous objects on the roadway to produce an operational
`signal when there is possibility of crash; and a safety driving
`ensuring device connected to the crash predicting circuit for
`actuating, in response to the operational signal, occupant
`protecting mechanism which is operatively connected
`thereto and equipped in the vehicle; wherein the crash
`predicting circuit comprises a neural network which is
`previously trained with training data to predict the possibil
`ity of crash, the training data representing ever-changing
`views previously picked‘up from the image picking-up
`device ‘during driving of the vehicle and just after actual
`crash.
`The neural network comprises at least an input layer and
`an output layer, and the training data are supplied to the
`input layer while the output layer is supplied with, as teacher
`data, ?ags representing expected and unexpected crash,
`respectively, of the vehicle. In addition, the neural network
`may comprise a two-dimensional self-organizing competi
`tive learning layer as an intermediate layer.
`
`9
`
`

`

`3
`Other advantages and features of the present invention
`will be described in detail in the following preferred
`embodiments thereof.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram of a conventional system for
`predicting and evading crash of a vehicle;
`FIG. 2 is a schematic view showing a processing element
`in atypical neural network;
`FIG. 3 is a graphical representation of a sigmoid function
`used as a transfer function for training neural networks;
`FIG. 4 is a block diagram of a system for predicting and
`evading crash of a vehicle using neural networks according
`to the ?rst embodiment of the present invention;
`FIG. 5(a) is a schematic structural diagram of a crash
`predicting circuit in FIG. 4 realized by a neural network of
`three layers;
`FIG. 5(b) shows an example of an input layer consisting
`of a two-dimensional array of processing elements of the
`neural network shown in FIG. 5(a);
`FIGS. 6(a) and 6(b) are exempli?ed views picked up, as
`the training image data supplied to the neural network, at
`different time instances during driving an experimental
`vehicle;
`FIG. 7 is a view showing an example of an image data
`obtained during driving a utility vehicle;
`FIG. 8 is a view showing another example of an image
`data obtained during driving a utility vehicle; and
`FIG. 9 is a block diagram of a system for predicting and
`evading crash using neural networks according to the second
`embodiment of the present invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`20
`
`25
`
`35
`
`A conventional system for predicting and evading crash of
`a vehicle is described ?rst to facilitate an understanding of
`the present invention. Throughout the following detailed
`description, similar reference numerals refer to similar ele
`ments in all ?gures of the drawing.
`In the following description, the term “crash” is used in a
`wider sense that relates to all unexpected tra?ic accidents.
`Accidents other than crash include a turnover or fall of a
`vehicle, with which the phenomenon of “crash” is associated
`in some degrees, so that the term crash is used as a cause of
`tra?ic accidents.
`As shown in FIG. 1, an image pick-up device 21 is
`mounted at a front portion of an automobile 10 to pick up
`ever-changing images as analog image data. This image
`pick-up device 21 is any one of suitable devices such as a
`charge-coupled-device (CCD) camera. The image data are
`subject to sampling for a sampling range AT at a predeter
`mined sampling interval At. The image data are collected up
`to crash. In this event, the image pick-up range of the image
`pick-up device 21 corresponds to a ?eld of view observed
`through naked eyes.
`The image pick-up device 21 is connected to an input
`interface 22. The analog image data obtained by the image
`pick-up device 21 are supplied to the input interface 22. The
`input interface 22 serves as an analog-to-digital converter
`for converting the analog image data into digital image data.
`More particularly, the picked up images are digitized by
`means of dividing the same into tiny pixels (data elements)
`isolated by grids. It is preferable to eliminate noises and
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,541,590
`
`4
`distortions at this stage. The input interface 22 is also
`connected to a speed sensor 23, a steering gear ratio sensor
`24 and a signal processor 30. The speed sensor 23 supplies
`velocity data to the signal processor 30 through the input
`interface 22. The velocity data represents an actual velocity
`of the automobile 10 at the time instant when the image
`pick-up device 21 picks up an image of a view. Likewise, the
`steering gear ratio sensor 24 supplies steering gear ratio data
`to the signal processor 30 through the input interface 22. The
`steering gear ratio data represents an actual steering gear
`ratio of the automobile 10.
`The signal processor 30 comprises a central processing
`unit (CPU) 31, a read-only memory (ROM) 32 and a
`random-access memory (RAM) 33. CPU 31, ROM 32 and
`RAM 33 are operatively connected to each other through a
`data bus 34. To evade potentially dangerous objects, CPU 31
`carries out calculation operation in response to the image,
`velocity and steering gear ratio data given through the input
`interface 22. CPU 31 performs proper functions according to
`programs stored in ROM 32 and RAM 33. The outputs of the
`signal processor 30 is transmitted through an output inter
`face 40. ROM 32 stores a table relating to numerical values
`required for the calculation. It also stores a table represent
`ing operational amount for a safety drive ensuring arrange
`ment 50. On the other hand, RAM 33 stores programs for
`use in calculating an optimum operational amount for the
`safety drive ensuring arrangement 50. A program for this
`purpose is disclosed in, for example, Teruo Yatabe, Auto
`mation Technique: Intelligent Vehicle, pages 22-28.
`The signal processor 30 ?rst determines, according to the
`picked up image data, whether there is a space available on
`the roadway to pass through. When there is enough space to
`pass through and a potentially dangerous object is present on
`the roadway, the signal processor 30 calculates optimum
`operational amount for the safety drive for ensuring arrange
`ment 50 to operate the same. In FIG. 1, the safety drive
`ensuring arrangement 50 consists of a steering actuator 51,
`a throttle actuator 52 and a brake actuator 53. If the signal
`processor 30 determines that it is necessary to operate these
`actuators, it produces steering gear ratio command, set
`velocity command, and brake operation command. The
`steering actuator 51, the throttle actuator 52 and the brake
`actuator 53 are operated depending on the condition in
`response to the steering gear ratio command, the set velocity
`command and the brake operation command, respectively.
`The actuators are for use in actuating occupant protecting
`mechanism such as a brake device. Operation of these
`actuators is described now.
`The steering actuator 51 is a hydraulic actuator for use in
`rotating steering wheel (not shown) in an emergency. In this
`event, the steering wheel is automatically rotated according
`to the‘ steering gear ratio and rotational direction indicated
`by the steering gear ratio command. The operational amount
`of the steering or hydraulic actuator can be controlled in a
`well-known manner through a servo valve and a hydraulic
`pump, both of which are not shown in the ?gure.
`The throttle actuator 52 acts to adjust opening amount of
`a throttle valve (not shown) to decrease speed while evading
`objects or so on.
`The brake actuator 53 performs a function to gradually
`decrease speed of a vehicle in response to the brake opera
`tional command. The brake actuator 53 is also capable of
`achieving sudden brake operation, if necessary.
`As mentioned above, CPU 31 carries out its operation
`with the tables and programs stored in ROM 32 and RAM
`33, respectively, calculating for all picked up image data.
`
`10
`
`

`

`5,541,590
`
`5
`The conventional system is thus disadvantageous in that the
`calculation operation requires relatively long time interval as
`mentioned in the preamble of the instant speci?cation.
`On the contrary, a system according to the present inven
`tion uses image data representing ever-changing views
`picked up from a vehicle until it suffers from an accident.
`These image data are used for training a neural network
`implemented in the present system. After completion of the
`training, the neural network is implemented in a utility
`vehicle and serves as a decision making circuit for starting
`safety driving arrangements to evade crash, which otherwise
`will certainly happen. The neural network predicts crash and
`evades the same by means of properly starting an automatic
`steering system or a brake system.
`A well-known neural network is described ?rst to facili
`tate an understanding of the present invention and, following
`which preferred embodiments of the present invention will
`be described with reference to the drawing.
`A neural network is the technological discipline con
`cerned with information processing system, which is still in
`a development stage. Such arti?cial neural network structure
`is based on our present understanding of biological nervous
`systems. The arti?cial neural network is a parallel, distrib
`uted information processing structure consisting of process
`ing elements interconnected unidirectional signal channels
`called connections. Each processing element has a single
`output connection that branches into as many collateral
`connections as desired.
`A basic function of the processing elements is described
`below.
`As shown in FIG. 2, each processing element can receive
`any number of incoming functions while it has a single
`output connection that can fan out to form multiple output
`connections. Thus the arti?cial neural network is by far more
`simple than the networks in a human brain. Each of the input
`data x1, x2,_ _
`_ , xi is multiplied by its corresponding weight
`coe?icient W1, W2, _
`_
`_ , wi, respectively, and the processing
`element sums the weighted inputs and passes the result
`through a nonlinearity. Each processing element is charac
`terized by an internal threshold or offset and by the type of
`nonlinearity and processes a predetermined transfer function
`to produce an output f(X) corresponding to the sum
`(X—Txi-wi). In FIG. 2, xi represents an output of an i-th
`processing element in an (s-1)-th layer and wi represents a
`connection strength or the weight from the (s-1)-th layer to
`the s-th layer. The output f(X) represents energy condition of
`each processing element. Though the neural networks come
`in a variety of forms, they can be generally classi?ed into
`feedforward and recurrent classes. In the latter, the output of
`each processing element is fed back to other processing
`elements via weights. As described above, the network has
`an energy or an energy function that will be minimum
`?nally. In other words, the network is considered to have
`converged and stabilized when outputs no longer change on
`successive iteration. Means to stabilize the network depends
`on the algorithm used.
`The back propagation neural network is one of the most
`important and common neural network architecture, which
`is applied to the present invention. In this embodiment, the
`neural network is used to determine if there is a possibility
`of crash. When the neural network detects the possibility of
`crash, it supplies an operational command to a safety ensur
`ing unit in a manner described below. As well known in the
`art, the back propagation neural network is a hierarchical
`design consisting of fully interconnected layers of process
`ing elements. More particularly, the network architecture
`
`15
`
`20
`
`25
`
`35
`
`40
`
`50
`
`55
`
`65
`
`6
`comprises at least an input layer and an output layer. The
`network architecture may further comprise additional layer
`or N hidden layers between the input layer and the output
`layer where N represents an integer that is equal to or larger
`than zero. Each layer consists of one or more processing
`elements that are connected by links with variable weights.
`The net is trained by initially selecting small random
`weights and internal thresholds and then presenting all
`training data repeatedly. Weights are adjusted after every
`trial using information specifying the correct result until
`weights converge to an acceptable value. The neural net
`work is thus trained to automatically generate and produce
`a desired output for an unknown input.
`Basic learning operation of the back propagation neural
`network is as follows. First, input values are supplied to the
`neural network as the training data to produce output values,
`each of which is compared with a correct or desired output
`value (teacher data) to obtain information indicating a
`difference between the actual and desired outputs. The
`neural network adjusts the weights to reduce the difference
`between them. More particularly, the difference can be
`represented by a well-known mean square error. During
`training operation, the network adjusts all weights to mini
`mize a cost function equal to the mean square error. Adjust
`ment of the weights is achieved by means of back propa
`gation transferring the error from the output layer to the
`input layer. This process is continued until the network
`reaches a satisfactory level of performance. The neural
`network trained in the above mentioned manner can produce
`output data based on the input data even for an unknown
`input pattern.
`The generalized delta rule derived with the steepest
`descent may be used to optimize the learning procedure that
`involves the presentation of a set of pairs of input and output
`patterns. The system ?rst uses the input data to produce its
`own output data and then compares this with the desired
`output. If there is no difference, no learning takes place and
`otherwise the weights are changed to reduce the difference.
`As a result of this it becomes possible to converge the
`network after a relatively short cycle of training.
`To train the net weights input data (training data) are
`successively supplied to the processing elements in the input
`layer. Each processing element is fully connected to other
`processing elements in the next layer where a predetermined
`calculation operation is carried out. In other words, the
`training input is fed through to the output. At the output layer
`the error is found using, for example, a sigmoid function and
`is propagated back to modify the weight on a connection.
`The goal is to minimize the error so that the weights are
`repeatedly adjusted and updated until the network reaches a
`satisfactory level of performance. A graphical representation
`of sigmoid functions is shown in FIG. 3.
`In this embodiment a sigmoid function as shown in FIG.
`3 is applied as the transfer function for the network. The
`sigmoid function is a bounded differentiable real function
`that is de?ned for all real input values and that has a positive
`derivative everywhere. The central portion of the sigmoid
`(whether it is near 0 or displaced) is assumed to be roughly
`linear. With the sigmoid function it becomes possible to
`establish effective neural network models.
`As a sigmoid function parameter in each layer, a y-direc
`tional scale and a y-coordinate offset are de?ned. The
`y-directional scale is de?ned for each layer to exhibit
`exponential variation. This results in improved convergence
`e?'iciency of the network.
`It is readily understood that other functions may be used
`as the transfer function. For example, in a sinusoidal func
`
`11
`
`

`

`5,541,590
`
`7
`tion a differential coe?icient for the input sum in each
`processing element is within a range equal to that for the
`original function. To use the sinusoidal function results in
`extremely high convergence of training though the hardware
`for implementing the network may be rather complex in
`structure.
`An embodiment of the present invention is described with
`reference to FIGS. 4 through 9.
`FIG. 4 is a block diagram of a system for predicting and
`evading crash of a vehicle using neural networks according
`to the ?rst embodiment of the present invention. A system in
`FIG. 4 is similar in structure and operation to that illustrated
`in FIG. 1 other than a crash predicting circuit 60. Description
`of the similar components will thus be omitted by the
`consideration of evading redundancy. FIG. 5 is a schematic
`structural diagram of the crash predicting circuit 60 illus
`trated in FIG. 4 formed by a neural network of three layers.
`The crash predicting circuit 60 in this embodiment is
`implemented by a neural network architecture of a hierar
`chical design with three layers as shown in FIG. 5(a). The
`input layer 61 consists of n processing elements 61—1
`through 61-n arranged in parallel as a one-dimensional
`linear form. Each processing element in the input layer 61 is
`fully connected in series to the processing elements in a
`hidden layer 62 of the network. The hidden layer 62 is
`connected to an output layer 63 of a single processing
`element to produce an operational command described
`below. FIG. 5(b) shows an input layer consisting of a
`two-dimensional array of processing elements. In this event,
`the image data are supplied to the input layer as a two
`dimensional data matrix of n divisions. Basically, the input
`and the hidden layers can have any geometrical form
`desired. With the two-dimensional array, the processing
`elements of each layer may share the same transfer function,
`and be updated together. At any rate, it should be considered
`that each processing element is fully interconnected to the
`other processing elements in the next layer though only a
`part of which are shown in FIG. 5(a) to evade complexity.
`Referring now to FIG. 6 in addition to FIG. 5, illustrated
`are views picked up, as the image data for use in training the
`neural network. The image pick-up device 21 picks up
`ever-changing images as analog image data as described
`above in conjunction with the conventional system. This
`image pick-up device 21 is also any one of suitable devices
`such as a CCD camera. The image pick-up operation is
`carried out during running of a vehicle at higher speed than
`a predetermined one. The image data are subject to sampling
`for a sampling range AT at a predetermined sampling inter
`val At. The image data are collected before and just after
`pseudo crash. The image pick-up range of the image pick-up
`device 21 corresponds to a ?eld of view observed through
`naked eyes. A view shown in FIG. 6(a) is picked up when
`a station wagon (estate car) 80a on the opposite lane comes
`across the center line. A view shown in FIG. 6(b) is picked
`up when an automobile 80b suddenly appears from a blind
`comer of a cross-street. These ever-changing images are
`collected as the training data for the neural network.
`The image data effectively used for the crash evasive
`purpose are those which allow continuous recognition of the
`ever-changing views before and just after pseudo crash.
`With this respect, the image pick-up device 21 picks up the
`images of a vehicle or other obstructions form a relatively
`short distance. In addition, the picked up images preferably
`are distinct re?ections from the outside views.
`The data elements consisting of one image are simulta
`neously supplied to the input layer 61 in parallel. In other
`
`20
`
`25
`
`30
`
`50
`
`55
`
`60
`
`65
`
`8
`words, each data element is supplied to the respective
`processing element of the input layer 61. The digital image
`data may be normalized before being supplied to the input
`layer 61 to increase a data processing speed. However, each
`processing element of the input layer 61 essentially receives
`the data element obtained by dividing the image data pre
`viously. The data elements are subjected to feature extrac
`tion when supplied to the hidden layer 62.
`In typical image processing, feature extraction is carried
`out according to any one of various methods of pattern
`recognition to clearly identify shapes, forms or con?gura
`tions of images. The feature-extracted data are quantized to
`facilitate subsequent calculations. In this event, separate
`analytical procedure is used for region partitioning or for
`extraction of con?guration strokes. In other words, a par
`ticular program is necessary for each unit operation such as
`region partitioning, feature extraction, vectorization and so
`on. Compared with this, the prediction system according to
`the present invention requires no program based on each
`operation or procedure because a unique algorithm is estab
`lished on completion of network training. This single algo
`rithm allows to perform necessary functions without using
`separate algorithms or programs.
`In a preferred embodiment, the feature extraction is
`directed to the con?guration of an object de?ning the driving
`lanes such as shoulders, curbs, guard rails or the center line.
`The feature may also be extracted on regions such as
`carriageways. The neural network learns these con?gura
`tions and regions during training process. This process is
`continued until the network reaches a satisfactory level of
`performance. The neural network is thus trained while
`carrying out feature extraction on the input image. Weights
`are adjusted after every trial on the quantized image data, so
`that the latest training data is weighted according to the latest
`result of adjustment and then supplied to the hidden layer 62.
`In addition, the neural network can be trained with image
`data including an object at time-varying positions. In this
`event, an