VWGoA - Ex. 1004
`Volkswagen Group of America, Inc. - Petitioner
`
`1
`
`

`
`
`
`U.S. PatentU.S. Patent
`
`
`
`Jun. 18, 1996Jun. 18, 1996
`
`
`
`Sheet 1 of 6Sheet 1 of 6
`
`
`
`5,528,6985,528,698
`
`2
`
`

`
`U.S. Patent
`
`Jun. 13, 1996
`
`Sheet 2 of 6
`
`5,528,698
`
`=o_m_x5
`
`m.o:
`
`
`
`
`$.as.__e2ao9__a$2._mo_==<
`sass“.\.4/
`
`
`§a>._8Ea
`
`an_2_§29__.__<282822:
`
`EE
`
`3
`
`
`
`
`
`

`
`
`
`U.S. PatentU.S. Patent
`
`
`
`Jun. 18, 1996Jun. 18, 1996
`
`
`
`Sheet 3 of 6Sheet 3 of 6
`
`
`
`5,528,6985,528,698
`
`
`
`FIG.4FIG.4
`
`
`<',_<',_
`
`‘<1’‘<1’
`
`4
`
`

`
`U.S. Patent
`
`Mm
`
`M._o_m_.8
`
`6.a__s=_m8_umas
`sm_§_._\m_._E§s_
`
`
`_m.>38=%_.E8u8=E_E
`
`5,528,698
`
`Q.9:
`
`2%
`
`N3
`
`NS
`
`a=._e_8
`
`
`
`a=_§_U228.283E5m:22:_
`
`em
`
`%=E_
`
`Sam
`
`a_E:_8m
`
`Exam
`
`%_.E_
`
`o S
`
`as
`
`5
`
`
`
`

`
`U.S. Patent
`
`Jun. 18, 1996
`
`Sheet 5 of 6
`
`5,528,698
`
`
`
`zo_§_%...§u_§z_as
`
`2.is.___2_
`
`E:2.zoEE.:ss2__:§_
`
`E5:3
`
`
`§=zo:_z:z=s__m_=z_\§§.__M_%_._aNon
`
`
`w§_=z_§2§_._aswe
`
`auxvi
`
`Z2>
`
`['15,
`
`.5
`
`
`=a=_>§_5.§_._.>m+I§_E=>>\_a_>§_=mE
`E22Aflsw
`
`
`§§=z§s_2m..%_ww
`292__z___._fi“%.
`
`
`II
`
`6
`
`
`
`

`
`U.S. Patent
`
`Jun. 18, 1996
`
`Sheet 6 of 6
`
`5,528,698
`
`310
`
`an
`
`814
`
`
`
`
`
`Child in rear-facing
`childcarrier seat in
`
`front passenger seat
`
`ignition
`
`Classify image based on
`interpretation and in
`accordance with other
`inputs
`
`
`
`
`
`822
`
`
`Backpround
`
`sel -test
`
`Corrimunicate classification
`decision to airbag electronics
`
`824
`
`326
`
`FIG. 8
`
`7
`
`

`
`1
`AUTOMOTIVE OCCUPANT SENSING
`DEVICE
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to in—vehicle imaging and
`sensing and, more particularly, to a device that determines
`whether a vehicle passenger seat is unoccupied, is occupied
`by a person, or is occupied by a child seated in a rear-facing
`child carrier.
`
`10
`
`2. Description of Related Art
`As the needs and desires for automobile safety enhance-
`ment features increase, injuries attributable to such safety
`equipment and devices themselves are exacerbated. For
`example, automobile passenger-side inflatable restraints
`(airbags) in automobiles equipped with such supplemental
`safety devices are generally deployed upon a sufliciently
`severe impact of the front section of the vehicle. However,
`it has been found that rear-facing child carriers, i.e., where
`the restrained child faces the passenger seat back of the
`automobile, may be potentially hazardous, and possibly
`perilous, upon deployment of an airbag. Serious damage to
`the child restrained in a typical child carrier could occur
`when the airbag first deploys, shooting out at speeds up to
`200 mph. As illustrated in FIG. 1, the inflating airbag 110
`deploys over the top of the child seat 112 and transfers a
`force to the back of the seated child’s head.
`
`In addition to possible injury of the child due to the direct
`contact with the airbag, injuries could also occur to a child
`in a rear-facing child carrier if a deploying air bag pushes the
`child restraint 112 into the passenger seat back 114.
`Although it will be recognized that a safer mode of travel for
`a child in a child carrier is in the back seat of a vehicle, some
`vehicles do not have back seats (e.g., trucks and small sports
`cars), while in other vehicles the back seat may be unable to
`accommodate a child carrier.
`
`Furthermore, in some vehicles which have small cab
`areas, such as compact pick-up trucks and sports cars,
`deploying may cause damage to the vehicle as well as injury
`to the driver if the vehicle windows are closed during an
`impact. In such instances, if both driver and passenger
`airbags deploy during impact,
`the side windows of the
`vehicle could be shattered and the eardrums of the driver
`
`ruptured due to the rapid air pressure increase in the small
`interior volume. Moreover, the replacement cost of a pas-
`senger airbag after deployment, which would otherwise not
`have been necessary due to the absence of such a passenger,
`may be substantial.
`Thus, to increase child safety in a rear-facing child carrier,
`as well as to lower the cost of unnecessarily deployed
`airbags, a variety of detection technologies have been sug-
`gested. For example, manual override switches may be
`installed to allow a driver to disable the passenger-side
`airbag manually. Such devices, however, become inefl“ective
`in instances where the driver or operator simply forgets to
`turn the switch on or off depending upon the existence of a
`passenger or a child in a rear-facing child carrier in the
`automobile passenger seat. Even such enhancements as
`dashboard indicators or automatic reset arrangements would
`not be foolproof. If a driver transporting a child in a
`rear-facing child carrier makes frequent stops, the require-
`ment that the driver continually manually reset the switch
`could be cumbersome.
`
`Other safety-enhancement schemes for occupant detec-
`tion include radar or ultrasonic technologies. Sensory
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,528,698
`
`2
`
`devices which detect radar or sound waves may be installed
`in the dashboard or in the passenger seat itself. However, if
`the dashboard is blocked or the seat is covered, accurate
`detection of a passenger in the passenger seat would be
`hindered. Moreover, it has been found that consumers gen-
`erally do not like the idea of “beams and waves” being
`directed at them. Consequently, such approaches are typi-
`cally not preferred.
`
`SUMMARY OF THE INVENTION
`
`Accordingly, an object of the present invention is to
`provide an automotive occupant safety system to enhance
`the safety of occupant restraint systems in vehicles. A further
`object of the invention is to provide a communication and
`detection system in which airbag control electronics in a
`vehicle communicate with the present invention to deter-
`mine the preferred airbag deployment depending on the
`occupancy status of the seating area.
`In accordance with these and other objects, an occupant
`safety system in accordance with the present
`invention
`includes image processing technology in conjunction with
`other sensors, such as seat belt extension, vehicle speed,
`door open status, seat weighting, etc.,
`to determine the
`preferred airbag deployment. In preferred embodiments, the
`image processing system generally comprises a phototrans-
`istor array sensor and lens assembly with image processing
`electronics to acquire a visual representation of the passen-
`ger seat area. The objects in the field of view are then
`discriminated to determine whether a person or a child in a
`rear-facing child carrier is present in the passenger seat.
`The photodetector array principally acquires images
`based upon photon reception in the near-infrared and visible
`light spectrum. To operate in dark or uneven lighting con-
`ditions without distracting the driver with a visible light
`source, the present invention directs an infared source at the
`passenger seat area. The infared emitter is mounted on a
`circuit board which preferably includes a processing ele-
`ment, memory devices that contain operational software and
`fault code information, interfaces to the vehicle electronic
`systems, and a power supply.
`Embodiments of the present invention are preferably
`located within the interior of a vehicle, and mounted in the
`roof headliner or overhead console. An occupant sensing
`system in accordance with the present invention is located to
`maximize viewing of the passenger seat area and minimize
`potential viewing obstructions such that the passenger-side
`airbag deployment can be optimized depending upon certain
`conditions.
`
`Other objects and aspects of the invention will become
`apparent to those skilled in the art from the detailed descrip-
`tion of the invention which is presented by way of example
`and not as a limitation of the present invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 depicts a typical operation of an inflatable restraint
`safety system in conjunction with a child carrier.
`FIG. 2 shows a perspective view of an automotive occu-
`pant sensor system in accordance with a preferred embodi-
`ment of the present invention.
`FIG. 3 shows one aspect of the sensor system in accor-
`dance with an embodiment of the invention.
`
`FIG. 4 shows an exploded view of the sensor arrangement
`of the automotive occupant sensor system embodiment of
`FIG. 3.
`
`8
`
`

`
`3
`
`4
`
`5,528,698
`
`FIG. 5 shows a lens arrangement in accordance with an
`embodiment of the present invention.
`FIG. 6 is a flow diagram of the occupant sensor image
`processing scheme in accordance with an embodiment of the
`invention.
`
`FIG. 7 is a block diagram of the occupant sensor accord-
`ing to a preferred embodiment of the present invention.
`FIG. 8 is a flow diagram of the general operation of the
`automotive occupant sensor system in accordance with an
`embodiment of the invention.
`
`10
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`The following is a description of the best presently
`contemplated mode of carrying out the invention. In the
`accompanying drawings, like numerals designate like parts
`of the several figures. This description is made for the
`purpose of illustrating the general principles of embodi-
`ments or the invention and should not be taken in a limiting
`sense. The scope of the invention is determined by reference
`to the accompanying claims.
`An automotive occupant sensor system 200 in accordance
`with a preferred embodiment of the present invention is
`shown generally in FIG. 2. Embodiments of the present
`invention are directed to the communication and coordina-
`tion with a vehicle airbag electronic control module 212 to
`periodically sense the presence or absence of a forward-
`facing human being in the front outboard, passenger seat of
`the vehicle. The existence or absence of a passenger then
`allows the occupant sensor system to make a determination
`whether to enable, inhibit, or otherwise modify the initiation
`or deployment of a passenger-side airbag during a collision
`or other impact.
`
`Preferred embodiments of the present invention perform
`a restraint system sensing function to control the deployment
`of the passenger-side airbag such that it is deployed on
`demand only if a forward-facing human being is present in
`the passenger seat. More particularly, embodiments of the
`invention are directed to differentiating between a child
`seated in a rear-facing child carrier and a forward-facing
`human in the passenger seat. It will be recognized that in
`alternate embodiments of the present invention, deployment
`of an air bag may be disabled only upon detection of a child
`sitting in a rear-facing child carrier in the passenger seat of
`the vehicle.
`In accordance with such embodiments,
`the
`airbag would deploy upon impact or collision if any for-
`ward-facing person or object
`is disposed in the sensor
`viewing area, e.g., the passenger seat. It also will be recog-
`nized that in alternate embodiments of the present invention,
`the relative size and position of occupants will be deter-
`mined by the occupant sensing system, with the information
`used to optimize the initiation, rate, and inflated volume of
`the deployed airbag.
`As shown in FIG. 2, preferred embodiments of the present
`invention 200 are located in the upper console 202 of the
`vehicle, above the driver’s line of vision as well as above the
`passenger seat 204. Mounting embodiments of the automo-
`tive occupant sensor system in the overhead console 202 is
`generally preferred for supporting embodiments of the
`present invention to allow the features of the most of the
`vehicle interior seating area to be seen. The configuration of
`the sensor embodiments must be such that arms and objects
`do not block the viewing area. Accordingly, as indicated in
`FIG. 2, embodiments of the present invention are preferably
`mounted at or near the center console area 202, rear-view
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`mirror (not shown), or pillars 206, 208, and 210, near the
`headliner on either the passenger or driver’s side of the
`vehicle. It will be recognized that other mounting configu-
`rations may be used wherever the sensor embodiments are
`able to adequately view the interior of the vehicle, and
`accurately distinguish passengers and/or objects.
`The general operation of preferred embodiments of the
`invention is described with reference to FIG. 3. An image
`310 (shown as a woman holding a book) is viewed through
`a lens 312. The lens 312 is provided to evenly focus the
`image 310 across a photodetector array 314. The photode-
`tector 314 may include analog and or digital processing
`capabilities. In preferred embodiments, the interface elec-
`tronics or image processing algorithms are incorporated into
`the sensor 314 itself or the sensor electronics, e.g., circuit
`board 410 of FIG. 4.
`
`Similarly, an analog-to-digital converter 316 coupled to
`the output of the photodetector array 314 may be incorpo-
`rated within the sensor or photodetector array 314. The
`output of the A/D converter 316 is input into a general
`purpose processor 318 which performs the image process-
`ing. Any image processing that was not accomplished by the
`photodetector array electronics would be performed by the
`processor 318, which ultimately determines whether the
`image of the passenger seat area represents a child seated in
`a rear-facing child carrier.
`More particularly, some of the components of an embodi-
`ment of the occupant sensor module 200 are shown in FIG.
`4. In the illustrated embodiment, multiple electronic com-
`ponents are mounted on a circuit board 410. A more detailed
`description of the circuit board electronics is discussed
`below-with reference to FIG. 7. In a preferred embodiment,
`the circuit board is made with integral interconnections of a
`flexible material allowing it to conform to restricted volume
`limitations for the system package. Preferably, the flexible
`circuit board 410, the sensor array/lens mount/lens assembly
`414, and the IR source 416 are manufactured as a single unit,
`so that the unit can be quickly and securely attached into a
`housing 418 using the appropriate fasteners. The housing
`may be formed of plastic, metal, or composite material,
`which may be integral with the mounting location, or may
`be removable or otherwise portable.
`In preferred embodiments,
`the housing and electronic
`sensor module 200 is securely and firmly mounted to the
`vehicle interior. The circuit board 410 is preferably installed
`in the housing 418 and coupled to the power source and
`vehicle electronics via a connector 412 which typically
`varies according to the vehicle manufacturer's specifica-
`tions. Thus, for example, the power supply for the sensor
`414 and all of the electronics on circuit board 410 can be
`shared with other vehicle control electronics. Preferably, the
`sensor 414 and circuit board 410 arrangement includes a
`discrete or other method of fault indication to communicate
`if a correct decision carmot be made by the system due, for
`example,
`to a blocked view or an internal fault. Other
`connection lines may include an enable/inhibit line to indi-
`cate to the airbag electronics 212 (FIG. 2) whether a forward
`facing passenger has been detected. The discrete indication
`electronics may include industry standard interfaces to pro-
`vide connection to other processing electronics within the
`vehicle.
`
`The sensor assembly 414 includes a lens 420, lens mount
`422, and the sensor itself 510. Although in preferred
`embodiments, the packaging of these components may be
`coupled integrally in the manufacturing process, the func-
`tions performed by each component are generally separate.
`
`9
`
`

`
`5
`
`6
`
`5,528,698
`
`For example, for low-cost, high-volume manufacturing, the
`sensor/lens/mounting assembly may be produced together in
`a single unit, e.g., an injection-molded acrylic embedded
`with the necessary contacts for electrical connections. By
`producing the lens and lens mounting integrally, manufac-
`turing costs can be reduced by minimizing assembly vari-
`ances to conform to required tolerances. Such a single unit
`configuration allows consistent placement and alignment of
`the individual components. It will be recognized, however,
`that a variety of other materials and substances may be used
`in the manufacture of the sensor, lens, and lens mounting
`assembly in a single unit, or as separable components.
`According to embodiments of the invention, an infrared
`source 416 provides illumination of the vehicle interior
`preferably by infrared LEDs, a bulb with proper filtering, or
`by another lighting device which can adequately illuminate
`the entire sensor field of view, enabling image acquisition in
`the absence of visible light. The sensor 414 is sensitive to
`illumination in the near-infrared band which is generally not
`visible to humans. An infrared bandpass filter may be placed
`in front of the lens 420 or be integrated with the system
`housing 418 to limit the operation of the sensor to the
`near-infrared region and to obscure the internal components
`of the system from vehicle occupants. The sensor 414 is
`preferably an image based sensor and ultimately translates
`captured light energy into a gray scale image. For example,
`the sensor array and support electronics may be used to
`transmit a 64x64 pixel image, with each pixel being repre-
`sented by one of a possible 256 gray shades to the processing
`components of the sensor module circuit board 410.
`FIG. 5 shows a more detailed side view of the sensor
`assembly 414 comprising a lens 420 mounted to a photo-
`detector array/circuit board 510. A cable 512 couples the
`sensor to the circuit board 410 (shown in FIG. 4). In
`preferred embodiments, the circuit board 410, photodetector
`array/circuit board 510 and the interconnection cable 512 are
`manufactured in a flex circuit board to simplify intercon-
`nections and assembly labor. More particularly, the sensor
`414 includes a photodetector array 510 which is preferably
`formed of multiple silicon phototransistors. The photode-
`tector array 510 acts as a light gathering tool which may
`include image processing capabilities of its own. The pho-
`todetector array 510 (shown in FIG. 5) is preferably of
`conventional CMOS design and therefore could include the
`functions of other (normally separate) electronic compo-
`nents.
`
`A processor 724 (shown in FIG. 7) included on the circuit
`board 410 analyzes the raw analog or digital image, and
`performs a histogram equalization to optimize the contrast
`between black and white. This is performed by shifting the
`whites whiter and blacks darker, and distributing gray
`shades evenly from white to black. Likely edges are then
`distinguished, and the angles of the edges are determined
`such that comparison may be made with stored templates
`describing the classifications of different objects, e.g., child
`carrier seats and empty passenger seats. Thus, since child
`seats will fall into a variety of difierent categories based on
`their general design, the distinguished refined edges of the
`image are compared to the different templates to determine
`if any match the characteristics of the current acquired
`image. When a match is found between the current image
`and a template, a degree of confidence is assigned to the
`match.
`
`Since movement or shifting of child restraints or an empty
`passenger seat will be limited, changes between consecutive
`captured images that represent dynamic motion (once the
`eifects of varying lighting conditions are eliminated) can be
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`used to gain confidence in the detection of human occupants.
`In addition, the history of prior classifications and their
`associated confidence values can be constantly analyzed to
`improve the reliability of the current classification decision
`generated by the occupant sensor. Thus,
`if the overall
`confidence, or calculated accuracy, is high based upon the
`conclusion that the sensed image is, for example, a rear-
`facing child carrier, the processor will communicate this
`status to the airbag control electronics to inhibit the deploy-
`ment of the passenger-side airbag.
`FIG. 6 describes steps for processing and classifying
`images viewed by the sensor according to a preferred
`embodiment of the present invention. A sensed input image
`610 is acquired by the photodetector array. The raw sensed
`image 610 is then input into a preprocessor 612 which
`applies certain intensity mapping functions to provide
`enhanced details, edges, and normalize intensity which
`minimizes threshold adaptation in later stages. The prepro-
`cessed image 612 then becomes the sub-image space 620
`which is input into a segmentor 622. The segmentor 622
`includes a spoke filter which can identify regions in an
`image which include parts of objects, passengers, and poten-
`tial “sunspots” which could lead to incorrect identification of
`false edges. The filter allows differentiation and clarification
`of unnatural edges due to things like shadows and sunlight
`streaming across the image space.
`The output of the segmentor 622 becomes an enhanced
`image 630. The enhanced image is then input into an interest
`point locator (IPL) 632. Preferably, the IPL 632 is a Sobel
`edge operator which examines the image and identifies the
`probable essential edges of the image. The edge operator
`632 also identifies the angles of the edges based upon the
`positions of the edge pixels of adjacent neighbors. Thus, the
`[PL 632 indicates whether the edges are vertical, horizontal
`or at 45° angles relative to the enhanced image to classify the
`image in accordance with stored data representing typical
`shapes and configurations of conventional child carriers,
`empty seats, and passengers.
`In preferred embodiments, typical shapes and configura-
`tions for comparison are included in a template or reference
`library indicating the various types of objects or images to
`be diiferentiated and distinguished. For example, in pre-
`ferred embodiments, the template or reference library is
`preferably stored in at least one memory device included in
`the sensor electronics on the circuit board 410 (FIG. 4). It
`will be recognized that a variety of filters may be imple-
`mented for the IPL 632. Both software and hardware ver-
`sions of edge detection filters have been examined, and may
`be incorporated into embodiments of the invention to maxi-
`mize the edge detection, distinction, and differentiation.
`As shown in FIG. 6, the output of the edge operator which
`performs the functions of the IPL 632 along with the output
`of the spoke filter becomes the feature vector space 640. The
`feature vector space 640 is then input
`into the feature
`correlator 642 which tries to match the feature vector space
`640 to the different stored templates, or reference library,
`which contains data describing idealized child carrier seat,
`empty seat, and passenger classifications. More particularly,
`the reference library includes specific descriptive informa-
`tion on image related characteristics of each object classi-
`fication. Consequently, groups of basic characteristics
`describing the edge shapes of such objects can be formulated
`for comparison with the sensed images.
`In preferred embodiments of the invention, the feature
`correlator 642 is capable of translating the reference library
`of data by shifting or rotating the templates in space to
`
`10
`
`10
`
`

`
`7
`
`8
`
`5,528,698
`
`account for inaccuracies in sensor alignment within the
`vehicle,
`the range of possible passenger seat positions,
`improper installation of child carrier seats, and random
`movement of passengers. The feature correlator 642 is
`further capable of examining only particular segments of the
`templates to correlate the segments individually or as a
`whole with the feature vector space 640. Thus, once the
`acquired sensed image has been distinguished with high-
`lighted or differentiated edges, the image can be matched
`with the data stored in the reference library. Because a
`family of child carriers has been stored in preferred embodi-
`ments, the sensor processor can adjust for movement or
`slight rotation of the person or object in the vehicle front
`passenger seat.
`
`Following the correlation, the output of the feature corr-
`elator 642 is directed into the classifier feature vector space
`650. The classifier feature vector space 650 provides a
`higher level of the decision-making process, and can further
`distinguish the sensed image in light of variations or incon-
`sistencies in lens angle, position, and focus. The feature
`vector space 650 is then input into the classifier 652, which
`provides an enhanced function of classifying the correlated
`image. Unlike the feature correlator 642 in which the feature
`vector space 640 is compared to the different stored tem-
`plates, the classifier 652 utilizes confidences assigned to
`each object to determine the most probable image classifi-
`cation.
`
`The classifier 652 also refers to other external inputs, such
`as history information or apriori information. For example,
`after observing five images in the passenger seat of a
`vehicle, the sensor may determine with a 99% confidence
`that a child carrier seat is being observed. For the next
`image, it may also determine with 77% confidence that the
`seat is empty. Using the information that a rear.facing child
`carrier was most likely observed in the past five images, a
`single empty seat indication may be questioned until more
`consistent and definite verification is achieved. Thus, the
`classifier 652 is capable of adapting its observations and
`determinations according to different vehicle seat positions
`and viewing perspectives.
`
`The output of the classifier 652 is ultimately communi-
`cated to the vehicle electronics and possibly the driver at the
`decision output 660. At this point, the communication of a
`final decision indicating the class of the image is made. In
`preferred embodiments, the confidence level, in terms of
`percentages, is translated into an airbag enable/disable deci-
`sion which is communicated to the vehicle airbag control
`electronics. Consequently, upon a determination,
`for
`example, that an image is a rear-facing child carrier seat, the
`classifier 652 elfectively disables the deployment of the
`passenger-side airbag upon an impact.
`FIG. 7 is a detailed description of preferred embodiments
`of the circuit board 410 shown in FIG. 4. A sensor 710 is
`coupled to an application specific integrated circuit (ASIC)
`708 that implements numerous functions. Part of the ASIC
`708, the sensor timing and control unit 712, is provided with
`several lines connecting the ASIC 708 to the sensor 710. The
`timing and control unit 712 provides horizontal and vertical
`controls necessary to acquire an image from the sensor pixel
`by pixel, and format the image into random access memory
`(RAM) 734. The image data is then combined to represent
`a complete image.
`A clock 714 is used to log time for a real time clock and
`diagnostic time logger (RTC/DTL) 716. The RTC/DTL 716
`provides the real time of events to a fault detection and
`diagnostic tool interface 718. Whenever the system is opera-
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`tional, the fault detection circuitry 718 continuously per-
`forms a se1f—test diagnostic function to verify the proper
`operation of the system. Preferably,
`the diagnostic tool
`interface is provided by the vehicle manufacturer to diag-
`nose problems. The diagnostic tool interface 718 represents
`the circuitry which accepts commands from the external
`diagnostic equipment provided by the vehicle manufacturer
`to initiate self-tests for calibration and communicate the
`determined status back to the diagnostic tool.
`An external bus interface 720 is coupled to the fault
`detection and diagnostic tool interface 718. The bus inter-
`face 720 is provides communications to other on-board
`electronics as required by the vehicle manufacturer. For
`example, this would represent the electronics necessary to
`perform as a SAEJ1850 bus interface, if chosen by the
`vehicle manufacturer as the data communication interface. It
`will be recognized, however, that other buses or connecting
`devices may be utilized.
`Two discrete lines extending from the fault detection and
`diagnostic tool interface 718 indicate whether a fault has
`been detected (line 730) and/or whether the airbag should be
`enabled or disabled (line 732). The fault line 730 signifies to
`the vehicle that a self-test problem has been diagnosed or
`that the image is not operational. The enable/inhibit line 732
`communicates with the airbag electronic control module to
`indicate that the airbag deployment should be disabled when
`a rear-facing child carrier seat or empty seat condition has
`been detected.
`
`The microcontroller macro 724 performs image process-
`ing on the image acquired by the sensor 710. A supervisorl
`watchdog timer 726 checks the system periodically to ensure
`that the ASIC 708 is functioning properly. In preferred
`embodiments, the watchdog timer 726 operates only in the
`key-on position, i.e., when the ignition is switched on, to
`reduce necessary battery power. A crystal 742 acts as the
`reference clock for the microcontroller 724. An automobile
`battery provides a nominally 12 volt DC power source that
`is regulated to -5 volts DC by the voltage regulator 740 for
`use by the occupant sensor electronics. The regulated +5 volt
`DC is supplied to the watchdog timer 726 as well as an
`EEPROM 736. The EEPROM -736 stores fault code infor-
`mation provided to the vehicle electronics that monitor
`status. RAM 734 may be a separate device or included in the
`ASIC 708 if sufficient capacity exists.
`Although in preferred embodiments of the invention, the
`ASIC 708 includes the several components as discussed
`above and illustrated in FIG. 7, it will be recognized that the
`various components. e.g., the clock, interfaces, microcon-
`troller, may be coupled together as discrete components.
`FIG. 8 is a flow diagram of the operation of a preferred
`embodiment of the present invention. Typically, three sce-
`narios are possible. In the first scenario, the front passenger
`seat is empty 810. Second, there is a rear-facing child carrier
`seat in the front passenger seat area 812. In the third scenario
`814, all other seating arrangements are represented, e.g.,
`forward facing passengers, child seats, and other objects. In
`general, if scenarios 810 and 812 are observed, the passen-
`ger side airbag will be disabled. In any other expected case
`814, the airbag would be enabled.
`In operation, when the driver of the vehicle turns on the
`ignition (Step 816), the decision-making process begins; at
`which time, the sensor initially acquires the image. The
`image will be in one of the three potential scenarios 810,
`812, or 814. Sensor embodiments of the present invention
`then interpret and classify (step 820) the image according to
`the process described with reference to FIG. 6. Steps 818
`
`11
`
`11
`
`

`
`9
`
`10
`
`5,528,698
`
`through 822 are repeated as rapidly as possible, the duration
`of each frame being a function of the software performed
`and the processing capability of the occupant sensor.
`Based upon the classification result of the particular frame
`or image, and considering any other inputs or additional
`stored information,
`it
`is determined whether the airbag
`should be enabled or disabled. Additional information may
`include inputs from other types of sensors. For example,
`dashboard-mounted ultrasonic sensors, a seat pressure sen-
`sor, or a seatbelt latch-sensing switch, among others, could
`be integrated. In addition, other inputs may include the past
`history of the most recent 5-10 decisions, as well as the
`confidence factors which may be associated with the earlier
`classifications. Accordingly, based on the information
`obtained and input, a decision of the current status of the
`passenger seat area can be made. The preferred embodi-
`ments of the present invention then communicate the deci-
`sion whether to enable or disable airbag deployment to the
`vehicle airbag electronics (212 in FIG. 2).
`Concurrent with the image processing, preferred embodi-
`ments of the present
`invention include a self-checking
`routine to verify continuous proper and reliable operation.
`The self-tests may include electronics checks, or basic image
`collection verification in which periodic checks of certain
`background features, or analysis of a histogram of pixel
`intensity is performed.
`In accordance with embodiments of the present invention,
`the preferred mounting location is in the overhead console of
`the vehicle interior headliner. Depending upon the vehicle
`configuration, alternate mounting locations may be prefer-
`able for more advantageous sensor viewing or for security-
`related concerns. As indicated in FIG. 2, pillars A, B, and C,
`206, 208, and 210 respectively on either the passenger or
`driver’s side of the vehicle, may be used for mounting the
`sensing module 200. In addition, it will be recognized that
`other arrangements may be implemented. For example,
`sensor embodiments may be included within the vehicle’s
`dome light stru