(12) United States Patent
`PrestOn et al.
`
`USOO6629033B2
`(10) Patent No.:
`US 6,629,033 B2
`(45) Date of Patent:
`Sep. 30, 2003
`
`(54) OPEN COMMUNICATION SYSTEM FOR
`REAL-TIME MULTIPROCESSOR
`APPLICATIONS
`
`(75) Inventors: Dan Alan Preston, Bainbridge Island,
`WA (US); Robert Pierce Lutter,
`Tacoma, WA (US)
`(73) Assignee: Medius, Inc., Seattle, WA (US)
`-
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/841,753
`
`(22) Filed:
`
`Apr. 24, 2001
`
`(65)
`
`Prior Publication Data
`US 2002/0156564 A1 Oct. 24, 2002
`
`(51) Int. Cl. .................................................. G06F 7700
`(52) U.S. Cl. ............................. 701/70; 701/33; 701/36;
`307/9.1
`(58) Field of Search .............................. 701.70, 33.36,
`701/45, 41; 340/425.5; 307/9.1, 10.1, 10.2;
`709/400
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`6,161,071. A 12/2000 Shuman et al. ............... 701/48
`6.243450 B1
`6/2001 Jansen et al.
`
`FOREIGN PATENT DOCUMENTS
`WO96/24229
`8/1996
`W:
`i9.
`WO99/65183
`12/1999
`WOO1/30061
`4/2001
`WOO1/58110
`8/2001
`
`WO
`W
`WO
`WO
`WO
`
`OTHER PUBLICATIONS
`Product description of Raytheon RT Secure, “Embedded
`Hard Real-Time Secure Operating System”, Copyright
`2000, pp. 1-2.
`Product description of Raytheon RT Secure, Copyright
`2001, pp. 1-2.
`Product description of Raytheon RT Secure, “Development
`Environment”, Copyright 2001, pp. 1-2.
`Product description of Raytheon Electronic Systems (ES),
`Copyright 2002, pp. 1-2.
`H. Chung, L. Ojeda, and J. Borenstein, “Sensor Fusion for
`Mobile Robot Dead-reckoning with a Precision-calibrated
`Fiber Optic Gyroscope', 2001 IEEE International Confer
`ence on Robotics and Automation, Seoul, Korea, May
`21-26, pp. 1-6.
`A. Das, R. Fierro, V. Kumar, J. Ostrowski, J. Spletzer, and
`C. Taylor, “A Framework for Vision Based Formation Con
`trol', IEEE Transactions on Robotics and Automation, vol.
`XX, No. Y, 2001, pp. 1-13.
`J. Takezaki, N. Ueki, T. Minowa, H. Kondoh, “Support
`System for Safe Driving A Step Toward ITS Autonomous
`Driving”, Hitachi Review, vol. 49, No. 3, 2000, pp. 1-8.
`(List continued on next page.)
`Primary Examiner-Yonel Beaulieu
`(74) Attorney, Agent, or Firm Marger Johnson &
`McCollom, PC
`ABSTRACT
`(57)
`A communication System for a mobile vehicle, home, or
`office environment includes multiple processors. The mul
`tiple processors each run an Open Communication System
`that controls how data is transferred between processors
`based on data content as opposed to the links that connect
`the processors together. The open communication System
`enables data or messages to be effectively transferred and
`processed for real-time applications or other Server based
`applications that may be running on the multiple processors
`in a Secure environment regardless of processors, locations,
`or data linkS.
`
`30 Claims, 9 Drawing Sheets
`
`22
`
`14
`ENGINE
`
`24-
`
`2O
`BRAKE
`BRAKE
`
`28
`
`\
`
`2
`
`is "
`
`oc
`OC
`
`26
`
`1 O
`USER
`INTERFACE
`
`1O
`
`1O
`
`J2
`
`JSO
`
`AHM, Exh. 1013, p. 1
`
`

`

`US 6,629,033 B2
`Page 2
`
`OTHER PUBLICATIONS
`S.G. Goodridge, “Multimedia Sensor Fusion for Intelligent
`Camera Control and Human-Computer Interaction', Dis
`Sertation Submitted to the Graduate Faculty of North Caro
`lina State University in partial fulfillment of the require
`ments for the degree of Doctor of Philosophy in Electrical
`Engineering, Raleigh, NC, 1997, pp. 1-5.
`M. Chantler, G. Russel, and R. Dunbar, “Probabilistic Sen
`sor Fusion for Reliable WorkSpace Sensing”, pp. 1-14.
`ISIS Project: Sensor Fusion, Linkoping University Division
`of Automatic Control and Communication Systems in coop
`eration with SAAB (Dynamics and Aircraft), 18 pp.
`Hitachi Automated Highway System (AHS), Automotive
`Products, Hitachi, Ltd., Copyright 1994-2002, 8 pages.
`Vehicle Dynamics Lab, University of California, Berkeley,
`funded by BMW, current members: D. Caveney and B.
`Feldman," Adaptive Cruise Control”, 17 pages.
`Counterair: The Cutting Edge, Ch. 2 “The Evolutionary
`Trajectory The Fighter Pilot-Here to Stay?” AF2025
`v3c8-2, Dec. 1996, pp. 1-7.
`Counterair: The Cutting Edge, Ch. 4“The Virtual Trajectory
`Air Superiority without an "Air" Force'?” AF2025 v3c8–4,
`Dec. 1996, pp. 1-12.
`
`TNO FEL Annual Review 1998: Quality works, 16 pages.
`Boeing News Release, “Boeing Demonstrates JSF Avionics
`Multi-Sensor Fusion”, Seattle, WA, May 9, 2000, pp. 1-2.
`Boeing Statement, “Chairman and CEO Phil Condit on the
`JSF Decision”, Washington, D.C., Oct. 26, 2001, pp. 1-2.
`Ada 95 Transition Support-Lessons Learned, Sections 3, 4,
`and 5, CACI, Inc. -Federal, Nov. 15, 1996, 14 pages.
`Joint Strike Fighter Terrain Database, ets-news.com “Simu
`lator Solutions' 2002, 3 pages.
`MSRC Redacted Proposal, 3.0 Architecture Development,
`pages 1-43.
`Powerpoint Presentation by Robert Allen-Boeing Phantom
`Works entitled “Real-Time Embedded Avionics System
`Security and COTS Operating Systems”, Open Group Real
`Time Forum, Jul. 18, 2001, 16 pages.
`Green Hills Software, Inc., “The AdaMULTI 2000 Inte
`grated Development Environment”, Copyright 2002, 7
`pageS.
`Luttge, Karsten; "E-Charging API: Outsource Charging to a
`Payment Service Provider” IEEE; 2001 (pp. 216–222).
`* cited by examiner
`
`AHM, Exh. 1013, p. 2
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 1 of 9
`
`US 6,629,033 B2
`
`O |
`
`Z |
`
`O |
`
`\ om
`
`
`
`
`
`?HO LINOW|
`
`AHM, Exh. 1013, p. 3
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 2 of 9
`
`US 6,629,033 B2
`
`ALTYOIYd vv
`SYSAV1OLC#YOSSIIONdYVONOILVOINNWNOOL#YOSSAD0UdYVO
`OrYAIOVNVAWALTYNOASYSOVNVW g¢ ALTYNOAS
`
` CvYIOVNVAWONIO9O1YSIVNVWONID9O1
`
`
`
`YSJOVNVAWALIYOIYdYADVNVW
`
`(S)NOILVOTIddv(S)NOILVOTIddv
`JOVAYALNIYVOJOVAYSLINIYVO
`
`
`
`(IdV)YSONVNVAN(IdV)YSOVNVA
`
`
`
`
`
`
`
`
`
`WALSASSNILVdsd0WSALSASSNILVdsdO
`
`SANTI1ZSUVMOYVHSANT1ZSYVMGYVH
`
`JOVAYALNIJOVAYSALNI
`
`Be
`
`YO4
`
`WALSAS90
`
`AHM, Exh. 1013, p. 4
`
`AHM, Exh. 1013, p. 4
`
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 3 of 9
`
`US 6,629,033 B2
`
`
`
`IDENTIFY PRIORITY
`VALUE ASSOCIATED
`WITH THE MESSAGE
`
`RANK PRIORITY
`FOR IDENTIFIED
`MESSAGE
`
`REJECT
`MESSAGE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IDENTIFY PRIORITY
`LABEL FOR
`IN COMING MESSAGE
`
`68
`
`PRIORITY SUFFICIENT
`TO RUN ON
`PROCESSOR
`
`
`
`7O
`
`
`
`RANK PRIORITY
`WITH PRIORITIES FOR
`OTHER MESSAGES
`
`72
`
`SCHEDULE PROCESSING
`OF MESSAGE ACCORDING
`TO PRIORITY RANKING
`
`
`
`
`
`
`
`
`
`AHM, Exh. 1013, p. 5
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 4 of 9
`
`US 6,629,033 B2
`
`42
`\
`
`
`
`
`
`
`
`RECEIVE MESSAGE
`
`8O
`
`READ LOGGING
`LABEL IN MESSAGE
`
`82
`
`MESSAGE RECQUIRE
`LOGGING 2
`
`84
`
`LOG MESSAGE IN
`MEMORY IDENTIFIED
`BY LOGGING LABEL
`
`86
`
`SEND MESSAGE TO
`NEXT COMMUNICATION
`MANAGER
`
`88
`
`FIG.5
`
`AHM, Exh. 1013, p. 6
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 5 of 9
`
`US 6,629,033 B2
`
`9 O
`
`RECEIVE MESSAGE
`
`92
`
`94
`
`SECURITY
`VALUE IDENTIFIED
`IN MESSAGE
`
`SECURITY VALUE Y
`REQUIRED FOR
`APPLICATIONS
`
`SECURITY VALUE
`ACCEPTABLE FOR
`
`CURRENT isney
`
`
`
`
`
`
`
`DROP
`MESSAGE
`
`PASS MESSAGE TO
`NEXT COMMUNICATION
`LAYER
`
`FIG.6
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`AHM, Exh. 1013, p. 7
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 6 of 9
`
`US 6,629,033 B2
`US 6,629,033 B2
`
`YLI
`
`cOl
`
`Odl
`
`ZL
`
`-20-N 138V1
` 6ELVIVONOILVOIIddv
`——————————————
`[z+Ort
`gi,2SNorsn]|BLL
`ac4GttTouo1
`JO3WILALIYOIYd|ONID9NOT|ALTYNOASANT]
`
`INAINAYNSVIWN
`OLLol
`LINYSLHS
`„–^??????????????????????????????????????????????????N.
`S1Z8V1001 m—vV—V"9@V~
`
`
`1598V1138V1SYJOVIH
`SZl©ZLOLOl
`PatPoe
`TViIuasJNVYE|~ZZ1|900|ToeTLEvavy
`
`8Cl9¢1OlZol
`90dObaay
`
`
`|ayvug|—>
`L014
`8Dld
`
`letVLVOWALSASSNILVUSdO
`ao,7La901
`Sel
`900\\
`
`oc1'8Z
`
`
`
`QVOIAVd
`
`ÇÇ I ST||E|8}\/T OO
`
`AHM, Exh. 1013, p. 8
`
`AHM, Exh. 1013, p. 8
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 7 of 9
`
`US 6,629,033 B2
`
`150
`
`es
`
`154
`
`156
`
`152
`
`
`
`
`
`158
`
`SMALL FAR
`AWAY OBJECT APPLICATION
`DETECTED
`
`LARGE, CLOSE
`IN OBJECT
`DETECTED
`
`17O
`
`16 O
`
`162
`
`164
`
`ADD
`SECURITY
`LABEL
`
`OC SYSTEM
`10
`
`ADD
`SECURITY
`LABEL
`
`DON'T
`LABEL FOR
`LOGGING
`
`- LABEL
`LABEL
`106,112
`AS HIGH
`AS LOW
`PRIORITY - PRIORITY
`
`172
`
`174
`
`176
`
`FORMAT FOR
`166-N-LINK TO FUSION
`PROCESSOR
`
`FORMAT FOR
`LINK TO FUSIONN-178
`PROCESSOR
`
`TRANSMIT
`
`TRANSMIT
`DATA
`
`18O
`
`FIG.9
`
`AHM, Exh. 1013, p. 9
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 8 of 9
`
`US 6,629,033 B2
`
`182
`
`4
`
`RECEIVE
`TRACK
`REPORT
`184
`
`PROCESS
`FOR RECEIVED
`INTERFACE
`186
`
`READ
`SECURITY
`LABEL
`188
`
`READ
`LOGGING
`LABEL
`
`190
`
`READ
`PRIORITY
`LABEL
`192
`
`SEND
`REPORT TO
`APPLICATION
`
`194
`
`196
`
`198
`
`SEND IMAGE
`TO DATA
`DISPLAY
`
`SEND
`BRAKE
`COMMAND
`
`FIG.10
`
`SEND AUDIO
`DATA TO CAR
`AUDIO SYSTEM
`
`AHM, Exh. 1013, p. 10
`
`

`

`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 9 of 9
`
`US 6,629,033 B2
`
`O27
`
`(NOILVOIIddv)
`
`
`
`N3ALSAS
`
`
`
`LYOd3uy4ANVud
`
`ALIYOIYd
`
`ALIYNIIAS
`
`
`
`LY¥Od3uYYOSNAS
`
`ALTYNDAS
`
`SAIFOIYOLLOANNOOD et
`
`
`(LYOd3¥YOSN3AS)NOLIVMALTYOIYd
`
`LOld()NOLIVM()}3ATI03YOLLOINNOD()ON3SOLLOINNOD
`
`912|d1aVl1OYLNOD
`
`(1LYO0d35YYOSNAS)
`
`
`GLZvie‘90UdaNvUa
`ONISSIOONdNOISN,|9YOSSS3I0Ud
`
`SOVAYSLNIJSYVMOUYVH
`
`NOILOSNNODN3dOONID901JY¥OSS390UdVYyOSS300Ud
`
`()}ON3SOLLOANNOD[Cleolz60Z
`
`
`J1gVLIOULNODg0zOl
`
`LYOd3YYOSNIS
`
` (NOLIVM()ONA4S1YOd3uYOSNIS()3AIZ03YOLLOINNOD
`
`
`‘dd¥NOISN4~JOVANALNIJYVMGUVH
`
`AlTYOIedNOILOSNNODN3dO
`
`
`(1LYOd3YYOSNIS)GNISOLLOSNNOOD
`o1ZWALSAS
`
`
`
`(NOILVOIIddv)VYOSSIO0Ud)NISS3IOONdUI
`
`YOSN3SUI
`
`
`
`(LYOd3Y¥YOSNIS)GNIS
`
`¥0Z
`
`90Z
`
`ALINNO|S
`
`9NID901
`
`LL?
`
`AHM, Exh. 1013, p. 11
`
`AHM, Exh. 1013, p. 11
`
`
`
`
`
`
`
`

`

`1
`OPEN COMMUNICATION SYSTEM FOR
`REAL-TIME MULTIPROCESSOR
`APPLICATIONS
`
`BACKGROUND
`Cars include many different electro-mechanical and elec
`tronic Systems. Examples include braking Systems, elec
`tronic Security Systems, radios, Compact Disc (CD) players,
`internal and external lighting Systems, temperature control
`Systems, locking Systems, Seat adjustment Systems, Speed
`control Systems, mirror adjustment Systems, directional
`indicators, etc. Generally the processors that control these
`different car Systems do not talk to each other. For example,
`the car radio does not communicate with the car heating
`System or the car braking System. This means that each one
`of these car Systems has to provide a separate Standalone
`operating System. For example, Separate processors and
`Separate user interfaces are required for the car temperature
`control System and for the car audio System. Many of these
`different car processors may be underutilized since they are
`only used intermittently.
`Even when Some processors in the car do talk to each
`other, they are usually So tightly coupled together that it is
`impossible to change any one of these processors without
`disrupting all of the Systems that are linked together. For
`example, Some cars may have an interface on the dashboard
`that controls both internal car temperature and a car radio.
`The car radio cannot be replaced with a different model and
`still work with the dashboard interface and the car tempera
`ture controller.
`Integration of new Systems into a car is also limited. Car
`Systems are designed and Selected well before the car is ever
`built. A custom wiring harness is then designed to connect
`all the car Systems Selected for the car. A car owner can not
`later incorporate new Systems into the existing car. For
`example, a car may not originally come with a car naviga
`tion System. An after market navigation System from another
`manufacturer cannot be integrated into the car.
`Because after market devices can not be integrated into
`car control and interface Systems, it is often difficult for the
`driver to try and operate these after market devices. For
`example, the car driver has to operate the after market
`navigation System from a completely new interface, Such as
`the keyboard and Screen of a laptop computer. The driver
`then has to operate the laptop computer, not from the front
`dashboard of the car, but from the passenger Seat of the car.
`This makes many after market devices both difficult and
`dangerous to operate while driving.
`The present invention addresses this and other problems
`asSociated with the prior art.
`SUMMARY OF THE INVENTION
`A communication System for a mobile vehicle, home, or
`office environment includes multiple processors. The mul
`tiple processors each run an Open Communication System
`that controls how data is transferred between processors
`based on data content as opposed to the links that connect
`the processors together. The open communication System
`enables data or messages to be effectively transferred and
`processed for real-time applications or other Server based
`applications that may be running on the multiple processors
`in a Secure environment regardless of processors, locations,
`or data linkS.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a diagram of a car that have multiple processors
`that each run an open communication System.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,629,033 B2
`
`2
`FIG. 2 is a block diagram of the open communication
`system shown in FIG. 1.
`FIG. 3 is a flow diagram showing how a priority manager
`processes outgoing data in the open communication System.
`FIG. 4 is a flow diagram showing how the priority
`manager receives data in the open communication System.
`FIG. 5 is a flow diagram showing how a logging manager
`processes data in the open communication System.
`FIG. 6 is a flow diagram Showing how a Security manager
`processes data in the open communication System.
`FIG. 7 is a diagram showing one example of how the open
`communication System is used by different processors.
`FIG. 8 is a diagram of a tracking report that is generated
`by the open communication System.
`FIG. 9 is a flow diagram showing how different image
`data is processed and transmitted using the open communi
`cation System.
`FIG. 10 is a flow diagram showing how the transmitted
`image data in FIG. 9 is received and processed using the
`open communication System.
`FIG. 11 is a block diagram showing another example of
`how the open connection System operates.
`
`DETAILED DESCRIPTION
`FIG. 1 shows a car 12 that includes multiple processors
`14, 16, 18 and 20. The engine monitor processor 14 in one
`configuration monitorS data from different Sensors 22 and 24
`in the car engine. The Sensors 22 and 24 can be any Sensing
`device Such as Sensors that monitor water temperature, oil
`temperature, fuel consumption, car Speed, etc. The brake
`control processor 20 monitors and controls an Automatic
`Braking System (ABS) 28. The display processor 16 is used
`to control and monitor a graphical or mechanical user
`interface. The Security processor 18 monitors and controls
`latches and Sensors 30 and 32 that are used in a car Security
`System.
`Typical networks, Such as in an office network
`environment, enable multiple computers to communicate
`with each other. Applications Such as printing jobs can be
`launched from any one of the networked computers. If one
`of the networked computers crashes or is busy, a user must
`manually Send the job to another computer. The other
`computer then handles the task like any other locally
`received task.
`In a car environment, tasks must be processed with
`different priorities in real-time. For example, the braking
`tasks in the brake processor 20 have to be processed with a
`high priority while a radio Selection task performed in the
`display processor 16 can be processed with a relatively low
`priority. The processors 14, 16, 18 and 20 all include
`software that runs an Open Communication (OC) system 10
`that enables the multiple processors to transfer data and
`eXchange messages for performing these real-time car appli
`cations.
`If the processor 20 currently running the high priority
`braking application fails, the OC system 10 allows the
`braking tasks to be offloaded to another processor in car 12,
`such as the display processor 16. The OC system 10 auto
`matically assigns a high priority to the braking tasks that
`allow the braking tasks to override lower priority tasks, Such
`as the radio application, that are currently being performed
`in display processor 16.
`The OC system 10 also ensures that data in each processor
`is processed in a Secure manner for the car environment. The
`
`AHM, Exh. 1013, p. 12
`
`

`

`US 6,629,033 B2
`
`15
`
`25
`
`35
`
`40
`
`3
`security portion of the OC system 10 prevents unauthorized
`devices from accessing the different car applications. The
`OC System 10 also includes a logging portion that allows
`data in the car System to be automatically logged. This is
`important for accident reconstruction purposes. The OC
`System 10 also allows different processors to communicate
`over different communication protocols and hardware inter
`faces. Any processor that includes an OC System 10 can be
`integrated in the system shown in FIG. 1. This allows
`different processors and different applications can be Seam
`lessly replaced and added to the Overall multiprocessor
`System.
`The description below gives only a few examples of the
`different processors and different applications that can
`implemented using the OC system 10. However, any single
`or multiprocessor System located either inside or outside of
`car 12 can communicate and eXchange data using the OC
`system 10. It should also be understood that the OC system
`10 can be used in any real-time network environment Such
`as between processors used in appliances and computers in
`the home.
`FIG. 2 is a block diagram of the communication managers
`used in the OC system 10 described in FIG.1. The different
`communication managers in the OC System 10 are config
`ured to provide the necessary control for operating a dis
`tributed processor System in a real-time car environment.
`Applications 48 are any of the different applications that can
`be performed for the car 12 shown in FIG. 1. For example,
`applications can include car displayS, braking control, Secu
`rity Systems, Sensor monitoring, airbag deployment, etc. One
`or more applications can be run in the same processor at the
`Same or at different times.
`A car interface manager 46 operates as an Application
`Programmers Interface (API) that can be implemented in
`any variety of different languages Such as Java, C++, Exten
`sible Markup Language (XML) or HyperText Markup Lan
`guage (HTML), etc. The car interface manager 46 enables
`applications 48 to be written in any variety of different
`languages. This prevents the applications 48 from having to
`be written specifically for the car environment or for a
`Specific communication protocol. Thus, applications written
`for other Systems can be reused in the car System described
`below. The car interface manager 46 reads basic processing
`and data transfer commands needed to transfer data and
`messages between different processors and Storage mediums
`inside or Outside the car 12.
`For clarity the terms 'message and data are used inter
`changeably below. After a message passes through the car
`interface manager 46, a priority manager 44 determines a
`priority value for the message that determines how the
`message is processed both in the local processor 50 and in
`other processors such as processor 52. Referring to FIG. 3,
`an outgoing message is identified by the priority manager 44
`in block 60. A priority for the message is identified in block
`62 by reading a priority value that the generic car interface
`manager 46 has attached to the message.
`In block 64, the priority manager 44 compares the priority
`value for the outgoing message with the priority values for
`other messages in the processor. The priority manager 44
`ranks the outgoing message with respect to the other mes
`Sages and then sends the message to the logging manager 42
`in block 66 (FIG. 2). For example, there may be several
`messages that either need to be output or received by a
`particular processor. An output message with a high priority
`value, Such as a crash indication message, will be assigned
`higher priority than other messages and will therefore be
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`immediately transmitted by the processor 50 before other
`lower priority messages.
`FIG. 4 shows how the priority manager 44 receives
`messages from other processors. There may be multiple
`applications running on the same processor and multiple
`messages and data Sent from other processors to those
`applications. For example, multiple Sensors may be sending
`different types of data to a Video display application running
`on one of the processor 50 (FIG. 2). That same processor 50
`may also be receiving different types of Sensor data for
`running an airbag deployment application. The priority
`manager 44 determines the order that messages are pro
`cessed by the different applications that reside on processor
`50.
`In block 68, the priority manager 44 reads the priority
`labels for incoming messages. If the priority of the message
`is not high enough to run on the processor in block 70, the
`data or message is rejected in block 76. The priority manager
`44 may send out a message to the Sending processor indi
`cating the message has been rejected. In Some situations, the
`message or data may have Such a low priority that an
`acknowledge message does not have to be sent back to the
`Sending processor. For example, inside temperature data
`from a temperature Sensor may be sent to one or more
`processors with no requirement that the processor accept or
`acknowledge the data. In this case the temperature data is
`sent with a very low priority value that indicates to the
`priority manager 44 that no message needs to be sent back
`to the temperature Sensor even if the data is rejected.
`The priority manager 44 in block 72 ranks the priority of
`the incoming message in relation to the priorities of all the
`other messages in the processor. The priority manager in
`block 74 decides according to the ranking whether the
`message should be put in a queue or Sent directly to the
`application for immediate processing. For example, a crash
`indication message may have a high enough priority to cause
`the priority manager 44 to delay all data currently being
`processed by all other applications in the same processor.
`The priority manager 44 directs all the applications to wait
`while the current high priority crash indication message is
`processed. The other data and messages are queued in the
`processor and processed after the crash indication message
`has been completed.
`Referring to FIGS. 2 and 5, a logging manager 42 controls
`what data is logged by different processors. It may be
`important to log critical failures that occur during an acci
`dent. For example, it may be important to verify that a
`particular processor Sent an airbag deployment message and
`that another processor Successfully received the airbag
`deployment message. This would allow insurance compa
`nies and other entities to reconstruct accidents by identifying
`when and where different messages were Sent and received.
`The logging manager 42 receives either an incoming
`message over a communications link for Sending to a local
`application 48 or receives an outgoing message from one of
`the local applications 48 for Sending out over the commu
`nications link to another processor in block 80. The logging
`manager 42 reads a logging label in the message in block 82.
`If the logging label indicates that no logging is required, the
`message is sent on to the next communication manager in
`block 88. If it is an outgoing message it is Sent to the Security
`manager 40 (FIG. 2). If it is a incoming message it is sent
`to the priority manager 44. If the message requires logging,
`the logging manager 42 Stores the message in a memory in
`block 86. The logging label may indicate a particular type of
`memory for logging, Such as a nonvolatile Flash memory or,
`if available, a high volume hard disk peripheral memory.
`
`AHM, Exh. 1013, p. 13
`
`

`

`S
`The logging manager 42 in each processor, provides the
`OC system 10 with the unique ability to track when and
`where messages are Sent and received at different processors
`in the multiprocessor car System. This is important in
`accident reconstruction allowing the logging managers 42 to
`identify which processors and applications failed and also
`the Sequence in which the different processors and asSoci
`ated applications failed.
`The logging manager 42 can also track unauthorized
`messages and data that may have caused any of the proces
`Sors in the car to crash. For example, an audio processor that
`handles audio applications in the car may crash due to
`unauthorized downloading of MP3 music from a laptop
`computer. The logging manager 42 can log the unauthorized
`data received from the laptop MP3 player. The logging
`manager 42 logs any data that does not have a particular
`Security or priority label value. A System administrator can
`then down load the MP3 data to identify what caused the
`audio processor to crash.
`Referring to FIGS. 2 and 6, a security manager 40
`provides Security for applications both receiving and trans
`mitting messages. For instance, a laptop computer may be
`connected to a Ethernet port in the car 12 (FIG. 1). If the
`laptop computer does not use the OC system 10, data from
`that laptop application is not allowed to access certain
`processors or certain applications in the car 12. For example,
`audio data should not be sent or processed by a processor
`that performs car braking control.
`The security manager 40 in block 90 reads a message
`either received from an application on the Same processor or
`received over a communication link from another processor.
`The Security manager 40 determines if there is a Security
`value associated with the message in block 92. If there is no
`Security value associated with the data, the Security manager
`40 may drop the data in block 100. However, some
`applications, Such as a processor that playS audio data may
`not require a Security label. In this case, the Security manager
`in block 94 allows the data to be passed on to the application
`in block 98.
`In other instances the data or message may have a Security
`value, but that security value is not sufficient to allow
`processing on the present applications. For example, data for
`car Security monitoring may be sent to a processor that
`controls air bag deployment and an automatic braking
`System. The two currently running applications may set a
`minimum Security level for receiving data. If data received
`from other processors do not have that minimum Security
`level in block 96, the data is dropped in block 100.
`Otherwise, the data or message is passed on to the next
`communication layer for further processing in block 98.
`Thus the Security manager 40 prevents unauthorized data or
`messages from effecting critical car applications.
`Referring back to FIG. 2, an operating system layer 38
`identifies the communication platform used for communi
`cating the data or message over a link identified in a
`hardware/link interface 36. The operating system 38 then
`formats the message for the particular communication Stack
`and medium used by the identified link 54. For example, the
`operating System layer 38 may identify a first message being
`transmitted over a Bluetooth wireleSS link and a Second
`message transmitted over a Transmission Control Protocol/
`Internet Protocol (TCP/IP) packet switched link. The data or
`message adds whatever headers and formatting is necessary
`for transmitting the first message over the Bluetooth wireleSS
`link and the second message over the TCP/IP hardwired link.
`The hardware/link interface 36 includes the Software and
`hardware necessary for interfacing with different communi
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,629,033 B2
`
`6
`cation links 54. For example, the two processors 50 and 52
`may communicate over a Ethernet link, 802.11 wireleSS link,
`or hardwired Universal Serial Bus link, etc. The Software
`necessary for the two processors to communicate over these
`different interfaces is known to those skilled in the art and
`is therefore not described in further detail.
`FIG. 7 describes one example of an application that uses
`the OC system 10 described above in FIGS. 1–6. A car 102
`includes an radar sensor 104 that is controlled by a radar
`processor 106. The radar sensor 104 is located in the front
`grill of car 102. An InfraRed (IR) sensor 110 is controlled by
`an IR processor 112 and is located on the front dash of car
`102. A braking system 123 is controlled by a brake control
`processor 122. The IR processor 112 is connected to a fusion
`processor 114 by an Ethernet link 116 and the radar proces
`Sor 106 is connected to the fusion processor 114 by a 802.11
`wireless link 108. The brake processor 122 is connected to
`the fusion processor 114 by a CAN serial link 120. The
`fusion processor 114 is also coupled to a display Screen 118.
`The radar sensor 104 in combination with the radar
`processor 106 generates Radar Track Reports (RTRs) 130
`that are sent to the fusion processor 114. The IR sensor 110
`in combination with the IR processor 112 generate Infrared
`Track Reports (ITRs) 128 that are sent to the fusion pro
`cessor 114.
`Referring to FIG. 8, each track report 128 and 130
`includes communication link headers 132 for communicat
`ing over an associated interface medium. In this example,
`the radar track report 130 includes the link headers 132
`necessary for transmitting data over the 802.11 link 108. The
`infrared track report 128 includes the link headers 132 for
`transmitting data over the Ethernet link 116.
`The track reports 128 and 130 include Open Communi
`cation (OC) labels 133 for performing the OC operations
`described above. A security label 134 is used by the security
`manager for preventing unauthorized data from being down
`loaded into one of the car processors and disrupting appli
`cations. A logging label 136 is used by the logging manager
`to identify data that needs to be logged in a local memory.
`The priority label 138 is used by the priority manager for
`Scheduling messages or data to the applications run by the
`processors. The link headers 132, Security label 134, logging
`label 136 and priority label 138 are all part of the data 131
`used by the open operating System 131.
`The radar processor 106 and IR processor 112 also send
`a time of measurement 140 and other data 142 from the radar
`sensor 104 and IR sensor 110, respectively. The data 142 can
`include kinematic States of objects detected by the Sensors.
`The time of measurement data 140 and other sensor data 142
`is referred to as application data 139 and is the actual data
`that is used by the application.
`FIGS. 9 and 10 show one example of how the radar and
`infrared sensor data is processed by the OC system 10. One
`or both of the radar processor 106 and the IR processor 112
`may generate image data 150 and 152 for the area in front
`of the car 102 (FIG. 7). For simplicity, the discussion below
`only refers to an image generated by radar Sensor 104. At a
`first time t=t, sensor 104 detects a small far away object
`154. At another time t=t, Sensor 104 detects a large up-close
`object 156.
`The applications described below are all performed by the
`OC System 10 thus preventing the applications from having
`to handle the tasks. This allows the applications to be written
`in a completely portable fashion with no knowledge of the
`network hardware, Security, priority and logging operations.
`This greatly reduces the cost of creating applications.
`
`AHM, Exh. 1013, p. 14
`
`

`

`15
`
`7
`An image processing application in the processor 106
`identifies the object 154 as a small far away object in block
`158. The image and kinematic data for the object is output
`by the OC system 10 as a radar track report 130. The security
`manager 40 (FIG. 2) in the radar processor 106 adds a
`security label 134 to the report in block 160 and the logging
`manager 42 may or may not add a logging label to the report
`in block 162. In this example, the object 154 has been
`identified by the image processing application as a Small far
`away object. Therefore, the logging manager does not label
`the track report for logging. The priority manager 44 (FIG.
`2) adds a priority label 138 (FIG. 8) to the report in block
`164. Because the image processing application identifies the
`object 154 as no critical threat (small far away object), the
`priority label 138 is assigned a low priority value in block
`164.
`The OC system 10 then formats the radar track report in
`block 168 according to the particular link used to send the
`report 130 to the fusion processor 114. For example, the
`operating system 38 and the hardware/link interface 36
`(FIG. 2) in the radar proces