(12)
`
`United States Patent
`Lutter
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,178,049 B2
`Feb. 13, 2007
`
`US007 178049B2
`
`(54) METHOD FOR MULTI-TASKING MULTIPLE
`UAVA VIRTUAL MACHINES IN A SECURE
`ENVIRONMENT
`
`75
`(75) Inventor: Rober Pierce Lutter, Tacoma, WA
`
`(73) Assignee: Medius, Inc., Bainbridge Island, WA
`(US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 449 days.
`
`(21) Appl. No.: 10/132,886
`(22) Filed:
`Apr. 24, 2002
`
`(65)
`
`(
`
`)
`
`(56)
`
`Prior Publication Data
`US 2003/020455O A1
`Oct. 30, 2003
`(51) Int. Cl.
`714/171.8/100
`(2006.01)
`52 section
`(58) Field of Classification Search ........ 718/100–103,
`718/1, 104, 106-107: 714/1-3, 10, 13
`See application file for complete search history.
`References Cited
`U.S. PATENT DOCUMENTS
`
`O
`
`- - - - - - - - -r - - - - - - - - - - - - - - - - - - - - - - - - -
`
`s
`
`4,591,976 A * 5/1986 Webber et al. ................ T14/20
`5,111,401. A * 5/1992 Everett et al. ................ TO1/24
`5,506,963 A * 4, 1996 Ducateau et al. ........... T18, 108
`6,061,709 A * 5/2000 Bronte ....................... T18, 103
`6,128,608 A * 10/2000 Barnhill ....................... TO6/16
`6,167.253 A * 12/2000 Farris et al. ............. 455,412.2
`6,243,450 B1
`6/2001 Jansen et al.
`
`WO
`WO
`WO
`WO
`WO
`WO
`
`6,374,286 B1 * 4/2002 Gee et al. ................... T18, 108
`2002/0012329 A1
`1/2002 Atkinson et al. .
`... 370,330
`2002fOO87886 A1* 7, 2002 Ellis .................
`... 713,201
`2002fO14401.0 A1* 10, 2002 Younis et al. .....
`... 709,314
`2002fO198925 A1* 12/2002 Smith et al. ......
`... 709/104
`2003/0065432 A1* 4/2003 Shuman et al. ............... TO1/48
`2003/01101 13 A1* 6, 2003 Martin ........................ TOS/36
`2003/0201365 A1 * 10, 2003 Nelson .................... 244/118.5
`SO
`FOREIGN PATENT DOCUMENTS
`WO96/24229
`8, 1996
`WO99,08436
`2, 1999
`WO99,57662
`11, 1999
`WO99,651.83
`12/1999
`WOO1/30061
`4/2001
`WOO1,58110
`8, 2001
`OTHER PUBLICATIONS
`Luttge, Karsten; "E-Charging API: Outsource Charging to a Pay
`ment Service Provider”; IEEE: 2001 (pp. 216-222).
`Product description of Raytheon RT Secure, "Embedded Hard
`Real-Time Secure Operating System”. Copyright 2000, pp. 1-2.
`(Continued)
`Primary Examiner Dieu-Minh Le
`McCollom, P.C.
`(57)
`
`(74) Attorney, Agent, Of Firm Marger Johnson &
`
`ABSTRACT
`
`The present invention allows construction of a secure,
`real-time operating system from a portable language Such as
`Java that appears to be a Java virtual machine from a top
`perspective but provides a secure operating system from a
`bottom perspective. This allows portable languages. Such as
`Java, to be used for secure embedded multiprocessor envi
`rOnmentS.
`
`40 Claims, 5 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`14
`
`as threa
`
`SNSRUSN
`THRA
`
`SNSCRFUSN .
`THREAD
`
`MESSAGE MANAGER
`
`CRTCADAAMANAGER
`
`SECURITY MANAGER
`
`DATAANAGER
`
`TASKANAGER
`
`CONFIGURAN WA
`NAGER
`
`Fa
`
`58
`
`AHM, Exh. 1001, p. 1
`
`

`

`US 7,178,049 B2
`Page 2
`
`OTHER PUBLICATIONS
`Product description of Raytheon RT Secure, Copyright 2001, pp.
`1-2.
`Product description of Raytheon RT Secure, “Development Envi
`ronment”. Copyright 2001, pp. 1-2.
`Product description of Raytheon Electronic Systems (ES), Copy
`right 2002, pp. 1-2.
`H. Chung, L. Ojeda, and J. Borenstein, “Sensor Fusion for Mobile
`Robot Dead-reckoning with a Precision-calibrated Fiber Optic
`Gyroscope', 2001 IEEE International Conference on Robotics and
`Automation, Seoul, Korea, May 21-26, pp. 1-6.
`A. Das, R. Fierro, V. Kumar, J. Ostrowski, J. Spletzer, and C. Taylor,
`“A Framework for Vision Based Formation Control', IEEE Trans
`actions on Robotics and Automation, vol. XX, No. Y. 2001, pp.
`1-13.
`J. Takezaki, N. Ueki, T. Minowa, H. Kondoh, “Support System for
`Safe Driving —A Step Toward ITS Autonomous Driving —”.
`Hitachi Review, vol. 49, No. 3, 2000, pp. 1-8.
`S.G. Goodridge, “Multimedia Sensor Fusion for Intelligent Camera
`Control and Human-Computer Interaction”. Dissertation submitted
`to the Graduate Faculty of North Carolina State University in partial
`fulfillment of the requirements for the degree of Doctor of Philoso
`phy in Electrical Engineering, Raleigh, NC, 1997, pp. 1-5.
`M. Chantler, G. Russel, and R. Dunbar, “Probabilistic Sensor Fusion
`for Reliable Workspace Sensing', pp. 1-14.
`ISIS Project: Sensor Fusion, Linkoping University Division of
`Automatic Control and Communication Systems in cooperating
`with SAAB (Dynamics and Aircraft), 18 pages.
`
`Hitachi Automated Highway System (AHS), Automotive Products,
`Hitachi, Ltd., Copyright 1994-2002, 8 pages.
`Vehicle Dynamics Lab, University of California, Berkeley, funded
`by BMW, current members: D. Caveney and B. Feldman, "Adaptive
`Cruise Control”, 17 pages.
`Counterair: The Cutting Edge, Ch. 2 “The Evolutionary Trajectory
`The Fighter Pilot-Here to Stay?” AF2025 v3c8-2, Dec. 1996, pp.
`1-7.
`Counterair: The Cutting Edge, Ch. 4 “The Virtual Trajectory Air
`Superiority without an “Air Force?” AF2025 v3c8-4, Dec. 1996,
`pp. 1-12.
`TNO FEL Annual Review 1998: Quality works, 16 pages.
`Boeing News Release, “Boeing Demonstrates JSF Avionics Multi
`Sensor Fusion'. Seattle, WA. May 9, 2000, pp. 1-2.
`Boeing Statement, “Chairman and CEO Phil Condit on the JSF
`Decision', Washington, D.C., Oct. 26, 2001, pp. 1-2.
`Ada 95 Transition Support- Lessons Learned, Sections, 3, 4, and 5.
`CACI, Inc. -Federal, Nov. 15, 1996, 14 pages.
`Joint Strike Fighter Terrain Database, ets-news.com “Simulator
`Solutions' 2002, 3 pages.
`MSRC Redacted Proposal, 3.0 Architecture Development, pp. 1-43.
`Powerpoint Presentation by Robert Allen - Boeing Phantom Works
`entitled “Real-Time Embedded Avionics System Security and
`COTS Operating Systems”. Open Group Real-Time Forum, Jul. 18.
`2001, 16 pages.
`Green Hills Software, Inc. “The AdaMULTI 2000 Integrated Devel
`opment Environment'. Copyright 2002, 7 pages.
`* cited by examiner
`
`AHM, Exh. 1001, p. 2
`
`

`

`U.S. Patent
`
`Feb. 13, 2007
`
`Sheet 1 of 5
`
`US 7,178,049 B2
`
`
`
`-
`
`AWA WRTUA, MACHINE
`
`-
`
`|
`
`-10
`
`FIG 1
`
`AHM, Exh. 1001, p. 3
`
`

`

`U.S. Patent
`U.S. Patent
`
`9 j.
`
`Feb. 13, 2007
`Feb. 13, 2007
`
`Sheet 2 of 5
`Sheet 2 of 5
`
`
`
`avdva
`
`YOSNAS
`
`dl
`
`al
`
`ALIAOAS
`
`ayvud
`
`oIany JOYLNOD
`
`US 7,178,049 B2
`US 7,178,049 B2
`
`¢Old
`
`AHM, Exh. 1001, p. 4
`
`
`
`
`
`SAILNOAXasSWIWayFYenoas
`
`AHM, Exh. 1001, p. 4
`
`

`

`U.S. Patent
`
`Feb. 13, 2007
`
`Sheet 3 of 5
`
`US 7,178,049 B2
`
`14
`
`
`
`MESSAGE
`MANAGER
`
`CRITICAL DATA
`MANAGER
`
`SECURITY
`MANAGER
`
`DATA MANAGER
`
`TASK MANAGER
`
`CONFIGURATION
`MANAGER
`
`FIG 3
`
`AHM, Exh. 1001, p. 5
`
`

`

`U.S. Patent
`U.S. Patent
`
`Feb. 13, 2007
`Feb. 13, 2007
`
`Sheet 4 of 5
`Sheet 4 of 5
`
`US 7,178,049 B2
`US 7,178,049 B2
`
`
`
`
`
`YAOVNVWSOVSSAW
`
`'
`
`|aVaMHL
`
`
`
`NOISN4HOSNAS|
`
`
`
`NOISN4aY¥OSNSS
`
`OVAMHL
`
`OVSYHLSdd
`
`vSls
`
`
`
`
`
`YAOVNVINVIVOTWOLLIEDS
`
`
`
`YAOVNVALIMNOAS
`
`
`
`YSaOVNVAVLVd
`
`
`
`YSOVNVASVL
`
`
`
`YAOVNVANOILLVYENSISANOD
`
`vi
`
`AHM, Exh. 1001, p. 6
`
`AHM, Exh. 1001, p. 6
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Feb. 13, 2007
`
`Sheet 5 of 5
`
`US 7,178,049 B2
`US 7,178,049 B2
`
`
`
`TASKMANAGER
`
`FIG5
`
`AHM, Exh. 1001, p. 7
`
`e
`89
`
`
`
`za
`9 §Qe
`
`—L
`
`oe
`oOa”
`Zz
`Mi
`a
`
`62
`
`GPSTHREAD
`
`AHM, Exh. 1001, p. 7
`
`

`

`US 7,178,049 B2
`
`1.
`METHOD FOR MULT-TASKING MULTIPLE
`UAVA VIRTUAL MACHINES IN A SECURE
`ENVIRONMENT
`
`2
`following detailed description of a preferred embodiment of
`the invention which proceeds with reference to the accom
`panying drawings.
`
`This application incorporates by reference U.S. patent
`application Ser. No. 09/841,753, filed Apr. 24, 2001, now
`U.S. Pat. No. 6,629,033 entitled: OPEN COMMUNICA
`TION SYSTEM FOR REAL-TIME MULTIPROCESSOR
`APPLICATIONS and pending U.S. patent application Ser.
`No. 09/841,915, filed Apr. 24, 2001 entitled: METHOD
`10
`AND APPARATUS FOR DYNAMIC CONFIGURATION
`OF MULTIPROCESSOR SYSTEM.
`
`5
`
`BACKGROUND OF THE INVENTION
`
`Java is a robust, object-oriented programming language
`expressly designed for use in the distributed environment of
`the Internet. Java can be used to create complete applications
`that may run on a single computer or be distributed among
`servers and clients in a network. A source program in Java
`is compiled into byte code, which can be run anywhere in a
`network on a server or client that has a Java virtual machine
`(JVM).
`A JVM describes software that is nothing more than an
`interface between the compiled byte code and the micro
`processor or hardware platform that actually performs the
`programs instructions. Thus, the JVM makes it possible for
`Java application programs to be built that can run on any
`platform without having to be rewritten or recompiled by the
`programmer for each separate platform.
`Jini is a distributed system based on the idea of federating
`groups of users and the resources required by those users.
`Resources can be implemented either as hardware devices,
`Software programs, or a combination of the two. The Jini
`system extends the Java application environment from a
`single virtual machine to a network of machines. The Java
`application environment provides a good computing plat
`form for distributed computing because both code and data
`can move from machine to machine. The Jini infrastructure
`provides mechanisms for devices, services, and users to join
`and detach from a network. Jini systems are more dynamic
`than is currently possible in networked groups where con
`figuring a network is a centralized function done by hand.
`However, the Java/Jini approach is not without its disad
`vantages. Both Java and Jini are free, open source applica
`tions. The Java application environment is not designed for
`controlling messaging between different machines. For
`example, the Java application is not concerned about the
`protocols between different hardware platforms. Jini has
`some built-in security that allows code to be downloaded
`and run from different machines in confidence. However,
`this limited security is insufficient for environments where it
`is necessary to further restrict code sharing or operation
`sharing among selected devices in a secure embedded sys
`tem.
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`SUMMARY OF THE INVENTION
`
`The present invention allows construction of a secure,
`real-time operating system from a portable language Such as
`Java that appears to be a Java virtual machine from a top
`perspective but provides a secure operating system from a
`bottom perspective. This allows portable languages. Such as
`Java, to be used for secure embedded multiprocessor envi
`rOnmentS.
`The foregoing and other objects, features and advantages
`of the invention will become more readily apparent from the
`
`60
`
`65
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram showing a java stack with an
`additional Secure Real-time Executive (SRE) layer.
`FIG. 2 is a diagram of a multiprocessor System that runs
`multiple Java Virtual Machines that each include a SRE.
`FIG. 3 is a detailed diagram of the managers in the SRE.
`FIG. 4 is a block diagram of how the SRE manages a
`multiprocessor System.
`FIG. 5 is a bock diagram showing how a task manager in
`the SRE operates the multiprocessor system in a lock-step
`mode.
`
`DETAILED DESCRIPTION
`
`A java application stack includes a Java layer 5 for
`running any one of multiple different applications. In one
`example, the applications are related to different vehicle
`operations such as Infrared (IR) and radar sensor control and
`monitoring, vehicle brake control, vehicle audio and video
`control, environmental control, driver assistance control,
`etc. A Java Virtual Machine (JVM) layer 16 provides the
`hardware independent platform for running the Java appli
`cations 5. A Jini layer 12 provides some limited security for
`the Java applications that run on different machines. How
`ever, the Jini layer 12 does not provide the necessary
`reconfiguration and security management necessary for a
`distributed real-time multiprocessor System.
`A Secure Real-time Executive (SRE) 14 provides an
`extension to the JVM 16 and allows Java to run on different
`processors for real-time applications. The SRE 20 manages
`messaging, security, critical data, file I/O multiprocessor
`task control and watchdog tasks in the Java environment as
`described below. The JVM 16, Jini 12 and SRE 14 can all
`be implemented in the same JVM 10. However, for expla
`nation purposes, the JVM 10 and the SRE 14 will be shown
`as separate elements.
`FIG. 2 shows a system 15 that includes multiple proces
`sors 16, 18, 20, 22 and 24. Each processor includes one or
`more JVMs 10 that run different Java applications. For
`example, processor 16 includes one Java application 28 that
`controls a vehicle security system and another Java appli
`cation 26 that controls the vehicles antilock brakes. A
`processor 18 includes a Java application 30 that controls
`audio sources in the vehicle. Other processors 20 and 22
`may run different threads 32A and 32B for the same sensor
`fusion Java application 32 that monitors different IR sensors.
`Another thread 32C on processor 24 monitors a radar sensor
`for the sensor fusion Java application 32.
`The SRE 14 runs below the JVMs 10 in each processor
`and control tasks, messaging, security, etc. For example, the
`Java application 26 controls vehicle braking according to the
`sensor data collected by the sensor fusion Java application
`32. The SRE 14 in one example prevents unauthorized data
`from being loaded into the processor 16 that runs brake
`control application 26. The SRE 14 also prevents other Java
`applications that are allowed to be loaded into processor 16
`from disrupting critical braking operations, or taking priority
`over the braking operations, performed by Java application
`26.
`
`AHM, Exh. 1001, p. 8
`
`

`

`US 7,178,049 B2
`
`3
`For example, the SRE 14 may prevent noncritical vehicle
`applications, such as audio control, from being loaded onto
`processor 16. In another example, noncritical operations,
`Such as security control application 28, are allowed to be
`loaded onto processor 16. However, the SRE 14 assigns the
`security messages low priority values that will only be
`processed when there are no braking tasks in application 26
`that require processing by processor 16.
`The SRE 14 allows any variety of real-time, mission
`critical, nonreal-time and nonmission critical Java applica
`tions to be loaded onto the multiprocessor system 15. The
`SRE 14 then automatically manages the different types of
`applications and messages to ensure that the critical vehicle
`applications are not corrupted and processed with the nec
`essary priority. The SRE 14 is secure software that cannot be
`manipulated by other Java applications.
`The SRE 14 provides priority preemption on a message
`scale across the entire system 15 and priority preemption on
`a task scale across the entire system 15. So the SRE 14
`controls how the JVMs 10 talk to each other and controls
`how the JVMs 10 are started or initiated to perform tasks.
`The SRE 14 allows programmers to write applications using
`Java in a safe and secure real time environment. Thus,
`viruses can be prevented by SRE 14 from infiltrating the
`system 15.
`While the explanation uses Java as one example of a
`programming environment where SRE 14 can be imple
`mented, it should be understood that the SRE 14 can be
`integrated into any variety of different programming envi
`ronments that may run in the same or different systems 15.
`For example, SRE 14 can be integrated into an Application
`Programmers Interface (API) for use with any programming
`language Such as C++.
`FIG.3 shows the different functions that are performed by
`the SRE 14. Any combination of the functions described
`below can be provided in the SRE 14. A message manager
`50 controls the order messages are received and transmitted
`by the different Java applications. A security manager 54
`controls what data and messages are allowed to be received
`or transmitted by different Java applications. A critical data
`manager 52 controls what data is archived by the different
`Java applications.
`A data manager 56 controls what data is allowed to be
`transferred between different processors. A task manager 58
`controls the order tasks are performed by the different JVMs.
`A configuration manager 60 monitors the operation of the
`different processors in the system and reassigns or recon
`figures Java applications and Java threads to different pro
`cessors according to what processors have failed or what
`new processors and applications have been configured into
`system 15.
`The message manager 50 partially corresponds to the
`priority manager 44 shown in FIG. 2 of U.S. Pat. No.
`6,629,033, the critical data manager 52 partially corresponds
`with the logging manager 44 shown in FIG. 2 of the 033
`patent, and the security manger 54 at least partially corre
`sponds with the security manager 40 shown in the 033
`patent. The data manager 56 at least partially corresponds
`with the data manager 42 shown in FIG. 2 of pending patent
`application Ser. No. 09/841,915, the task manager 58 par
`tially corresponds to the device manager 46 shown in FIG.
`2 of the 915 application, and the configuration manager 60
`at least partially corresponds to the configuration manager
`44 shown in FIG. 2 of the 915 patent application. The
`descriptions of how the different managers 50–60 operate
`similarly to the corresponding managers in the ’033 patent
`
`40
`
`45
`
`4
`and the 915 patent applications are herein incorporated by
`reference and are therefore not described in further detail.
`However, Some specific tasks performed by the managers
`50–60 are described below in further detail.
`FIG. 4 shows in more detail how the SRE 14 operates.
`One of the operations performed by the task manager 58 is
`to control when different tasks are initiated on different
`processors. For example, a first Global Positioning System
`(GPS) thread 62 is running on a JVM in a processor 80.
`Another sensor fusion thread 64 is running on a different
`processor 82. Block 74 represents the Java Virtual Machine
`operating in each of processors 80 and 82. A master JVM 74
`may run on either processor 80, processor 82 or on some
`other processor.
`The task manager 58 sends an initiation command 66 to
`the GPS thread 62 to obtain location data. The task manager
`58 then directs the obtained GPS data 68 through a link to
`the sensor fusion thread 64 for Subsequent processing of
`GPS data 68. The link may be any bus, such as a PCI bus,
`serial link such as a Universal Serial Bus, a wireless link
`such as blue tooth or IEEE 802.11, or a network link such
`as Ethernet, etc.
`The configuration manager 60 acts as a watchdog to make
`sure that the GPS thread 62 and the sensor fusion thread 64
`are each running correctly. In one example, separate con
`figuration managers 60 in each processor 80 and 82 sends
`out periodic signals to the other configuration managers 60
`in the other processors. Any one of the configuration man
`agers 60 can detect a processor or application failure by not
`receiving the periodic 'ok' signals from any one of the other
`processors for some period of time. If a failure is detected,
`then a particular master configuration manager 60 in one of
`the processors determines where the task in the failed
`processor is going to be reloaded. If the master configuration
`manager 60 dies, then Some conventional priority Scheme,
`Such as round robin, is used to select another configuration
`master.
`If a failure is detected, say in the processor 82 that is
`currently performing the sensor fusion thread 64, a message
`is sent from the configuration manager 60 notifying the task
`manager 58 which processor is reassigned the sensor fusion
`thread. In this example, another sensor fusion thread 76 in
`processor 84 is configured by the configuration manager 60.
`The critical data manager 52 manages the retention of any
`critical data 72 that was previously generated by the sensor
`fusion thread 64. For example, the critical data manager 54
`automatically stores certain data and state information that
`was currently being used in the sensor fusion thread 64. The
`critical data may include GPS readings for the last 10
`minutes, sensor data obtained from sensors in other proces
`sors in the vehicle over the last 10 minutes. The critical data
`may also include any processed data generated by the sensor
`fusion thread 64 that identifies any critical vehicle condi
`tions.
`The critical data manager 52 also determines which data
`to archive generally for vehicle maintenance and accident
`reconstruction purposes.
`The configuration manager 60 directs the critical data 72
`to the new sensor fusion thread 76. The task manager 58 then
`redirects any new GPS data obtained by the GPS thread 78
`to the new sensor fusion thread 76 and controls sensor fusion
`tasks from application 76. Thus, the configuration manager
`60 and the task manager 58 dynamically control bow
`different Java threads are initialized, distributed and acti
`vated on different processors.
`The message manager 50 determines the priority of sent
`and received messages. If the data transmitted and received
`
`10
`
`15
`
`25
`
`30
`
`35
`
`50
`
`55
`
`60
`
`65
`
`AHM, Exh. 1001, p. 9
`
`

`

`5
`by the sensor fusion thread 76 is higher priority than other
`data transmitted and received on the processor 84, then the
`sensor fusion data will be given priority over the other data.
`The task manager 58 controls the priority that the sensor
`fusion thread 76 is giving by processor 84. If the sensor
`fusion thread 76 has higher priority than, for example, an
`audio application that is also being run by processor 84, then
`the sensor fusion thread 76 will be performed before the
`audio application.
`The SRE 14 can be implemented in any system that needs
`to be operated in a secure environment. For example,
`network servers or multiprocessors operating in a home
`environment. The multiprocessors in home appliances. Such
`as washer and dryers, home computers, home security
`systems, home heating systems, can be networked together
`and operate Java applications. The SRE 14 prevents these
`multiple processors and the Software that controls these
`processors from being corrupted by unauthorized software
`and also allows the applications on these different processors
`to operate as one integrated system.
`The SRE 14 is a controlled trusted computing based that
`is not accessible by non-authorized application program
`mers and anyone in the general public. Therefore, the SRE
`14 prevents backing or unauthorized control and access to
`the processors in the vehicle.
`Task Controlled Applications
`Debugging is a problem with multiprocessor Systems. The
`task manager 58 allows the Java applications to be run in a
`lock-step mode to more effectively identify problems in the
`multiprocessor system 15.
`FIG. 5 shows a path 90 taken by a vehicle 92. In one
`application, the position of the vehicle 92 is sampled every
`second t, t t t etc. The position of the vehicle 92 is
`sampled by a GPS receiver in vehicle 92 that reads a
`longitudinal and latitudinal position from a GPS satellite.
`The GPS receiver is controlled by the GPS thread 62 that
`receives the GPS data and then sends the GPS data to a
`sensor fusion thread 64 that may run on the same or a
`different processor in the vehicle 92. The sensor fusion
`thread 64 can performany one of many different tasks based
`on the GPS data. For example, the sensor fusion thread 64
`may update a map that is currently being displayed to the
`driver of vehicle 92 or generate a warning signal to the
`vehicle driver.
`For each sample period to the task manager 58 sends a
`request 94 to the GPS thread 62 to obtain GPS data. The task
`manager 58 uses a clock 95 as a reference for identifying
`each one second sample period. Each time a second passes
`according to clock 95, the task manager 58 sends out the
`request 94 that wakes up the GPS tread 62 to go read the
`GPS data from the GPS satellite. Once the GPS data has
`been received, the GPS thread 62 passes the GPS data 96 to
`the sensor fusion tread 64. The GPS thread 62 then goes
`back into an idle mode until it receives another activation
`command from the task manager 58.
`The task manager 58 can control when the GPS thread 62
`is woken up. Instead of the GPS thread 62 being free
`running, the GPS thread 62 is operating according to a
`perceived time controlled by the task manager 58. The task
`manager 58 may send the activation request 94 to the GPS
`thread 62 once every second during normal sensor fusion
`operation. When the system is in a debug mode, however,
`the task manager 58 may only send one activation command
`94. This allows the other operations performed by the
`system 89 to be monitored and determine how the single
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 7,178,049 B2
`
`5
`
`10
`
`15
`
`6
`sampling of GPS data 96 propagates through system 89. The
`task manager 58 may also delay or disable task initiation to
`other threads, so that the processing of the GPS data 96 can
`be isolated.
`The task manager 58 can isolate any state in the overall
`system 89, such as the state of system 89 after a first GPS
`reading by GPS thread 62 or the state of system 89 after the
`thirty second GPS reading by GPS thread 62 by controlling
`when and how often activation commands 94 are sent to
`GPS thread 62. In a similar manner, the task manager 58 can
`control when other tasks are performed by the system 89.
`such as when the sensor fusion thread 64 is activated.
`Thus, the task manager 58 controls when Java applica
`tions are activated effectively running the overall system 89
`in a lock-step mode. The task manager 58 can control the
`initiation of multiple tasks at the same time. This allows the
`task manager to control what parameters and operations are
`performed and used by the different Java threads so that
`different states in the multiprocessor system 89 can be
`detected and monitored more effectively.
`One application for the task controlled applications is for
`accident reconstruction. The critical data manager 52 (FIG.
`3) may save different vehicle parameters from a vehicle that
`has been in an accident. For example, sensor data, brake
`data, speed data, etc. The task manager 58 can feed the saved
`data into the different Java applications in a lock-step mode
`to determine how each Java thread processes the saved data.
`This can then be used to identify any failures that may have
`occurred in the system 89.
`The system described above can use dedicated processor
`systems, micro controllers, programmable logic devices, or
`microprocessors that perform some or all of the communi
`cation operations. Some of the operations described above
`may be implemented in Software and other operations may
`be implemented in hardware.
`For the sake of convenience, the operations are described
`as various interconnected functional blocks or distinct soft
`ware modules. This is not necessary, however, and there may
`be cases where these functional blocks or modules are
`equivalently aggregated into a single logic device, program
`or operation with unclear boundaries. In any event, the
`functional blocks and software modules or described fea
`tures can be implemented by themselves, or in combination
`with other operations in either hardware or software.
`Having described and illustrated the principles of the
`invention in a preferred embodiment thereof, it should be
`apparent that the invention may be modified in arrangement
`and detail without departing from Such principles. Claim is
`made to all modifications and variation coming within the
`spirit and scope of the following claims.
`The invention claimed is:
`1. A portable processing system for controlling operations
`in a vehicle, comprising:
`multiple processors located in the vehicle that transmit
`and receive messages over a wired or wireless network
`located within the vehicle, the processors executing
`different vehicle applications associated with critical
`operations associated with primary vehicle functions
`used for driving and controlling the vehicle and non
`critical operations associated with providing and con
`trolling driver ergonomic functions not required for
`operating the primary vehicle driving and control func
`tions;
`at least one of the processors operating as an executive
`that dynamically initiates the execution of the applica
`tions on different processors according to processor
`availability and the critical and noncritical operations
`
`AHM, Exh. 1001, p. 10
`
`

`

`7
`associated with the vehicle applications so that identi
`fied failure of at least one of the vehicle applications
`associated with one of the critical operations is auto
`matically transferred to a selected at least one of the
`processors currently executing non-critical operations
`thereby dynamically reassigning the selected at least
`one of the processors as a backup processor for one or
`more of the processors executing critical operations.
`2. A portable processing system according to claim 1
`wherein the vehicle applications are implemented using a
`Java or C++ operating language.
`3. A portable processing system according to claim 1
`wherein the executive identifies failure of a first processor
`executing one of the vehicle applications associated with
`one of the critical operations and in response to the identified
`failure automatically initiating the vehicle application on a
`second processor that is executing an application associated
`with one of the non-critical operations.
`4. A portable processing system according to claim 1
`wherein the vehicle applications are used for controlling
`real-time vehicle braking, driving or object detection opera
`tions.
`5. A portable processing system according to claim 4
`wherein the non-critical operations include operation of a
`Car Stereo.
`6. A portable processing system according to claim 1
`wherein the executive assigns priority values to the mes
`sages according to the critical and non-critical operations
`sending and receiving the messages.
`7. A portable processing system according to claim 6
`30
`wherein the executive assigns priority preemption to mes
`sages and tasks for controlling how different applications
`communicate with each other.
`8. A portable processing system according to claim 1
`wherein the executive automatically archives vehicle data
`during vehicle operation for use in accident reconstruction
`or vehicle maintenance.
`9. A portable processing system according to claim 1
`including:
`a first processor operating a first critical operation;
`a second processor operating one of the non-critical
`operations;
`wherein the executive detects a failure of the first pro
`cessor, or failure in the critical operation on the first
`processor, and
`in response to the failure detection the executive identi
`fying the second processor as available as a backup
`processor for running the critical operation and accord
`ingly directing the second processor to start operating
`the first critical operation while transferring data
`obtained from the first processor while operating the
`first critical operation to the second processor Such that
`the second processor operates one of the non-critical
`operations and also provides redundancy for the failed
`first critical operation.
`10. A portable processing system according to claim 1
`wherein the executive further includes a message manager
`configured to control how messages are received and trans
`mitted between different vehicle applications operating on
`different processors.
`11. A portable processing system according to claim 10
`wherein the executive further includes a security manager
`configured to control what data and messages are allowed to
`be received and transmitted between the different vehicle
`applications operating on the different processors.
`12. A portable processing system according to claim 11
`wherein the security manager prevents non-critical opera
`
`50
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`US 7,178,049 B2
`
`10
`
`15
`
`25
`
`8
`tions and messages from non-critical operations from being
`loaded onto processors running critical operations.
`13. A portable processing system according to claim 1
`wherein the executive further comprises a critical data
`manager that identifies critical operations and periodically
`stores data associated with the identified critical operations.
`14. A portable processing system according to claim 13
`wherein the critical data manager manages the retention of
`data generated by a sensor fusion application operating on a
`first processor by transferring the data to a second processor
`that