THE NETWORK VEHICLE - A GLIMPSE INTO THE FUTURE OIF
`MOBILE MULTI-MEDIA
`
`R. Lind, R. Schumacher, R. Reger, R. Olney, H. Yen and M. Laur
`Delphi Delco Electronics Systems, Kokomo, Indiana
`
`R. Freeman
`IBM Corporation, white Plains, New York
`
`ABSTRACT
`The Network Vehicle is the Delphi
`Automotive Systems’ vision for the future
`convergence of computers, the communications
`infrastructure, and the automobile. It features
`many advanced functions such as: satellite
`video, Internet access, virtual navigation,
`remote vehicle diagnostics and control, games,
`mobile office, automotive web site, and
`customized real-time stock quotes and sports
`scores. These features are enabled by an
`integrated planar antenna that is capable of
`multiple satellite reception, a client-server
`network architecture, and unique human-
`vehicle-interfaces such as color reconfigurable
`head up and head down displays, steering
`wheel controls, voice recognition,
`text-to-
`speech, and large touch screen active matrix
`liquid crystal displays (LCD’s). The software
`applications are written
`in
`Java, using
`Application Programming Interfaces (API’s) to
`reduce the complexity and cost of the source
`code.
`
`entertainment to millions of commuters who
`spend hours each day cruising the roads or
`stuck in traffic. It is designed to demonstrate
`what technologies and software can do for the
`vehicle of the future. The Network Vehicle,
`pictured in Figure 1, made
`its debiut at
`COMDEX ’97 and has since been all over the
`world appearing in demos, conferences, and
`technology shows, including SAE Conigress
`’98, and CeBit ‘98.
`
`Figure 1. The Network Vehicle
`
`INTRODUCTION
`new
`a
`The Network Vehicle,
`initiative by Delphi Delco
`technology
`Electronics Systems and its partners (IBM,
`Netscape
`Communications,
`and
`Sun
`Microsystems) is aimed at offering more
`productivity tools, convenience, safety, and
`0-7803-5086-3 /98/$10.00 01998 IEEE
`
`SYSTEM OVERVIEW
`The Network Vehicle is created by
`integrating existing hardware and software
`recognition,
`technologies
`including voice
`wireless communications, global positioning
`via
`satellite, head-up
`displays,
`JavaTM
`technology, microprocessors, Web access:, and
`121-1
`
`IPR2017-02022
`Unified EX1016 Page 1
`
`

`

`other Internethtranet features. The voice
`recognition
`technology allows drivers and
`passengers to verbally request and listen to e-
`mail messages, locate a restaurant or hotel, ask
`for navigation help or for specific music or
`sports scores, and use voice-activated telephone
`services, all done safely without interfering
`with driving. Drivers can use a head-up display
`projected through the windshield to navigate to
`their destination or check vehicle functions
`without taking their eyes off the road.
`Passengers using individual terminals
`next to their seats can do even more, including
`interacting with
`the
`Internet, watching
`television or playing games. Existing services
`such as
`theft deterrent
`technologies and
`emergency services could also be integrated.
`These "smart" features are enabled mainly
`because of
`the
`real-time data-streaming
`capabilities over a wireless network using Java-
`based technology. The Network Vehicle also
`has an integrated cellular phone, Netscape
`Communicator software for Web browsing and
`e-mail, a removable personal digital assistant
`(PDA) and docking station. There is also a
`vehicle web site that provides driver support for
`a wide array of customized capabilities such as
`enabling remote monitoring and control of
`vehicle systems in emergency situations.
`
`Onboard And Off-Board Communications
`Networks
`The key to the functions of the Network
`Vehicle is its ability to communicate efficiently
`both onboard and off-board of the vehicle.
`Figure 2 depicts
`the onboard network
`architecture of the vehicle.
`At the heart of the onboard system is a
`network computer that links up a number of
`sub-networks, including a Class I1 bus (for
`engine, head-up and head-down displays, door
`lock and door actuator, WID, and cell phone,
`etc.), an Ethernet (for passenger computers and
`displays), and a mobile media
`link
`(for
`speakers and CD players).
`Since
`the
`subnetworks all have different clock speed and
`0-7803-5086-3 /98/$10.00 01998 IEEE
`
`functionality, gateways are used to translate the
`messages from one sub-network to another so
`as to ensure harmonious operations.
`
`Dir CP y
`
`AmplifierdSpeakers
`
`Figure 2. Onboard Network Architecture
`The off-board network is shown in
`Figure 3. The high bandwidth requirements for
`on-demand audio and video functions are
`fulfilled with DirecPC and DirecTV satellite
`broadcasts. A wireless modem provides the
`uplink out of the vehicle directly to Internet
`service providers. The downlink return path
`from the Internet to the Network Vehicle can
`come through either the satellite (400 kbps), or
`through the wireless modem. The downlink
`satellite video and data signals are collected by
`the Network Vehicle's roof-mounted antenna
`and processed by their respective receivers.
`
`Figure 3. Off-Board Network Architecture
`
`V Y
`
`*
`
`Internet S ~ N I C ~
`
`Web Server
`
`HomBlOtflce
`Computer
`
`In order to take full advantages of the
`resources and services available to the drivers
`and passengers, the vehicle must have robust
`external communications paths.
`It is also
`vitally important that the vehicle configuration
`includes a firewall to ensure the integrity of the
`
`121-2
`
`IPR2017-02022
`Unified EX1016 Page 2
`
`

`

`vehicle operation. This means that regardless
`of what happens to the multimedia applications
`that are running in the vehicle, the powertrain
`system control, the chassis control, brakes,
`airbags, and other critical functions must
`maintain their integrity.
`In the Network
`Vehicle, this isolation is implemented through
`multiple gateways as shown in Figure 2.
`
`KEY TECHNOLOGIES AND SYSTEM
`COMPONENTS
`technologies and system
`The key
`components behind the Network Vehicle are
`advanced speech recognition software, mobile
`media link (high-speed fiber optic data bus),
`reconfigurable
`head-up
`display,
`high-
`bandwidth communications that connect the
`vehicle with the outside world, and a suite of
`automotive computer software. These elements
`are described below.
`
`Speech Recognition and Text-to-Speech
`System
`IBM has modified its advanced speech
`recognition
`and
`text-to-speech
`system,
`ViaVoiceTM, for the automotive environment.
`ViaVoice allows the driver to access virtually
`all
`the vehicle's
`features
`through voice
`commands and enables the vehicle to talk back
`using synthesized speech. For example, the
`driver can: execute vehicle system commands
`such as lock doors, play CD, and change radio
`station, request travel directions and traffic
`updates from the Web or other sources, check
`e-mail and voicemail, request news, sports, and
`stock information. The speech recognition
`system can understand most drivers instantly,
`with no system training required, and it has
`been tuned to offer optimal performance even
`in a potentially noisy vehicle environment.
`
`Driver and Center Console Displays
`The Network Vehicle is equipped with
`three displays for the driver: the head-down
`
`0-7803-5086-3 /98/$10.00 0 1998 BEE
`
`display (HDD), the head-up display (HUD),
`and the center console display.
`The HDD
`standard
`system displays
`graphics for an instrument panel: road speed,
`engine speed, engine status, door IocWajar, and
`fuel level. It also displays the functions for the
`steering wheel buttons since these functions
`vary with the entertainment mode that is
`selected. The HUD projects a virtual image
`through the windshield that gives information
`to drivers without requiring them to take their
`eyes off the road.
`It displays road speed,
`engine
`status, waiting e-mail
`indication,
`navigation information, microphone ordoff, and
`a text message area for giving feedback to the
`driver.
`
`The center console's touch-screen LCD
`serves as a user interface for controlling nearly
`all of
`the Network Vehicle's multimedia
`functions, (office, navigation, entertainment,
`and information). When in entertainment
`mode, the display is configured as a radio- or
`CD-style faceplate whose buttons and controls
`are activated by touch (see Figure 4).
`It is
`reconfigured
`to display e-mail, navigation
`maps, Web browser, cellular phone faceplate
`(shown in Figure 5), and more. Voice-
`activated commands can be used with all (of the
`center console functions and, in some cases, is
`
`coiribineci wilh Lexl-to-speech 1npUl
`minimize driver distraction.
`Figure 4. Audio on Demand Faceplate
`I2 1 -3
`
`to
`
`IPR2017-02022
`Unified EX1016 Page 3
`
`

`

`“extrusion” of a one-dimensional (constant
`cross-section) geometry. This has the effect of
`
`replacing a conventional i C ~ ~ ~ = ~ 2element
`r
`,
`
`structure (of discrete radiators, couplers, etc.)
`with a
`less complex “monolithic” array
`comprised of “N” integrated coupler/radiator
`features. The simple “tee” cross-section of the
`integrated CTS coupler/radiator
`forms an
`inherently
`low-”Q”
`(non-resonant) element
`which exhibits significant advantages
`(as
`compared to slot or patch radiators) in terms of
`wide-angle scanning capability, polarization
`purity,
`bandwidth,
`and
`dimensional
`insensitivity.
`
`RadiatinglCoupling Stub
`
`(Coupled) E-Field
`
`Plane-Wave
`
`Parallel-Plate Region
`
`~~
`
`~~
`
`~
`
`~
`
`Figure 6. CTS Antenna Cross Section
`
`Customer Application Web Site
`the
`The driver and passengers of
`Network Vehicle can take advantage of Internet
`services provided by automotive-oriented ISPs
`(Internet Service Providers). The customer
`service site developed for the Network Vehicle
`showcases the types of features that could be
`offered when intelligent vehicle systems are
`connected to the Internet.
`Because the Network Vehicle’s systems
`are accessible via its command and control
`application, which also has secure access to the
`Internet, many
`scenarios
`are possible.
`Functions like parking lights and door locks on
`the Network Vehicle can be controlled
`remotely, providing safety and convenience.
`An example of such a web page is shown in
`Figure 7.
`
`121-4
`
`Figure 5. Cellular Phone Faceplate Display
`
`Passenger Displays
`The Network Vehicle has color LCD
`touch panels for the back-seat and front-seat
`passengers. As with the center console display,
`the passenger displays serve as user interfaces
`for controlling nearly all of the functions on the
`Network Vehicle. Unlike the center console
`display, however, the passenger displays can
`show video from DirecTV or DVD players, and
`they allow touch-screen access to all functions.
`
`Planar Satellite Antenna
`The antenna technology used in the
`Network Vehicle is the Continuous Transverse
`Stub (CTS) array. It is chosen because of its
`simplicity, planar construction, and potential
`for being an inexpensive product. Figure 6
`shows the cross sectional view of a typical CTS
`realized as an array of broad
`antenna,
`continuous transverse radiating stubs, finite in
`height, extending from the upper conductive
`plate of an open parallel-plate transmission-line
`structure .
`As a receiving antenna, the induced
`longitudinal current components in the parallel
`plate structure are interrupted by the transverse-
`oriented stubs and excite propagating waves in
`the parallel-plate structure.
`This simple
`architecture allows
`for a complex
`two-
`dimensional planar array to be realized as an
`0-7803-5086-3 /98/$10.00 01998 EEE
`
`IPR2017-02022
`Unified EX1016 Page 4
`
`

`

`take
`features demonstrated
`Other
`advantage of Internet service providers' ability
`to communicate with the Network Vehicle.
`Trip plans created by vehicle owners or
`concierge services can be stored at the In ternet
`service provider.
`By accessing global
`positioning satellite (GPS) data directly from
`the vehicle, the driver and passengers can
`obtain maps, route guidance, fuel, lodging, and
`restaurant information while traveling. 0 wners
`of the Network Vehicle can also use their Web
`browser
`to set up a profile,
`including
`preferences like radio stations, personalized
`audio content, service records, and emergency
`service numbers.
`
`Personal Digital Assistant Docking
`An
`important role of
`the Network
`Vehicle is providing office functions to mobile
`users, using the same data the users have in
`their office desktop computers. This function
`is mainly
`provided
`through
`vehicle
`applications,
`Internet
`connectivity,
`and
`advanced, voice-based user interfaces.
`It is
`important for the Network Vehicle to access
`and synchronize personal and business data
`with PDAs, which is becoming increasingly
`important in our mobile, connected world.
`A slot built into the center console of
`IBM
`the Network Vehicle accepts
`the
`WorkPadTM and uses the Network Vehicle's
`computer to add speech recognition and text-to-
`speech functions. This enables the driver to
`listen to schedules and to update files and other
`S o the
`office data stored in the WorkPad.
`driver could, for example, issue a voice
`command to read calendar entries from the
`WorkPad or dictate a to-do list while driving.
`As PDA and smart card technology advances,
`the Network Vehicle's PDA integration could
`support the use of PDA-type devices for the
`secure
`transport and access of personal,
`financial, and business information as people
`move from one network access point
`to
`another.
`
`Figure 7. Monitor and Control of Vehicle
`Status via Internet
`Under voice control of the driver, the
`Network Vehicle can upload vehicle status
`according to a profile set up in advance. An
`application monitors the diagnostic information
`available from components in
`the engine
`compartment. Should the diagnostics indicate
`that the latest reading from engine sensors falls
`outside a normal range, the application can
`notify the customer service Web site, where the
`severity of the reading can be determined.
`According to this determination, the control on
`the Network Vehicle could be used by the
`service representative to alert the driver; the
`alert could be done using text-to-speech and
`head-up display, and then features of
`the
`application, e-mail, and office could be used to
`arrange a service appointment. Figure 8
`illustrates the sensor data that are being
`monitored through the vehicle web site.
`
`Figure 8. Remote Monitoring of Vehicle
`Gauges via a Web Site
`
`0-7803-5086-3 /98/$10.00 0 1998 JEEE
`
`121-5
`
`IPR2017-02022
`Unified EX1016 Page 5
`
`

`

`Automotive Computer Software
`Virtual machine-based, object-oriented
`technologies were applied to the Network
`Vehicle's development because they support
`the special requirements posed by the vehicle.
`Smalltalk, a well proven object-oriented
`environment and the latest JavaTM technologies
`were chosen to develop the Network Vehicle's
`software.
`A command and control application,
`written
`in Smalltalk and running on
`the
`vehicle's main processor, monitors vehicle
`status, controls devices such as cell phone and
`microphone, supplies driver information via
`head-up
`display, manages
`off-vehicle
`information
`flow,
`and
`controls vehicle
`software, such as the voice recognition and
`various convenience passenger applications.
`This application coordinates with the voice
`recognition, the head-up display, and the touch
`screen to provide feedback and control of the
`various systems, helping to keep the driver's
`eyes on the road. For driver convenience and
`safety, this application also interfaces via the
`Internet to a customer service provider, who
`can remotely monitor the vehicle status, unlock
`doors, turn on exterior lighting, etc.
`Graphical user interfaces for Network
`Vehicle's center console and passenger displays
`are developed with Java to provide access to
`various in-vehicle and off-vehicle applications,
`such as navigation, audiolvideo entertainment,
`and office. An Internet server application was
`developed with VisualAge@ for Java to provide
`communication
`between
`Internet-based
`customer applications and the vehicle systems.
`These applications support a message-based
`protocol communicating through the Internet to
`the
`in-vehicle
`command
`and
`control
`application, which in turn interfaces to the
`vehicle
`systems and
`the user
`interface
`application. Java applets were also developed
`for the customer site in order to provide
`animated vehicle status and control graphics.
`
`Media BUS and In- Vehicle Electronics
`The Network Vehicle contains a
`number of audio and video components that are
`configured and controlled via a high-speed
`(100 Mbps) digital fiber-optic mobile media
`link (MML). Audio and video devices like CD
`players can send their output signals via the
`multiplexed, fiber-optic bus to drive other
`components like amplifiers and displays. A
`control signal is also carried on the MML bus
`to configure and control the devices - configure
`the components, switch signals, and adjust
`variables such as volume, fade, and balance.
`The MML gives the application full use of the
`vehicle's multimedia
`systems
`to develop
`functions and user interfaces tailored to driver
`and passenger use.
`Combining a command and control
`application with the MML in the Network
`Vehicle made possible features such as context-
`sensitive volume reduction during the use of
`the phone and the text-to-speech feature.
`It
`also allows "faceplate-style" user interface to
`integrate Internet radio with the sound system,
`and common and consistent user interfaces to
`define, review, and play the audio and video
`content preferred by the driver and passengers.
`The vehicle also features a satellite receiver
`system, which provides DirecTV and DirecPC
`access. DirecPC offers high-speed (440 kbps)
`Internet data access while DirecTV offers over
`175
`channels
`of
`digital
`entertainment
`programming.
`
`FEATURES AND FUNCTIONS OF THE
`NETWORK VEHICLE
`the Network
`The main features of
`Vehicle are manifested in the following three
`areas: (1) Driving Aids, (2) Business Tools, and
`(3) News and Entertainment. The Network
`Vehicle is designed such that, through software
`upgrades the customer can be provided with
`new features and functions as they become
`available.
`
`0-7803-5086-3 /98/$10.00 01998 lEEE
`
`121-6
`
`IPR2017-02022
`Unified EX1016 Page 6
`
`

`

`Driving Aids - Get Where You’re Going
`Efficiently
`The Network Vehicle can access a simulated
`integrated global positioning system and hence
`advanced navigation capabilities via Internet.
`Therefore, it literally knows where it is and
`where it’s going. It can display the appropriate
`maps or simply provide route directions on the
`head-up display, command console or as
`spoken instructions. Drivers could control the
`navigation
`system using commands
`like
`“destination” to select a new destination or
`“route
`guidance”
`to
`get
`turn-by-turn
`instructions. An example of such a display is
`shown in Figure 9.
`
`Figure 9. Example of a Navigation Function
`Display
`The ability to monitor key vehicle systems
`means that the Network Vehicle can detect and
`avoid problems before they happen.
`If key
`vehicle systems show signs of failing, a
`warning light appears on the head-up display,
`or the vehicle tells the driver what the problem
`In the future, a technician or a vehicle
`is.
`manufacturer could take a look under the hood
`remotely to identify and diagnose problems,
`determine whether you can keep driving, or
`recommend obtaining service.
`
`If you
`the Network
`left
`think you
`Vehicle’s doors unlocked or the lights on, just
`visit the vehicle web site from your home or
`office computer to check, then fix the problem
`0-7803-5086-3 /98/$10.00 01998 E E E
`
`The Network Vehicle’s future
`remotely.
`remote control capabilities not only could allow
`you to actually start the vehicle in the parking
`lot from the office, but also to set the heater or
`air conditioner from your office computer, so
`that the vehicle is comfortable and ready to roll1
`when you are. You could even plan trips on the
`vehicle web site, then download them to your
`vehicle. Figure 9 shows an example page of
`the vehicle web site where remote control
`parameters can be set.
`
`Business Tools - You ’re Never Out of Touch
`The Network Vehicle is on the Web,
`with a DirecPC satellite link beaming the
`Internet to the vehicle at 400 kbps, more: than
`15 times the speed of a standard 28.8 kbps
`modem. Web sites can be viewed on the color
`reconfigurable displays at each of the three
`passenger seats, and the Network Vehicle can
`download selected information to the tlriveir
`such as maps for navigation.
`The Network Vehicle is an extension of
`your office, seamlessly offering access to
`everything from e-mail and address book to the
`critical files you need. The driver cain be
`alerted to new e-mail via the head-up display
`and can ask the vehicle to read e-mail. With
`voice commands, a to-do list can be called up
`from the docked Workpad.
`E-mails and
`appointments can be downloaded to the docked1
`Workpad PDA, for review after leaving the
`vehicle.
`In the future, the Network Vehicle
`will be able to communicate pages and play
`voice mail. The driver will be able to use voice:
`commands to call up a personal address book,
`then dictate e-mail messages which the veihicle
`can turn into text and send. Passengers will be
`able to view faxes on their LCD displays.
`The Network Vehicle’s cell phoine is
`fully integrated with other systems, so a simple
`voice command tells the phone to dial any
`number. The Network Vehicle uses its own in-
`vehicle audio system as a speakerphone,
`allowing hands-free conversations.
`
`I2 1 -7
`
`IPR2017-02022
`Unified EX1016 Page 7
`
`

`

`affordable and robust mobile satellite tracking
`antenna system is yet to be realized. It is clear
`though,
`the
`integration
`of
`computer,
`communications, and software technologies
`with vehicles will have a dramatic impact on
`the future of automotive industry.
`
`News and Entertainment - Getting There is
`More Than Half the Fun
`Via
`the DirecTV
`satellite
`link,
`passengers will be able to view hundreds of
`broadcast and cable channels. Each passenger
`can watch a different channel at the same time,
`with headsets to keep the noise level to a
`minimum. Using the Network Vehicle web
`site, you can preprogram your favorite channels
`and shows. In the future, the Network Vehicle
`will include a DVD player so you’ll be able to
`watch the latest movies on disc.
`The Network Vehicle offers AM/FM
`radio, a CD player and audio from the Internet,
`all accessible with voice commands. From the
`Network Vehicle web site, one can program
`personal radio presets. Or, take advantage of
`various capabilities on the Internet, such as
`downloadable audio books, or audio-on-
`demand for an
`infinite choice of music
`selections or radio stations worldwide that are
`now broadcasting over
`the Web.
`Each
`passenger can also use an LCD display to
`watch movies, surf the Net or play the latest
`computer
`games.
`Using Netscape
`Communicator, the Network Vehicle will allow
`you to speak “Read Stocks” and hear the
`current value of each stock in your portfolio.
`In the future, the vehicle will read you the latest
`news from Web sites, or provide customized
`news from the vehicle web site.
`
`CONCLUSION
`The Network Vehicle has proven that
`by properly
`integrating
`several existing
`functions and technologies one can get a good
`glimpse of what
`the
`future of mobile
`multimedia holds. The range of applications
`enabled by
`the availability of wideband
`communications
`to
`the car
`is enormous.
`Reconfigurable
`displays
`and
`upgradable
`software will allow vehicle owners to take
`the
`latest applications and
`advantages of
`services. However, to make this vision a
`reality, a great deal of
`improvement and
`development is still needed. For example, an
`
`0-7803-5086-3 /98/$10.00 01998 IEEE
`
`121-8
`
`IPR2017-02022
`Unified EX1016 Page 8
`
`