Invalidity of U.S. Patent No. 6,748,317 by
`The Cyberguide System (“Cyberguide”) and Abowd, Gregory D., A Mobile Context-Aware Tour Guide, Baltzer
`Journals (September 23, 1996) (“Abowd”)
`
`I have provided below a claim chart comparing the disclosures of Cyberguide and Abowd to the ’317 Asserted Claims, as well the
`combination of Cyberguide or Abowd and U.S. Patent No. 6,067,502 to Hayashida et al. (“Hayashida”).
`
`As described in my Report, the Cyberguide system was made publicly available at least as early as September 23, 1996. The features
`
`and functionalities of the Cyberguide system are described in the following printed publications:
`
`• Abowd, Gregory D., A Mobile Context-Aware Tour Guide, Baltzer Journals (September 23, 1996 (“Abowd”)
`• Long, Sue, Cyberguide: Prototyping Context-Aware Mobile Applications, Future Computing Environments,
`https://www.cc.gatech.edu/fce/cyberguide/pubs/chi96-cyberguide.html (“Cyberguide Prototyping”)
`• CyBARguide
`Project
`Notes
`by
`Mike
`Pinkerton,
`Gregory
`Abowd,
`https://www.cc.gatech.edu/fce/cyberguide/cybarguide/CyBARguide.html (“CyBARguide”)
`
`Long,
`
`and
`
`Sue
`
`Because the Cyberguide product itself was known and used by others prior to the ‘317 Patent’s priority date, it constitutes prior art
`under 35 U.S.C. § 102(a) and (b)(pre-AIA). Additionally, because Abowd published on September 23, 1996, it independently
`constitutes prior art under 35 U.S.C. § 102(b) (pre-AIA).
`
`Hayashida was filed on August 21, 1997 and issued on May 23, 2000. Hayashida therefore qualifies as prior art with regard to the ’317
`patent under 35 U.S.C. § 102(e) (pre-AIA).
`
`U.S. Patent No. 6,748,317
`Claim 1
`1[P]. A portable
`comprising:
`
`terminal,
`
`Cyberguide / Abowd
`
`The Cyberguide is a portable terminal. For example, Abowd describes Cyberguide being deployed
`on popular mobile devices at the time, such as the Apple MessagePad.
`
`Future computing environments will free the user from the constraints of the
`desktop. Applications for a mobile environment should take advantage of
`contextual information, such as position, to offer greater services to the user.
`In this paper, we present the Cyberguide project, in which we are building
`
`1
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 1 of 30
`
`

`

`prototypes of a mobile context-aware tour guide. Knowledge of the user’s
`current the location, as well as a history of past the location, as used to provide
`more of the kind of services that we come to expect from a real tour guide. We
`describe the architecture and features of a variety of Cyberguide prototypes
`developed for indoor and outdoor use on a number of different hand-held
`platforms. We also discuss the general research issues that have emerged in
`our context-aware applications development in a mobile environment.
`Abowd at 1; see also id. at 2.
`
`
`We will describe in Section 5 the initial realization of the generic components
`of the Cyberguide architecture, a series of prototypes developed for the Apple
`MessagePad
`Abowd at 2-3.
`
`This section outlines some possible uses for future mobile context-aware
`applications. Some of these uses are currently being implemented and some are
`futuristic. We begin with our initial assumptions about what technology we
`expect Cyberguide to use. Tourists are usually quite happy to carry around a
`book that describes the location they are visiting, so a reasonable packaging
`would be in the form of a hand-held device. The ideal hand-held device will
`have a screen and pen/finger interface, access to substantial storage resources
`|possibly through an internal device such as a CD drive, or through substantial
`communication and networking resources (cell phone, pager, data radio
`interface) providing access to other storage servers (such as the Web)| an audio
`input and output interface with speech generation and potentially sophisticated
`voice recognition, and a video input and output interface.
`Abowd at p. 3.
`
`
`One major application of mobile context-aware devices are personal guides.
`Museums could provide these devices and allow users to take personalized
`tours seeing any exhibits desired in any order, in contrast to today's taped
`tours. In fact, many museums now provide portable devices for just such a
`purpose, but what we are envisioning is a device that would allow the tourist
`
`
`
`2
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 2 of 30
`
`

`

`[1(a)] a device for getting the
`location information denoting
`a resent1 place of said portable
`terminal;2
`
`to go anywhere she pleases and be able to receive information about anywhere
`she is. Walking tours of cities or historical sites could be assisted by these
`electronic guidebooks. The hand-held devices could use position measurement
`systems such as indoor beacons or the Global Positioning System (GPS) to
`locate the user, and an electronic compass or inertial navigation system to find
`user orientation.
`Abowd at 3.
`
`Under the Court’s construction of this limitation, Cyberguide discloses the function of getting
`location information denoting a present place of said portable terminal using a structure of a
`compass, gyroscope, and/or sensor such as a clinometer in conjunction with a CPU, or equivalents
`thereof. Cyberguide includes a device (i.e., the CPU of the Apple Message Pad with Newton
`Operating System, a GPS, and an IR-beacon system) that performs the claimed function of getting
`the location information denoting a present place of said portable terminal. The present location is
`then depicted using a pointer symbol on a map.
`
`
`The hand-held devices could use position measurement systems such as
`indoor beacons or the Global Positioning System (GPS) to locate the user,
`and an electronic compass or inertial navigation system to find user
`orientation. Objects of interest could be marked with visual markers or active
`beacons or recognized using computer vision.
`Id. at 3.
`
`In thinking about and developing a location-aware application, we were greatly
`influenced by work such as the PARCTab at XeroxPARC [14], the InfoPad
`project at Berkeley [7], the Olivetti Active Badge system [14] and the Personal
`Shopping Assistant proposed at AT&T [3]. We wanted to build useful
`
`1 Based on Plaintiff’s Infringement Contentions and subsequent claim limitations referring to a present place, Defendant understands
`this to mean “present.”
`
`“Function: getting location information denoting a present place of said portable terminal
`Structure: a compass, gyroscope, and/or sensor such as a clinometer in conjunction with a CPU, or equivalents thereof.”
`3
`
`
`2 The Court construed this element as:
`
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 3 of 30
`
`

`

`applications that might take advantage of the hardware developed in the
`PARCTab and InfoPad projects. We did not want to build our own hardware,
`so we have a different focus from all of these projects. There are a number of
`commercially available and relatively inexpensive hand-held units that would
`suffice for our purposes, such as the Apple MessagePad with the Newton
`Operating System,a MagicCap machine or a pen-based palmtop/tablet PC.
`We chose to work initially with the Apple MessagePad 100 with Newton 1.3 and
`pen-based PCs running Windows for Pen Computing 1.0. Each platform was
`available for $ 150-500 with relatively powerful development environments.
`This low cost of hardware was critical to the success of Cyberguide because it
`made it possible to put a number of units in the hands of many students, all with
`different ideas that they were allowed to investigate.
`Abowd at 3-4.
`
`
`5.4 Position Component
`
`Position is the obvious starting point for a context-aware mobile device. We
`considered several methods for sensing the user the location. Outdoors,
`continuous services, such as GPS, can be used. Indoors, however, GPS
`signals are weak or not available.
`Abowd at 8.
`
`
`One solution for an indoor positioning system was to use infrared (IR). Our
`first positioning system was based on using TV remote control units as active
`beacons, and using a special IR receiver tuned to the carrier frequency (~
`40kHz) of those beacons (Figure 3). A microcontroller (Motorola 68332)
`interfaced the IR receiver to the serial port of the Apple MessagePad. We
`deployed an array of remote controls hanging from the ceiling (Figure 3 right),
`each remote control acting as a position beacon by repeatedly beaming out a
`unique pattern. The 68332 translates the IR pattern into a unique cell identifier
`that is sent to the Apple MessagePad's serial port. As the tourist moves around
`the room and passes into the range of a new cell, the position (indicated by an
`arrowhead) is updated on the map. Keeping track of the last recorded cell the
`
`
`
`4
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 4 of 30
`
`

`

`location provides a good guess as to the location the tourist is heading, so we
`indicate an assumed orientation by pointing the position icon accordingly.
`Abowd at 9.
`
`
`
`
`Figure 3: IR positioning prototype (left) and the array of positioning beacons
`in the GVU Lab (right).
`Abowd at 9.
`
`
`
`
`8.5 Use of Vision
`
`In the extreme case, we can think of a communion between the physical and
`electronic worlds, as suggested by work in augmented reality. Replace the
`hand-held unit with a pair of goggles and as the user wanders around,
`information about certain exhibits can be summoned up and overlaid on top of
`the actual image. Vision techniques can be used to augment the positioning
`system to inform the system and tourist what the tourist is looking at. We have
`experimented with vision systems as an extension to Cyberguide. Ultimately,
`we want to move to ward personalized vision systems.
`
`8.6 Ubiquitous Positioning System
`
`Our current prototypes are exclusively indoor or outdoor, not both. This is
`mainly because we had no one positioning system that worked in both
`conditions. GPS is unreliable indoors and the IR-based beacon system is
`impractical for us to implement outdoors. We intend to integrate both
`5
`
`
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 5 of 30
`
`

`

`positioning systems into one application, to allow a tourist to wander in and
`out of buildings and have Cyberguide automatically switch the positioning
`system.
`Abowd at 19.
`
`
`The map is the view the visitor is using to navigate. Visualizing and
`manipulating the map dominates the user interface of Cyberguide. It can be
`viewed at varying levels of detail and scrolled around. The visitor is indicated
`by location and orientation on the map (the arrowhead in Figure 1) and various
`demonstrations are also marked (as stars in Figure 1).
`Cyberguide Prototyping at p. 3
`
`
`The positioning component provides constantly updated information on the
`location and orientation of the tourist. Our current prototype implements
`indoor positioning via a collection of TV remote control beacons broadcasting
`separate location IDs. When within range of a beacon, a custom IR transceiver
`unit (consisting of a separate IR sensor and a Motorola 68332 processor
`connected via serial port to the Newton) translates the ID into a map location
`and orientation. the additional processor unit allows for further customized
`extensions to the positioning system, such as an electronic compass. Optionally,
`we could use the built-in Newton IR transceiver coupled with individual Newton
`beacons. This option requires no additional hardware, but is less flexible.
`Cyberguide Prototyping at p. 3.
`
`
`Our positioning system gives you fairly accurate readings, but you are still off
`by a block or so from your actual position. We have a few suggestions as to how
`to correct this delta. First, you could try to allow the user to modify or correct
`your current position, adding a delta factor to the latitude and longitude read
`in from the GPS unit. In addition you could try getting futher latitude and
`longitude from other NMEA formats. We are using the most frequently occuring
`encoding, there are however several other formats we are recieving from the
`GPS unit, but ignoring. There may be additional information in these other
`formats that may more accurately report our current position.
`
`
`
`6
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 6 of 30
`
`

`

`[1(b)] a device for getting a
`direction
`information
`denoting an orientation of said
`portable terminal;3
`
`
`CyBARguide at p. 3.
`
`Under the Court’s construction of this limitation, Cyberguide discloses the function of getting
`direction information denoting an orientation of said portable terminal using a structure of a
`compass, gyroscope, and/or sensor such as a clinometer in conjunction with a CPU, or equivalents
`thereof. Cyberguide includes a device (i.e., the CPU of the Apple Message Pad with Newton
`Operating System and internal compass) that performs the claimed function of getting direction
`information denoting an orientation of said portable terminal.
`
`
`Applications for a mobile environment should take advantage of contextual
`information, such as position, to offer greater services to the user. In this paper,
`we present the Cyberguide project, a series of prototypes of a mobile, hand-
`held context-aware tour guide. Initially, we are concerned with only a small
`part of the user's context, specifically the location and orientation.
`Abowd at 2.
`
`
`The long-term goal is an application that knows where the tourist is, what she
`is looking at, can predict and answer questions she might pose, and provide the
`ability to interact with other people and the environment. Our short-term goal
`was to prototype versions of Cyberguide on commercially available PDAs and
`pen-based PCs in which context-awareness simply meant the current physical
`position and orientation of the Cyberguide unit (and since it is hand-held, this
`locates the user as well).
`Id. at 2.
`
`
`The hand-held devices could use position measurement systems such as indoor
`beacons or the Global Positioning System (GPS) to locate the user, and an
`electronic compass or inertial navigation system to find user orientation.
`
`“Function: getting direction information denoting an orientation of said portable terminal
`
`
`3 The Court construed this element as:
`Structure: a compass, gyroscope, and/or sensor such as a clinometer in conjunction with a CPU, or equivalents thereof.”
`
`
`7
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 7 of 30
`
`

`

`Objects of interest could be marked with visual markers or active beacons or
`recognized using computer vision.
`Id. at 3.
`
`
`In thinking about and developing a location-aware application, we were greatly
`influenced by work such as the PARCTab at XeroxPARC [14], the InfoPad
`project at Berkeley [7], the Olivetti Active Badge system [14] and the Personal
`Shopping Assistant proposed at AT&T [3]. We wanted to build useful
`applications that might take advantage of the hardware developed in the
`PARCTab and InfoPad projects. We did not want to build our own hardware,
`so we have a different focus from all of these projects. There are a number of
`commercially available and relatively inexpensive hand-held units that would
`suffice for our purposes, such as the Apple MessagePad with the Newton
`Operating System1,a MagicCap2 machine or a pen-based palmtop/tablet PC.
`We chose to work initially with the Apple MessagePad 100 with Newton 1.3 and
`pen-based PCs running Windows for Pen Computing 1.0. Each platform was
`available for $ 150-500 with relatively powerful development environments.
`This low cost of hardware was critical to the success of Cyberguide because it
`made it possible to put a number of units in the hands of many students, all with
`different ideas that they were allowed to investigate.
`Abowd at 3-4.
`
`
`Navigator (Positioning Component) The interests of the tourist lie relatively
`close to their physical the location. Therefore, it is important to know exactly
`where the tourist is, in order to show the immediate surroundings on the map
`or answer questions about those surroundings (“What am I looking at?"). The
`navigator is responsible for charting the location of the tourist within the
`physical surroundings. This component is realized by a positioning module that
`delivers accurate information on tourist the location and orientation.
`Id. at 6.
`
`
`The map is the view the visitor is using to navigate. Visualizing and
`manipulating the map dominates the user interface of Cyberguide. It can be
`
`
`
`8
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 8 of 30
`
`

`

`viewed at varying levels of detail and scrolled around. The visitor is indicated
`by location and orientation on the map (the arrowhead in Figure 1) and various
`demonstrations are also marked (as stars in Figure 1).
`Cyberguide Prototyping at p. 3
`
`
`The positioning component provides constantly updated information on the
`location and orientation of the tourist. Our current prototype implements
`indoor positioning via a collection of TV remote control beacons broadcasting
`separate location IDs. When within range of a beacon, a custom IR transceiver
`unit (consisting of a separate IR sensor and a Motorola 68332 processor
`connected via serial port to the Newton) translates the ID into a map location
`and orientation. The additional processor unit allows for further customized
`extensions to the positioning system, such as an electronic compass.
`Optionally, we could use the built-in Newton IR transceiver coupled with
`individual Newton beacons. This option requires no additional hardware, but
`is less flexible.
`Cyberguide Prototyping at p. 3.
`
`
`It would have been obvious to a PHOSITA to modify the portable terminal taught by the
`Cyberguide to include an internal compass for getting a direction information denoting an
`orientation of said portable terminal. A PHOSITA would have been motivated to make this
`modification in order to find a user’s orientation as expressly suggested by the Cyberguide. A
`PHOSITA would have a reasonable expectation of success in modifying the Cyberguide to include
`an internal compass because of the modularity of the portable terminal taught by the Cyberguide.
`Accordingly, such a modification would not have required undue experimentation and would have
`yielded predictable results.
`
`
`The utility of this architectural decomposition for Cyberguide is that it provides
`an extensible and modular approach to system development. It is extensible
`because we can always add further services. For example, we have considered
`adding an historian whose purpose is to document where the tourist has been
`and what her reactions were to the things she saw. It is modular because it has
`
`
`
`9
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 9 of 30
`
`

`

`[1(c)] an input device for
`inputting a destination; and
`
`allowed us to change the implementation of one component of the system with
`minimal impact on the rest of the system. For example, we have implemented
`different versions of the navigator and the librarian without having to alter the
`other components. Of course, these components are related in some ways; for
`instance, position information ultimately has to be translated into a location on
`the physical map. Defining standard interfaces between the components is the
`means by which we achieve separation between and coordination among the
`various components.
`Abowd at p. 6, see also id. at p. 7-8, and p. 10.
`
`Cyberguide discloses an input device for inputting a destination (e.g., screen and pen/finger
`interface). Specifically, the Cyberguide includes a screen and pen/finger interface that is the
`structure performing the function of inputting a destination.
`
`
`The ideal hand-held device will have a screen and pen/finger interface, access
`to substantial storage resources—possibly through an internal device such as
`a CD drive, or through substantial communication and networking resources
`(cell phone, pager, data radio interface) providing access to other storage
`servers (such as the Web)| an audio input and output interface with speech
`generation and potentially sophisticated voice recognition, and a video input
`and output interface.
`Abowd at 3.
`
`
`In thinking about and developing a location-aware application, we were greatly
`influenced by work such as the PARCTab at XeroxPARC [14], the InfoPad
`project at Berkeley [7], the Olivetti Active Badge system [14] and the Personal
`Shopping Assistant proposed at AT&T [3]. We wanted to build useful
`applications that might take advantage of the hardware developed in the
`PARCTab and InfoPad projects. We did not want to build our own hardware,
`so we have a different focus from all of these projects. There are a number of
`commercially available and relatively inexpensive hand-held units that would
`suffice for our purposes, such as the Apple MessagePad with the Newton
`Operating System1,a MagicCap2 machine or a pen-based palmtop/tablet PC.
`
`
`
`10
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 10 of 30
`
`

`

`We chose to work initially with the Apple MessagePad 100 with Newton 1.3 and
`pen-based PCs running Windows for Pen Computing 1.0. Each platform was
`available for $ 150-500 with relatively powerful development environments.
`This low cost of hardware was critical to the success of Cyberguide because it
`made it possible to put a number of units in the hands of many students, all with
`different ideas that they were allowed to investigate.
`Abowd at 3-4.
`
`
`
`
`
`
`
`Figure 4: The outdoor Cyberguide (left) with GPS unit (right).
`Abowd at 10.
`
`
`Figure 10 shows the map interface on the left and a view of a user-modifiable
`database for interesting establishments on the right. The tourist can indicate a
`desired destination and as she moves around, CyBARguide automatically
`chooses the map of the highest detail that contains both the traveler (indicated
`by a triangle in Figure 10), and the destination (the beer mug with the
`emboldened border in Figure 10). Along the way, if the tourist eyes another
`interesting establishment that is not currently highlighted on the map, it could
`be added. Each establishment has a user-modifiable database entry
`associated with it that reflects both objective (e.g., availability of parking,
`average price of drinks) and subjective (e.g., ambiance or other comments)
`information that can be used in the future to plan an evening's excursion.
`
`
`
`11
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 11 of 30
`
`

`

`querying of a large amount of information and some minimal routing facilities.
`We also plan to make the data within the information module modifiable so the
`user can add personalized information including personal impressions that may
`be useful for future reference, a type of virtual graffiti. We envision the use of
`CyBARguide in a mode in which both tourist and proprietors are able to modify
`the information base. A simple searching capability can assist revellers in
`search of a certain kind of entertainment experience, and additional contextual
`information, such as the knowledge of where the traveller has been already, the
`time of day, and what special events (information provided by the proprietors)
`are currently scheduled, can be used to deliver suggestions for where to go.
`Id. at 14-15.
`
`
`
`Figure 10: The CyBARguide interface. The left shows the interactive map
`indicating the user's the location (the triangle) and the location of
`establishments previously visited (the beer mugs). The user modifiable database
`shown on the right supports the long-term development of touring information
`for a location.
`Id. at 15.
`
`The Cyberguide includes a display.
`
`
`12
`
`[1(d)] a display, wherein
`
`
`
`
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 12 of 30
`
`

`

`The ideal hand-held device will have a screen and pen/finger interface, access
`to substantial storage resources—possibly through an internal device such as
`a CD drive, or through substantial communication and networking resources
`(cell phone, pager, data radio interface) providing access to other storage
`servers (such as the Web)| an audio input and output interface with speech
`generation and potentially sophisticated voice recognition, and a video input
`and output interface. The video input (a video camera) could be pointed at the
`user to interpret user gestures, or pointed at the environment to interpret
`objects or symbols in the environment. The video output could be integrated
`into the main screen or be a separate video display device, such as an attached
`screen or heads up display on glasses worn by the user.
`Abowd at 3.
`
`
`In thinking about and developing a location-aware application, we were greatly
`influenced by work such as the PARCTab at XeroxPARC [14], the InfoPad
`project at Berkeley [7], the Olivetti Active Badge system [14] and the Personal
`Shopping Assistant proposed at AT&T [3]. We wanted to build useful
`applications that might take advantage of the hardware developed in the
`PARCTab and InfoPad projects. We did not want to build our own hardware,
`so we have a different focus from all of these projects. There are a number of
`commercially available and relatively inexpensive hand-held units that would
`suffice for our purposes, such as the Apple MessagePad with the Newton
`Operating System1,a MagicCap2 machine or a pen-based palmtop/tablet PC.
`We chose to work initially with the Apple MessagePad 100 with Newton 1.3 and
`pen-based PCs running Windows for Pen Computing 1.0. Each platform was
`available for $ 150-500 with relatively powerful development environments.
`This low cost of hardware was critical to the success of Cyberguide because it
`made it possible to put a number of units in the hands of many students, all with
`different ideas that they were allowed to investigate.
`Abowd at 3-4.
`
`
`
`13
`
`
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 13 of 30
`
`

`

`Figure 1: The map (left) and information (right) interfaces of the initial
`Cyberguide MessagePad prototype.
`
`
`
`
`
`
`Abowd at p. 7.
`
`
`
`
`
`[1(e)] said display displays
`positions of said destination
`and said present place, and a
`relation of said direction and a
`
`
`
`
`
`Figure 4: The outdoor Cyberguide (left) with GPS unit (right).
`Abowd at 10.
`
`The Cyberguide displays positions of said destination and said present place, and a relation of said
`direction and a direction from said present place to said destination. For example, Cyberguide
`displays a map showing the user’s present location and direction (or orientation) using a pointer
`icon and uses symbols (e.g., stars) to mark destinations.
`14
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 14 of 30
`
`

`

`direction from said present
`place to said destination, and
`
`
`
`
`
`One major application of mobile context-aware devices are personal guides.
`Museums could provide these devices and allow users to take personalized
`tours seeing any exhibits desired in any order, in contrast to today's taped tours.
`In fact, many museums now provide portable devices for just such a purpose,
`but what we are envisioning is a device that would allow the tourist to go
`anywhere she pleases and be able to receive information about anywhere she
`is. Walking tours of cities or historical sites could be assisted by these electronic
`guidebooks. The hand-held devices could use position measurement systems
`such as indoor beacons or the Global Positioning System (GPS) to locate the
`user, and an electronic compass or inertial navigation system to find user
`orientation. Objects of interest could be marked with visual markers or active
`beacons or recognized using computer vision. Some objects, such as animals
`at a zoo or aquarium, might be difficult to mark but could be recognized with
`simple computer vision and some assistance from the environment (indications
`that this is the elephant cage, for example). The personal guide could also assist
`in route planning and providing directions. Some of these functions are
`currently being provided by automobile on-board navigation systems.
`Abowd at 3.
`
`
`The overall system serves as a tour guide, but we can think of a tour guide as
`playing the role of cartographer, librarian, navigator and messenger. The
`services provided by these components are:
`
`Cartographer (Map Component) This person has intimate knowledge of the
`physical surroundings, such as the location of buildings, interesting sights
`within a building, or pathways that the tourist can access. This component is
`realized in our systems by a map (or maps) of the physical environments that
`the tourist is visiting.
`
` …
`
`
`
`
`
`
`
`15
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 15 of 30
`
`

`

`Navigator (Positioning Component) The interests of the tourist lie relatively
`close to their physical the location. Therefore, it is important to know exactly
`where the tourist is, in order to show the immediate surroundings on the map
`or answer questions about those surroundings (“What am I looking at?"). The
`navigator is responsible for charting the location of the tourist within the
`physical surroundings. This component is realized by a positioning module that
`delivers accurate information on tourist the location and orientation.
`Id. at 5-6.
`
`
`The initial map module, shown on the left side of Figure 1, contains a map of
`the entire GVU Center. Passageways and demonstration stations (stars in
`Figure 1) are shown. Only a limited view of the lab can be seen at any given
`time. The user can scroll the map around and zoom in and out to see alternative
`views. There is an icon to show the user's the location on the map. Using
`information from the positioning module, we implemented automatic
`scrolling of the map. If desired, the user's position is updated automatically
`and the map is scrolled to ensure that the user's current position remains on
`the visible portion of the map.
`Id. at 6-7.
`
`
`
`
`Figure 1: The map (left) and information (right) interfaces of the initial
`Cyberguide MessagePad prototype.
`Id. at 7.
`
`
`16
`
`
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 16 of 30
`
`

`

`
`
`
`
`Figure 4: The outdoor Cyberguide (left) with GPS unit (right).
`Abowd at 10.
`
`
`There were several motivations for building a Cyberguide prototype for
`outdoor use (Figure 4). First, we wanted to use Cyberguide over a wider area
`than the relatively small GVU Center. We also wanted to test the modularity of
`our design by having to change critical features. The two features that were
`changed on this prototype were the underlying map and the physical
`positioning system. We obtained a different map and inserted that into the map
`module without any problems. For positioning, we replaced the IR positioning
`module with a Trimble GPS unit attached to the Apple MessagePad serial port.
`(see right side of Figure 4). The GPS unit sends a position in latitude and
`longitude which was then translated into a pixel coordinate representing the
`user's current position on the map.
`Id. at 10.
`
`
`The Delphi prototype uses vector-based maps, allowing for arbitrary scaling
`and rotation of the map and well as path generation.
`Id. at 11.
`
`
`
`
`17
`
`Apple v. Maxell
`IPR2020-00407
`Maxell Ex. 2009
`
`Page 17 of 30
`
`

`

`
`Figure 5: The main map interface of the PC Cyberguide. Checks on demo
`sites indicate the user has been to visit that demo already, indicating history-
`sensitive interface.
`Id. at 12.
`
`
`Figure 10 shows the map interface on the left and a view of a user-modifiable
`database for interesting establishments on the right. The tourist can indicate a
`desired destination and as she moves around, CyBARguide automatically
`chooses the map of the highest detail that contains both the traveler (indicated
`by a triangle in Figure 10), and the destination (the beer mug with the
`emboldened border in Figure 10). Along the way, if the tourist eyes another
`interesting establishment that is not currently highlighted on the map, it could
`be added. Each establishment has a user-modifiable database entry associated
`with it that reflects both objective (e.g., availability of parking, average price
`of drinks) and subjective (e.g., ambiance or other comments) information that
`can be used in the future to plan an evening's excursion. querying of a large
`amount of information and some minimal routing facilities. We also plan to
`make the data within the information module modifiable so the user can add
`personalized informat