Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 1 of 22 PageID #: 25551
`Case 5:19-cv-00036—RWS Document 457-2 Filed 07/30/20 Page 1 of 22 PageID #: 25551
`
`EXHIBIT 5
`
`EXHIBIT 5
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 2 of 22 PageID #: 25552
`
`APL-MAXELL_00713087
`
`Baltzer Journals
`
`September  , 
`
`Cyberguide: A Mobile Context-Aware Tour Guide
`
`Gregory D. Abowd , Christopher G. Atkeson , Jason Hong , Sue Long
`Rob Kooper and Mike Pinkerton
`
`;
`
`,
`
` Graphics, Visualization and Usability Centre
`College of Computing
`Georgia Institute of Technology
`Atlanta, GA -
`E-mail: kooper,mpinkert,hong,abowd,cga@cc.gatech.edu
`
`Wink Communications, Alameda, CA
`E-mail: sue.long@wink.com
`
`Future computing environments will free the user from the constraints of the desk-
`top. Applications for a mobile environment should take advantage of contextual
`information, such as position, to oer greater services to the user. In this paper,
`we present the Cyberguide project, in which we are building prototypes of a mo-
`bile context-aware tour guide. Knowledge of the user’s current location, as well
`as a history of past locations, are used to provide more of the kind of services
`that we come to expect from a real tour guide. We describe the architecture and
`features of a variety of Cyberguide prototypes developed for indoor and outdoor
`use on a number of dierent hand-held platforms. We also discuss the general
`research issues that have emerged in our context-aware applications development
`in a mobile environment.
`
`Keywords: Mobile computing, context-awareness, location-dependent applica-
`tions, hand-held devices
`
` Introduction
`
`Future computing environments promise to free the user from the constraints of station-
`ary desktop computing, yet relatively few researchers are investigating what applications
`maximally benet from mobility. Current use of mobile technology shows a slow evolution
`from our current desktop paradigm of computing, but the history of interaction shows
`that the adoption of new technology usually brings about a radical revolution in the way
`humans use and view technology  . Whereas the eective use of mobile technology
`will give rise to an interaction paradigm shift, it is dicult to predict what that shift
`will be. We follow the advice of Alan Kay, therefore, and choose to predict the future by
`inventing it. Our approach is to think rst about what activities could be best supported
`by mobile technology and then determine how the technology would have to work. This
`applications focus is important to distinguishing our work in mobile computing.
`In April , we formed the Future Computing Environments FCE Group within
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 3 of 22 PageID #: 25553
`
`APL-MAXELL_00713088
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`the College of Computing and the Graphics, Visualization and Usability GVU Center at
`Georgia Tech to promote such an applications focus. Our group is committed to the rapid
`prototyping of applications that benet from the use of emerging mobile and ubiquitous
`computing technologies. Quick development of these futuristic applications allows us to
`predict and shape what our everyday lives will be like when today’s novel technology
`becomes commonplace.
`Applications for a mobile environment should take advantage of contextual informa-
`tion, such as position, to oer greater services to the user.
`In this paper, we present
`the Cyberguide project, a series of prototypes of a mobile, hand-held context-aware tour
`guide. Initially, we are concerned with only a small part of the user’s context, specically
`location and orientation. Knowledge of the user’s current location, as well as a history
`of past locations, are used to provide more of the kind of services that we come to ex-
`pect from a real tour guide. We describe the architecture and features of a variety of
`Cyberguide prototypes developed for indoor and outdoor use on a number of dierent
`hand-held platforms. We also discuss the general research issues that have emerged in
`our experience of developing context-aware applications in a mobile environment. Some
`of these research issues overlap with those that we have considered in applying other
`applications of ubiquitous computing technology.
`The general application domain which has driven the development of Cyberguide
`is tourism, but we have found it necessary to be even more focused in our research.
`The initial prototypes of Cyberguide, therefore, were designed to assist a very specic
`kind of tourist |a visitor in a tour of the GVU Center Lab during our monthly open
`houses. Visitors to a GVU open house are typically given a map of the various labs
`and an information packet describing all of the projects that are being demonstrated
`at various sites. Moving all of the paper-based information into a hand-held, position-
`aware unit provided a testbed for research questions on mobile, context-aware application
`development.
`The long-term goal is an application that knows where the tourist is, what she is
`looking at, can predict and answer questions she might pose, and provide the ability
`to interact with other people and the environment. Our short-term goal was to pro-
`totype versions of Cyberguide on commercially available PDAs and pen-based PCs in
`which context-awareness simply meant the current physical position and orientation of
`the Cyberguide unit and since it is hand-held, this locates the user as well. Position
`information improves the utility of a tour guide application. As the prototypes of Cyber-
`guide evolve, we have been able to handle more of the user’s context, such as where she
`and others have been, and we have increased the amount in which the tourist can interact
`and communicate with the place and people she is visiting.
`
` . Overview
`
`This paper is an extended version of an earlier report on Cyberguide , we discuss the
`evolution of the Cyberguide design and prototype as well as what future research areas
`our experience has uncovered. We begin in Section  by describing scenarios for the use
`of context-aware mobile applications. In Section , we provide context for our research
`within the area of applications-centered mobile computing. The generic architecture of
`Cyberguide is explained in Section . We will describe in Section  the initial realiza-
`tion of the generic components of the Cyberguide architecture, a series of prototypes
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 4 of 22 PageID #: 25554
`
`APL-MAXELL_00713089
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`developed for the Apple MessagePad. We will then describe in Section  how the initial
`indoor prototypes were extended for use outdoors and for greater interaction with the
`environment. We conclude in Sections  and  with a discussion of signicant issues for
`context-aware applications development and how our past experience will inuence our
`future development plans.
`
` Scenarios for a mobile context-aware application
`
`This section outlines some possible uses for future mobile context-aware applications.
`Some of these uses are currently being implemented and some are futuristic. We be-
`gin with our initial assumptions about what technology we expect Cyberguide to use.
`Tourists are usually quite happy to carry around a book that describes the location they
`are visiting, so a reasonable packaging would be in the form of a hand-held device. The
`ideal hand-held device will have a screen and pennger interface, access to substantial
`storage resources |possibly through an internal device such as a CD drive, or through
`substantial communication and networking resources cell phone, pager, data radio in-
`terface providing access to other storage servers such as the Web| an audio input
`and output interface with speech generation and potentially sophisticated voice recogni-
`tion, and a video input and output interface. The video input a video camera could be
`pointed at the user to interpret user gestures, or pointed at the environment to interpret
`objects or symbols in the environment. The video output could be integrated into the
`main screen or be a separate video display device, such as an attached screen or heads up
`display on glasses worn by the user.
`One major application of mobile context-aware devices are personal guides. Museums
`could provide these devices and allow users to take personalized tours seeing any exhibits
`desired in any order, in contrast to today’s taped tours.
`In fact, many museums now
`provide portable devices for just such a purpose, but what we are envisioning is a device
`that would allow the tourist to go anywhere she pleases and be able to receive information
`about anywhere she is. Walking tours of cities or historical sites could be assisted by these
`electronic guidebooks. The hand-held devices could use position measurement systems
`such as indoor beacons or the Global Positioning System GPS to locate the user, and
`an electronic compass or inertial navigation system to nd user orientation. Objects
`of interest could be marked with visual markers or active beacons or recognized using
`computer vision. Some objects, such as animals at a zoo or aquarium, might be dicult
`to mark but could be recognized with simple computer vision and some assistance from
`the environment indications that this is the elephant cage, for example. The personal
`guide could also assist in route planning and providing directions. Some of these functions
`are currently being provided by automobile on-board navigation systems.
`There are other ways to assist users. Consider a traveler in Japan that does not speak
`or read Japanese. The hand-held device could act as a pocket multilingual dictionary,
`actually speaking the appropriate phrase with the appropriate pronunciation to a taxi
`driver, for example or even showing the appropriate Kanji and an associated map on
`the screen. A device that included video input or a scanner could assist in reading signs
`or menus. A device that could show stored images might be able to show a shopkeeper
`the desired object or favorite meal. Another more futuristic use is to assist the user by
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 5 of 22 PageID #: 25555
`
`APL-MAXELL_00713090
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`recognizing faces at a cocktail party and reminding the user who people are.
`Real-time communication allows a personal device to act as an agent for the user. A
`personal guide to a theme park could make reservations at particular rides, and alert the
`user when the reservation was available. The device could also tell the user which rides
`had the shortest lines. Similar approaches are currently being used for automobile trac
`management in major cities.
`An important application of context-aware devices is enhanced reality. A heads up
`display could provide X-ray" vision for the user. While surveying a building for reno-
`vation, the location of hidden plumbing or electrical conduits could be indicated to the
`user, based on information from sensors andor building plans. At an archeological site
`a visitor could be provided with various overlays indicating what used to be above the
`current ground level as well as what is below the current ground level.
`Context-aware devices can also be used as tools. Simple sonar devices are used to
`make room measurements today. It would not take much to have a hand-held device that
`both videotaped and mapped a room along with user commentary. An ecological eld
`study or an archeological dig could be assisted by a device that automatically recorded
`the context of a particular nd, including noting the surrounding objects. Consider an
`electronic eld guide that assisted the user in recognizing plants or insects.
`One of the most interesting applications of context-aware devices is to support group
`interaction on a tour or in a classroom, for example. Participants in a live demonstration of
`some new technology could use their personal device to help steer the demo using majority
`voting or consensus among the viewers. Each participant could run a personalized version
`of the same demo by expressing their own choices. In this case context is which demo a
`participant is participating in or attending to, and the personal machine may switch to
`another context if it detects the user is attending to that context instead.
`Many tourists take records of some sort of their travelling experiences, either by taking
`pictures or videos or by composing a travel diary. Imagine the possibilities if the recording
`of these experiences could be more eciently and accurately recorded. A drive across the
`country could result in a trail superimposed upon a map, and clicking on the trail would
`reveal an image of what you could see at that moment |an automatically-generated
`spatial index into your memories.
`These are but a few of the possibilities we can imagine that a context-aware applica-
`tion can provide for the tourist. We have investigated many of these possibilities already
`and report on them later.
`
` Related Work
`
`In thinking about and developing a location-aware application, we were greatly inuenced
`by work such as the PARCTab at Xerox PARC  , the InfoPad project at Berkeley ,
`the Olivetti Active Badge system   and the Personal Shopping Assistant proposed
`at AT&T  . We wanted to build useful applications that might take advantage of the
`hardware developed in the PARCTab and InfoPad projects. We did not want to build our
`own hardware, so we have a dierent focus from all of these projects. There are a number
`of commercially available and relatively inexpensive hand-held units that would suce
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 6 of 22 PageID #: 25556
`
`APL-MAXELL_00713091
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`for our purposes, such as the Apple MessagePad with the Newton Operating System , a
`MagicCap machine or a pen-based palmtoptablet PC. We chose to work initially with
`the Apple MessagePad with Newton . and pen-based PCs running Windows for
`Pen Computing .. Each platform was available for $  with relatively powerful
`development environments. This low cost of hardware was critical to the success of
`Cyberguide because it made it possible to put a number of units in the hands of many
`students, all with dierent ideas that they were allowed to investigate.
`For positioning, we considered the Active Badge system, but rejected it for reasons
`of cost and long-term objectives. The Active Badge system combines position detection
`with communication. For room-level granularity of position, this is reasonable since the
`communications range is on par with the position resolution. With Cyberguide, it is not
`clear that positioning and communication systems should always share physical resources.
`Certain versions of our prototype did; other prototypes did not. We provided for the
`separation of the wireless communications capabilities from the positioning system, so we
`could seek out more cost-eective solutions for both.
`We tried to pay attention to the higher level conceptual design of Cyberguide, but we
`have not been as general in our handling of context-aware mobile objects as has Schilit
` .
`
` Architecture of Cyberguide
`
`From the beginning, we have viewed Cyberguide as a family of prototypes and not just
`a single prototype, so it made sense to think about a conceptual design, or architec-
`ture, that captured the essence of the mobile tour guide. We have divided the system
`into several independent components, or building, and have found it useful to present
`those components both in terms of the generic function and personied in terms of the
`people a tourist would like to have available while exploring unfamiliar territory. The
`overall system serves as a tour guide, but we can think of a tour guide as playing the
`role of cartographer, librarian, navigator and messenger. The services provided by these
`components are:
`
` Cartographer Map Component This person has intimate knowledge of the
`physical surroundings, such as the location of buildings, interesting sights within
`a building, or pathways that the tourist can access. This component is realized in
`our systems by a map or maps of the physical environments that the tourist is
`visiting.
`
` Librarian Information Component This person provides access to all of the
`information about sights that a tourist might encounter during their visit. This
`would include descriptions of buildings or other interesting sights and the identities
`of people associated with the areas. The librarian can answer specic question
`about certain sights Who works in that building?" or What artist painted that
`
` MessagePad and the Newton Operating System are registered trademarks of Apple Com-
`puter, Inc.
`MagicCap is a registered trademark of General Magic, Inc.
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 7 of 22 PageID #: 25557
`
`APL-MAXELL_00713092
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`picture?" or What other demonstrations are related to what I am looking at?".
`This component is realized as a structured repository of information relating to
`objects and people of interest in the physical world.
`
` Navigator Positioning Component The interests of the tourist lie relatively
`close to their physical location. Therefore, it is important to know exactly where
`the tourist is, in order to show the immediate surroundings on the map or answer
`questions about those surroundings What am I looking at?". The navigator is
`responsible for charting the location of the tourist within the physical surround-
`ings. This component is realized by a positioning module that delivers accurate
`information on tourist location and orientation.
`
` Messenger Communications Component A tourist will want to send and
`receive information, and so the messenger provides a delivery service. For example,
`when visiting an exhibit or demonstration, the tourist might want to speak with the
`owner of the exhibit. If the owner is not present, the tourist can leave a message.
`In order to nd out where other tourists are located, each tourist can communicate
`her current location to some central service that others can access. It might also
`be desirable to broadcast information to a set of tourists The bus will be leaving
`from the departure point in  minutes.". This component is realized as a set of
`wireless communications services.
`
`The utility of this architectural decomposition for Cyberguide is that it provides an ex-
`tensible and modular approach to system development. It is extensible because we can
`always add further services. For example, we have considered adding an historian whose
`purpose is to document where the tourist has been and what her reactions were to the
`things she saw. It is modular because it has allowed us to change the implementation of
`one component of the system with minimal impact on the rest of the system. For exam-
`ple, we have implemented dierent versions of the navigator and the librarian without
`having to alter the other components. Of course, these components are related in some
`ways; for instance, position information ultimately has to be translated into a location on
`the physical map. Dening standard interfaces between the components is the means by
`which we achieve separation between and coordination among the various components.
`
` The Indoor Cyberguide
`
`In this section, we describe how each of the separate modules in the conceptual archi-
`tecture have been realized in the initial series of prototypes developed on the Apple
`MessagePad for use indoors during GVU open houses.
`
`. Map Component
`
`The initial map module, shown on the left side of Figure , contains a map of the entire
`GVU Center. Passageways and demonstration stations stars in Figure  are shown.
`Only a limited view of the lab can be seen at any given time. The user can scroll the map
`around and zoom in and out to see alternative views. There is an icon to show the user’s
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 8 of 22 PageID #: 25558
`
`APL-MAXELL_00713093
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`Figure : The map left and information right interfaces of the initial Cyberguide
`MessagePad prototype.
`
`location on the map. Using information from the positioning module, we implemented
`automatic scrolling of the map. If desired, the user’s position is updated automatically
`and the map is scrolled to ensure that the user’s current position remains on the visible
`portion of the map.
`
`.
`
`Information Component
`
`The information module shown on the right side of Figure  contains information about
`each of the demos on display at the GVU open house. This includes abstracts of the
`project being demoed, background information on those involved with the project, as
`well as where to get further information. The location of each demo is marked on the
`map by a star. The user selects the star icon for a demo to reveal its name. Selecting
`the name brings up the information page for that demo. The user can also go directly
`into the information module and search for information for specic demo pages either by
`category or by project name.
`One version of the information module was hard-coded, providing very fast response
`but requiring a recompilation every time demo information needed to be updated. An-
`other implementation used Newton les, called soups, to store information. The use of
`soups avoided hard-coding data into the application and simplied demo information
`updates, but did not have adequate response time. Our third implementation of the in-
`formation module used Newton Books, the Newton platform documentation viewer, to
`store the demo information. The use of Newton Books improved our access time consid-
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 9 of 22 PageID #: 25559
`
`APL-MAXELL_00713094
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`Figure : Questionnaire using communications module for delivery.
`
`erably, allowing for an automated information update process without requiring data be
`hard-coded directly into the application. Throughout all three versions of our information
`module, we were able to modify the information module independent of the development
`eorts of the other modules, validating the modularity of part of our design.
`
`. Communication Component
`
`Our initial implementation of a communication module consisted of a wired Internet
`Appletalk connection from a Apple MessagePad through a Unix Appletalk Gateway. We
`designed an application level protocol on top of a public domain implementation of the
`Appletalk protocol for Solaris. This allows us to open a connection-based Appletalk
`stream from the Apple MessagePad to a UNIX platform. We then invoked our gateway
`application to repacketize Appletalk packets into TCPIP packets for transmission over
`the Internet. This allowed for TCPIP connectivity from a Apple MessagePad via an
`Appletalk connection. We could then fetch HTML documents as well as send and receive
`e-mail. We utilized this functionality within Cyberguide by providing a questionnaire for
`users to complete, which was sent to the developers as an e-mail message. see Figure 
`
`. Position Component
`
`Position is the obvious starting point for a context-aware mobile device. We considered
`several methods for sensing the user location. Outdoors, continuous services, such as GPS,
`can be used. Indoors, however, GPS signals are weak or not available. We considered RF
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 10 of 22 PageID #: 25560
`
`APL-MAXELL_00713095
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`Figure : IR positioning prototype left and the array of positioning beacons in the GVU
`Lab right.
`
`for indoor position measurement, but found o the shelf solutions too expensive.
`One solution for an indoor positioning system was to use infrared IR. Our rst
`positioning system was based on using TV remote control units as active beacons, and
`using a special IR receiver tuned to the carrier frequency  kHz of those beacons
`Figure . A microcontroller Motorola   interfaced the IR receiver to the serial
`port of the Apple MessagePad. We deployed an array of remote controls hanging from
`the ceiling Figure right, each remote control acting as a position beacon by repeatedly
`beaming out a unique pattern. The   translates the IR pattern into a unique cell
`identier that is sent to the Apple MessagePad’s serial port. As the tourist moves around
`the room and passes into the range of a new cell, the position indicated by an arrowhead
`is updated on the map. Keeping track of the last recorded cell location provides a good
`guess as to the location the tourist is heading, so we indicate an assumed orientation by
`pointing the position icon accordingly.
`The remote control system is too expensive for large scale use as the cost of the  
`microcontroller is roughly equivalent to that of the MessagePad.
`
` Extending the initial prototype
`
`The rst Cyberguide prototypes were completed within  months. To test out the gener-
`icity of our architectural approach, we decided to develop further prototypes that altered
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 11 of 22 PageID #: 25561
`
`APL-MAXELL_00713096
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`Figure : The outdoor Cyberguide left with GPS unit right.
`
`one or more of the major components described in Section  and increased overall func-
`tionality. We describe these extended prototypes here.
`
`. Outdoor positioning
`
`There were several motivations for building a Cyberguide prototype for outdoor use Fig-
`ure . First, we wanted to use Cyberguide over a wider area than the relatively small
`GVU Center. We also wanted to test the modularity of our design by having to change
`critical features. The two features that were changed on this prototype were the underly-
`ing map and the physical positioning system. We obtained a dierent map and inserted
`that into the map module without any problems. For positioning, we replaced the IR
`positioning module with a Trimble GPS unit attached to the Apple MessagePad serial
`port. see right side of Figure . The GPS unit sends a position in latitude and longitude
`which was then translated into a pixel coordinate representing the user’s current position
`on the map.
`The outdoor positioning system has been tested by two prototypes. We rst built a
`proof of concept tour of the Georgia Tech campus shown in Figure . We also developed
`a more functional outdoor prototype that covered three surrounding neighborhoods of the
`campus, described later
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 12 of 22 PageID #: 25562
`
`APL-MAXELL_00713097
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`. Alternate platforms
`
`In order to verify the platform independence of our conceptual design, we initiated two
`separate eorts building pen-based PC versions of Cyberguide. These limited functional-
`ity PC versions were written using Borland’s Delphi environment and Microsoft’s Visual
`Basic. Both were initially installed on Dauphin DTR- palmtops running Pen for Win-
`dows Computing ..
`The Delphi version implements the map and information module. Web pages contain-
`ing demo information are stored locally as database objects using a stand-alone Borland
`database engine. The information base is extracted from the collection of existing Web
`pages for GVU projects but stored locally. This information is viewed using a public
`domain Delphi HTML viewer. Though this provided a very fast response for information
`queries, it is a long-term disadvantage to have the information base stored locally. Too
`much in our environment is subject to change. A local and static database is only slightly
`more useful than a book. This approach to static information storage is used currently
`for on-board navigation systems on certain rental cars.
`The Delphi prototype uses vector-based maps, allowing for arbitrary scaling and
`rotation of the map and well as path generation. While there are several sources for
`obtaining vector maps for outdoor regions, it is not so easy for indoors. Consequently,
`there is a trade-o between the easily generated but limited functionality of bitmap images
`and the highly functional but hard to generate vector maps for indoor use.
`The Visual Basic prototype, shown in Figures  realizes all four components of Cy-
`berguide, including two-way communications, which is discussed next. We implemented
`historical context by predicting when a user had visited a demo, based on time spent
`in the area of the demo and interaction with the map. In Figure , a visited demo is
`indicated on the map by a checkmark. There is also a separate panel that lists the demos
`visited. This information could be used, for example, to generate a summary of the day’s
`visit to GVU open house and then mailed o to the visitor. We again made use of a
`publicly available HTML rendering component to display the project descriptions, still
`stored locally.
`
`.
`
`Increased communication
`
`A number of interesting possibilities are enabled for the tourist in a wireless communi-
`cation mode. We have spent a good deal of eort building an indoor, low-cost wireless
`communications infrastructure for Cyberguide. We have built a serial IR network us-
`ing inexpensive modules from Sharp the same modules used in the MessagePad. We
`have written UNIX server software and client software for both the MessagePad and PC.
`Figure  shows how our homemade network connects mobile units to the departmental
`network. We have dened a simple protocol to support three dierent kinds of messages:
`mailing out from a mobile unit to the network as shown in Figure ; broadcasting
`from the network to all mobile units Figure ; and updating positioning information.
`We implemented this protocol over serial IR instead of some other standard so that we
`could immediately use all platforms UNIX, Newton and Windows. Given the appro-
`priate hardware and protocol support perhaps IRDA, we sencould provide the same
`functionality more robustly.
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 13 of 22 PageID #: 25563
`
`APL-MAXELL_00713098
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
` 
`
`Figure : The main map interface of the PC Cyberguide. Checks on demo sites indicate
`the user has been to visit that demo already, indicating history-sensitive interface.
`
`Figure : The information browser interface of the PC Cyberguide.
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 14 of 22 PageID #: 25564
`
`APL-MAXELL_00713099
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
`
`
`Figure : Home-made ir units allow cross-platform communication.
`
`Figure : The mail interface of the PC Cyberguide. Addresses are automatically lled in
`and message templates are supplied upon request.
`
`

`

`Case 5:19-cv-00036-RWS Document 457-2 Filed 07/30/20 Page 15 of 22 PageID #: 25565
`
`APL-MAXELL_00713100
`
`G. D. Abowd et al. Cyberguide: A Mobile Context-Aware Tour Guide
`
` 
`
`Figure : A demonstration of receiving a broadcast message from the Network to an
`individual Cyberguide unit.
`
`.
`
`Increased interaction with environment
`
`All of our applications of Cyberguide so far have restricted the role of the tourist to
`browsing, but it is likely that as she visits some place, the tourist will want to keep
`a record of her experience as advice for herself or others later on. With this idea of
`increased interaction with the environment and recording in mind, we created another
`Cyberguide prototype, called CyBARguide, to assist a tourist in pursuit of refreshment at
`neighborhood establishments in Atlanta. This prototype covers approximately  square
`miles of midtown Atlanta, using multiple maps at varying levels of detail.
`Figure shows the map interface on the left and a view of a user-modiable database
`for interesting establishments on the right. The tourist can indicate a desired destination
`and as she moves around, CyBARguide automatically chooses the map of the highest
`detail that contains both the traveler indicated by a triangle in Figure , and the
`destination the beer m