Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 1 of 22 PageID #: 24183
`Case 5:19-cv-00036—RWS Documen|!,4lj:a€f_-IA&3 \EflfigRZfimo Page 1 of 22 PageID #: 24183
`PUBLIC VERSION
`
`EXHIBIT C
`
`EXHIBIT C
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 2 of 22 PageID #: 24184
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 \lfgfibiglg‘l/ZO Page 2 of 22 PageID #: 24184
`PUBLIC VERSION
`
`Baltzer Journals
`
`September ‘23, 1996
`
`Cyberguide: A Mobile Context-Aware Tour Guide
`
`GREGORY I). ABOWDI. CITRIS'I‘OPHFR G. AT-Kas0tx". JASON IIONG‘. SUE LONG“,
`Ron KoornaI AND MIKE PINKERTONl
`
`lephies. l-"esunltzot‘ton and t-"sat’iitity Centre
`College of (.'-'r)m.pur‘ing
`Georgia Institute of Technology
`:‘ltltmm. Gil 30332-0280
`
`E—I‘nail: {kooper ,mpinkert , hong , abowd , aged-ace . gatech .edu
`
`.‘ltt‘tmt-‘fl‘fi‘ Cxi
`L’ Wink Comm imitations.
`[i-mail: sue.long@uink.com
`
`Put—urn computing environ ments will free the. user from the constraints of the desk—
`top.
`.-'\p].1lieations For a mobile environment. should take advantage. of contextual
`information. snel': as position. to offer greater services to the. user.
`In this paper,
`we present the Cybergnide project. in which we are building prototypes of a mo-
`bile context—aware Lonr guide. Knowledge of the user‘s current location. as well
`as a liifi'tt'n'y of ].?iL‘-il- locations. are used to provide more of the kind of st'rz‘vices
`that We come to expect from a. real tour guide. we deseribe the architecture and
`features of a variety of ('Tyhergnirle prototypes developed for indoor and outdoor
`use on a number of rlifi’el’ent hand—held plat-forms. We also discuss the general
`research issues that haw. emerged in our context—aware applications development.
`in a mobile environment-t
`
`Keywords: Mobile computing, rontext—awaIr-ness, location—dependent applica—
`tions, hand—held devices
`
`1
`
`Introduction
`
`Future computing environments promise to free the user from the constraints of station—
`ary desktop computing, yet relatively few researchers are investigating what applications
`maximally benefit from mobility. (Jurrent use ol'mobiletechnologyr shows a slow evolution
`from our enrrenl desktop paradigl'n ofeol'npuling, but
`the history of inleraction shows
`that the adoption of new technology usually brings about a radical revolution in the way
`humans use and view technology [11]. Whereas the elt‘ective use of mobile technology
`will give rise to an interaction paradigm shift, it
`is difficult
`to predict what
`that shift
`will be. we follow the advice of Alan Kay. therefore. and choose to predict the future by
`inventing it. Our approach is to think lirst about what activities could be best supported
`by mobile technology and then determine how the. technology would have to work. This
`applications focus is important to distinguishing our work in mobile computing.
`In April “395. we formed the Future. Computing lsiuvironments {l’Cl-I) Group within
`
`APL-MAXELL_00713087
`APL-MAXELL_OO713087
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 3 of 22 PageID #: 24185
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 \tfliztfigpafi4/2O Page 3 of 22 PageID #: 24185
`PUBLIC VERSION
`
`G. D. Aboard et at. Q‘berguidc: A Art’obitc Context-Aware Tour Guide
`
`2
`
`the College of Computing and the Graphics. Visualization and Usability [GVIF Center} at
`Georgia Tech to promote such an applications focus. Our group is committed to the rapid
`prototyping of applications that benefit from the use of emerging mobile and ubiquitous
`computing technologies. Quick development of these futuristic applications allows us to
`predict and shape what our everyday lives will be like when today‘s novel
`technology
`becomes commonplace.
`Applications for a mobile environment should take advantage of contextual informa—
`tion, such as position. to offer greater services to the user.
`In this paper. we present
`the (i‘yberguide project. a series of prototypes of a mobile. hand—held context—aware tour
`guide. Initially, we are concerned with only a small part of the user‘s context, specifically
`location and orientation. Knowledge of the user‘s current location. as well as a history
`of past locations, are used to provide more of the kind of services that we collie to ex-
`pect from a real tour guide. We describe the architecture and features of a variety of
`Cyberg‘nide prototypes developed for indoor and outdoor Ilse on a number of different
`hand—held platforn'is. We also discuss the general research issues that have emerged in
`our experience of developing context-aware applications in a mobile environment. Some
`of these research issues overlap with those that we have considered in applying other
`applications of ubiquitous computing technology.
`The general application domain which has driven the development of Cyberguide
`is tourism. but we have found it necessary to be even more focused in our research.
`The initial prototypes of (.‘yberguide.
`therefore. were designed to assist a very specific
`kind of tourist —a visitor in a tour of the. GVU (i'entcr Lab during our monthly open
`houses. Visitors to a GVU open house are typically given a map of the various labs
`and an information packet describing all of the projects that are being demonstrated
`at various sites. Moving all of the paper-based information into a hand-held. position-
`aware unit provided a testbed for research questions on mobile. context—awarc application
`development .
`The long—term goal is an application that knows where the tourist is what she is
`looking at. can predict and answer questions she might pose. and provide the ability
`to interact with other people and the environment. Our short-term goal was to pro-
`totype versions of (Z‘yberguide on commercially available I’DAs and pen—based PCs in
`which context-awareness simply meant
`the current physical position and orientation of
`the (i'yberguide unit [and since it is hand—held. this locates the user as well). Position
`information improves the utility ofa tour guide application. As the prototypes of Cyber-
`guidc evolve. we have been able to handle more of the user’s context. such as where she
`and others have been. and we have increased the amount in which the tourist can interact
`and communicate with the place and people she is visiting.
`
`1.1 One-mien:
`
`This paper is an extended version of an earlier report on (.‘yberguide [T]. WL‘ discuss the
`evolution of the Cyberguide design and prototype as well as what future research areas
`our experience has uncovered. We begin in Section 2 by describing scenarios for the use
`of context—aware mobile applications.
`In Section 3. we provide context for our research
`within the area of applit.‘ations-centered mobile computing. The generic architecture of
`Cyberguide is explained in Section fl. We will describe in Section 5 the initial realiza—
`tion of the generic components of the Cyberg‘uide architecture. a series of prototypes
`
`APL-MAXELL_00713088
`APL-MAXELL_OO713088
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 4 of 22 PageID #: 24186
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 fflfiéhiW/ZO Page 4 of 22 PageID #: 24186
`PUBLIC VERSION
`
`G. D. Abowd et at. Q‘bergnidc: A .irt’obilc Context-Aware Tour Guide
`
`3
`
`developed for the Apple lt-IessagePad. We will then describe in Section (3 how the initial
`indoor prototypes were extended for use outdoors and for greater interaction with the
`environment. We conclude in Sections 7 and 8 with a discussion of significant
`issues for
`context—aware applications development and how our past experience will influence our
`future development plans.
`
`2 Scenarios for a mobile context—aware application
`
`This section outlines some possible uses for future mobile. context—aware applications.
`Some of these uses are currently being implemented and some are futuristic. We be
`gill with our initial assumptions about what technology we expect Cyberguide to use.
`Tourists are usually quite happy to carry around a book that: describes the location they
`are visiting, so a reasonable packaging would be in the form of a hand—held device. The
`ideal hand-held device will have a screen and pen/finger interface, access to substantial
`storage resources —possihty through an internal device such as a CD drive, or through
`substantial communication and networking resources [cell phone. pager. data radio in—
`terface} providing access to other storage servers {such as the VVebl— an audio input
`and output
`interface with speech generation and potentially sophisticated voice recogni—
`tion. and a video input and output interface. The video input (a video camera) could be
`pointed at
`the user to interpret user gestures, or pointed at the environment to interpret
`objects or symbols in the environment. The video output could be integrated into the
`main screen or be a separate video display device. such as an attached screen 01' heads up
`display on glasses worn by the user.
`One major application of mobile context-aware devices are personal guides. Museums
`could provide these. devices and allow users to take personalized tours seeing any exhibits
`desired in any order.
`in contrast
`to today’s taped tours.
`In fact. many museums now
`provide portable devices for just such a purpose. but what we. are envisioning is a device
`that would allow the tourist to go anywhere she pleases and be able to receive information
`about anywhere she is. Walking tours ofcities or historical sites could be assisted by these
`elect ronlc. guidebooks. The hand-held devices could use position measurement systems
`such as indoor beacons or the Global Positioning System [UPS] to locate the user. and
`an electronic compass or inertial navigation system to find user orientation. Objects
`of interest could be marked with visual markers or active beacons 01' recognized using
`computer vision. Some objects, such as animals at a zoo or aquarium, might be difficult
`to mark but could be recognized with simple computer vision and some assistance from
`the environment (indications that this is the elephant cage. for example]. The: personal
`guide could also assist in route: planning and providing directions. Some ofthese functions
`are currently being provided by automobile on—board navigation systems.
`There are other ways to assist users. Consider a traveler in Japan that does not speak
`or read Japanese. The hand—held device could act as a pocket multilingual dictionary.
`actually speaking the appropriate phrase with the appropriate pronunciation to a taxi
`driver, for example (or even showing the appropriate lianji and an associated map on
`the screen). A device that included video input or a scanner could assist in reacting signs
`or menus. A device that could show stored images might be able to show a shopkeeper
`the desired object or favorite meal. Another more futuristic use is to assist
`the user by
`
`APL-MAXELL_00713089
`APL-MAXELL_OO713089
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 5 of 22 PageID #: 24187
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 \lfliztfigpafiflzo Page 5 of 22 PageID #: 24187
`PUBLIC VERSION
`
`G. D. Aboard et at. Q‘berguidc: A .lrtobitc Context-Aware Tour Guide
`
`4
`
`recognizing faces at a cocktail party and reminding the user who people are.
`Real—time communication allows a personal device to act as an agent for the user. A
`personal guide to a l'llen'ic park could make reservations at particular rides. and alert the
`user when the reservation was available. The device could also tell the user which rides
`
`had the shortest lines. Similar approaches are currently being used for automobile Iratl'ic
`t'nanagement in major cities.
`An important application of context-aware devices is enhanced reality. A heads up
`display could provide "X—ray“ vision for the user. While surveying a building for reno—
`vation, the location of hidden plumbing or elect rica] conduits could be indicated to the
`user. based on information from sensors and for building plans. At an archeological site
`a visitor could be provided with various overlays indicating what used to be above the
`current ground level as well as what is below the current ground level.
`Context-aware devices can also be used as tools. Simple sonar devices are used to
`make room measurements today. It would not take much to have a hand-held device that
`both videotaped and mapped a room along with user commentary. An ecological field
`study or an archeological dig could be assisted by a device that automatically recorded
`the context of a particular find,
`including noting the surrounding object's. Consider an
`electronic field guide that assisted the user in recognizing plants or insects.
`One of the tnost interesting applications of context—aware devices is to support group
`interact ion on a tour or in a classroom. for example. Participants in alive demonstration of
`some new technology could use their personal device to help steer the demo using majority
`voting or consensus among the viewers. Each participant could run a personalized version
`of the same demo by expressing their own choices.
`[11 this case context
`is which demo a
`participant is participating in or attending to. and the personal machine may switch to
`another context it' it detects the user is attending to that context instead.
`Many tourists take records ofsome sort ot'their travelling experiences. either by taking
`pictures or videos or by composing a travel diary. Imagine the possibilities itt'lie recording
`of these experiences could be more efficiently and accurately recorded. A drive across the
`country could result in a trail superimposed upon a map, and clicking on the trail would
`reveal an image of what you could see at that moment —an autonialically-generated
`spatial index into your memories.
`These are but a few of the possibilities we can imagine that a context—aware applica—
`tion can provide for the tourist. We have investigator! many ol‘tlicse possibilities already
`and report on them later.
`
`3 Related Work
`
`In thinking about and developing a location—aware application. we were greatly influenced
`by work such as the PA RC'Tab at Xerox PARC- [14]. the lnfoPad project at Berkeley [T],
`the Olivetti Active Badge system [1/1] and the. Personal Shopping Assistant proposed
`at AleT [3]. We wanted to build useful applications that might take advantage of the
`hardware developed in the PARC-Tab and Int'oPad projects. We did not want to build our
`own hard ware. so we have a different focus from all of these projects. There are a number
`of commercially available and relatively inexpensive hand—held units that would suffice
`
`APL-MAXELL_00713090
`APL-MAXELL_OO713090
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 6 of 22 PageID #: 24188
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 fflfiéttiW/ZO Page 6 of 22 PageID #: 24188
`PUBLIC VERSION
`
`G. D. Aboard et at. Cybergnide: A .li’obilc Context-A were Tour Guide
`
`(,7!
`
`for our purposes, sucll as tlle Apple MessagePad with the Newton Operating Systenll, a
`lelgicCap2 machine or a pen-based palmtop/tablet PC. We chose to work initially with
`tile Apple MessagePad 100 with Newton 1.3 and pen—based PCs running Windows for
`Pen Computing 1.0. Each platform was available for S 150—500 with relatively powerful
`development environments. This low cost of hardware was critical
`to the success of
`(.‘yberguide because it lllade it possible to put a number of units in tile hands of many
`students, all with different
`ideas that
`they were allowed to investigate.
`For positioning, we considered the Active Badge systelll, but rejected it for reasons
`of cost and long-term objectives. The Active Badge system combines position detection
`with communication.
`["or room—level granularity of position, this is reasonable since the
`conlmunicatiolls range is on par with the position resolution. With Cyberguide, it is not
`clear that positioning alld communication systems should always sllare physical resources.
`Certain versions of our prototype did; or her prototypes did not. We provided for the
`separation of' the wireless COI'IIIt'llltlitfatiUIIS capabilities from the positioning system. so we
`could seek out more eost-effertive solutions for both.
`
`We tried to pay attention to the higher level conceptual design of (i'yberguide, but we
`have not been as general ill our handling of context-aware mobile objects as has Schilit
`[13].
`
`4 Architecture of Cyberguidc
`
`From the beginning, we have viewed Cyberguide as a family of prototypes and not just
`a single prototype, so it made sense to tllink about a conceptual design, or architec-
`ture. that captured tlle essence of the mobile tour guide. We have divided the system
`ilrto several
`illdependellt components. or building, and have found it useful to present
`those components bot ll
`ill terms of the generic function and personified ill ternls of the
`people a tourist would like to have available while exploring unfamiliar territory. The
`overall systerll serves as a tour guide, but we can think of a tour guide as playing tlle
`role of cartograpller, librarian, navigator and messenger. The services provided by these
`components are:
`
`0 Cartographer (Map Component.) Tllis person has intimate knowledge of the
`physical surroundings, such as the location of buildings, interesting sights within
`a building. or pathways that the tourist can access. This component is realized in
`our systems by a map (or maps) of the physical environments that the tourist
`is
`visiting.
`
`o Librarian (Information Component) This person provides access to all of the
`information about sights that a tourist might encounter during their visit. This
`would include descriptions of building's or other interesting sights alld the identities
`of people associated with the areas. The librariall can answer specific question
`about certain sights ("W'llo works ill that building?" or "VVllat al't'ist' painted that
`
`lh-lessagePatl and the. Newton Operating System are registered trademarks of Apple (:DI'II-
`pllter, lnc.
`2lt-lagictlflap is a register-art trademark of ('trrnel‘al Magic, lnc.
`
`APL-MAXELL_00713091
`APL-MAXELL_OO713091
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 7 of 22 PageID #: 24189
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 \Elgfibiaig‘l/ZO Page 7 of 22 PageID #: 24189
`PUBLIC VERSION
`
`G. D. Abowd et al. Cybergnide: A .irfobtlc Context-A ware Tour Guide
`
`6
`
`picture?” or "What other demonstrations are related to what I am looking at‘?“).
`This component is realized as a structured repository of information relating to
`objects and people of interest in the physical world.
`
`a Navigator (Positioning Component) The interests of the tourist lie relatively
`close to their physical location. Therefore.
`it
`is irnporlanl
`to know exactly where
`the tourist is in order to show the immediate surroundings 011 the map or answer
`questions about those surroundings (“What am I looking at?“}. The navigator is
`responsible for charting the location of the tourist within the physical surround-
`ings. This component
`is realized by a positioning module that delivers accurate
`information on tourist location and orientation.
`
`0 Messenger (Communications Component) A tourist will want to send and
`receive information. and so the messenger provides a delivery service. For example.
`when visiting an exhibit or demonstration, the tourist might want to speak with the
`owner ofthe exhibit.
`lfthe. owner is not present, the tourist can leave a message.
`In order to find out where other tourists are located. each tourist can communicate
`her current location to some. central service that others can access.
`It might also
`be desirable to broadcast
`informal ion to a set of tourists (“The bus will be leaving
`from the departure point
`in 15 minutes"). This component
`is realized as a set of
`[wireless] communications services.
`
`The utility of this arcl'iitectural decomposition for (ST-yberguide is that it provides an ex—
`tensible and modular approach to system development.
`lt
`is extensible because we can
`always add further services. For example, we have considered adding an historian whose
`purpose is to document where the tourist has been and what her reactions were to the
`things she saW. It is I'nodular because it has allowed us to change the implementation of
`one component of the system with I'ninimal impact on the rest of the system. For exam—
`ple, we have inn'Jlemented different versions of the navigator and the librarian without
`having to alter the other components. 01' course. these components are related in some
`ways; for instance, position informal ion ultimately has to be translated into a location on
`the physical map. Defining standard interfaces between the components is the means by
`which we achieve separation between and coordination among the various components.
`
`5 The Indoor Cyberguide
`
`In this section. we describe how each of the separate modules in the conceptual archi-
`tecture have been realized in the initial series of prototypes developed on the Apple
`li-IessagePad for use. indoors during GV U open houses.
`
`5.1 Map Chromite-n!
`
`The initial map module. shown on the left side of Figure L contains a map of the entire
`GVL' Center. Passageways and demonstration stations [stars in Figure l) are shown.
`Only a limited view ofthe lab can be seen at any given time. The user can scroll the map
`around and zoom in and out to see alternative views. There is an icon to show the user's
`
`APL-MAXELL_00713092
`APL-MAXELL_OO713092
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 8 of 22 PageID #: 24190
`Case 5:19-cv-00036—RWS Documenlt,4lj:a€f_-IA&3 6Efi§96fi4l20 Page 8 of 22 PageID #: 24190
`PUBLIC VERSION
`
`G. l). Jibowd et al. Clyt'mrgnide: A .i'i'otnl'r- Context-A were Tour Guide
`
`t
`
`.Categuies all demos
`Ooemos Cybergurde : Intelligent Mobne‘l‘
`The Cynergutde project has recently
`calm to investigate the issues
`ndeveluping mobile computing
`pplicalions. the initial (yberguide
`rotntype was
`pecifically built to support the GVU
`errm Days as amil alternative to me
`wet-based map inlarrnatlon provlded
`0 visitors. The initial position-aware
`mtotype contains a map of the GW
`emu area together with linked
`nfarrnatinn and the abilily to send
`' etess requests for e-mail and primed
`nformation. future versions at
`
` .
`
`7
`
`obile Tour Guides
`a C;bergunde . Inteliq
`
`FI
`
`“95"“
`
`
`—A-CT|°.. .- -
`
`moo: sue-(I; cesee‘ao
`lN'W‘ Date: Extras
`Undo
`Flnd
`“5““
`Names Date: Extras
`Fnd
`
`Figure l: The map (left) and information {right} interfaces of the initial Cyberguide
`h-lessagel’ad j'u'ototype.
`
`location on the map. Using infori'nation from the pesitioning I'nodule, we implemented
`antot'natic scrolling of the map.
`Iftlt_~.si1't.‘d._ the user‘s position is updated atltol'natieally
`and the map is scrolled to ensure that the user‘s current position remains on the visible
`port ion of the map.
`
`5.2
`
`hrjormm‘nm Component
`
`The information module [shown on the right side of Figure 1) contains information about
`each oi” the demos on display at the (“’1' open house. This includes abstracts of the
`project being demoed. background information on those involved with the project. as
`well as where to get further information. The local ion of each demo is marked on the
`map by a star. The user selects the star icon for a demo to reveal its name. Selecting
`the name. brings up the informal ion page for that demo. Tile user can also go directly
`into the information module and search for information for specific demo pages either by
`category or by project name.
`One version of the information module was hard—coded. providing very last response
`but requiring a reeompilation every time. demo inlormation ne:_~.ded to be updated. An—
`other implementation used Newton files. railed soups, to store inlbrmation. The use of
`soups avoided hard—coding data into the application and sin'iplilied demo iiilbI'ii'ial ion
`updates. but did not have adequate response time. Our third implennmtat ion of the in—
`fimnalimt I'nm‘lule used Newton Books. the Newton platform documentation viewer. to
`store the demo information. The use of Newton Books improved our access time consid-
`
`APL-MAXELL_00713093
`APL-MAXELL_OO713093
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 9 of 22 PageID #: 24191
`Case 5:19-cv-00036—RWS Documenlt,AL.J:a€f_-IA&3 \l/lezlfihlgig‘l/ZO Page 9 of 22 PageID #: 24191
`PUBLIC VERSION
`
`G. D. :Umwd et al. Cb‘bei‘guide: A Mobile C-‘ontext-.~l ware Tour Guide
`
`8
`
`I: yber \"lail VII?
`
`Cyberguide E—mail
`
`I erg welt)
`
`Current value
`
`ill-“MS Dates Extras ' Undo
`
`find
`
`out“?
`
`Figure 2’1 Questionnaire using eornn'lunieations I'nodnle for delivery.
`
`erably. allowing for an automated information update process without requiring data be
`hard-coded directly into the applieal ion. Throughout all three versions of our information
`module, we were. able to modify the information module independent of the development
`efforts of the other modules. validating the modularity of part of our design.
`
`.5, 3 Com in imitation Com pone-Ht
`
`Our initial implementation of a communieation module consisted of a wired Internet
`Appletalk connection from at Apple MessagePad through a Unix Appletalk ('a'ateway. We
`designed an application level protocol on top of a public domain implementation of the
`Appletalk protocol for Solaris[d]_ This allows us to open a connection-based Appletalk
`stream from the Apple h'lessagel’ad to a UNIX platform. We then invoked our gateway
`application to repaeketize Appletalk packets into TCPIIP packets for transmission over
`the Internet. This allowed For 'I‘CP/ll’ connectivity from a Apple Messagel’ad via an
`Appletalk connect ion.
`life. could then fetch HTML documents as well as sand and reeeive
`e—mail. We utilized this fun:_-tionality within C.‘-yberguide by providing a questionnaire for
`users to complete. which was sent to the developers as an e-mail message. (see Figure 2}
`
`5.4
`
`Position(L'r)mponuit‘
`
`for a context—aware mobile device. We considered
`Position is the obvious starting point
`several methods for sensing the user location. Outdoors. continuous services. such as GPS.
`can be used. indoors. however. (”’3 signals are weak or not available. We considered ill"
`
`APL-MAXELL_00713094
`APL-MAXELL_OO713094
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 10 of 22 PageID #: 24192
`Case 5:19-cv-00036—RWS Documenbfigfic gagsgng/m Page 10 of 22 PageID #: 24192
`PUBLIC VERSION
`
`9 F
`
`igure 3: IR. positioning prototype (left) and the array of positioning beacons in the GVU
`Lab (right).
`
`G. D. Abowd et. al. Cb'berguide: A Mobile Context-Aware Tour Guide
`
`for indoor position measurement. but found off the shelf solutions too expensive.
`One solution {or an indoor positioning system was to use infrared ([11). Our first
`positioning system was based on using TV remote control units as active beacons‘ and
`using a special
`IR. receiver tuned to the carrier frequency (a: dflkllz) of those beacons
`[Figure 3). A microcontroller [Motorola 68332) interfaced the IR. receiver to the serial
`port of the Apple MessagePad. We deployed an array of remote controls hanging from
`the ceiling (Figure 3 right], each remote control acting as a position beacon by repeatedly
`beaming out a unique pattern. The 68332 translates the TH. pattern into a unique cell
`identifier that is sent to the Apple MessagePad’s serial port. As the tourist moves around
`the room and passes into the range of a new cell, the position {indicated by an arrowhead)
`is updated on the map. Keeping track of the last recorded cell location provides a good
`guess as to the location the tourist is heading, so we indicate an assumed orientation by
`pointing the position icon accordingly.
`The remote control system is too expensive for large scale use as the cost of the 68332
`microcontroller is roughly equivalent
`['0 that of the Messagel’ad.
`
`6 Extending the initial prototype
`
`The first Cyberguide prototypes were completed within 6 months. To test out the gener—
`icity of our architectural approach, we decided to develop further prototypes that altered
`
`APL-MAXELL_00713095
`APL-MAXELL_OO713095
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 11 of 22 PageID #: 24193
`Case 5:19-cv-00036—RWS Documenbfigfic gagsggKfl/m Page 11 of 22 PageID #: 24193
`PUBLIC VERSION
`
`G. D. Aboard et al. Cb‘herguide: A Mobile Context-A were Tour Guide
`
`It)
`
` .
`
`................... CyberGuItle
`
`
`
`Figure fl: The outdoor Cyberguide (left) with GPS unit {right}.
`
`one or more of the major components deseribed in Section 4 and increased overall fune-
`tionalily. We describe these extended prototypes here.
`
`6.1 Outdoor linsittontngr
`
`There were several motivations for building a (.‘yberguide prototype for outdoor use [Fig—
`ure (I}. First. we wanted to use Cybergnide over a wider area than the relatively small
`GVLJ Center. We also wanted to test the modularity of our design by having to change
`critical features. The two features that were changed on this prototype were the underly—
`ing map and the physical posit ioning system. We obtained a different map and inserted
`that into the map module without any problems. For positioning. we replaced the [It
`positioning module with a Trin'IIJle {JPS unit attached to the Apple MessagePad serial
`port.
`(see right side of [Figure II]. The G PS unit sends a position in latitude and longitude
`which was then translated into a pier coordinate representing the user's :_‘.urreut position
`on the map.
`The outdoor positioning system has been tested by two prototypes. We first built a
`proof of voucepl tour of t he Georgia Tech campus (shown in Figure 4). we also developed
`a more functional outdoor prototype that covered three surrounding neighborhoods of the
`campus. deseribed later
`
`APL-MAXELL_00713096
`APL-MAXELL_OO713096
`
`

`

`Case 5:19-cv-00036-RWS Document 438-4 Filed 07/24/20 Page 12 of 22 PageID #: 24194
`Case 5:19-cv-00036—RWS Documengfigfic gagsgng/m Page 12 of 22 PageID #: 24194
`PUBLIC VERSION
`
`G. D. :Umwd et at. C3bergnidc: A firtobitc Context-A ware Tour Guide
`
`1
`
`l
`
`6.2 :1 tfffi‘f‘tttltt’ pirstfm-ms
`
`In order to verify the platform independence of our conceptual design. we initiated two
`separate efforts building pen—based PC- vcrsions of (if-ybcrguide. These limited functional-
`ity PC versions were written using Borland‘s Delphi environment and Microsoft‘s Visual
`Basic. Both were initially installed on Dauphin DTR.—l palmtops mulling Pen for Win—
`(lows Computing 1.0.
`The Delphi version t]l]|'JlEI'llE'l'lt':S the map and information module. Web pages contain—
`ing demo information are stored locally as database objects using a stand—alone Borland
`database engine. The information base is extracted from the collection of existing “Feb
`pages for GVL' projects but stored locally. This information is viewed using a public
`domain Delphi HTML viewer. Though this provided a very fast response for information
`queries.
`it
`is a long-term disadvantage to have the information base stored locally. Too
`much in our environment is subject to change. A local and static database is only slightly
`more useful than a book. This approach to static information storage is used currently
`for on—board navigation systems on certain rental cars.
`The Delphi prototype uses vector-based maps. allowing for arbitrary scaling and
`rotation of the map and well as path generation. While there are several sources for
`obtaining vector maps for outdoor regions. it is not so easy for indoors. Consequently,
`there is a trade—off between the easily generated but limited fun:'_-.tionality ot' bitmap images
`and the highly functional but hard to generate vector maps for indoor Ilse.
`The Visual Basic prototype, shown in Figures 5—9 realizes all four components of Cy—
`berguide, including two—way communications. which is discussed next. We implemented
`historical context by predicting when a user had visited a demo, based on time spent
`in the area of the demo and interaction with the map.
`In Figure 5, a visited demo is
`indicated on the map by a cheekmark. There is also a separate panel that lists the demos
`visited. This information could be used. for example. to generate a summary of the days
`visit
`to GVU open house and then mailed off to the visitor. we again made Ilse of a
`publicly available HTML rendering component to display the project descriptions. still
`Stored locally.
`
`6,3 Increased cominmnimtirm
`
`A number of interesting possibilities are enabled for the tourist in a wireless comlnuni—
`cation mode. we have spent a good deal of effort building an indoor. low-cost wireless
`communications infrastructure for (if-yberguide. We have built a serial IR. network us—
`ing inexpensive modules from Sharp (the same modules used in the Message-Pad). we
`have written UNIX server software and client software for both the h'lessagel’ad and PC.
`Figure 7 shows how our homemade