Context-Aware Computing Applications
`Bill Schilit*
`Roy Want
`Norman Adams
`Palo Alto Research Center
`Computer Science Dept
`Palo Alto Research Center
`Xerox Corporation
`Xerox Corporation
`Columbia University
`Palo Alto, CA 94304
`Palo Alto, CA 94304
`New York, NY 10025
`
`Abstract
`This paper describes systems that examine and re-
`act to an individual's changing context. Such systems
`can promote and mediate people's interactions with de-
`vices, computers, and other people, and they can help
`navigate unfamiliar places. We believe that a limited
`amount of information covering a person's proximate
`environment is most important for this form of com-
`puting since the interesting part of the world around us
`is what we can see, hear, and touch. In this paper we
`define context-aware computing, and describe four cat-
`egories of context-aware applications: proximate selec-
`tion, automatic contextual reconfiguration, contextual
`information and commands, and context-triggered ac-
`tions. Instances of these application types have been
`prototyped on the PARCTAB, a wireless, palm-sized
`computer.
`1
`Introduction
`Our investigation focuses on an extended form of
`mobile computing in which users employ many differ-
`ent mobile, stationary and embedded computers over
`the course of the day. In this model computation does
`not occur at a single location in a single context, as in
`desktop computing, but rather spans a multitude of
`situations and locations covering the office, meeting
`room, home, airport, hotel, classroom, market, bus,
`etc. Users might access their computing resources
`from wireless portable machines and also through sta-
`tionary devices and computers connected to local area
`networks.
`We call this collection of mobile and stationary
`computing devices that are communicating and coop-
`erating on the user's behalf a mobile distributed com-
`puting system. This form of computing is broader than
`mobile computing because it concerns mobile people
`not just mobile computers. These systems aim to pro-
`vide ubiquitous access to information, communication,
`and computation.
`One significant aspect of this emerging mode of
`computing is the constantly changing execution en-
`vironment. The processors available for a task, the
`devices accessible for user input and display, the net-
`work capacity, connectivity, and costs may all change
`• Visiting researcher Xerox Palo Alto Research Center.
`1 This work was supported by Xerox. Portions were also sup-
`ported by ARPA under contract DABT63-91-C-0027.
`
`over time and place. In short, the hardware configura-
`tion is continually changing. Similarly, the computer
`user
`move from one location to another, joining
`and leavmg groups of people, and frequently interact-
`ing with computers while in changing social situations.
`2 Context-Aware Computing
`One challenge of mobile distributed computing is
`to exploit the changing environment with a new class
`of applications that are aware of the context in which
`they are run. Such context-aware systems adapts ac-
`cording to the location of use, the collection of nearby
`people, hosts, and accessible devices, as well as to
`changes to such things over time. A system with these
`capabilities can examine the computing environment
`and react to changes to the environment.
`Three important aspects of context are: where you
`are, who you are with, and what resources are nearby
`(see Figure 1). Context encompasses more than just
`the user's location, because other things of interest
`are also mobile and changing. Context includes light-
`ing, noise level, network connectivity, communication
`costs, communication bandwidth, and even the social
`situation; e.g., whether you are with your manager or
`with a co-worker.
`We are investigating these kinds of applications us-
`ing the PARCTAB [1, 7], a small hand held device
`which uses an infrared-based cellular network for com-
`munication. The tab acts as a graphics terminal, and
`most applications run on remote hosts. This design
`exploits remote processing power to achieve a smaller
`and cheaper device. For input, the tab has three
`finger-operated buttons on the grip, and a touch sen-
`sitive screen. For output, the tab has a 128x64 pixel
`display and a piezo-electric speaker. When wired with
`an infrared transceiver, a room becomes a cell in the
`infrared network. The total bandwidth available for
`all tabs in a cell is 19,200 bps.
`The combination of room-sized cells, and tabs that
`periodically send an identifying packet (i.e., beacon),
`permits accurate location monitoring even when the
`device is not being used. The system notifies applica-
`tions of location changes, and also provides location
`information to a public service that collects and redis-
`tributes information about objects and their locations
`[9]. Other systems might learn about location by other
`means: for example, by using global positioning (GPS)
`or dead-reckoning, or simply by monitoring worksta-
`
`0-8186-6345-6/95 $04.00 © 1995 IEEE
`
`85
`
`Oracle Exhibit 1005, Page 1
`
`

`

`tion interactions. An alternative to the tab svstem's
`locating scheme - where the mobile devices
`-
`is a scheme where stationarv devices broadcast a mes-
`sage identifying their fixed l;cation. Mobile computers
`can listen to these broadcasts to determine their own
`locations.
`The devices and locating strategy described above
`combine to form a mechanism for building context-
`aware applications. In the following sections we de-
`scribe four categories of context-aware applications.
`These categories are the product of two points along
`two orthogonal dimensions (see Table 1): whether the
`task at hand is getting information or carrying out
`a command, and whether it is effected manually or
`automatically.
`2.1 Proximate Selection
`Proximate selection is a user interface technique
`where the located-objects that are nearby are empha-
`sized or otherwise made easier to choose. In general.
`proximate selection involves entering a "locus" and a
`"selection." However, of particular interest are user
`interfaces that automatically default the locus to the
`user's current location.
`There are at least three kinds of located-objects
`that are interesting to select using this technique. The
`first kind is computer input and output devices that
`require physical interaction. This includes printers,
`displays, speakers, facsimiles, video cameras, ther-
`mostats, and so on. Another example is people in the
`same room to whom you would like to "beam" a doc-
`ument. The second kind is situated non-physical ob-
`jects and services that are routinely accessed from par-
`ticular locations; for example, bank accounts, menus,
`and lists of instructions or regulations. The third
`kind is the set of places one wants to find out about:
`restaurants, night clubs, gas stations, and stores, or
`more generically, exits and entrances. Consider an
`electronic "yellow pages" directory that, instead of
`the subject divisions of information, sorts represented
`businesses according to their distance from the reader.
`Location information can be used to weight the
`choices of printers that are nearby. Figure 2 shows
`proximate selection dialogs for printers using three
`columns: the name of the printer, the location, and
`a distance from the user. One interface issue is how
`to navigate dialogs that contain this additional loca-
`tion information. For example, should dialogs use the
`familiar alphabetical ordering by name or should they
`be ordered by location. Shown here are (a) alpha-
`betically ordering by name; (b) ordered by proximity:
`(c) alphabetical with nearby printers emphasized; (d)
`alphabetical with font size scaled by proximity, some-
`thing like a perspective view.
`Another factor that proximate selection interfaces
`must take into account is bandwidth requirements.
`Presenting information that changes, either due to the
`user moving or the contents of the dialog changing
`(e.g. other people moving) will cause update network
`traffic. One approach is to view location information
`with more or less precision based on the situation. The
`interfaces in Table 2 are fine-grained -
`the distance
`column requires updating for each change in location
`
`of the locus. In contrast a coarser-grained view of the
`same information might show a zone rather than a dis-
`tance. Driving around town with such a dialog would,
`for example, change only when the viewer, or the ob-
`jects in the selection dialog, crossed the city limits.
`Proximate selection may also be used to choose vir-
`tualobjects. Using the PARcTAB voting application,
`users select previously created ballots either alphabet-
`ically or by the current location. This use of proximate
`selection is helpful when ballots are carried out at par-
`ticular locations - e.g., voting on what snacks to have
`at High Tea - or when you are meeting with a group
`that has just created a ballot.
`User interfaces for proximate selection pose some
`challenges. For example, how can a UI display both al-
`phabetical and proximity information simultaneously.
`imagery may provide a good UI metaphor. Since
`proximate selection may occur on a mobile host, the
`l'I techniques developed must take into account device
`capabilities such as SCreen real-estate and communica-
`tion bandwidth.
`2.2 Automatic Contextual Reconfigura-
`tion
`Reconfiguration is the process of adding new com-
`ponents, removing existing components. or altering
`the connections between components. Typical com-
`ponents are servers and connections are their commu-
`nication channels to clients. However reconfigurable
`components may also include loadable device drivers,
`program modules, hardware elements, etc. In the case
`of context-aware systems, the interesting aspect is how
`context of use might bring about different system con-
`figurations and what these adaptions are.
`\"Ihen a group of people is in one place, the peo-
`ple can easily share the physical objects in that place.
`For example, people in a meeting room share a ta-
`ble that might hold scattered papers, and a white-
`board with diagrams. To promote similar sharing, we
`wrote a multi-user drawing program for the PARCTAB
`which provides a workspace for each room, a sort of
`virtual whiteboard. Entering a room causes an auto-
`matic binding between the mobile host and the room's
`virtual whiteboard. In this way people in the same
`room can easily collaborate using the virtual white-
`board. Moving to a different room brings up a differ-
`ent drawing surface. Automatic reconfiguration cre-
`ates the illusion of accessing this virtual object as if it
`were physical.
`Reconfiguration could be based on other informa-
`tion in addition to location, for example, the people
`present in a room. If a project group is meeting then
`the project whiteboard is active. This change makes
`virtual whiteboards more powerful than their physical
`analogues since a virtual whiteboard can persist from
`meeting to meeting, and can follow participants from
`room to room.
`Contextual reconfiguration might also include oper-
`ating system functions: for example, leaving the disk
`spinning when the mobile has a power connection.
`Schilit and Duchamp [8] describe how an operating
`system can use the memory of nearby idle comput-
`ers for backing store, rather than swapping to a local
`
`86
`
`Oracle Exhibit 1005, Page 2
`
`

`

`Figure 1: A Context-Aware Computing System (PARCTAB)
`
`information
`command
`
`Table 1: Context-Aware Software Dimensions
`
`or remote disk. The context of use, i.e., the hosts
`in the vicinity, define the system configuration, and
`when hosts change location, the configuration adapts
`accordingly.
`Systems that reconfigure based on context are sub-
`ject to the same problems faced by reconfigurable sys-
`tems in general. In addition, if the context is changing
`rapidly it may be distracting to the user or impractical
`(due to performance) to adapt to every change. Also,
`certain adaptions may confuse users, particularly if the
`context is incorrectly reported, if the user is unaware
`of what context an application considers relevant, or
`if the context changes during use. Future work should
`address these issues.
`2.3 Contextual Information and Com-
`mands
`People's actions can often be predicted by their sit-
`uation. There are certain things we regularly do when
`in the library, kitchen, or office. Contextual inform a-
`
`tion and commands aim to exploit this fact. Queries
`on contextual information can produce different re-
`sults according to the context in which they are is-
`sued. Similarly, context can parameterize "contextual
`commands," for example, the print command might,
`by default, print to the nearest printer.
`The location browser is a PARcTAB application
`that views a "location-based filesystem." Directories
`are named after locations and contain files, programs,
`and links. When moving from room to room, the
`browser changes the displayed directory to match the
`viewer's location. For example, when in an office we
`see the occupant's finger plan and calendar files. In the
`public area of our lab we see a general description of
`the research group, and when near the kitchen we see
`directions for making coffee and finding supplies. Lo-
`cation directories are writable so anyone in our group
`can add customized information. A researcher might
`leave an electronic post-it note for colleagues telling
`them when she plans to return to the office.
`
`87
`
`Oracle Exhibit 1005, Page 3
`
`

`

`35-2-1-08
`35-2-3-01
`35-2-1-03
`a
`
`caps
`b
`Room
`Name
`Istance
`t
`caps
`-
`-
`0-
`35-2-1-08
`claudia
`35-2-1-08 30ft
`claudia
`perfector 35-2-3-01
`perfector 35-2-3-01 20ft
`35-2-1-03
`100ft
`snoball
`snoball
`35-2-1-03
`c)
`( d)
`Table 2: UI Techniques for Proximate Selection
`
`caps
`
`35-2-2-00
`
`35-2-1-08
`35-2-1-03
`35-2-2-00
`
`200ft
`
`DIstance
`30ft
`20ft
`100ft
`
`Aside from displaying data files parameterized by
`the viewer's location, the location browser also runs
`programs. Contextual commands of this kind may
`take two forms. First, the appearance of the com-
`mand itself might change depending on context of use.
`For example, when in the library the button to invoke
`a card catalogue database might appear prominently
`whereas it is normally hidden. Second, a command
`may appear the same but produce parameterized re-
`sults. For example the location browser presents a
`migrate button in it's user interface that appears iden-
`tical from room to room. When this button is pressed
`the user's workstation application migrates its win-
`dows to a nearby display.
`The location browser contrasts with Scoreboard
`[15], an application that takes advantage oflarge (3x4
`foot), stationary, displays in public areas. The pro-
`gram monitors which users pass by and are in the
`vicinity and then shows information that they are in-
`terested in, e.g., the latest hockey scores. A third
`example of exploiting contextual information is Lam-
`ming and Newman's activity-based information re-
`In this application, information entered
`trieval [6].
`into the computer is tagged with context keys facili-
`tating future retrieval by using those keys.
`Contextual information and commands pose some
`challenges. Consider businesses and government agen-
`cies that would export contextual information and
`commands to inexpensively advertise and assist large
`numbers of potential customers. People interacting
`in this manner need to ensure securitv and authentic-
`ity of the information. Also, personal customizations
`must somehow coordinate with service providers.
`2.4 Context-Triggered Actions
`Context-triggered actions are simple IF-THE:\'
`rules used
`to specify how context-aware systems
`should adapt. Information about context-of-use in a
`condition clause triggers consequent commands; some-
`thing like living in a rule-based expert system' A num-
`ber of applications can be organized in this way. The
`category of context-triggered actions is similar to con-
`textual information and commands, except that these
`actions are invoked automatically according to previ-
`
`ously specified rules. A sample predicate set might
`include the forms in Figure 3.
`\Ve have experimented with two context-triggered
`action applications, Active Badgel based "Watchdog"
`and tab based "Contextual Reminders." The watch-
`dog program monitors Active Badge activity and exe-
`cutes arbitrary Unix shell commands. On startup the
`program reads a user's configuration file containing a
`description of Active Badge events and actions. En-
`tries are of the form:
`badge location event-type action
`The badge and location are strings that match the
`badge wearer and sighting location. The event-type
`is a badge event type: arriving, departing, settled-in,
`missing, or attention 2 . When a matching event occurs,
`Watchdog invokes the action with a set of Unix envi-
`ronment variables as parameters. These include the
`badge owner, owner's office, sighting location, and the
`name of the nearest host. For example, see Table 4.
`The first example monitors the "coffee" badge-which
`is attached to the coffee maker in the kitchen-and
`plays the rooster sound whenever anyone makes cof-
`fee. The second starts an Emacs window at a nearby
`host whenever the attention signal is received.
`In addition to generating the normal functions of
`reminders at certain dates and times, Contextual Re-
`minders permit a fuller description of the situation for
`when a reminder should occur. A message pops up
`according to when, where, who and what is with you.
`For example, the "next time in the library" or the
`"next time I see Marvin," or "when I'm back at my
`desk." The PARcTAB implementation of Contextual
`Reminders pops up a message on the tab when differ-
`ent situations occur. The user may edit the reminder,
`dismiss it, or ignore it.
`
`1 The Active Badge [14] is a tag that periodically broadcasts
`a unique identifier for the purpose of determining the location
`of the wearer.
`2The badge incorporates a button, and the attention event
`is reported when this button is clicked twice.
`
`88
`
`Oracle Exhibit 1005, Page 4
`
`

`

`I Category
`Example
`after April 15
`Uate and tIme
`between 10 and 12noon
`in room 35-2-2-00
`Location
`with {User Adams}
`Co-location
`Display} having {Features Color}
`L-__________
`Table 3: Predicates for Context-Triggered Actions
`
`Ki tchen arriving
`Coffee
`schilit *
`attention
`
`"play -v 50 isounds/rooster.au"
`"emacs -display $NEARESTHOST:O.O"
`
`Table 4: Watchdog Configuration
`
`The problems of building context-triggered actions
`include how to balance the requirement of timely ex-
`ecution with the need for predictable behavior, when
`systems transition between a number of states it may
`not be desirable to have all the intermediary actions
`triggered, but delaying too long will make the sys-
`tem seem sluggish. Two problems to be addressed are
`the expressiveness of the predicate language, and the
`accuracy and timeliness of the underlying context in-
`formation.
`3 Related Work
`This research is inspired by the ubiquitous comput-
`ing vision [16] and our experiences with PARcTAB, the
`first mobile computing system built to explore and ex-
`ploit context-aware software [1, 7).
`Active Badges developed at Olivetti Research Lab
`(ORL) [14) focused on the hardware design and im-
`plementation of badges and networks of infrared re-
`ceivers. The main software application, an "aid for a
`telephone receptionist," shows a table of names along-
`side a dynamically updating display of locations and
`telephone extensions. Staff wearing badges can have
`telephone calls directed to their current location. The
`original ORL system did not take context into ac-
`count. Badge wearers expressed a desire to control
`call forwarding using context information: who they
`are with, where they are, and the time of day. "Per-
`sonal control scripts" were added to a later version of
`the system to address this issue [13).
`A mechanism for application contextual customiza-
`tion using the idea of a user global execution envi-
`ronment, was described in [10). Similarly "comput-
`ing personae" [2) address how people moving between
`systems can have a persistent computing environment.
`Adaption of user interfaces for wireless mobile systems
`is described in [3]. An application that uses context
`to tag information and facilitate retrieval is presented
`in [6J. In contrast, system level contextual reconfigu-
`ration has been examined in [8].
`Infrastructure issues in location-aware computing
`are described in [11, 4, 12) and issues of information
`dissemination for context-aware software is presented
`
`in [9]. "Situation awareness" for mobile communica-
`tion systems is advocated in [5].
`4 Acknowledgments
`A number of people worked on the location sys-
`tems in our lab, including Marvin Theimer, Mike Spre-
`itzer, Karin Petersen, David Nichols and Phil James.
`Thanks to Rich Gold for Figure 1. John Ellis wrote
`the PARCTAB voting application. David Nichols and
`Marvin Theimer wrote the Scoreboard application.
`Thanks to Natalie J eremijenko and Douglas Terry for
`comments and approval. Finally, we appreciate Mark
`Weiser's leadership in pursuit of the Ubiquitous Com-
`puting vision.
`References
`[1] Norman Adams, Rich Gold, Bill N. Schilit,
`Michael Tso, and Roy VVant. An infrared network
`In Proceedings USENIX
`for mobile computers.
`Symposium on Mobile Cf Location-independent
`Computing, pages 41-52. USENIX Association,
`August 1993.
`[2] Arindam Banerji, David Cohn, and Dinesh
`Kulkarni. Mobile computing personae. In Pro-
`ceedings Fourth Workshop on Workstation Oper-
`ating Systems (WWOS-IV), pages 14-20. IEEE,
`October 1993.
`[3) David Goldberg and Michael Tso. How to pro-
`gram networked portable computers. In Proceed-
`ings Fourth Workshop on Workstation Operating
`Systems (WWOS-IV), pages 30-33. IEEE, Octo-
`ber 1993.
`[4] Andy Harter and Andy Hopper. A distributed
`location system for the active office. IEEE Net-
`work, pages 62-70, January/February 1994.
`[5] Randy H. Katz. Adaption and mobility in wire-
`less information systems. IEEE Personal Com-
`munications, 1(1):6-17,1994.
`
`89
`
`Oracle Exhibit 1005, Page 5
`
`

`

`[10]
`
`[11]
`
`[12]
`
`[6] Michael G. Lamming and William M. l\ewman.
`Activity-based information retrieval: Technology
`in support of personal memory. In F .H. Vogt,
`editor, Personal Computers and Intelligent Sys-
`tems., volume A-14 of IFIP 12th World Congress.
`Proceedings of Information Processing 92, pages
`IFIP, Elsevier Science Publishers (N orth-
`Holland), 1992.
`[7] Bill N. Schilit, Norman Adams, Rich Gold,
`Michael Tso, and Roy Want. The PARCTAB mo-
`bile computing system.
`In Proceedings Fourth
`Workshop on Workstation Operating Systems
`(WWOS-IV), pages
`IEEE, October 1993.
`[8] Bill N. Schilit and Daniel Duchamp. Adaptive
`remote paging for mobile computers. Technical
`Report CUCS-004-91, Columbia Univ. Computer
`Science Dept., February 1991.
`[9] Bill N. Schilit and Marvin M. Theimer. Dissem-
`inating active map information to mobile hosts.
`IEEE Network, 1994. to appear.
`Bill N. Schilit, Marvin M. Theimer, and Brent B.
`In
`Welch. Customizing mobile application.
`Proceedings USENIX Symposium on Mobile cJ
`Location-Independent Computing, pages
`138.
`USENIX Association, August 1993.
`Mike Spreitzer and Marvin Theimer. Providing
`location information in a ubiquitous computing
`environment.
`In Proceedings of the Fourteenth
`A CM Symposium on Operating System Princi-
`ples, pages
`Asheville,
`Dec 1993.
`SIGOPS, ACM.
`Mike Spreitzer and Marvin Theimer. Scalable,
`secure, mobile computing with location informa-
`tion. CACM, 36(7):27, July 1993. In Special Is-
`sue, Computer-Augmented Environments.
`Row Want and Andy Hopper. Active badges and
`personal interactive computing objects.
`IEEE
`Transactions on Consumer Electronics,
`20, Feb 1992.
`Roy Want, Andy Hopper, Veronica Falcao. and
`Jonathan Gibbons. The active badge location
`system. ACM Transactions on Information Sys-
`Jan 1992.
`tems,
`[15] Mark Weiser. The computer for the 21st century.
`Scientific American, 265(3):94-104, September
`1991.
`[16] Mark Weiser. Some computer science issues in
`ubiquitous computing. CA CM, 36(7):74-83, July
`1993. In Special Issue, Computer-Augmented En-
`vironments.
`
`[13]
`
`[14]
`
`90
`
`Oracle Exhibit 1005, Page 6
`
`