s
`
`Fourth Annual
`Symposium on
`User interface
`Software and
`
`Technology
`
`-
`
`Proceedings of the
`ACM Symposium on
`User Interface Software
`
`and Technology
`
`Hilton Head
`
`-
`
`South Carolina, USA
`November 11-13, 1991
`
`Sponsored by ACM SIGGRAPH
`and ACM SIGCHI
`
`Page 1 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`89060653300
`
`I I Ill lllll lllll ll llllll ll llllll ll llll llllll llll lll I I
`b89060653300a
`
`Page 2 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`□
`□
`rri
`rri
`I.J'J
`JJ
`□
`JJ
`□ er
`cO
`
`==co
`
`m
`
`‘ ,fi
`
`..
`
`~
`- ( t l
`
`_
`
`'WgEAJ—ur
`
`=:=::o
`==== o
`===M = I t )
`==:==M
`===o (0 ~o
`=
`_ co =.o
`=====
`
`new1111!)!» 1User Interface Software andTechnolog NoVember1113,1991
`
`Page 3 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`,- - - - - - - - - - - - - - - - - - -
`
`IIH
`<C -o
`WNW
`==== o
`==:=M
`-
`M
`==L I ' )
`===co
`-
`0
`(0
`::::::= 0
`==== CJ)
`::::::= CX)
`al
`
`B89060653300A
`
`D
`D
`n,
`n,
`U"J
`....!l
`D
`....!l
`D
`a(cid:173)
`ct:)
`
`lHllll BHUEDESBBUU
`WIIIIIIHHHlHlllHlHll
`
`Page 4 of 20
`
`GOOGLE EXHIBIT 1005
`
`Page 4 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`OfMCO
`
`KURT F. WENDT UIRARV
`COLLEGE OF ENGINll"ING
`AUG 2 0 1998
`
`UW-MADISON, WI 13701
`
`Page 5 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Fourth Annual
`Symposium on
`User Interface
`Software and
`Technology
`
`Proceedings of the
`ACM Symposium on
`User Interface Software
`and Technology
`
`Hilton Head
`South Carolina, USA
`November 11-13, 1991
`
`Sponsored by ACM SIGGRAPH
`and ACM SIGCHI
`
`Page 6 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`The Association for Computing Machinery
`11 West 42nd Street
`New York, New York 10036
`
`© 1991 by the Association for Computing Machinery, Inc. Copying without fee is permitted
`provided that the copies are not made or distributed for direct commercial advantage, and credit to the
`source is given. Abstracting with credit is permitted. For other copying of articles that carry a code at
`the bottom of the first page, copying is permitted provided that the per-copy fee indicated in the code
`is paid through the Copyright Clearance Center, 27 Congress Street, Salem, MA 01970. For permission
`to republish write to: Director of Publications, Association for Computing Machinery. To copy
`otherwise, or republish, requires a fee and/or specific permission.
`
`ISBN 0-89791-451-1
`
`Additional copies may be ordered prepaid from:
`
`ACM Order Department
`P.O. Box 64145
`Baltimore, MD 21264
`
`ACM Order Number: 429913
`
`Page 7 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Contents
`Symp'osium Committees ...................................................................................... 7
`
`Symposium Program, Monday 11 November 1991
`
`Plenary Speaker
`Software Interfaces for Users with Special Needs
`Robert C. Williges
`Virginia Polytechnic Institute and State University
`
`Virtual Workspaces
`Hybrid Interfaces: Breeding Virtual Bigger Interfaces for Physically
`Smaller Computers
`Steven Feiner & Ari Shamash, Columbia University ..................................................... 9
`
`On Temporal-Spatial Realism in the Virtual Reality Environment
`Jiandong Liang, Chris Shaw, & Mark Green, University of Alberta ........... ...................... 19
`
`The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display
`Pierre Wellner, Rank Xerox EuroPARC & University of Cambridge ................................ 27
`
`Interactive Components
`Buttons as First Class Objects on an X Desktop
`George G. Robertson, D. Austin Henderson, Jr., & Stuart K. Card, Xerox PARC ....... ; ....... 35
`
`EmbeddedButtons: Documents as User Interfaces
`Eric A. Bier, Xerox PARC .................................................. ...... ......................... 45
`
`Interactive Graph Layout
`Tyson R. Henry & Scott E. Hudson, University of Arizona ........................................... 55
`
`Panel: Disabled Users: Why Should We Care, and How Can We Help
`Chair:
`Randy Pausch, University of Virginia
`Alistair Edwards, University of York
`Panelists:
`Beverly H. Williges, Virginia Polytechnic Institute
`Robert C. Williges, Virginia Polytechnic Institute
`
`Banquet
`A Nose Gesture Interface Device: Extending Virtual Realities
`Tyson R. Henry, Scott E. Hudson, & Andrey K. Yeatts, University of Arizona
`Brad A. Myers, Carnegie Mellon University
`Steven Feiner, Columbia University ...................................................................... 65
`
`November 11-13, 1991
`
`UIST'91
`
`3
`
`Page 8 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Symposium Program, Tuesday 12 November 1991
`cscw
`Primitives for Programming Multi-User Interfaces
`Prasun Dewan & Raj iv Choudhary, Purdue University ................................................ 69
`
`MMM: A User Interface Architecture for Shared Editors on a Single Screen
`Eric A. Bier, Xerox PARC
`Steve Freeman, Xerox PARC & Cambridge University ................................................ 79
`
`Comparing the Programming Demands of Single-User and Multi-User Applications
`John F. Patterson, Bellcore ................................................................................. 87
`
`UI Frameworks
`The PICASSO Application Framework
`Lawrence A. Rowe, Joseph A. Konstan, Brian C. Smith, Steve Seitz, & Chung Liu
`University of California, Berkeley .............................................. ........................... 95
`
`An Event-Object Recovery Model for Object-Oriented User Interfaces
`Haiying Wang & Mark Green, University of Alberta .................................................. 107
`
`SUIT: The Pascal of User Interface Toolkits
`Randy Pausch, Nathaniel R. Young II, & Robert DeLine, University of Virginia ................ 117
`
`Input Techniques
`Stylus User Interfaces for Manipulating Text
`David Goldberg, Xerox PARC
`Aaron Goodisman, Xerox PARC & MIT ............................................................... .127
`
`Issues in Combining Marking and Direct Manipulation Techniques
`Gordon Kurtenbach & William Buxton, University of Toronto ...................................... 137
`
`Smoothly Integrating Rule-Based Techniques Into A Direct Manipulation Interface Builder
`Scott H. Hudson & Andrey K. Yeatts, University of Arizona ........................................ 145
`
`Constraint Techniques
`The Importance of Pointer Variables in Constraint Models
`Brad Vander Zanden, University of Tennessee
`Brad A. Myers & Dario Giuse, Carnegie Mellon University
`Pedro Szekely, USC/ISI ................................................................................... 155
`
`A General Framework for Bi-Directional Translation Between Abstract and Pictorial Data
`Shin Takahashi, Satoshi Matsuoka, Akinori Yonezawa, University of Tokyo
`Tomihisa Kamada, Access Co. Ltd ....................................................................... 165
`
`Plenary Speaker
`Escaping Flatland
`Andries van Dam
`Brown University
`
`4
`
`UIST'91
`
`Hilton Head, South Carolina
`
`Page 9 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Symposium Program, Wednesday 13 November 1991
`
`Internationalization
`A Model for Input and Output of Multilingual Text in a Windowing Environment
`Yutaka Kataoka, Waseda University, & Masato Morisaki, NTT
`Hiroshi Kuribayashi, Omron Corporation
`Hiroyoshi Ohara, Waseda University .................................................................... 175
`
`XJp System: An Internationalized Language Interface for the X Window System
`Masato Morisaki & Etsuo Kawada, NTT
`Hiroshi Kuribayashi & Seiji Kuwari, Omron Corp
`Masahiko Narita, Fujitsu Limited ......................................................................... 185
`
`A Flexible Chinese Character Input Scheme
`S. C. Hsu, Cambridge University ........................................................................ 195
`
`UI Builders
`A Unidraw-Based User Interface Builder
`John M. V/issides & Steven Tang, Stanford University ............................................... 201
`
`Separating Application Code from Toolkits: Eliminating the Spaghetti of Call-Backs
`Brad A. Myers, Carnegie Mellon University ................................................ .. .......... 211
`
`A Demonstrational Technique For Developing Interfaces With Dynamically Created Objects
`David Wolber, University of California, Davis
`Gene Fisher, California Polytechnical University ...................................................... 221
`
`Panel: Interface Development Tools - Feast or Famine
`Chair:
`Ed Lee, Hewlett-Packard
`Panelists: Mark Linton, Silicon Graphics
`John Ousterhout, University of California, Berkeley
`Len Bass, Software Engineering Institute
`Frank Hall, Hewlett-Packard
`
`Author Index ..................................................................................................... 231
`
`November 11-13, 1991
`
`UIST'91
`
`5
`
`Page 10 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Buttons as First Class Objects on an X Desktop
`
`George G. Robertson, D. Austin Henderson, Jr., and Stuart K. Card
`Xerox Palo Alto Research Center
`3333 Coyote Hill Road
`Palo Alto, CA 94304
`
`Abstract
`
`A high-level user interface toolkit, called XButtons, has
`been develop ed to support on-screen buttons as first
`class objects on an X window system desktop. With
`the toolkit, buttons can be built that connect user in(cid:173)
`teractions with procedures specified as arbitrary Unix
`Shell scripts. As first class desktop objects, these but(cid:173)
`tons encapsulate appearance and behavior that is user
`tailorable. They are persistent objects and may store
`state relevant to the task they perform . They can also
`be mailed to other users electronically. In add ition to
`being first class desktop objects, XButtons are gesture(cid:173)
`based with multiple actions. They support other inter(cid:173)
`action styles, like the drag and drop metaphor, in addi(cid:173)
`tion to simple button click actions. They also may be
`concurrently shared among users, with changes reflected
`to all users of the shared buttons. This paper describes
`the goals of XButtons and the history of button develop(cid:173)
`ment that led to XButtons . It also describes XButtons
`from the user's point of view. Finally, it discusses some
`implementation issues encountered in building XBut(cid:173)
`tons on top of the X window system.
`
`1
`
`Introduction
`
`Physical buttons have been around since the first elec(cid:173)
`trical devices were built . They are so common that we
`never think about them; push a button and some ac(cid:173)
`tion will take place. On-screen buttons in one form or
`another have been around since the mid-1960 's. Their
`appeal as a human computer interaction technique is
`obvious; arbitrary actions can be invoked by a simple
`
`Permission to copy without fee all or part of this material is
`granted provided that the copies are not made or distributed for
`direct commercial advantage, the ACM copyright notice and the
`title of the publication and its date appear, and notice is given
`that copying is by permissio n of the Association for Computing
`Machinery. To copy otherwise, or to repub lish, requires a fee
`and/or spec ific permission .
`© 1991 ACM 0-89791-451-1/91/0010/0035 ... $1.50
`
`interaction with a displ ay object that looks pressable,
`and the style of interaction is familiar to everyone. It is
`no surprise that many computer systems use on-screen
`buttons as part of their interface. On the other hand,
`very few systems provide buttons that stand on their
`own ( "first class objects") or that allow an end-user to
`create and adapt buttons for their own needs. This kind
`of user tailorable button is what this paper focuses on.
`In this pap er, we defin e a class of on-screen buttons
`that are easy for a user to create and adapt at a very
`high level. For current window systems like X [12], this
`means that the user must perceive the buttons to be
`at the same level as windows, icons, and menus on the
`desktop. Ideally, the user should be able to easily cre(cid:173)
`ate and modify such buttons so that a wide range of
`desired appearances and behaviors is possible. A high(cid:173)
`level user interface toolkit, called XButtons, implements
`this class of tailorable buttons as first class objects on
`an X desktop.
`In the first part of this paper, we describe the goals
`of buttons as first class desktop objects, and some ad(cid:173)
`ditional goals that m ake XButtons even more powerful.
`In the second part, we describe related work on buttons
`and how XButtons is different. The third part of the pa(cid:173)
`per describes how XButtons works from a user's point
`of view. Finally, we discuss some of the implementa(cid:173)
`tion issues for building XButtons within the X window
`system.
`
`1.1 Basic Goals: Buttons as First Class
`Desktop Objects
`
`In general, a button is a display object that has an
`appearance, accepts some triggering input (i.e. , it is
`"pressed"), and has some associated behavior. In most
`current systems, buttons are part of applications; an
`application programmer defines the appearance, place(cid:173)
`ment, and behavior of the button ( often a part of some
`control panel) when the application is built. In a case
`like this, the user has little control over the nature of
`the button, and the button only exists if the applica(cid:173)
`tion is running . Such buttons are clearly not first cl ass
`
`November 11-13, 1991
`
`UIST'91
`
`35
`
`Page 11 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`desktop objects; th ey are second class at least to the
`window in which the app li cation is running .
`To make buttons into first class desktop objects , we
`must first encapsulate the appearance and behavior of
`the button into an object that the user can manipu(cid:173)
`late. We must also ensure that these buttons can exist
`independent of any particular running application. To
`distinguish these first class desktop objects from but(cid:173)
`tons as simple display objects, we will refer to them as
`Desktop Buttons (DButtons, for short).
`To capture behavior independent of application re(cid:173)
`quires that DButtons have a language for expressing
`action. This action language must also be independent
`of specific applications.
`To maximize the flexibility of DButtons, they must
`have some means of storing state. For example, the
`user m ay want to have a DButton start or control some
`process that requires parameters stored with the DBut(cid:173)
`ton. Also, the appearance of the DButton may need to
`reflect the stored state.
`To ensure that the needs of the end-user are ulti(cid:173)
`mately met, DButtons need to be user tailorable [8).
`Users ( not necessarily programmers) should be able
`to easily create and manipulate DButtons. It should
`be easy to change the appearance and placement of a
`DButton as well as its behavior and stored state. To
`make it easy for non-programmers to change the be(cid:173)
`havior of DButtons, the button action language must
`be simple or already familiar to the end-user. Another
`way to ensure tailorability is to make it easy to copy a
`DButton and make incremental changes (for example,
`with a structured editor).
`In use, experience shows that users grow to depend on
`DButtons quickly, so reliability is important. Also, peo(cid:173)
`ple forget where buttons came from, so once lost they
`are hard to replace. Thus, DButtons must be persistent
`across system failures and restarts. Once a user creates
`an interesting DButton, he or she may want to transmit
`it to colleagues . Hence it should be possible to store a
`DButton in a document or send a DButton in an elec(cid:173)
`tronic mail message. Finally, it should be possible to
`group DButtons to perform or support some particular
`task. Hence it should be easy to initiate presentation of
`a group of DButtons simultaneously.
`The XButtons system described in this paper meets
`all of the.se basic goals. The following list summarizes
`the desired attributes for buttons as first class objects
`on a desktop:
`
`• Are independent of applications,
`
`• Encapsulate everything · they need ( appearance,
`placement, behavior, and state),
`
`• Are user tailorable (appearance, placement, behav(cid:173)
`ior, and state),
`
`• Have a simple or fam iliar act ion language,
`
`• Are persistent,
`
`• Can be copied and incrementally changed,
`
`• Can be transmitted (e.g., mailed),
`
`• Can be grouped .
`
`1.2 Additional Goals for XButtons
`
`XButtons is not the first system to implement the goals
`described above. Rooms Buttons [4) and EuroPARC
`Buttons [8) also meet these goals. In both of these sys(cid:173)
`tems, the desktop environment is the Xerox Lisp win(cid:173)
`dow system and the button action language is Lisp.
`XButtons is a furth er evolution of Rooms Buttons. This
`section describ es additional goals for XButtons that dis(cid:173)
`tinguishes them from Rooms Buttons.
`Rooms Buttons have a single action that results from
`pressing the button. XButtons support multiple ac(cid:173)
`tions. Some actions are based on gestures. That is,
`there are a number of possible actions that can result
`from interacting with the button, and the nature of
`the interaction determines the action. For example, a
`back and forth rubbing gesture invokes a rubout action
`(which by default deletes the button after confirmation).
`Other actions are agent based (e.g., time based) .
`The interactions style supported by Rooms Buttons
`is limited to invoking unary operations. XButtons sup(cid:173)
`ports a richer set of interactions. To support binary op(cid:173)
`erations, XButtons supports a drag and drop metaphor.
`For example, you can drag a document icon and drop
`it onto a printer icon to print the docum ent, with
`both icons implemented as XButtons. XButtons sup(cid:173)
`ports building of property sheets, by allowing an ed(cid:173)
`itable type-in fi eld on a button. XButtons supports
`progress indicators, by making the appearance and be(cid:173)
`havior modifiable by actions or other processes.
`Previous DButtons systems supported independent
`buttons with no sharing. XButtons supports a form of
`sharing, so that changes made by one user of a shared
`XButton are seen by other users.
`XButtons are tailorable by non-programmers (mak(cid:173)
`ing it a high-level toolkit). For example, the button
`action language is familiar; the default is the operat(cid:173)
`ing system command language (for X, this is the Unix
`Shell) . Also, it is obvious how to change all the proper(cid:173)
`ties of the XButton; a structured button editor makes
`all potential changes self-evident.
`Finally, XButtons is window manager independent.
`This is particularly important in the X window system
`world , because there are so many window m anagers and
`no obvious industry leader .
`
`36
`
`UIST'91
`
`Hilton Head, South Carolina
`
`Page 12 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`-
`
`The following list summarizes the additional goals for
`XButtons:
`
`• Supports ·multiple actions,
`
`• Is gesture-based,
`
`• Supports common interaction styles,
`
`• Supports shared buttons,
`
`• Is a high-level toolkit,
`
`• Is window manager independent .
`
`2 Related Work
`
`The concept of a button as a display object that ex(cid:173)
`ecutes some action when pressed has been around at
`least since the mid-1960's. Sutherland's SketchPad [19)
`and other Lincoln Lab applications had light but tons.
`These were text display objects that were selected with
`a light pen. They had some of the properties of Desktop
`Buttons, but were part of an application and were not
`user tailorable.
`The early hypertext systems, like PROMIS in 1974
`[16) and ZOG in 1975 [13), had selections or function
`pads that had many of the properties of Desktop But(cid:173)
`tons. These early systems did not have the benefit of
`graphics or bitmap displays, hence were text based and
`did not have the appearance of buttons that is com(cid:173)
`mon today. Even so, they used touch screens for input ,
`so they did support the concept of a pressable display
`object. Later hypertext systems, like Apple's Hyper(cid:173)
`card [2), had similar mechanisms to these early systems
`but capitalized on bitmap displays to give their buttons
`more interesting appearances.
`The development of bitmap display technology in the
`Xerox Alto [20) encouraged further development of the
`button concept. In 1976, the Learning Research Group
`at Xerox PARC [21) built a display simulation of the
`MIT Logo project's Turtle button box. The original
`button box was a physical device. LRG 's simulation
`was one of the earliest versions of button display ob(cid:173)
`jects that actually looked like physical buttons. These
`buttons were far from first class desktop objects, but
`they represent a turning point in the history of buttons,
`because of their appearance.
`One of the first published uses of the term "buttons"
`for button display objects was in the Xerox Star user
`interface [17) in 1982. This system had a "Customer
`Programming Language" option (CUSP) that allowed
`users to defin e their own commands and place CUSP
`buttons into documents. This was close to the first class
`desktop object concept, except that it was embedded in
`
`an application (i.e., the buttons only a ppeared in do c(cid:173)
`uments) and the action language was unfamiliar and
`different from the language used in the rest of the sys(cid:173)
`tem. Metaphor Capsules [7) extended t he idea of CUSP
`buttons. However, Capsules are not directly invoked
`(you open one and then press a "run" button) and their
`appearance is not tailorable.
`Also in the early-1980 's, Trillium [5) had Button ob(cid:173)
`jects that had many of the properties of Desktop But(cid:173)
`tons. Trillium was a user interface design environment
`that was used for designing copier interfaces. Trillium
`Buttons were tailorable objects, but were embedded
`in applications, had no state, were not persistent, and
`could not be mailed .
`The development of icons happened in the same en(cid:173)
`vironment as the development of buttons, and they
`evolved together.
`Icons are like degenerate buttons .
`They typically are used only to start up an applica(cid:173)
`tion or represent a file, fold er , or process. Icons have
`very limited user tailorability.
`
`2.1 Rooms Buttons and Their Progeny
`
`Rooms [4) is a workspace m anager designed to signifi(cid:173)
`cantly reduce display space contention and support lrn(cid:173)
`man multitasking in window-based user interfaces. De(cid:173)
`veloped in the mid-1980's, Rooms introduced the idea of
`buttons as first class objects on a desktop. Rooms But(cid:173)
`tons are used for a variety of purposes, such as moving
`from room to room, building new interfaces, and as(cid:173)
`sisting with user tasks. For example, a door in Rooms
`is a button - when selected, the user passes from one
`Room to another. Rooms Buttons can be passed from
`one Room to another, and from one user to another via
`electronic mail.
`EuroPARC Buttons [8) are extensions of Rooms But(cid:173)
`tons. EuroPARC Buttons are designed to exist in doc(cid:173)
`uments as well as on the desktop . They are still imple(cid:173)
`mented in the Xerox Lisp environment , but have been
`designed to increase the amount of flexibility and tai(cid:173)
`lorability a programmer has in building buttons. For
`exam pie, parameters ( distinguished stored state) and
`specialized structure editors improve tailorability, and
`agency provides a means of timed action.
`In 1989, The Cognitive Coprocessor [14) introduced
`Int eractive Objects, a generalization of Rooms Buttons
`that supports gestures and multiple actions, animation,
`2D or 3D appearances, and an extensible set of inter(cid:173)
`action styles.
`Interactive Objects are also first class
`desktop objects, where th e "desktop" is a 3D Rooms
`world .
`XButtons is based on Rooms Buttons and Interac(cid:173)
`tive Objects. It takes the basic ideas of Rooms Buttons
`as first class objects on a desktop and moves this to
`
`November 11-13, 1991
`
`UIST'91
`
`37
`
`Page 13 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`Figure 1: Sample Set of XButtons.
`
`the Unix and X environment. It then extends the ideas
`to include gestures and multiple actions and some of
`the expanded set of interaction styles from Interactive
`Objects. XButtons goes beyond Interactive Objects by
`supporting shared buttons and window manager inde(cid:173)
`pendence.
`
`2.2 Buttons Embedded in Applications
`
`As mentioned earlier, CUSP buttons [17] are an example
`of the class of buttons that are embedded in some appli(cid:173)
`cation. They only exist in documents. The idea of in(cid:173)
`cluding buttons in documents has been nicely extended
`in the EmbeddedButtons system (1], where document
`editing tools are used to tailor the appearance and be(cid:173)
`havior of buttons. These EmbeddedButtons share most
`of the properties of buttons as first class desktop ob(cid:173)
`jects. The major difference is that these buttons cannot
`exist on the desktop; they are dependent on an app li(cid:173)
`cation (a text editor). Another difference is that the
`button action language is an unfamiliar (although sim(cid:173)
`ple) new language.
`The most common form of buttons embedded in ap(cid:173)
`plications are those based on the wide range of user
`interface toolkits that support buttons . Trillium But(cid:173)
`tons (5] is an early example. Also, all of th e X toolkits
`
`(e.g., Xt [10], XView [3], OLIT (1 8], Motif [11]) pro(cid:173)
`vide some kind of button widget. An application built
`with these toolkits typically provides buttons whose ap(cid:173)
`pearance and behavior is determined by the application
`programmer . The buttons are dependent on a running
`application. They also are not user tailorable and can(cid:173)
`not be moved or mailed.
`XButtons breaks free of this dependence on an em(cid:173)
`bedding application by providing buttons as first class
`objects on the X desktop.
`
`3 Look and Feel of XButtons
`
`There are essentially two sorts of users of XButtons.
`The person who creates an XButton (from whole cloth,
`or via modifying another) sees the XButton "from the
`inside". The person who invokes the behavior of a ere-
`. ated XButton or passes it on to a fri end sees the button
`"from the outside".
`From the point of view of either of these users, XBut(cid:173)
`tons typically appear as small rectan'gular screen ob(cid:173)
`jects, similar to familiar icons. They usually have some
`text t hat indicates what their action is, and may have
`a bitmap displayed with the text . They may also have
`a field for some editable text . Figure 1 shows a group
`
`38
`
`UIST'91
`
`Hilton Head, South Carolina
`
`Page 14 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`of XButtons used for test purposes. Notice that XBut(cid:173)
`tons, like icons, do not have any of the normal window
`dressing provided by most X window managers. The
`appearance of an XButton is completely under control
`of the end-user.
`
`XButtons have multiple actions, which are selected by
`simple mouse or pen gestures. There are two classes · of
`gestures, touching and grasping. The left mouse button
`is used for touching gestures, and the middle mouse
`button supports grasping. Grasping a button is used for
`two purposes: (1) moving a button to a new location;
`and (2) dropping a button onto another button with a
`drop action (e.g., mailing or printing a button).
`
`The two examples in the right column of Figure 1
`show a printer icon XButton and a document icon
`XButton; the user can use drag and drop to print the
`document . The middle column shows a number of other
`XButtons with drop actions: mail a button, print a
`button, and speak the title or text field of a button
`(using a text to speech synthesizer). The left column
`shows a range of appearances and actions for XButtons.
`The "Sample Button" will respond to every gesture by
`speaking the name of the gesture. The "Phone" but(cid:173)
`ton will let you type the name of someone, then pop
`up a window with their phone number (by clicking)
`or dial the number (with the flick right gesture). The
`"Count" button increments a counter and causes the
`current count to be spoken. The button with cherries
`will cycle through different appearances as it is clicked.
`
`XButtons are self-revealing. If the user is unfamiliar
`with the action of a particular button, a menu can be
`popped up to reveal which gestures are supported (and
`what they do). This is done with the right mouse button
`(or by holding down the left mouse button). Figure 2
`shows what a typical button menu looks like. The menu
`items can be selected to execute the actions. In addition
`to the user-defined actions, menu items are provided for:
`copying, deleting, editing, and mailing XButtons; pop(cid:173)
`ping up help information associated with an XButton;
`and sending comments to the XButtons maintainer.
`
`From the user's point of view, XButtons exist on the
`desktop, can be modified there, are persistent, and can
`be mailed from one user's desktop to another. To imple(cid:173)
`ment this, each button has a backing fi le that is trans(cid:173)
`parently maintained (i .e., any change to the button by
`the user or a process is automatically reflected in that
`backing file). When a button is electronically mailed,
`it is actually the contents of this backing file that is
`mailed. However, the end-user need never be aware of
`backing files. In essence, these files provide the encapsu(cid:173)
`lation and persistent of XButtons as first class desktop
`objects.
`
`3.1 XButton Gestures
`
`XButtons support mouse-based or pen-based gestural
`input in addition to simple "pressing". Whenever a
`user gestures at an XButton, a gesture parser inter(cid:173)
`prets mouse or pen movement and classifies it as one
`of a small set of easily differentiated gestures (flick left,
`flick right, flick up, flick down, click, rubout, check, or
`insert). Once a gesture has been identified, the XButton
`executes the appropriate action (i.e., there is an action
`for each of the gestures). During the gesture, feedback
`is provided in the form of a mouse track displayed on
`the screen. As soon as the gesture is completed, the
`feedback is erased.
`Much interesting work has been done recently on
`gesture-based systems. For example, the GEdit sys(cid:173)
`tem [6] uses circling gestures for grouping, moving, and
`copying graphical objects. Since XButtons lives in the
`X world, a gesture must start in an XButton (so that
`the proper process receives the events). This prohibits
`some potentially interesting gestures from being used in
`XButtons (like the circling gesture just described).
`XButtons currently supports a fixed set of gestures.
`Ideally, the set of gestures should be extensible, with a
`trainable gesture recognizer (see [15]) . In spite of the
`small fixed set of initial gestures supported, XButtons
`does provide a testbed for exploring the use of gestures
`on the desktop. At some point in the future, an exten(cid:173)
`sible set of trainable gestures can be added.
`
`3.2 XButton Actions
`
`To make the button action language as familiar as pos(cid:173)
`sible and as high level as possible, the default in XBut(cid:173)
`tons is to use the operating system command language
`(i .e., the Unix Shell). This means that the full range
`of actions the user can do from the Shell can be encap(cid:173)
`sulated into on-screen buttons. To provide maximum
`flexibility, an end-user can specify that some other lan(cid:173)
`guage be used for the action language (e.g., Lisp). Of
`course, to use this flexibility, care must be taken to en(cid:173)
`sure that the selected action language will work in the
`environment of the end-user of the button.
`When the button action shell command is executed,
`arguments are passed to it through environment vari(cid:173)
`ables. Some of these are pre-defined: the button's title
`and identifier, an indication of whether or not the but(cid:173)
`ton has been checked (with the check gesture), and the
`content of the editable text field ( if there is one). The
`user can provide additional arguments (and maintain
`stored state in the button) by specifying any number of
`user-defined properties on the button. Whenever a but(cid:173)
`ton action is executed, all user-d efin ed properti es are
`passed to it in environment variables.
`
`November 11-13, 1991
`
`UIST'91
`
`39
`
`Page 15 of 20
`
`GOOGLE EXHIBIT 1005
`
`

`

`( Click - Get phone It
`
`Flick Right - Dial phone It
`
`Comment
`Copy
`Delete
`Edit
`Help
`
`Figure 2: Sample Button Menu.
`
`A number of button action commands are supported
`to make tailoring buttons easier. Most of these com(cid:173)
`mands ( xbcopy, xbdelete, xbedit, xbhelp, xbmail, and xb(cid:173)
`menu) are designed to ensure that every action that is
`possible from the interface (e.g., from the menu) can
`be assigned to a gesture if the end-user desires. In ad(cid:173)
`dition, there are commands for getting and setting any
`property of a button ( xbgetprop, xbsetprop, and xbprop) .
`These commands allow actions ( or other processe