Exhibit 1014 – Part 8
`
`Exhibit 1014 — Part 8
`
`

`
`5.2 Examples of Direct-Manipulation Systems
`
`F-'ileManager ConlIu|Puue| PnnlMum:ger
`@
`DOS Prompt Windows Setup
`
`Gupta
`'
`
`Paintbrush
`
`Terminal
`
`Notepad
`
`awHe murder
`Uardtule
`Calendav
`Calculator
`
`Figure 5.11
`
`Microsoft Windows 3.0 and other systems on IBM's PS/2 offered variations on the
`direct-manipulation style popularized by the Macintosh. (Screen shot ©1985-1991
`Microsoft Corporation. Reprinted with permission from Microsoft Corporation,
`Redmond, WA.)
`
`ing, and customizing information," HyperCard quickly spawned variants
`such as SuperCard and ToolBook. Each has a scripting language to enable
`users to create appealing graphics applications.
`Checkbook maintenance and searching can be done in a direct manipula-
`tion manner by displaying a checkbook register with labeled columns for
`check number, date, payee, and amount, as is done in the Quicken product
`from Intuit, Inc. Changes can be made in place, new entries can be made at
`the first blank line, and a check mark can be made to indicate verification
`against a monthly report or bank statement. Users can search for a particular
`payee by filling in a blank payee field and then typing a ?.
`Bibliographic searching has more elaborate requirements, but a basic
`system could be built by first showing the user a wall of labeled catalog-
`index drawers. A cursor in the shape of a human hand might be moved over
`to the section labeled Author Index and to the drawer labeled F—L.
`
`Depressing the button on the joystick or mouse would cause the drawer to
`open, revealing an array of index cards with tabs offering a finer index. By
`moving the cursor-hand and depressing the selection button, the user would
`make the individual index cards appear. Depressing the button while
`
`

`
`202
`
`Chapter 5
`
`Direct Manipulation
`
`holding a card would cause a copy of the card to be made in the users
`notebook, also represented on the screen. Entries in the notebook might be
`edited to create a printed bibliography, or combined with other entries to
`perform set intersections or unions. Copies of entries could be stored on user
`files or transmitted to colleagues by electronic mail. It is easy to visualize
`many alternate approaches, so careful design and experimental testing
`would be necessary to sort out the successful comprehensible approaches
`from the idiosyncratic ones.
`Why not do airline reservations by showing the user a map and prompt-
`ing for cursor motion to the departing and arriving cities? Then, use a
`calendar to select the date, and a clock to indicate the time. Seat selection is
`
`done by showing the seating plan of the plane on the screen, with a diagonal
`line to indicate an already-reserved seat.
`The term direct manipulation is accurately applied to describe the program-
`ming of some industrial robot tools. The operator holds the robot ”hand”
`and guides it through a spray painting or welding task while the controlling
`computer records every action. The control computer can then operate the
`robot automatically and repeat the precise action as many times as neces-
`sary.
`Why not teach students about polynomial equations by letting them
`move sliders to set values for the coefficients and watch how the graph
`changes, where the y-axis intercept occurs, or how the derivative equation
`reacts (Shneiderman, 1974). Similarly, direct manipulation of sliders for red,
`green, and blue is a satisfying way to explore color space. Slider-based
`dynamic queries are a powerful tool for information exploration (Section
`11.8).
`
`These ideas are sketches for real systems. Competent designers and
`implementers must complete the sketches and fill in the details. Direct
`manipulation has the power to attract users because it is comprehensible,
`natural, rapid, and even enjoyable.
`If actions are simple, reversibility
`ensured, and retention easy, then anxiety recedes and satisfaction flows in.
`
`Explanations of Direct Manipulation
`
`Several authors have attempted to describe the component principles of
`direct manipulation. An imaginative observer of interactive system designs,
`Ted Nelson (1980), perceives user excitement when the interface is con-
`structed by what he calls the principle of virtual1'ty—-a representation of reality
`that can be manipulated. Rutkowski (1982) conveys a similar concept in his
`principle of transparency: ”The user is able to apply intellect directly to the
`task; the tool itself seems to disappear.“
`
`

`
`5.3 Explanations of Direct Manipulation
`
`203
`
`Heckel (1991) laments that ”Our instincts and training as engineers
`encourage us to think logically instead of visually, and this is counterpro-
`ductive to friendly design.” He suggests that thinking like a filmmaker can
`be helpful for interactive systems designers: "When I design a product, I
`think of my program as giving a performance for its user."
`Hutchins et al. (1986) review the concepts of direct manipulation and offer
`a thoughtful decomposition of concerns. They describe the ”feeling of
`involvement directly with a world of objects rather than of communicating
`with an intermediary,” and clarify how direct manipulation breaches the gulf
`of execution and the gulf of explanation.
`These writers and others (Ziegler and Fahnrich, 1988; Thimbleby, 1990;
`Phillips and Apperley, 1991) support the growing recognition that a new
`form of interactive system is emerging. Much credit also goes to the
`individual designers who have created systems that exemplify aspects of
`direct manipulation.
`Another perspective on direct manipulation comes from the psychology
`literature on problem-solving and learning research. Suitable representations of
`problems have been clearly shown to be critical to solution finding and to
`learning. Polya (1957) suggests drawing a picture to represent mathematical
`problems. This approach is in harmony with Maria Montessori’s teaching
`methods for children (Montessori, 1964). She proposed use of physical
`objects, such as beads or wooden sticks, to convey such mathematical
`principles as addition, multiplication, or size comparison. Bruner (1966)
`extended the physical-representation idea to cover polynomial factoring and
`other mathematical principles. Carroll, Thomas, and Malhotra (1980) found
`that subjects given spatial representation were faster and more successful in
`problem solving than were subjects given an isomorphic problem with a
`temporal representation. Similarly, Te'eni (1990) found that the feedback in
`direct-manipulation designs was effective in reducing users’ logical errors in
`a task requiring statistical analysis of student grades. The advantage appears
`to stem from having the data entry and display combined in a single location
`on the display. Deeper understanding of the relationship between problem
`solving and visual perception can be obtained from Arnheim (1972) and
`McKirn (1972).
`
`Physical, spatial, or visual representations also appear to be easier to
`retain and manipulate than do textual or numeric representations.
`Wertheimer (1959) found that subjects who memorized the formula for the
`area of a parallelogram, A = h X b, rapidly succeeded in doing such
`calculations. On the other hand, subjects who were given the structural
`understanding of cutting off a triangle from one end and placing it on the
`other end could more effectively retain the knowledge and generalize it to
`solve related problems. In plane-geometry theorem proving, spatial repre-
`sentation facilitates discovery of proof procedures over a strictly axiomatic
`representation of Euclidean geometry. The diagram provides heuristics that
`
`

`
`204
`
`Chapter 5
`
`Direct Manipulation
`
`are difficult to extract from the axioms. Similarly, students are often encour-
`aged to solve algebraic word problems by drawing a picture to represent
`that problem.
`Papert’s (1980) LOGO language creates a mathematical microworld in
`which the principles of geometry are visible. Based on the Swiss psychologist
`Jean Piaget's theory of child development, LOGO offers students the oppor-
`tunity to create line drawings easily with an electronic turtle displayed on a
`screen. In this environment, users derive rapid feedback about their pro-
`grams, can determine what has happened easily, can spot and repair errors
`quickly, and can gain satisfaction from creative production of drawings.
`These features are all characteristic of a direct—manipulation environment.
`
`5.3.1
`
`Problems with direct manipulation
`
`Spatial or visual representations are not necessarily an improvement over
`text, because they may be too spread out, Causing off-page connectors on
`paper or tedious scrolling on displays. In professional programming, use of
`high-level flowcharts and database-schema diagrams can be helpful for
`some tasks, but there is a danger that they will be confusing. Similarly,
`direct—manipulation designs may consume valuable screen space and thus
`force valuable information offscreen, requiring scrolling or multiple actions.
`Studies of flowchart usage in programming (Shneiderman, 1982) and in
`business graphics (Tullis, 1981) show that these visual representations can
`produce poorer performance than the equivalent program text or tabular
`presentation, probably because of the low density of information in the
`visual displays. For experienced users, a tabular textual display of 50
`document names may be more appealing than only 10 document graphic
`icons with the names abbreviated to fit the icon size.
`A second problem is that users must learn the meaning of components of
`the visual representation. A graphic icon may be meaningful to the designer,
`but may require as much or more learning time than a word. Some airports
`that serve multilingual communities use graphic icons extensively, but the
`meanings of these icons may not be obvious. Similarly, some computer
`terminals designed for international use have icons in place of names, but
`the meaning is not always clear.
`A third problem is that the visual representation may be misleading.
`Users may grasp the analogical representation rapidly, but then draw
`incorrect conclusions about permissible actions. Ample testing must be
`carried out to refine the displayed objects and actions and to Ininimize
`negative side effects.
`A fourth problem is that, for experienced typists, moving a mouse or
`raising a finger to point may sometimes be slower than typing. This problem
`is especially likely to occur if the user is familiar with a compact notation,
`
`

`
`5.3 Explanations of Direct Manipulation
`
`205
`
`such as arithmetic expressions, that is easy to enter from a keyboard, but
`may be more difficult with mouse-based selection. The keyboard remains
`the most effective direct-manipulation device for some tasks.
`Choosing the right objects and actions is not an easy task. Simple
`metaphors, analogies, or models with a minimal set of concepts seem most
`appropriate to start. Mixing metaphors from two sources may add complex-
`ity that contributes to confusion. The emotional tone of the metaphor should
`be inviting rather than distasteful or inappropriate (Carroll and Thomas,
`1982)—sewage—disposal systems are an inappropriate metaphor for elec-
`tronic message systems. Since the users may not share the metaphor,
`analogy, or conceptual model with the designer, ample testing is required.
`For help in training, an explicit statement of the model, the assumptions, and
`the limitations is necessary.
`
`5.3.2 The SSOA model explanation of direct manipulation
`
`The attraction of direct manipulation is apparent in the enthusiasm of the
`users. The designers of the examples in Section 5.2 had an innovative
`inspiration and an intuitive grasp of what users would want. Each example
`has features that could be criticized, but it seems more productive to
`construct an integrated portrait of direct manipulation:
`
`1. Continuous representation of the objects and actions of interest
`2. Physical actions or presses of labeled buttons instead of complex syntax
`3. Rapid incremental reversible operations whose effect on the object of
`interest is immediately visible
`
`Using these three principles, it is possible to design systems that have
`these beneficial attributes:
`
`0 Novices can learn basic functionality quickly, usually through a dem-
`onstration by a more experienced user.
`Experts can work rapidly to carry out a wide range of tasks, even
`defining new functions and features.
`Knowledgeable intermittent users can retain operational concepts.
`Error messages are rarely needed.
`Users can immediately see if their actions are furthering their goals,
`and, if the actions are counterproductive, they can simply change the
`direction of their activity.
`Users experience less anxiety because the system is comprehensible and
`because actions can be reversed so easily.
`
`Users gain confidence and mastery because they are the initiators of
`action, they feel in control, and the system responses are predictable.
`
`

`
`206
`
`Chapter 5
`
`Direct Manipulation
`
`ACTION
`
`0 B1 ECT
`
`TASK
`
`OBJECT |
`ACT|0N
`COMPUTER
`
`W
`
`SEMANTIC
`
`SYNTACTIC
`
`Figure 5.12
`Users of direct-manipulation systems, may need to have substantial task-domain
`semantic knowledge. However, users must acquire only a modest amount of
`computer-related semantic knowledge and syntactic knowledge.
`
`The success of direct manipulation is understandable in the context of the
`SSOA model. The object of interest is displayed so that actions are directly in
`the high-level task domain. There is little need for the mental decomposition
`of tasks into multiple commands with a complex syntactic form. On the con-
`trary, each action produces a comprehensible result in the task domain that is
`visible immediately. The closeness of the task to the action syntax reduces
`operator problem-solving load and stress. This principle is related to the
`principle of stimu1us—response compatibility in the human-factors literature.
`The task semantics dominate the users’ concerns, and the distraction of
`dealing with the computer semantics and the syntax is reduced (Figure 5.12).
`Dealing with representations of objects may be more "natural” and closer
`to innate human capabilities: Action and visual skills emerged well before
`language in human evolution. Psychologists have long known that spatial
`relationships and actions are grasped more quickly with visual rather than
`linguistic representations. Furthermore, intuition and discovery are often
`promoted by suitable visual representations of formal mathematical systems.
`The Swiss psychologist Iean Piaget described four stages of development:
`sensorimotor (from birth to approximately 2 years), preopcrational (2 to 7
`years), concrete operational (7 to 11 years), and formal operations (begins at
`approximately ll years) (Copeland, 1979). According to this theory, physical
`
`

`
`5.4 Visual Thinking and Icons
`
`207
`
`actions on an object are comprehensible during the concrete operational
`stage, and children acquire the concept of conservation or invariance. At
`about age 11, children enter the formal-operations stage, in which they use
`symbol manipulation to represent actions on objects. Since mathematics and
`programming require abstract thinking, it is more difficult for children, and
`a greater effort must be made to link the symbolic representation to the
`actual object. Direct manipulation brings activity to the concrete-operational
`stage, thus making certain tasks easier for children and adults.
`It is easy to envision use of direct manipulation in cases where the task is
`confined to a small number of objects and simple actions. In complex
`applications,
`it may be more difficult
`to design a direct-manipulation
`interface. On the other hand, display editors provide impressive functional-
`ity in a natural way. The limits of direct manipulation will be determined by
`the imagination and skill of the designer. With more examples and experi-
`ence, researchers should be able to test competing theories about the most
`effective metaphor or analogy. Familiar visual analogies may be more
`appealing to users in the early stages of learning about the system; more
`specific, abstract models may be more useful during regular use.
`
`5.4 Visual Thinking and Icons
`
`The concepts of a visual language and of visual thinking were promoted by
`Arnheim (1972), and were embraced by commercial graphic designers
`(Verplank, 1988), semiotically oriented academics (semiotics is the study of
`signs and symbols), and data-Visualization gurus. The computer provides a
`remarkable visual environment for revealing structure, showing relation-
`ship, and enabling interactivity that attracts users with artistic, right-
`brained, holistic, intuitive personalities. The increasingly visual nature of
`computer interfaces can sometimes challenge or even threaten the logical,
`linear, text-oriented, left-brained, compulsive, rational programmers who
`were the heart of the first generation of hackers. Although these stereo-
`types——or caricatures—will not stand up to scientific analysis, they do
`convey the dual paths that computing is following. The new visual direc-
`tions are sometimes scorned by the traditionalists as WIMP (windows, icons,
`mouse, and pull-down menus) interfaces, whereas the command-line devo-
`tees are seen as inflexible, or even stubborn.
`There is evidence that different people have different cognitive styles, and
`it is quite understandable that individual preferences may vary. Just as there
`are multiple ice-cream flavors or car models, so too there will be multiple
`interface styles. It may be that preferences will vary by user and by tasks. So
`
`

`
`Chapter 5
`
`Direct Manipulation
`
`Figure 5-13
`
`A small set of small, but
`
`effective, icons with no E I3
`
`labels to convey the
`formatting actions of left,
`center, right, and dual
`justification, and a pair of
`action icons plus a label to
`convey printing
`orientation. (©Apple
`Computer, Inc., Cupertino,
`CA. Used with
`permission.)
`
`[Irientation
`
`_-I-_
`
`'
`
`respect is due to each community, and the designer's goal is to provide the
`best of each style and the means to cross over when desired.
`The conflict between text and graphics becomes most heated when the
`issue of icons is raised. Maybe it is not surprising the dictionary definitions of
`icon usually refer to religious images, but the central notion is that an icon is
`an image, picture, or symbol representing a concept (Gittins, 1986; Rogers,
`1989). In the computer world, icons are usually small (approximately 1 inch
`square, or 64 by 64 pixels) representations of a file or program (an object or
`action; see Figures 5.13, 5.14, and 5.15). Smaller icons are often used to save
`space or to be integrated in other objects, such as a window border (see
`Chapter 9). It is not surprising that icons are often used in painting programs
`to represent the tools or actions (lasso or scissors to cut out an image, brush
`for painting, pencil for drawing, eraser to wipe clean), whereas word
`
`DTUIE
`
`5
`
`Mac‘-n-‘rite ||
`
`"“{€~:-xi
`
`."‘-.-"
`Ma-3Pa1'nt
`
`slider‘
`
`9r'U'iiTI f'+.T'
`
`-9-WINDOWS
`
`gru-:|1'n2 rtf
`
`grudin 4 rtf
`
`Figure 5.14
`
`Macintosh icon collection: folder with the manuscript for this book, applications
`MacWrite II and MacPaint, a MacPaint image. On the second line are four
`documents in different formats, as indicated by their varied styles. (Courtesy of
`Claris Corp. Microsoft Word: Screen shot ©1984~1990 Microsoft Corporation.
`Reprinted with permission from Microsoft Corporation, Redmond, WA.)
`
`

`
`5.4 Visual Thinking and Icons
`
`209
`
`Figure 5.15
`
`SAS uses icons with labels to the right to identify different applications. (Screens
`reprinted by permission. Copyright 1991 by SAS Institute Inc.)
`
`processors usually have textual menus for their actions. This difference
`appears to reflect the differing cognitive styles of visually and textually
`oriented users, or at least differences in the tasks. Maybe, while you are
`working on visually oriented tasks, it is helpful to "stay visual” by using
`icons, whereas, while you are working on a text document, it is helpful to
`”stay textual” by using textual menus.
`For situations where both a visual icon or a textual item are possible—for
`example, in a directory listing—designers have two interwoven issues: how
`to decide between icons and text and how to design icons. The well-
`established highway signs are a useful source of experience. Icons are
`unbeatable for showing things such as a road curve, but sometimes a phrase
`such as ONE WAY—DO NOT ENTER is more comprehensible than an icon. Of
`course, the smorgasbord approach is to have a little of each (as with, for
`example, the octagonal STOP sign) and there is evidence that icons plus
`words are effective in computing situations (Norman, 1991). So the answer
`to the first question (deciding between icons and text) depends not only on
`the users and the tasks, but also on the quality of the icons or the words that
`are proposed. Textual menu choices are covered in Chapter 3; many of the
`principles carry over. In addition, these icon-specific guidelines should be
`
`

`
`210
`
`Chapter 5
`
`Direct Manipulation
`
`considered:
`
`0 Represent the object or action in a familiar and recognizable manner.
`Limit the number of different icons.
`
`Make the icon stand out from its background.
`
`Consider three-dimensional icons; they are eye-catching, but also can
`be distracting.
`
`Ensure that a single selected icon is clearly visible when surrounded by
`unselected icons.
`
`Make each icon distinctive from every other icon.
`Ensure the harmoniousness of each icon as a member of a family of
`icons.
`
`Design the movement animation: when dragging an icon, the user
`might move the whole icon, just a frame, possibly a grayed—out version,
`or a black box.
`
`Add detailed information, such as shading to show size of a file (larger
`shadow indicates larger file), thickness to show breadth of a directory
`folder (thicker means more files inside), color to show the age of a
`document (older might be yellower or grayer), or animation to show
`how much of a document has been printed (a document folder is
`absorbed progressively into the printer icon).
`Explore the use of combinations of icons to create new objects or
`actions—for example, dragging a document icon to a folder, trashcan,
`outbox, or printer icon has great utility. Can a document be appended
`or prepended to another document by pasting of adjacent icons? Can
`security levels be set by dragging a document or folder to a guard dog,
`police car, or vault icon? Can two database icons be intersected by
`overlapping of the icons?
`
`Marcus (1992) applies semiotics as a guide to four levels of icon design:
`
`. Lexical qualities: Machine—generated marks—pixel shape, color, bright-
`ness, blinking
`
`. Syntacticsz Appearance and movement—lines, patterns, modular parts,
`size, shape
`
`3. Semantics: Objects represented—concrete, abstract, part—whole
`4. Pragmatics: Overall legible, utility, identifiable, memorable, pleasing
`
`He recommends starting by creating quick sketches, pushing for consistent
`style, designing a layout grid, simplifying appearance, and evaluating the
`designs by testing with users. We might consider a fifth level of icon
`design:
`
`5. Dynamics: Receptivity to clicks—highlighting, dragging, combining
`
`

`
`5.5 Direct-Manipulation Programming
`
`211
`
`The dynamics of icons might also include a rich set of gestures with a mouse,
`touchscreen, or pen. The gestures might indicate copy (up and down), delete
`(a cross), edit (circle), etc. Icons might also have associated sounds, For
`example, if each document icon had a tone associated with it (the lower the
`tone, the bigger the document), then, when a directory was opened, each
`tone might be played simultaneously or sequentially. Users might get used
`to the symphony played by each directory and could detect certain features
`or anomalies.
`
`Icon design becomes more interesting as computer hardware improves
`and as designers become more creative. Animated icons that demonstrate
`their function improve online help capabilities (see Chapter 12). Beyond
`simple icons, there have been increasing numbers of visual programming
`languages (Chang, 1990; Shu, 1988; Glinert et al., 1990) and specialized
`languages for mechanical engineering, circuit design, and database query
`(see Chapter 11).
`____
`
`5.5 Direct-Manipulation Programming
`
`Performing tasks by direct manipulation is not the only goal. It should be
`possible to do programming by direct manipulation as well, for at least some
`problems. Robot programming is sometimes done by moving the robot arm
`through a sequence of steps that are later replayed, possibly at higher speed.
`This example seems to be a good candidate for generalization. How about
`moving a drill press or a surgical tool through a complex series of motions
`that are then repeated exactly? How about programming a car by driving it
`once through a maze and then having the car repeat the path? In fact, these
`direct—manipulation programming ideas are implemented in modest ways
`with automobile radios that the user presets by turning the frequency
`control knob and then pulling out a button. When the button is depressed,
`the radio tunes to the frequency. Some professional
`television-camera
`supports allow the operator to program a sequence of pans or zooms and
`then to replay it smoothly when required.
`Programming of physical devices by direct manipulation seems quite
`natural, but an adequate visual representation of information may make
`direct-manipulation programming possible in other domains. Several word
`processors allow users to create macros by simply performing a sequence of
`commands that is stored for later reuse. WordPerfect enables the creation of
`
`macros that are sequences of text, special function keys such as TAB, and
`other WordPerfect commands (Figure 5.16). EMACS allows its rich set of
`functions, including regular expression searching to be recorded into mac-
`ros. Macros can invoke each other,
`leading to complex programming
`possibilities. These and other systems allow users to create programs with
`
`

`
`212
`
`Chapter 5
`
`Direct Manipulation
`
`{DISPLAY OFF}
`{Tab}{Tab}{Tab){Tab}Ben-Shne1derman{Enter}
`{Tab}{Tab}tTab){Tab}Department-of»Computer-Science{Enter}
`{Enter}
`tTab}{TaL}{Tab}{Tab3May-95,~199E{Entaw}
`(Enter)
`{Enter}
`Dear-~,tEnter:
`€Enter§
`{Enter}
`{Enter}
`{Tab}{Tab}{Tah1{Tab}Sincerely"your5,{Enter}
`
`Figure 5.16
`
`This WordPerfect macro produces a template of a letter.
`
`nonvarying action sequences using direct manipulation, but strategies for
`including loops and conditionals vary. EMACS allows macros to be encased
`in a loop with simple repeat factors. By resorting to textual programming
`languages, EMACS and WordPerfect allow users to attach more general
`control structures.
`
`Spreadsheet packages, such as LOTUS 1-2-3, Excel, and Quattro, have rich
`programming languages and allow users to create portions of programs by
`carrying out standard spreadsheet operations. The result of the operations is
`stored in another part of the spreadsheet and can be edited, printed, and
`stored in a textual form. Macro facilities in graphic user interfaces are more
`challenging to design than are macro facilities in traditional command
`interfaces, but MacroMaker and Tempo2 on the Macintosh, and the Hewlett-
`Packard NewWave Agent facility on the IBM PCS, are current standouts. The
`MACRO command of Direct Manipulation Disk Operating System (DMDOS)
`(lseki and Shneiderman, 1986) was an early attempt to support a limited
`form of programming for file movement, copying, and directory commands.
`A delightful children's program, Delta Drawing from Spinnaker, enables
`children to move a cursor and to draw on the screen by typing D to draw one
`unit, R to rotate right 30 degrees, and so on. The 40 commands provide rich
`possibilities for drawing various kinds of screen images. In addition, Delta
`Drawing allows users to save, edit, and then invoke programs. For example,
`a user can draw a circle by saving the program consisting of a D and a R.
`Invoking the program with the argument 1 2 then produces a rough l2—sided
`circle.
`
`Smith (1977) inspired work in this area with his Pygmalion system that
`allowed arithmetic programs to be specified visually with icons. A number
`of research projects have attempted to create direct manipulation program-
`ming systems (Rubin et al., 1985). Halbert’s Smallstar (1984) was a program-
`ming-by-example system to enable programming of Xerox Star actions.
`Maulsby and Witten (1989) developed a system that could induce or infer a
`
`

`
`5.6 Home Automation
`
`213
`
`program from examples, questioning the users to resolve ambiguities. Myers
`(1991) coined the phrase demonstrutional programming to characterize these
`efforts as programming—by-example or programming-with-examples in
`which users can create macros by simply doing their tasks and letting the
`system construct the proper generalization automatically to form a macro.
`Cypher (1991) built and ran a usability test with seven subjects for his
`EAGER system that monitored user actions within Hype1Card. When
`EAGER recognized two similar sequences, a small smiling cat appeared on
`the screen to offer the users help in carrying out further iterations. Cypher’s
`success with two specific tasks is encouraging, but more work is needed to
`generalize this approach.
`If designers are to create a general tool that works reliably in many
`situations, it they must meet the five challenges of programming in the user
`interface (PITUI) (Potter, 1992):
`
`. Sufficient computational generality (conditionals, iteration)
`. Access to the appropriate data structures (file structures for directories,
`structural representations of graphical objects) and operators (selectors,
`Booleans, specialized operators of applications)
`. Base in programming (by example, by demonstration, modularity,
`argument passing) and editing programs
`. Simplicity in invocation and assignment of arguments (direct manipula-
`tion, simple library strategies with meaningful names or icons, in-context
`execution, and availability of result)
`. Low risk (high probability of bug-free programs, halt and resume facili-
`ties to permit partial executions, undo operations to enable repair of
`unanticipated damage)
`
`The goal of PITUI is to allow users easily and reliably to repeat automatically
`the actions that they can perform manually in the user interface. Rather than
`depending on unpredictable inferencing, users will be able to indicate their
`intentions explicitly by manipulating objects and actions. The design of
`direct-manipulation systems will undoubtedly be influenced by the need to
`support PITUI. This influence will be a positive step that will also facilitate
`history keeping, undo, and online help.
`
`Home Automation
`
`Internationally, many companies have logically concluded that the next big
`market will be the inclusion of richer controls in homes. Simple ideas such as
`turning off all the lights with a single button or remote control of devices
`(either from one part of the home to another, from outside, or by pro-
`
`

`
`Chapter 5
`
`Direct Manipulation
`
`grammed delays) are being extended in elaborate systems that channel
`audio and video throughout the house, schedule lawn watering as a function
`of ground moisture, offer video surveillance and burglar alarms, and
`provide mu1tiple—zone environmental controls plus detailed maintenance
`records. Demonstrations such as the Smart House project
`in Upper
`Marlboro, Maryland, and installations such as those by Custom Command
`Systems, are a testing ground for the next generation.
`Some futurists and marketing specialists promote voice controls and
`home robots, but the practical reality is more tied to traditional pushbuttons,
`remote controllers, telephone keypads, and especially touchscreens, with the
`latter proving to be the most popular. Installations with two to 10
`touchscreens spread around the house should satisfy most homeowners.
`Providing direct-manipulation controls with rich feedback is vital in these
`applications. Users are willing to take training, but operation must be rapid
`and easy to remember even if used only once or twice a year (such as spring
`and fall adjustments for daylight-savings time).
`Our studies (Plaisant et a1., 1990; Plaisant and Shneiderman, 1991)
`explored four touchscreen designs, all based on direct—manipulation prin-
`ciples, for scheduling operations such as VCR recording or light switching:
`
`1. A digital clock that the user sets by pressing step keys (similar to onscreen
`programming in current video—cassette players)
`. A 24-hour circular clock whose hands can be dragged with the fingers
`
`. A 12-hour circular clock (plus A.M.—-l’.M.
`dragged with the fingers (Figure 5.17)
`. A 24-hour time line in which ON—OFF flags can be placed to indicate
`start-stop times (Figure 5.18)
`
`toggle) whose hands can be
`
`Our results indicated that the 24-hour time line was easiest to understand
`
`and use. Direct-manipu