Tools, techniques,
`and concepts to
`optimize user
`interfaces.
`
`Ben Shneidemnan, University of Maryland
`
`TOUCH SCREENS NOW
`OFFER COMPELLING USES
`Of course, touch screens have some problems:
`+ Users’ hands may obscure the screen.
`+ Screens need to be installed at a lower position
`and tilted to reduce ann fatigue.
`+ Some reduction in image brightness may
`occur.
`+ They cost more than alternative devices.
`These are real problems, but they can be ad-
`dressed successfully. Some critics suggest that
`smudges on the screen may be a problem, but we
`clean our touch screens no more frequently than our
`standard monitors or our mice.
`
`Ifyou thought touch scyem were a thing of the pm,
`this essay will bring you up to date on improvements to
`this input dmicek mer intwface. I suspect we will be
`seeing towh screens medfor more applications than ma,
`bef0re.
`In this essay, Ben Shneidemnan shares some of his
`latest reseur-ch on impwving the mer intetfimes of touch
`screen applications. He is a professor of computer science,
`bead ofthe Human-Computer Interaction Lboratory
`ut the Univwsity ofMaThnd, and author of Sofcware
`Psychology (Little, B r m , 1980) and Designing the
`Uswlnterjace (Addiron- Wesley, 1987).
`
`MICHELANGELO’S FRESCO OF GOD’S
`finger reaching down to touch a person’s hand is
`compelling. The process of touching is immediately
`recognizable as the gift oflife. Inventors of the touch
`screen in the 1960s may have been inspired by this
`image in their cultural unconscious. Touch screens
`have an unrivaled immediacy, a rewarding sense of
`control, and the engaging experience of direct ma-
`nipulation.
`First-generation touch screens have been suc-
`cessfully applied in sales kiosks, public information
`services, and computer-aided instruction - in spite
`of poor precision, slow and erratic activation, and
`poorly designed displays. Now, second-generation
`touch screens are supporting novel applications that
`are likely to enormously expand access to comput-
`ing and information resources as well as enjoyable
`entertainment, art, and music applications.
`
`WHY TOUCH SCREENS? Touch screens have several
`advantages over other pointing devices:
`+ Touching a visual display of choices requires
`little thinking and is a form of direct manipulation
`that is easy to learn.
`+ Touch screens are the fastest pointing device.
`+ Touch screens have easier hand-eye coordina-
`tion than mice or keyboards.
`+ No extra workspace is required as with other
`pointing devices.
`+ Touch screens are durable in public-access and
`in high-volume usage.
`These advantages mean that touch screens are
`highly effective in public-access information sys-
`tems, cash machines, home automation, museums
`and libraries, medical instruments, education, and
`many other domains.
`
`WHAT’S NEW? The second generation of touch
`screens uses several techniques to overcome previ-
`ous limits. Lift-off strategies was one such technique
`that offers several advantages in precision of item
`selection and the movement of elements.
`The use of lift-off strategies allows higher preci-
`sion by showing users a cursor on the screen slightly
`above their fingers. (My colleagues and I compared
`three lift-off strategies in “Improving the Accuracy
`of Touch Screens: An Experimental Evaluation of
`Three Strategies,” Pr-oc. Conf Human Factm in
`Computing Systems, ACM, 1988, pp. 27-32.) With
`lift-off, you can drag the cursor smoothly and con-
`tinuously along the screen’s surface. Functions can
`be activated when users lift their fingers off the sur-
`face - something we call the “untouch screen.”
`Our early study showed that, with lift-off, people
`could easily selecr targets the size of a pair of letters.
`However, we had to add stabilization software to
`allow single-pixel selection on a 640 x 480 display (a
`VGA-resolution display) - or less than a square
`millimeter. Improved hardware and software sup-
`porting this high-precision strategy is now available
`in commercial touch screens (vendors include
`Microtouch Systems of Wilmington, Mass., and
`Elographics of Oak Ridge, Tenn.).
`Dragging a cursor is only one use of the lift-off
`sn-ategy. The most engaging applications are those
`that let users drag icons, buttons, sliders, words,
`flags, or clock hands. But why not allow dragging of
`musical notes, paint brushes, or large sections of the
`screen image? In our experience, there is a delightful
`sense of magic about dragging images around the
`screen.
`
`WHAT’S POSSIBLE? Designers’ imaginations be-
`
`9 3
`
`Editor: Kothleen Potosnok
`Pldus Corp.
`4 1 1 First Ave. S.
`WA 98104-2871
`Internet k.potosnok@compmoil.com
`
`I E E E S O F T W A R E
`
`SCEA Ex. 1025 Page 1
`
`

`
`Su N o T u -= lh F r Si.
`
`19 11 12 13 1.
`11 18 19 28 21
`
`15 16
`29
`
`/=/
`
`VCR SCHEDULING
`
`I
`
`Figure 1. The boxesarozlnd datesand the handr on the clock may be dragged via Figure 2. The me of on and offjags was the nzoa effectine scheduling metaphor
`for touch-screen usen tested.
`touch screen.
`
`come freer when they enter the world of
`touching, dragging, and drawing with these
`improved touch screens. Our first applia-
`tion was with our Hyperties hypermedia sys-
`tem (available from Cognetics of Princeton
`Junction, N.J.) applied to a Smithsonian In-
`stitution installation containing information
`on 200 archaeological dig sites that accept
`volunteers. Users could touch words in the
`text for more information or locations on the
`11 world maps.
`Most users succeeded in using the kiosk
`immediately. About 15 percent were mo-
`mentarily confused by the lift-off strategy,
`but they quickly learned it after one or two
`touches. We observed and interviewed early
`users to make improvements and analyzed
`the log data for the 4,461 users in the first four
`weeks of the 18-month six-city tour.
`As we became more comfortable with the
`
`idea of high-preasion touch screens and lift-
`off, we developed several versions of home-
`control scheduler tasks like scheduling
`VCRS. Pointing at a day on a monthlycalen-
`dar was very natural when the user could
`smoothly drag a box-shaped cursor. Then to
`choose the time, we let the users drag the
`hands on an analog clock, as Figure 1 shows.
`Participants in OUT usability test had great fun
`doing this, but the most effective scheduler
`used a 24-hour time-line with on and off flags
`(as Figure 2 shows). Users could drag the
`flags onto the time-line, slide them around to
`adjust, or drag them off to delete.
`A common pursuit with touch screens is
`developing visually appealing metaphors that
`react predictably. Opening a book, touching
`lettered tabs, and turning pages are natural in
`the touch-screen environment. While we
`built two museum versions of books,
`
`Cognetics's amst, Paul Hoffman, made a
`, strikingly realistic ring-binder telephone
`book for a conference messaging system that
`eliminated the keyboard and used touch
`screens and scanners only, as Figure 3 shows.
`Smiles were common when demonstrat-
`ing an art and music environment that al-
`lowed electronic finger painting. In Playpen
`II, created by Andrew Sears, users select col-
`ors, textures, sounds, and shapes with their
`fingers. Figure 4 shows an example. The re-
`sults depend not only on finger position but
`also on the velocity and direction of motion.
`This additional information can be used in
`other applications, such as touch-screen ver-
`sions of musical instruments in which the
`volume depends on the velocity of touch on a
`set of strings or piano keys.
`Touch-screen keyboard replacements be-
`Continued on p. 107
`
`Figure 3. The ring-binder metaphor worked ve?y well asa toucb-srreen applica-
`tion.
`
`Figure 4. The Playpen tool combines traditional touch-and-select technology with
`velocity and motion sensing. This added infirmation can be used to control volume,
`pitch, brightness, speed and othw such attributes.
`
`9 4
`
`~
`
`M A R C H 1 9 9 1
`
`SCEA Ex. 1025 Page 2
`
`

`
`early as during critical-design review.
`With simple project data and the in-pro-
`cess efficiency achieved from it, a total-qual-
`ity-management system can learn the unique
`behavior of a project using expert methods.
`This lets you predict when the remaining
`milestones will occur, how quickly the code
`will be written, what the optimal staffing
`strategy for the rest of the project is, and
`what the defect rates during systems test-
`ing will be.
`These methods are more than simple data
`extrapolation. They use statistical curve-fit-
`ting techniques and take into account h d a -
`mental process behavior. These methods
`make it a design discipline based on the expe-
`rience &om thousands of completed projects.
`Adaptive control of ongoing projects is
`possible; moreover, it is absolutely necessary
`to do if we are to control runaway projects.
`
`OUTlOOlGDuringthe 1990s, Isee software
`management maturing, moving from ama-
`teurism to professionalism.
`Control officeswill be established to mea-
`sure, plan, and control projects. This office
`will be the software-data repository, the met-
`ria group that measures process-productiv-
`ity improvement; the center that generates
`realistic, consistent work plans for schedule,
`cost, staffing, code production, and defect
`prehction; and a center of expertise in re-
`source management available to all project
`teams.
`Executive managers will start to take a
`part in development because it is now strate-
`gic to the organization. They will expect and
`will be able to get good answen to the man-
`agement questions: How long is it going to
`take? How many people? How much is it
`going to cost? What savings am I getting?
`What is the r e m on investment? Will it be
`good enough to satisfy my best customer?
`Can I optimize these factors to get the best
`value for my money?
`Professionalism implies measuring prog-
`ress, estimating and controlling, taking pro-
`ductivity seriously, taking quality seriously,
`and making it happen.
`We will see many organizations doing
`this, not just a few. We will see real gains in
`process improvement. Good products will
`start to be delivered on time, within budget.
`Customers will start to become happy with
`+
`their software. It can happen in the next 10
`years.
`
`probably be in innovative applications, like
`controlling 3D artificial realities (letyourh-
`gers do the walhng), selecting irregular
`shaped objects (for example, pointing at
`human body parts and getting lab results), or
`selecting moving objects (for example, point-
`ing at fish swimming in a pool to find out
`more about the species, or pointing at a rotat-
`ing globe to select countries).
`We found that the challenge was to break
`free from the older notion that touch screens
`are for buttons and to explore how we might
`use sliding, dragging, and other gestures to
`move objects and invoke actions.
`Who knows what new fonns of video
`games are possible if we let our imaginations
`go free? Why not a touch-screen Ouija
`board or labyrinth? Who will be the first to
`make a magical Aladdin’s lamp with a genie
`that pops-out when you rub it?
`Soon enough we can envision a pocket-
`sized computer with two folding halves each
`having at least 80-character hy 25-line high-
`precision touch screens. Your calendar, ad-
`dress book, current projects, and the morn-
`ing newspaper could all be a touch or an
`untouch away. And why not high-resolution
`LCD touch screens next to museum art-
`works to give you the artist’s biography,
`provenance, and description? Every refriger-
`ator door, automobile dashboard, household
`main entry doorway, or T V is a potential
`place for untouch screens with useful infor- +
`mation, assistance, and data entry.
`
`come attractive when only occasional data
`enay is necessary. Typists achieved 25-word-
`per-minute speeds with our nearly normal-
`sized touch-screen keyboards, compared
`with 58 words per minute using standard
`keyboards and 17 words per minute using a
`mouse to select the keys (Figure 5 shows the
`touch-screen typing screen). However the
`touch-screen keyboard can be adjusted to re-
`duce the size to less than 2.5 inches wide and
`still preserve reasonable typing speed. A
`small keyboard is applicable for portable or
`pocket-sized computers, as a pop-up tool to
`enter data on a medical form or sales receipt,
`or to enter a search string in an electronic book
`
`WHAT’S NEXT? Further advances with
`high-precision touch screens seem very
`likely, in both the hardware technology and
`the software designs that apply lift-off. While
`some touch screens can provide three to four
`levels of touch pressure, improvements are
`needed to make this notion viable. Another
`improvement would be to allow multiple si-
`multaneous touches to support the pressing
`of a Shift key while typing, selecting colors
`while drawing, touch-typing, or selecting an
`object and an action simultaneously.
`The most exciting breakthroughs will
`
`Figure 5. A touch-screen keyboard. Although sloww than a traditional keyboard, it works wellfw- note-taking,
`fms-enhy, and other applications that require small amounts of input in a portable orsnlall device.
`
`I E E E S O F T W A R E
`
`1 0 7
`
`SCEA Ex. 1025 Page 3