`Human-Computer
`Interaction Technology
`
`Brad A. Myers
`
`44
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`TRADING TECH EXHIBIT 2069
`CQG & CQGT v. Trading Technologies
`CBM2015-00057
`
`Page 1 of 11
`
`
`
`
`
`a r t i c l e
`
`Brad A. Myers
`Human–Computer
`Interaction Institute
`School of Computer
`Science
`Carnegie Mellon
`University
`Pittsburgh, PA 15213-
`3891
`
`bam@cs.cmu.edu
`
`Research in human–computer interac-
`
`tion (HCI) has been spectacularly suc-
`
`cessful and has fundamentally changed
`
`computing. One example is the ubiqui-
`
`tous graphical
`
`interface used by
`
`Microsoft Windows 95, which is based
`
`on the Macintosh, which is based on
`
`work at Xerox PARC, which in turn is
`
`based on early research at the Stanford
`
`Research Laboratory (now SRI) and at
`
`Massachusetts Institute of Technology.
`
`Another example is that virtually all soft-
`ware written today employs user interface
`toolkits and interface builders—concepts that
`were developed first at universities. Even the
`remarkable growth of the World Wide Web is
`a direct result of HCI research: applying
`hypertext technology to browsers allows one
`to traverse a link across the world with a click
`of the mouse. More than anything else,
`improvements to interfaces have triggered this
`explosive growth. Furthermore, the research
`that will lead to the user interfaces for the
`computers of tomorrow is being carried out
`today at universities and a few corporate
`research labs.
`This paper attempts to briefly summarize
`many of the important research developments
`in HCI technology, emphasizing the role of
`university research, which may not be widely
`recognized. By “research,” I mean exploratory
`work at universities and government and cor-
`porate research labs (such as Xerox PARC)
`that is not directly related to products. By
`“HCI technology,” I am referring to the com-
`puter side of HCI. A companion article on the
`history of the “human side,” discussing the
`contributions
`from psychology, design,
`human factors, and ergonomics would also be
`appropriate.
`
`Figure 1 shows the time span for some of
`the technologies discussed in this article.
`including when they were introduced. Of
`course, a deeper analysis would reveal signifi-
`cant interaction among the university, corpo-
`rate research, and commercial activity lines. It
`is important to appreciate that years of
`research are involved in creating and making
`these technologies ready for widespread use.
`The same will be true for the HCI technolo-
`gies currently being developed that will pro-
`vide the interfaces of tomorrow.
`Clearly it is impossible to list every system
`and source in a paper of this scope, but I have
`tried to represent the earliest and most influ-
`ential systems. Further information can be
`found in other surveys of HCI topics (see, for
`example, [1, 11, 36, 41]). Another useful
`resource is the video All The Widgets, which
`shows the historical progression of a number
`of user interface ideas [27].
`The technologies discussed in this paper
`include fundamental interaction styles such as
`direct manipulation, the mouse pointing
`device, and windows; several important kinds
`of application areas, such as drawing, text
`editing, and spreadsheets; the technologies
`that will likely have the biggest impact on
`interfaces of the future, such as gesture recog-
`nition, multimedia, and three-dimensionality;
`and the technologies used to create interfaces
`using the other technologies, such as user
`interface management systems, toolkits, and
`interface builders.
`
`Basic Interactions
`Direct Manipulation of
`Graphical Objects
`The now ubiquitous direct manipulation inter-
`face, where visible objects on the screen are
`directly manipulated with a pointing device,
`was first demonstrated by Ivan Sutherland in
`Sketchpad [47], the thesis of his doctoral dis-
`sertation in 1963. Sketchpad supported the
`manipulation of objects using a light pen,
`including grabbing objects, moving them,
`changing size, and using constraints. It con-
`tained the seeds of myriad important interface
`ideas. The system was built at Lincoln Labs
`with support from the U.S. Air Force and the
`National Science Foundation (NSF).
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`45
`
`Page 2 of 11
`
`
`
`
`
`KEY:
`
`University Research
`
`Corporate Research
`
`Commercial Products
`
`Direct Manipulation of Graphical Objects
`
`Figure 1. Approximate time lines showing where
`and when work was performed on some major
`technologies discussed in this article.
`
`1945
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`The Mouse
`
`1945
`
`Windows
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`1945
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`Text Editing
`
`1945
`
`HyperText
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`1945
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`Gesture Recognition
`
`1945
`
`1955
`
`1965
`
`1975
`
`1985
`
`1995
`
`46
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`Page 3 of 11
`
`
`
`
`
`a r t i c l e
`
`William Newman’s Reaction Handler [33],
`created at Imperial College, London during
`1966 and 1967, provided direct manipulation
`of graphics and introduced Light Handles
`[32], a form of graphical potentiometer that
`was probably the first “widget.” Another early
`system was AMBIT/G (implemented at the
`Massachusetts Institute of Technology’s
`(MIT) Lincoln Labs in 1968 and
`funded by the Advanced Research
`Projects Agency (ARPA). It
`employed, among other inter-
`face techniques, iconic repre-
`sentations, gesture recog-
`nition, dynamic menus with
`items selected using a point-
`ing device, selection of icons
`by pointing, and moded
`and mode-free styles of
`interaction.
`David Canfield Smith
`coined the term “icons” in his
`1975 doctoral thesis on Pyg-
`malion [44] (funded by ARPA
`and National Institute of Mental
`Health – NIMH). Smith later popu-
`larized icons as one of the chief designers
`of the Xerox Star [45]. Many of the interac-
`tion techniques popular in direct manipula-
`tion interfaces, such as how objects and text
`are selected, opened, and manipulated, were
`researched at Xerox PARC in the 1970s. In
`particular, the idea of “WYSIWYG” (what
`you see is what you get) originated there with
`systems such as the Bravo text editor and the
`Draw drawing program [11]
`The concept of direct manipulation inter-
`faces for everyone was envisioned by Alan Kay
`of Xerox PARC in a 1977 article about the
`Dynabook [18]. The first commercial systems
`to use direct manipulation extensively were the
`Xerox Star (1981) [45], the Apple Lisa (1982)
`[54], and the Macintosh (1984) [55]. Ben
`Shneiderman at the University of Maryland
`coined the term “direct manipulation” in 1982,
`identified the components, and gave psycholog-
`ical motivations for direct manipulation [43].
`
`NLS project (funding from ARPA, NASA,
`and Rome ADC) [10]. It was intended to be a
`cheap replacement for light pens, which had
`been used at least since 1954 [11, p. 68].
`Many of the current uses of the mouse were
`demonstrated by Doug Engelbart as part of
`NLS in a movie created in 1968 [9]. The
`mouse was then made famous as a
`practical input device by Xerox
`PARC in the 1970s. It first
`appeared commercially as part
`of the Xerox Star (1981), the
`Three Rivers Computer
`Company’s PERQ (1981)
`[25],
`the Apple Lisa
`(1982), and the Apple
`Macintosh (1984).
`
`Windows
`Multiple tiled windows
`were demonstrated
`in
`Engelbart’s NLS in 1968
`[9]. Early research at Stan-
`ford on systems like COPI-
`LOT (1974) [49] and at MIT
`with the EMACS text editor (1974)
`[46] also demonstrated tiled windows.
`Alan Kay proposed the idea of overlapping
`windows in his 1969 doctoral thesis [17], and
`overlapping windows first appeared in 1974 in
`his Smalltalk system [12] at Xerox PARC, and
`soon afterward in the InterLisp system [50].
`Some of the first commercial uses of win-
`dows were on Lisp Machines Inc. (LMI) and
`Symbolics Lisp Machines (1979), which grew
`out of MIT Artificial Intelligence (AI) Lab pro-
`jects. The Cedar Window Manager from Xerox
`PARC was the first major tiled window manag-
`er (1981) [48], followed soon by the Andrew
`window manager [35] by Carnegie Mellon
`University’s (CMU) Information
`Technology Center (1983, funded
`by IBM). The main commercial
`systems popularizing windows
`were the Xerox Star (1981), the
`Apple Lisa (1982), and most
`important, the Apple Macin-
`tosh (1984). The early versions of
`the Star and Microsoft Windows
`were tiled, but eventually they sup-
`ported overlapping windows like the
`
`“THE SKETCHPAD
`SYSTEM MAKES IT
`POSSIBLE FOR A
`MAN AND A COM-
`PUTER TO CON-
`VERSE RAPIDLY
`THROUGH THE
`MEDIUM OF LINE
`DRAWINGS.”
`
`— IVAN SUTHERLAND
`[47, P. 329]
`
`Mouse
`The mouse was developed at Stanford
`Research Laboratory in 1965 as part of the
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`47
`
`Page 4 of 11
`
`
`
`
`
`Lisa and Macintosh. The X Window System, a
`current international standard, was developed
`at MIT in 1984 [42]. For a survey of window
`managers, see [26].
`
`based display editors that was widely used
`[51]. The Hypertext Editing System [53, p.
`108] from Brown University had screen edit-
`ing and formatting of arbitrary-sized strings
`with a light pen in 1967 (funding from
`IBM). NLS demonstrated mouse-
`based editing in 1968 (Figure 4).
`TECO from MIT was an early
`screen editor (1967), and
`EMACS [46] was developed
`from it in 1974. Xerox
`PARC’s Bravo [11, p. 284]
`was the first WYSIWYG
`editor-formatter (1974).
`It was designed by Butler
`Lampson and Charles
`Simonyi, who had started
`working on these con-
`cepts about 1970 while at
`Berkeley. The first com-
`mercial WYSIWYG edi-
`tors were
`the
`Star,
`LisaWrite,
`and,
`later,
`MacWrite. For surveys of
`text editors, see [24] and [53].
`
`“THE DIFFICULTY
`OF MANKIND’S
`PROBLEMS WAS
`INCREASING AT A
`GREATER RATE THAN
`OUR ABILITY TO
`COPE. (WE ARE IN
`TROUBLE.) [SO] I ...
`COMMITTED MY
`CAREER TO
`‘AUGMENTING THE
`HUMAN INTELLECT.’”
`
`— DOUG ENGELBART, ON THE
`NLS SYSTEM [8, P. 189]
`
`Applications
`Drawing Programs
`Much of the current technology
`for drawing programs was
`demonstrated in Sutherland’s
`1963 Sketchpad
`system
`[47]. The use of a mouse to
`manipulate graphics was
`demonstrated
`in NLS
`(1965). In 1968 Ken
`Pulfer
`and
`Grant
`Bechthold at the Nation-
`al Research Council of
`Canada built a mouse out
`of wood patterned after
`Engelbart’s mouse and
`used it with a key-frame
`animation system to draw
`all the frames of a movie. A
`subsequent movie in 1971,
`Hunger, won a number of
`awards and was drawn using a
`tablet instead of the mouse (fund-
`ing by the National Film Board of
`Canada) [3]. William Newman’s Markup
`(1975) was the first drawing program for
`Xerox PARC’s Alto, followed shortly by
`Patrick Baudelaire’s Draw, which added han-
`dling of lines and curves [11, p. 326]. The
`first computer painting program was probably
`Dick
`Shoup’s
`Superpaint
`at PARC
`(1974–1975). The first widely used drawing
`programs were MacPaint and MacDraw on
`the Macintosh (1984).
`
`Spreadsheets
`The initial spreadsheet was Visi-
`Calc which was developed by
`Frankston and Bricklin (1977–1978) for
`the Apple II while they were students at MIT
`and the Harvard Business School, respectively.
`The solver was based on a dependency-direct-
`ed backtracking algorithm by Sussman and
`Stallman at the MIT AI Lab.
`
`HyperText
`The idea for hypertext (by which documents are
`linked to related documents) is credited to Van-
`nevar Bush’s famous MEMEX idea from 1945
`[4]. Ted Nelson coined the term “hypertext” in
`1965 [31]. Engelbart’s NLS system [9] at the
`Stanford Research Laboratories in 1965 made
`extensive use of linking (funding from ARPA,
`NASA, and Rome ADC). The “NLS Journal”
`[11, p. 212] was one of the first on-line journals
`and included full linking of articles (1970). The
`Hypertext Editing System, jointly designed by
`Andy van Dam, Ted Nelson, and two students
`at Brown University (funding from IBM) was
`distributed extensively [52].
`
`Text Editing
`In 1962, at the Stanford Research
`Laboratory, Engelbart proposed,
`and later implemented, a word
`processor with automatic word
`wrap, search and replace, user-
`definable macros, scrolling text,
`and commands to move, copy,
`and delete characters, words, or
`blocks of text. Stanford’s TVEdit
`(1965) was one of the first CRT-
`
`48
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`Page 5 of 11
`
`
`
`
`
`The University of Vermont’s
`PROMIS (1976) was the first
`hypertext system released to
`the user community. It
`was used to link patient
`and patient care infor-
`mation at the Univer-
`sity of Vermont’s
`medical center. The
`ZOG project (1977) from
`CMU was another early hypertext
`system and was funded by Office of
`Naval Research (ONR) and ARPA [39]. Ben
`Shneiderman’s Hyperties was the first system
`in which highlighted items in the text could
`be clicked on to go to other pages (1983, Uni-
`versity of Maryland) [19]. HyperCard from
`Apple (1988) significantly helped to
`bring the idea to a wide audience.
`Tim Berners-Lee used the
`hypertext idea to create the
`World Wide Web in 1990 at
`the
`government-funded
`European Particle Physics
`Laboratory
`(CERN).
`Mosaic, the first popular
`hypertext browser for
`the World-Wide Web,
`was developed at the
`University of Illinois’
`National Center
`for
`Supercomputer Appli-
`cations (NCSA). For a
`more complete history of
`HyperText, see [34].
`
`“MULTIPLE
`WINDOWS ALLOW A
`DOCUMENT
`(COMPOSED OF TEXT,
`PICTURES, MUSICAL
`NOTATION) TO BE
`CREATED AND
`VIEWED
`SIMULTANEOUSLY AT
`SEVERAL LEVELS OF
`REFINEMENT.”
`
`— ALAN KAY,
`ON SMALLTALK-72
`[18, P. 34].
`
`Computer-Aided Design
`The same 1963 International
`Federation of Information Pro-
`cessing Societies (IFIPS) confer-
`ence at which Sketchpad was
`presented also contained a number of com-
`puter-aided design (CAD) systems, includ-
`ing Doug Ross’s Computer-Aided Design
`Project at MIT in the Electronic Systems Lab
`[40] and Coons’s work at MIT with Sketch-
`Pad [7]. Timothy Johnson’s pioneering work
`on the interactive 3D CAD system Sketch-
`pad 3 [15] was his doctoral thesis in 1963
`(funded by the U.S. Air Force). The first sys-
`tem in industry using CAD and computer-
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`49
`
`a r t i c l e
`
`aided manufacturing
`(CAM) was probably
`General Motor’s DAC-
`1 (about 1963).
`
`Video Games
`The first graphi-
`cal video game
`was probably
`SpaceWar by Slug
`Russel of MIT in 1962 for
`the PDP-1 [21, p. 49], including the
`first computer joysticks. The early computer
`Adventure game was created by Will
`Crowther at Bolt, Beranek & Newman
`(BBN), and Don Woods developed it into a
`more sophisticated Adventure game at Stan-
`ford in 1966 [21, p. 132]. Conway’s
`game of LIFE was developed on
`computers at MIT and Stan-
`ford in 1970. The first popu-
`lar commercial game was
`Pong (about 1976).
`
`Up-and-Coming Areas
`Gesture Recognition
`The
`first pen-based
`input device, the Rand
`tablet, was funded by
`ARPA. Sketchpad used
`light-pen
`gestures
`(1963). Teitelman in
`1964 developed the first
`trainable gesture recog-
`nizer. A very early demon-
`stration
`of
`gesture
`recognition was Tom Ellis’s
`GRAIL system on the Rand
`tablet (1964, funded by ARPA).
`It was quite common in light-
`pen–based systems to include some
`gesture recognition, for example in the
`AMBIT/G system (1968, funded by ARPA). A
`gesture-based text editor using proofreading
`symbols was developed at CMU by Michael
`Coleman in 1969. Bill Buxton at the Universi-
`ty of Toronto has been studying gesture-based
`interactions since 1980. Gesture recognition
`has been used in commercial CAD systems
`since the 1970s and came to universal notice
`with the Apple Newton in 1992.
`
`Page 6 of 11
`
`
`
`
`
`by NASA, U.S. Navy, and others). Another
`important center of current research on 3-D
`interfaces is Fred Brooks’s lab at University of
`North Carolina at Chapel Hill (e.g., [2]).
`
`Multimedia
`The FRESS project at Brown University used
`multiple windows and integrated text and
`graphics (1968, funding from industry). The
`Interactive Graphical Documents project at
`Brown was the first hypermedia (as opposed
`to hypertext) system and used raster graphics
`and text but not video (1979–1983, funded
`by ONR and NSF). The Diamond
`project at BBN (starting in 1982,
`funded by ARPA) explored
`combining multimedia infor-
`mation (text, spreadsheets,
`graphics, speech). The
`Movie Manual at the
`Architecture Machine
`Group (MIT) was one
`of the first to demon-
`strate mixed video
`and computer graph-
`ics in 1983 (funded
`by ARPA).
`
`THE ‘MOUSE’ WAS
`DEVELOPED BY SRI. IT
`IS CONSTRUCTED FROM
`TWO POTENTIOMETERS,
`MOUNTED ORTHOGONALLY,
`EACH OF WHICH HAS A
`WHEEL ATTACHED TO ITS
`SHAFT. THE MOUNTING
`FRAME ... IS ENCLOSED
`IN A 2 INCH X 2 INCH X 4
`INCH WOODEN CASE.”
`
`Virtual Reality and “Augmented Reality”
`The original work on virtual reality (VR) was
`performed by Ivan Sutherland when he was at
`Harvard (1965–1968, funding by Air
`Force, Central Intelligence Agency,
`and Bell Laboratories). Very
`important early work was
`done by Tom Furness when
`he was at Wright-Patter-
`son
`AFB. Myron
`Krueger’s early work at
`the University of Con-
`necticut on interfaces
`that incorporate and
`interpret a video of
`the user was influen-
`tial. Fred Brooks’s and
`Henry Fuch’s groups
`at UNC did a lot of
`early research, includ-
`ing the study of force
`feedback (1971, fund-
`ing from U.S. Atomic
`Energy Commission and
`NSF). Much of the early
`research on head-mounted
`displays and on the DataGlove
`was supported by NASA.
`
`— W.K. ENGLISH, D.C. ENGELBART, AND
`M.L. BERMAN, ON THE NLS SYSTEM
`[10, P. 7]
`
`Three-
`Dimensionality
`The first 3D system
`was probably Timothy
`Johnson’s 3D CAD sys-
`tem mentioned earlier
`(1963, funded by the U.S.
`Air Force) [15]. The Lin-
`coln Wand by Larry Roberts
`was an ultrasonic 3D location
`sensing system developed at Lincoln
`Labs (1966, funded by ARPA). That system
`also had the first interactive 3D hidden line
`elimination. An early use was for molecular
`modeling [20]. The late 1960s and early
`1970s saw the flowering of 3D raster graph-
`ics research at the University of Utah with
`Dave Evans, Ivan Sutherland, G.W.
`Romney, Henri Gouraud, Bui-Tuong
`Phong, and G.S. Watkins, much
`of it government funded. Also,
`the military-industrial flight
`simulation work of the 1960s
`and 1970s led the way to mak-
`ing 3D interfaces operate in real
`time on commercial systems
`from General Electric, Evans &
`Sutherland, Singer/Link (funded
`
`Computer-Supported
`Cooperative Work
`Doug Engelbart’s 1968 demonstration of
`NLS [9] included the remote participation of
`multiple people at various sites (funding from
`ARPA, NASA, and Rome ADC). Licklider
`and Taylor predicted online interactive com-
`munities in a 1968 article [22] and speculat-
`ed about the problem of access being limited
`to the privileged. Electronic mail, still the
`most widespread multiuser software, was
`enabled by the ARPAnet, which became
`operational in 1969, and by the Ethernet
`from Xerox PARC in 1973. An early comput-
`er conferencing system was Turoff ’s EIES sys-
`tem at the New Jersey Institute of Technology
`(1975).
`
`50
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`Page 7 of 11
`
`
`
`
`
`a r t i c l e
`
`Natural Language and Speech
`The fundamental research for speech and nat-
`ural language understanding and generation
`has been performed at CMU, MIT, SRI,
`BBN, IBM, AT&T Bell Labs and BellCore,
`much of it government funded. See, for exam-
`ple, [37] for a survey of the early work.
`
`Software Tools and Architectures
`The area of user interface (UI) software tools
`is quite active now, and many companies are
`selling tools. Most of today’s applications are
`implemented using various forms of software
`tools. For a more complete survey and discus-
`sion of UI tools, see [28].
`
`User Interface Management Systems
`and Toolkits
`User interface management systems (UIMS)
`and toolkits are software libraries and tools
`that help to create user interfaces. The first
`UIMS was William Newman’s Reaction Han-
`dler [33] created at Imperial College, London
`(1966–1967 with SRC funding). Most of the
`early work was done at universities (University
`of Toronto with Canadian government fund-
`ing; George Washington University with
`NASA, NSF, Department of Energy, and
`National Bureau of Standards (NBS) funding;
`Brigham Young University with industrial
`funding; and so on). The term “UIMS” was
`coined by David Kasik at Boeing (1982) [16].
`Early window managers such as Smalltalk
`(1974) and InterLisp, both from Xerox PARC,
`came with a few widgets, such as popup menus
`and scrollbars. The Xerox Star (1981) was the
`first commercial system to have a large collec-
`tion of widgets. The Apple Macintosh (1984)
`was the first to actively promote its toolkit for
`use by other developers to enforce a consistent
`interface. An early C++ toolkit was InterViews
`[23], developed at Stanford (1988, industrial
`funding). Much of the modern research is
`being performed at universities, for example,
`the Garnet (1988) [29] and Amulet (1994)
`[30] projects at CMU (funded by DARPA)
`and SubArctic at Georgia Tech (1996, funded
`by Intel and NSF) [14].
`
`Interface Builders
`Interface builders are interactive tools that
`
`allow interfaces composed of widgets such as
`buttons, menus, and scrollbars to be placed
`using a mouse. The Steamer project at BBN
`(1979–1985; funded by ONR) demonstrated
`many of the ideas later incorporated into
`interface builders and was probably the first
`object-oriented graphics system. Trillium [13]
`was developed at Xerox PARC in 1981.
`Another early interface builder was the
`MenuLay system [5] developed by Bill Buxton
`at the University of Toronto (1983, funded by
`the Canadian Government). The Macintosh
`(1984) included a Resource Editor that
`allowed widgets to be placed and edited. Jean-
`Marie Hullot created SOS Interface in Lisp
`for the Macintosh while working at The
`French National Institute for Research in
`Computer Science and Control (INRIA)
`(1984, funded by the French government)
`which was the first modern interface builder.
`Hullot built this into a commercial product in
`1986 and then went to work for NeXT and
`created the NeXT Interface Builder (1988),
`which popularized this type of tool. Now
`there are literally hundreds of commercial
`interface builders.
`
`Component Architectures
`The idea of creating interfaces by connecting
`separately written components was first
`demonstrated in the Andrew project [35] by
`Carnegie Mellon University’s Information
`Technology Center (1983, funded by IBM). It
`is now being widely popularized by Microsoft’s
`object linking and embedding (OLE), Apple
`and IBM’s OpenDoc, and the Java Beans
`architectures.
`
`Discussion
`It is clear that all of the most impor-
`tant innovations in HCI have
`benefited from research at
`both corporate research labs
`and universities. One moti-
`vation for this article is to
`overcome the impressions
`some people may have that
`much of the important work
`in human–computer interac-
`tion occurred in industry and
`that if university research in
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`51
`
`Page 8 of 11
`
`
`
`
`
`PERMISSION TO MAKE DIGITAL/ HARD
`
`COPY OF PART OR ALL OF THIS WORK
`
`FOR PERSONAL OR CLASSROOM USE IS
`
`GRANTED WITHOUT FEE PROVIDED
`
`THAT COPIES ARE NOT MADE OR
`
`DISTRIBUTED FOR PROFIT OR COM-
`
`MERCIAL ADVANTAGE, THE COPYRIGHT
`
`NOTICE, THE TITLE OF THE PUBLICA-
`
`TION AND ITS DATE APPEAR, AND
`
`NOTICE IS GIVEN THAT COPYING IS BY
`
`PERMISSION OF ACM, INC. TO COPY
`
`OTHERWISE, TO REPUBLISH, TO POST
`
`ON SERVERS, OR TO REDISTRIBUTE TO
`
`LISTS REQUIRES PRIOR SPECIFIC PERMIS-
`
`SION AND/OR A FEE.
`
`© ACM 1072-5220/98/0300 $3.50
`
`HCI is not supported, then industry will just
`carry on anyway. This is simply not true. I
`hope that this paper has shown that many of
`the most famous HCI successes developed by
`companies are deeply rooted in univer-
`sity research. In fact, virtually all of
`today’s major interface styles
`and applications have been
`significantly influenced by
`research at universities
`and labs, often with gov-
`ernment funding. To
`illustrate this point,
`this article has listed
`the funding sources
`of some of the major
`advances. Without
`this research, many
`of the advances in
`the field of HCI
`would probably not
`have taken place, and
`as a consequence, the
`user interfaces of com-
`mercial products would
`be far more difficult to
`use and learn than they are
`today. As described by Stu
`Card [6, p. 162]:
`of
`funding
`Government
`advanced human-computer interac-
`tion technologies built the intellectual capi-
`tal and trained the research teams for pioneer
`systems that, over a period of 25 years, revolu-
`tionized how people interact with computers.
`Industrial research laboratories at the corporate
`level in Xerox, IBM, AT&T, and others played a
`strong role in developing this technology and
`bringing it into a form suitable for the commer-
`cial arena.
`Another important argument in favor
`of HCI research at universities is that
`computer science students need to
`know about user interface issues.
`User interfaces are likely to be
`one of the main value-added
`competitive advantages of the
`future as both hardware and basic
`software become commodities. If
`students do not know about user
`interfaces, they will not serve
`
`“IN EMBEDDED
`MENUS, HIGHLIGHTED
`OR UNDERLINED
`WORDS OR PHASES
`WITHIN THE TEXT
`BECOME THE MENU
`ITEMS, AND ARE
`SELECTABLE USING THE
`COMMONLY USED
`TOUCH SCREEN,
`CURSOR AND MOUSE
`METHODS.”
`
`— L. KOVED AND
`B. SHNEIDERMAN ON THE
`HYPERTIES SYSTEM [19, P. 312].
`
`industry needs. It seems that only through
`computer science does HCI research dissemi-
`nate to products. Furthermore, without
`appropriate levels of funding of academic
`HCI research, there will be fewer doc-
`toral graduates in HCI to perform
`research in corporate labs, and
`fewer top-notch graduates in
`this area will be interested
`in being professors, so
`the needed user inter-
`face courses will not be
`offered.
`The conventional
`style of graphical
`user interfaces that
`use windows, icons,
`menus, and a mouse
`are in a phase of
`stan dardizat ion,
`where almost every-
`one
`is using
`the
`same, standard tech-
`nology and making
`small,
`incremental
`changes. As computers
`perform faster, more of
`the processing power is
`being devoted to the user
`interface. The interfaces of the
`future will use gesture recognition,
`speech recognition, “intelligent agents,”
`adaptive interfaces, video, and many other
`technologies now being investigated by
`research groups at universities and corporate
`labs [38]. Therefore, it is imperative that uni-
`versity, corporate, and government-supported
`research continue and be well-supported, so
`that we can develop the science and technolo-
`gy needed for the user interfaces of the future.
`
`Acknowledgments
`I must thank a large number of people who
`responded to posts of earlier versions of this
`article on the announcements.chi mailing list
`for their very generous help, and to Jim Hollan
`who helped write the short excerpt of this arti-
`cle. Some of the information in this article was
`supplied by (in alphabetical order): Stacey Ash-
`lund, Meera M. Blattner, Keith Butler, Stuart
`K. Card, Bill Curtis, David E. Damouth, Dan
`
`52
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`Page 9 of 11
`
`
`
`
`
`a r t i c l e
`
`Diaper, Dick Duda, Tim T.K.
`Dudley, Steven Feiner, Harry
`Forsdick, Bjorn Freeman-
`Benson, John Gould,
`Wayne Gray, Mark
`Green, Fred Hansen,
`Bill Hefley, D.
`Austin Henderson,
`Jim Hollan, Jean-
`Marie Hullot, Rob Jacob,
`Bonnie
`John,
`Sandy
`Kobayashi, T.K. Landauer, John
`Leggett, Roger Lighty, Marilyn Mantei, Jim
`Miller, William Newman, Jakob Nielsen, Don
`Norman, Dan Olsen, Ramesh Patil, Gary Perl-
`man, Dick Pew, Ken Pier, Jim Rhyne, Ben
`Shneiderman, John Sibert, David C. Smith,
`Elliot Soloway, Richard Stallman, Ivan Suther-
`land, Dan Swinehart, John Thomas, Alex
`Waibel, Marceli Wein, Mark Weiser, Alan
`Wexelblat, and Terry Winograd. Editorial
`comments were also provided by the forego-
`ing as well as by Jim Hollan, Ellen Borison,
`
`Rich McDaniel, Rob
`Miller, Bernita Myers,
`Yoshihiro Tsujino, and
`the reviewers.
`excerpt
`A
`short
`article
`from
`this
`appeared as part of
`“Strategic Direc-
`tions in Human
`Computer Inter-
`action,” B. Myers, J. Hollan,
`I. Cruz (eds.). ACM Computing Sur-
`veys 28, 4 (Dec. 1996), pp. 794–809.
`This research was sponsored partly by
`NCCOSC under Contract No. N66001-94-
`C-6037, ARPA Order No. B326, and partly
`by the National Science Foundation under
`Grant No. IRI-9319969. The views and con-
`clusions contained in this document are those
`of the author and should not be interpreted as
`representing the official policies, either
`expressed or implied, of NCCOSC or the
`U.S. Government.
`
`References
`1. Baecker, R., et al. “A Historical and Intel-
`lectual Perspective.” In Readings in
`Human–Computer Interaction: Toward the Year
`2000. (R. Baecker et al., eds.). 2nd ed. Mor-
`gan Kaufmann Publishers, Inc., San Francisco,
`1995, pp. 35–47.
`2. Brooks, F. “The computer ‘scientist’ as tool-
`smith—Studies in interactive computer graph-
`ics.” In Proceedings of IFIP Conference, 1977,
`pp. 625–634.
`3. Burtnyk, N. and Wein, M. “Computer
`generated key frame animation.” Journal of the
`Society of Motion Picture and Television Engi-
`neers 8, 3 (1971), pp. 149–153.
`4. Bush, V. “As we may think.” The Atlantic
`Monthly 176 (July 1945), pp. 101–108.
`Reprinted and discussed in interactions, 3, 2
`(Mar. 1996), pp. 35–67.
`5. Buxton, W. et al. “Towards a comprehen-
`sive user interface management system.” In
`Computer Graphics 17 (1983), pp. 35-42. Pro-
`ceedings of SIGGRAPH’83, Detroit, MI.
`6. Card, S.K. “Pioneers and Settlers: Methods
`Used in Successful User Interface Design.” In
`Human-Computer Interface Design: Success Sto-
`ries, Emerging Methods, and Real-World Con-
`
`text (M. Rudisill et al., eds.). Morgan Kauf-
`mann Publishers, San Francisco, 1996, pp.
`122–169.
`7. Coons, S. “An Outline of the Requirements
`for a Computer-Aided Design System.” In
`Proceedings of AFIPS Spring Joint Computer
`Conference 23 (1963), pp. 299–304.
`8. Engelbart, D. “The Augmented Knowledge
`Workshop.” In A History of Personal Worksta-
`tions (A. Goldberg, ed.). Addison-Wesley Pub-
`lishing Company, New York, 1988, pp.
`187-232.
`9. Engelbart, D. and English, W. “A Research
`Center for Augmenting Human Intellect.”
`Reprinted in ACM SIGGRAPH Video Review
`106 (1994), video made in 1968
`10. English, W.K., Engelbart, D.C., and
`Berman, M.L. “Display selection techniques
`for text manipulation.” IEEE Transactions on
`Human Factors in Electronics HFE-8, 1 (1967),
`pp. 5–15.
`11. Goldberg, A., ed. A History of Personal
`Workstations. Addison-Wesley Publishing
`Company, New York, 1988.
`12. Goldberg, A. and Robson, D. “A
`Metaphor for User Interface Design.” In Pro-
`ceedings of the 12th Hawaii International Con-
`
`ference on System Sciences 1 (1979), pp.
`148–157.
`13. Henderson Jr., D.A. “The Trillium user
`interface design environment.” In Proceedings
`of SIGCHI ’86, Human Factors in Computing
`Systems, Boston, MA, 1986, pp. 221–227.
`14. Hudson, S.E. and Smith, I. “Ultra-
`Lightweight Constraints.” In Proceedings of
`ACM SIGGRAPH Symposium on User Inter-
`face Software and Technology (UIST ’96), Seat-
`tle, WA, 1996, pp. 147–155. System available
`from
`http://www.cc.gatech.edu/gvu/ui/sub_arctic/
`15. Johnson, T. “Sketchpad III: Three
`Dimensional Graphical Communication with
`a Digital Computer.” In Proceedings of AFIPS
`Spring Joint Computer Conference 23 (1963),
`pp. 347–353.
`16. Kasik, D.J. “A user interface management
`system.” In Computer Graphics 16 (1982),
`Boston, MA, pp. 99–106. Proceedings SIG-
`GRAPH’82.
`17. Kay, A. The Reactive Engine. Doctoral
`dissertation, Electrical Engineering and Com-
`puter Science, University of Utah, 1969.
`18. Kay, A. “Personal dynamic media.” IEEE
`Computer 10, 3 (1977), pp. 31–42.
`
`i n t e r a c t i o n s . . . m a r c h + a p r i l 1 9 9 8
`
`53
`
`Page 10 of 11
`
`
`
`
`
`19. Koved, L. and Shneiderman, B. “Embed-
`ded menus: Selecting items in context.” Com-
`munications of the ACM 4, 29 (1986), pp.
`312–318.
`20. Levinthal, C. “Molecular model-building
`by computer.” Scientific American 214, 6
`(1966), pp. 42–52.
`21. Levy, S. Hackers: Heroes of the Computer
`Revolution. Anchor Press/Doubleday, Garden
`City, NY, 1984.
`22. Licklider, J.C.R. and Taylor, R.W. “The
`computer as communication device.” Sci. Tech
`(April 1968), pp. 21–31.
`23. Linton, M.A., Vlissides, J.M., and Calder,
`P.R. “Composing user interfaces with Inter-
`Views.” IEEE Computer 22, 2 (1989), pp.
`8–22.
`24. Meyrowitz, N. and Van Dam, A. “Interac-
`tive editing systems: Parts 1 and 2.” ACM
`Computing Surveys 14, 3 (1982), pp.
`321–352.
`25. Myers, B.A. “The user interface for Sap-
`phire.” IEEE Computer Graphics and Applica-
`tions 4, 12 (1984), pp. 13–23.
`26. Myers, B.A. “A taxonomy of user inter-
`faces for window managers.” IEEE Computer
`Graphics and Applications 8, 5 (1988), pp.
`65–84.
`27. Myers, B.A. “All the Widgets.” SIG-
`GRAPH Video Review 57 (1990). Available at
`http://www.acm.org/sigchi/video.
`28. Myers, B.A. “User interface software
`tools.” ACM Transactions on Computer
`Human Interaction 2, 1 (1995), 64–103.
`29. Myers, B.A. et al. “