`IBG ET AL. v. TRADING TECH
`CBM2015-00179
`
`
`
`artlcle —
`
`
`
`Brad A. Myers
`
`Human—Computer
`Interaction Institute
`
`School of Computer
`Science
`
`Carnegie Mellon
`University
`Pittsburgh, PA 15213-
`3891
`
`bam6cs.cmu.edu
`
`Research in human—computer
`
`interac-
`
`tion (HCI) has been spectacularly suc-
`
`cessful and has fundamentally changed
`
`computing. One example is the ubiqui-
`
`tous
`
`graphical
`
`interface
`
`used
`
`by
`
`Microsoft Windows 95, which is based
`
`on the Macintosh, which is based on
`
`work at Xerox PARC, which in turn is
`
`based on early research at the Stanford
`
`Research Laboratory (now SRI) and at
`
`Massachusetts Institute of Technology.
`
`Another example is that virtually all soft-
`ware written today employs user
`interface
`toolkits and interface builders—concepts that
`were developed first at universities. Even the
`remarkable growth of the World Wide Web is
`a direct
`result of HCI research: applying
`hypertext technology to browsers allows one
`to traverse a link across the world with a click
`
`of the mouse. More than anything else,
`improvements to interfaces have triggered this
`explosive growth. Furthermore, the research
`that will lead to the user interfaces for the
`
`computers of tomorrow is being carried out
`today at universities and a few corporate
`research labs.
`
`This paper attempts to briefly summarize
`many of the important research developments
`in HCI technology, emphasizing the role of
`university research, which may not be widely
`recognized. By “research,” I mean exploratory
`work at universities and government and cor-
`porate research labs (such as Xerox PARC)
`that is not directly related to products. By
`“HCI technology,” I am referring to the com-
`puter side of HCI. A companion article on the
`history of the “human side,” discussing the
`contributions
`from psychology, design,
`human factors, and ergonomics would also be
`appropriate.
`
`Figure 1 shows the time span for some of
`the technologies discussed in this article.
`including when they were introduced. Of
`course, a deeper analysis would reveal signifi-
`cant interaction among the university, corpo-
`rate research, and commercial activity lines. It
`is
`important
`to appreciate that years of
`research are involved in creating and making
`these technologies ready for widespread use.
`The same will be true for the HCI technolo-
`
`gies currently being developed that will pro-
`vide the interfaces of tomorrow.
`
`Clearly it is impossible to list every system
`and source in a paper of this scope, but I have
`tried to represent the earliest and most influ-
`ential systems. Further information can be
`found in other surveys of HCI topics (see, for
`example,
`[1, ll, 36, 41]). Another useful
`resource is the video All 771:’ Wtdgm, which
`shows the historical progression of a number
`of user interface ideas [27].
`
`The technologies discussed in this paper
`include fimdamental interaction styles such as
`direct manipulation,
`the mouse pointing
`device, and windows; several important kinds
`of application areas, such as drawing,
`text
`editing, and spreadsheets;
`the technologies
`that will
`likely have the biggest impact on
`interfaces of the future, such as gesture recog-
`nition, multimedia, and three-dimensionality;
`and the technologies used to mate interfaces
`using the other technologies, such as user
`interface management systems, toolkits, and
`interface builders.
`
`Basic Interactions
`Duu-:c'r MANn>UumoN op
`
`GRAPHICAI. OBJECTS
`
`The now ubiquitous direct manipulation inter-
`face, where visible objects on the screen are
`directly manipulated with a pointing device,
`was first demonstrated by Ivan Sutherland in
`Sketchpad [47], the thesis of his doctoral dis-
`sertation in 1963. Sketchpad supported the
`manipulation of objects using a light pen,
`including grabbing objects, moving them,
`changing size, and using constraints. It con-
`tained the seeds of myriad important interface
`ideas. The system was built at Lincoln [abs
`with support from the U.S. Air Force and the
`National Science Foundation (NSF).
`
`interactions..
`
`.march + april
`
`I998
`
`Page 2 of 11
`
`
`
`KEY:
`
`‘ University Research
`
`technologies discussed in this article.
`
`2 C°'P°'at° R°5°a"h
`2 Commercial Products
`
`Figure 1. Approximate time lines showing where
`and when work was performed on some major
`
`Direct Manipulation of Graphical Objects
`
`The Mouse
`
`interactions...march + april
`
`1998
`
`Page 3 of 11
`
`
`
`Wdliam Newman’s Reaction Handler [33],
`
`created at Imperial College, London during
`1966 and 1967, provided direct manipulation
`of graphics and introduced Light Handles
`[32], a form of graphical potentiometer that
`was probably the first “widget.” Another early
`system was AMBIT/G (implemented at the
`Massachusetts Institute of Technology’s
`(MIT) Lincoln Labs in 1968 and
`
`artlcle —
`
`NLS project (funding from ARPA, NASA.
`and Rome ADC) [10]. It was intended to be a
`
`cheap replacement for light pens, which had
`been used at least since 1954 [11, p. 68].
`Many of the current uses of the mouse were
`demonstrated by Doug Engelbart as part of
`NLS in a movie created in 1968 [9]. The
`mouse was then made famous as a
`
`input device by Xerox
`practical
`PARC in the 19705.
`It
`first
`
`firnded by the Advanced Research
`Projects Agency (ARPA).
`It
`employed, among other inter-
`face techniques, iconic repre-
`sentations, gesture recog-
`nition, dynamic menus with
`items selected using a point-
`ing device, selection of icons
`by pointing, and moded
`and mode-free styles of
`interaction.
`
`"1115 5|(E1'cHpAD
`
`SYSTEM MAKES IT
`
`appeared commercially as part
`of the Xerox Star (1981), the
`
`POSSIBLE FOR A
`
`MAN AND A coM_
`
`PUTER TO CON-
`
`VERSE RAPIDLY
`
`THROUGH THE
`
`Three Rivers Computer
`Company’s PERQ (1981)
`[25],
`the Apple Lisa
`(1982), and the Apple
`Macintosh (1984).
`
`\W1N1)oWs
`
`Multiple tiled windows
`were
`demonstrated
`in
`
`David Canfield Smith
`
`MEDIUM OF LINE
`
`coined the term “icons” in his
`1975 doctoral
`thesis on Pyg-
`malion [44] (funded by ARPA
`and National Institute of Mental
`
`DRAw|NG5_n
`
`-"";,',",,“,’,_",'§,',',““""’
`
`Engelbart’s NLS in 1968
`[9]. Early research at Stan-
`ford on systems like com-
`LOT (1974) [49] and at MIT
`with the EMACS text editor (1974)
`[46] also demonstrated tiled windows.
`
`Alan Kay proposed tl1e idea of overlapping
`windows in his 1969 doctoral thesis [17], and
`
`overlapping windows first appeared in 1974 in
`his Smalltalk system [12] at Xerox PARC, and
`soon afterward in the InterLisp system [50].
`Some of the first commercial uses of win-
`
`dows were on Lisp Machines Inc. (LMI) and
`Symbolics Lisp Machines (1979). which grew
`out ofMIT Artificial Intelligence (AI) Lab pro-
`jects. The Cedar Window Manager from Xerox
`PARC was the first major tiled window manag-
`er (1981) [48], followed soon by the Andrew
`window manager [35] by Carnegie Mellon
`University’s
`(CMU)
`Information
`Technology Center (1983, funded
`by IBM). The main commercial
`systems popularizing windows
`were the Xerox Star (1981), the
`
`Apple Lisa (1982), and most
`important,
`the Apple Macin-
`tosh (1984). The early versions of
`the Star and Microsoft Windows
`
`were tiled, but eventually they sup-
`ported overlapping windows like the
`
`
`
`Health — NIMH). Smith later popu-
`larized icons as one of the chief designers
`of the Xerox Star [45]. Many of the interac-
`tion techniques popular in direct manipula-
`tion interfaces, such as how objects and text
`are selected, opened, and manipulated, were
`researched at Xerox PARC in the 1970s. In
`
`particular, the idea of “WYSIWYG” (what
`you see is what you get) originated there with
`systems such as the Bravo text editor and the
`
`Draw drawing program [1 1]
`The concept of direct manipulation inter-
`faces for everyone was envisioned by Alan Kay
`of Xerox PARC in a 1977 article about the
`
`Dynabook [18]. The first commercial systems
`to use direct manipulation extensively were the
`Xerox Star (1981) [45], the Apple Lisa (I982)
`[54], and the Macintosh (1984)
`[55]. Ben
`
`Shneiderman at the University of Maryland
`coined the term “direct manipulation” in 1982,
`identified the components, and gave psycholog-
`ical motivations for direct manipulation [43].
`
`MOUSE
`
`The mouse was developed at Stanford
`Research Laboratory in 1965 as part of the
`
`inreractions...march 4- april
`
`I998
`
`Page 4 of 11
`
`
`
`Lisa and Macintosh. The X Window System, a
`current intemational standard, was developed
`at MIT in 1984 [42]. For a survey of window
`managers, see [26].
`
`based display editors that was widely used
`[51]. The Hypertext Editing System [53, p.
`108] from Brown University had screen edit-
`ing and formatting of arbitrary-sized strings
`with a light pen in 1967 (firnding from
`IBM). NLS demonstrated mouse-
`
`Applications
`DRAWING PROGRAMS
`
`Much of the current technology
`for drawing programs was
`demonstrated in Sutherland’s
`
`1963 Sketchpad system
`[47]. The use of a mouse to
`
`manipulate graphics was
`demonstrated in NLS
`
`In 1968 Ken
`(1965).
`and
`Grant
`Pulfer
`Bechthold at the Nation-
`
`al Research Council of
`Canada built a mouse out
`
`of wood patterned after
`Engelbart’s mouse and
`used it with a key-frame
`animation system to draw
`all the frames of a movie. A
`
`"THE DIFFICULTY
`
`OF MANKIND'S
`
`PROBLEMS WAS
`
`INCREASING AT A
`
`based editing in 1968 (Figure 4).
`TECO from MIT was an early
`screen editor
`(1967), and
`
`EMACS [46] was developed
`from it
`in 1974. Xerox
`
`PARC’s Bravo [11, p. 284]
`was the first WYSIWYG
`
`GREATER RATE THAN
`
`editor-formatter (1974).
`
`OUR ABILITY TO
`
`COPE.
`
`(WE ARE IN
`
`TROUBLE.) [SO] I
`
`COMMITTED MY
`
`CAREER TO
`
`'AUGMENTING THE
`
`HUMAN lNTELLECI'.’"
`
`- nous ENGELBART, on THE
`NLS SYSTEM [8, P. 189]
`
`It was designed by Butler
`Lampson and Charles
`Simonyi, who had started
`working on these con-
`cepts about 1970 while at
`Berkeley. The first com-
`mercial WYSIWYG edi-
`
`tors were
`LisaWrite,
`
`the
`and,
`
`Star,
`later,
`
`MacWrite. For surveys of
`‘ext °di‘°'5* See [24] and [53]-
`
`SPREADSHEETS
`
`The initial spreadsheet was Visi-
`Calc which was developed by
`Frankston and Bricklin (1977-1978) for
`
`the Apple II while they were students at MIT
`and the Harvard Business School, respectively.
`The solver was based on a dependency-direct-
`ed backtracking algorithm by Sussman and
`Stallman at the MIT AI Lab.
`
`l'IYPI-ZRTI-IX'l'
`
`The idea for hypertext (by which documents are
`linked to related documents) is credited to Van-
`nevar Bush’s famous MEMEX idea from 1945
`
`[4]. Ted Nelson coined the term “hypertext” in
`1955 [31]. Engelbart’s NIS system [9] at the
`Stanford Research Laboratories in 1965 made
`
`extensive use of linking (fimding from ARPA,
`NASA, and Rome ADC). The “NLS Journal”
`
`[1 1, p. 212] was one of the first on-line journals
`and included full
`of articles (1970). The
`Hypertext Editing System, jointly designed by
`Andy van Dam, Ted Nelson, and two students
`at Brown University (funding from IBM) was
`distributed extensively [52].
`
`subsequent movie in 1971,
`Hunger, won a number of
`awards and was drawn using a
`tablet instead of the mouse (fund-
`
`ing by the National Film Board of
`Canada) [3]. William Newmans Markup
`(1975) was the first drawing program for
`Xerox PARC’s Alto,
`followed shortly by
`Patrick Baudelaire’s Draw, which added han-
`
`dling of lines and curves [1], p. 326]. The
`first computer painting program was probably
`Dick
`Shoup’s
`Superpaint
`at
`PARC
`(1974-1975). The first widely used drawing
`programs were MacPaint and MacDraw on
`the Macintosh (1984).
`
`
`
`TEXT EDITING
`
`In 1962, at the Stanford Research
`
`laboratory, Engelbart proposed,
`and later implemented, a word
`processor with automatic word
`wrap, search and replace, user-
`definable macros, scrolling text,
`and commands to move, copy,
`and delete characters, words, or
`blocks of text. Stanford's TVEdit
`
`(1965) was one of the first CRT-
`
`interactions...marcII + april
`
`I998
`
`Page 5 of 11
`
`
`
`artlcle —
`
`manufacturing
`aided
`(CAM) was probably
`General Motors DAC-
`
`1 (about 1963).
`
`VIDEO GAMES
`
`The first graphi-
`cal video game
`was
`probably
`SpaceWar by Slug
`Russel of MIT in 1962 for
`
`the PDP-1 [21, p. 49], including the
`first computer joysticks. The early computer
`Adventure game was
`created by Will
`Crowther at Bolt, Beranek 8c Newman
`
`
`
`The University of Vermont’s
`PROMIS (1976) was the first
`
`hypertext system released to
`the user community.
`It
`was used to link patient
`and patient care infor-
`mation at the Univer-
`
`sity of Vermont’s
`medical
`center. The
`
`from
`(1977)
`ZOG project
`CMU was another early hypertext
`system and was funded by Oflice of
`Naval Research (ONR) and ARPA [39]. Ben
`
`Shneiderman’s Hyperties was the first system
`in which highlighted items in the text could
`be clicked on to go to other pages (1983, Uni-
`versity of Maryland) [19]. HyperCard from
`Apple (1988) significantly helped to
`bring the idea to a wide audience.
`Tim Berners-Lee used the
`
`(BBN), and Don Woods developed it into a
`more sophisticated Adventure game at Stan-
`ford in 1966 [21, p. 132]. Conway's
`game of LIFE was developed on
`computers at MIT and Stan-
`ford in 1970. The first popu-
`lar commercial game was
`Pong (about 1976).
`
`-Mu|_'np|_E
`
`WINDOWS ALLOW A
`DOCUMENT
`(COMPOSED OF TEXT.
`
`idea to create the
`hypertext
`World Wide Web in 1990 at
`the
`government-funded
`European Particle Physics
`Laboratory
`(CERN).
`Mosaic, the first popular
`hypertext browser for
`the World-Wide Web,
`
`the
`was developed at
`University of Illinois’
`National Center
`for
`Supercomputer Appli-
`cations (NCSA). For a
`more complete history of
`HyperText, see [34].
`
`COMPUTI-1R-AIDED DESIGN
`The same 1963 International
`Federation of Information Pro-
`
`PICTURES. MUSICAL
`
`NOTATION) TO BE
`
`CREATED AND
`
`VIEWED
`s'MU|_1-ANEOUSLY AT
`
`SEVERAL LEVELS OE
`REFINEMENT. "
`
`— ALAN KAY,
`0" 5MAu_m,_K.72
`['3' "- 341-
`
`Up-and-Coming Areas
`GESTURE RECOGNITION
`The
`first
`pen-based
`input device, the Rand
`tablet, was funded by
`ARPA. Sketchpad used
`light-pen
`gestures
`(1963). Teitelman in
`1964 developed the first
`trainable gesture recog-
`nizer. A very early demon-
`stration
`of
`gesture
`_
`_
`_
`l’€COgIlltl0n was Tom Ellis’s
`GRAIL system on the Rand
`tablet (1964, fiinded by ARPA).
`It was quite common in light-
`pen—based systems to include some
`gesture recognition,
`for example in the
`AMBIT/G system (1968, funded by ARPA). A
`gesture-based text editor using proofreading
`symbols was developed at CMU by Michael
`Coleman in 1969. Bill Buxton at the Universi-
`
`cessing Societies (IFIPS) confer-
`ence
`at which Sketchpad was
`presented also contained a number of com-
`puter—aided design (CAD) systems, includ-
`ing Doug Ross’s Computer-Aided Design
`Project at MIT in the Electronic Systems Lab
`[40] and Coons’s work at MIT with Sketch-
`
`Pad [7]. Timothy Johnson's pioneering work
`on the interactive 3D CAD system Sketch-
`pad 3 [15] was his doctoral thesis in 1963
`(fimded by the U.S. Air Force). The first sys-
`tem in industry using CAD and computer-
`
`ty of Toronto has been studying gesture-based
`interactions since 1980. Gesture recognition
`has been used in commercial CAD systems
`since the 19705 and came to universal notice
`
`with the Apple Newton in 1992.
`
`interactions..
`
`.march 4- april
`
`I998
`
`Page 6 of 11
`
`
`
`MULTIMEDIA
`
`The FRESS project at Brown University used
`multiple windows and integrated text and
`graphics (1968, funding from industry). The
`Interactive Graphical Documents project at
`Brown was the first hypermedia (as opposed
`to hypertext) system and used raster graphics
`and text but not video (1979-1983, fimded
`
`by NASA, U.S. Navy, and others). Another
`important center of current research on 3-D
`interfaces is Fred Brooks’s lab at University of
`North Carolina at Chapel Hill (e.g., [2]).
`
`Vntruu. REALITY AND “AUGMENTED R1-:AuTv”
`
`The original work on virtual reality (VR) was
`performed by Ivan Sutherland when he was at
`Harvard (1965-1968, funding by Air
`Force, Central Intelligence Agency,
`and Bell Laboratories). Very
`important early work was
`done by Tom Fumess when
`he was at \Wright-Patter-
`son
`AFB.
`Myron
`
`IT
`
`IS CONSTRUCTED FROM
`
`Group (MIT) was one
`of the first to demon-
`strate mixed video
`
`and computer graph-
`ics in 1983 (funded
`
`by ARPA).
`
`THREE-
`DIMENSIONALITY
`
`The first 3D system
`was probably Timothy
`Johnson's 3D CAD sys-
`tem mentioned earlier
`
`(1963, funded by the U.S.
`Air Force)
`[15]. The Lin-
`
`TWO POTENTIOMETERS.
`
`MOUNTED ORTHOGONALLY.
`
`EACH OF WHICH HAS A
`
`WHEEL ATTACHED TO ITS
`
`SHAFT. THE MOUNTING
`
`FRAME
`
`IS ENCLOSED
`
`INAZINCHXZINCHX4
`
`INCH WOODEN CASE."
`
`— W.K. ENGLISH, D.C. ENGELBAR1; AND
`M.L. HERMAN, ON THE NLS SYSTEM
`110.!’-7]
`
`Kruegers early work at
`the University of Con-
`necticut on interfaces
`
`incorporate and
`that
`interpret a video of
`the user was influen-
`tial. Fred Brooks?» and
`
`Henry Fuch’s groups
`at UNC did a lot of
`
`early research, includ-
`ing the study of force
`feedback (1971, fund-
`
`ing from U.S. Atomic
`Energy Commission and
`NSF). Much of the early
`research on head-mounted
`
`coln Wand by Larry Roberts
`was an ultrasonic 3D location
`
`displays and on the DataGlove
`was supported by NASA.
`
`sensing system developed at Lincoln
`Labs (1966, funded by ARPA). That system
`also had the first interactive 3D hidden line
`
`COMPUTER-SUP:-oim-:0
`COOPERATIVE WORK
`
`Doug Engelbart’s 1968 demonstration of
`NLS [9] included the remote participation of
`multiple people at various sites (funding from
`ARPA, NASA, and Rome ADC). Licklider
`
`and Taylor predicted online interactive com-
`munities in a 1968 article [22] and speculat-
`ed about the problem of access being limited
`to the privileged. Electronic mail, still the
`most widespread multiuser software, was
`enabled by the ARPAnet, which became
`operational
`in 1969, and by the Ethernet
`from Xerox PARC in 1973. An early comput-
`er conferencing system was Turoff’s EIES sys-
`tem at the New Jersey Institute ofTechnology
`( 1975).
`
`elimination. An early use was for molecular
`modeling [20]. The late 1960s and early
`1970s saw the flowering of 3D raster graph-
`ics research at the University of Utah with
`Dave Evans,
`Ivan Sutherland, G.\W.
`
`Romney, Henri Gouraud, Bui-Tuong
`Phong, and G.S. Watkins, much
`of it government funded. Also,
`the military-industrial
`flight
`simulation work of the 1960s
`
`
`
`and 1970s led the way to mak-
`ing 3D interfaces operate in real
`time on commercial
`systems
`from General Electric, Evans 85
`
`Sutherland, SingerILink (funded
`
`
`
`interactions...march + april
`
`I998
`
`Page 7 of 11
`
`by ONR and NSF). The Diamond
`project at BBN (starting in 1982,
`funded by ARPA) explored
`combining multimedia infor-
`mation (text, spreadsheets,
`graphics,
`speech). The
`Movie Manual at
`the
`Architecture Machine
`
`THE ‘MOUSE’ WAS
`
`DEVELOPED BY SRI.
`
`
`
`artlcle —
`
`NATURAL LANGUAGE AND SPEECH
`
`The fundamental research for speech and nat-
`ural language understanding and generation
`has been performed at CMU, MIT, SR1,
`BBN, IBM, AT&T Bell Labs and BellCore.
`
`much of it government funded. See, for exam-
`ple, [37] for a survey of the early work.
`
`Software Tools and Architectures
`
`The area of user interface (UI) software tools
`
`is quite active now, and many companies are
`selling tools. Most of today’s applications are
`implemented using various forms of software
`tools. For a more complete survey and discus-
`sion of U1 tools, see [28].
`
`USER INTERFACE MANAGEMENT SYSTEMS
`AND Tooucrrs
`
`User interface management systems (UIMS)
`and toolkits are software libraries and tools
`
`allow interfaces composed of widgets such as
`buttons, menus, and scrollbars to be placed
`using a mouse. The Steamer project at BBN
`(1979-1985; funded by ONR) demonstrated
`many of the ideas later incorporated into
`interface builders and was probably the first
`object-oriented graphics system. Trillium [13]
`was developed at Xerox PARC in 1981.
`Another early interface builder was the
`MenuI..ay system [5] developed by Bill Buxton
`at the University of Toronto (1983, funded by
`the Canadian Government). The Macintosh
`(1984)
`included a Resource Editor
`that
`
`allowed widgets to be placed and edited. jean-
`Marie Hullot created SOS Interface in Lisp
`for the Macintosh while working at The
`French National
`Institute for Research in
`
`(INRIA)
`Computer Science and Control
`(1984, funded by the French government)
`which was the first modern interface builder.
`
`that help to create user interfaces. The first
`UIMS was VVilliam Newman’s Reaction Han-
`
`Hullot built this into a commercial product in
`1986 and then went to work for NeXT and
`
`created the NeXT Interface Builder (1988),
`
`which popularized this type of tool. Now
`there are literally hundreds of commercial
`interface builders.
`
`COMPONENT ARCHITECTURES
`
`The idea of creating interfaces by connecting
`separately written components was
`first
`demonstrated in the Andrew project [35] by
`Carnegie Mellon University’s Information
`Technology Center (1983, funded by IBM). It
`is now being widely popularized by Microsofts
`object linking and embedding (OLE), Apple
`and IBM’s OpenDoc, and the Java Beans
`architectures.
`
`dler [33] created at Imperial College, London
`(1966-1967 with SRC funding). Most of the
`early work was done at universities (University
`of Toronto with Canadian government fimd-
`ing; George Washington University with
`NASA, NSF, Department of Energy, and
`National Bureau of Standards (NBS) funding;
`Brigham Young University with industrial
`funding; and so on). The term “UIMS” was
`coined by David Kasik at Boeing (1982) [16].
`Early window managers such as Smalltalk
`(1974) and InterLisp, both from Xerox PARC,
`came with a few widgets, such as popup menus
`and scrollbars. The Xerox Star (1981) was the
`
`first commercial system to have a large collec-
`tion of The Apple Macintosh (1984)
`was the first to actively promote its toolkit for
`use by other developers to enforce a consistent
`interface. An early C++ toolkit was InterViews
`[23], developed at Stanford (1988, industrial
`funding). Much of the modern research is
`being performed at universities, for example,
`the Garnet (1988) [29] and Amulet (1994)
`
`[30] projects at CMU (fimded by DARPA)
`and SubArctic at Georgia Tech (1996, funded
`by Intel and NSF) [14].
`
`INTERFACE BUILDERS
`
`Interface builders are interactive tools that
`
`inreractions..
`
`.Inarch 4- april
`
`I998
`
`Page 8 of 11
`
`Discussion
`
`It is clear that all of the most impor-
`tant
`innovations in HCI have
`
`benefited from research at
`
`both corporate research labs
`and universities. One moti-
`vation for this article is to
`
`overcome the impressions
`some people may have that
`much of the important work
`in human—computer interac-
`tion occurred in industry and
`that
`if university research in
`
`
`
`
`
`HCI is not supported, then industry will just
`carry on anyway. This is simply not true. I
`hope that this paper has shown that many of
`the most famous HCI successes developed by
`companies are deeply rooted in univer-
`sity research. In fact, virtually all of
`today's major
`interface styles
`and applications have been
`significantly influenced by
`research at universities
`
`industry needs. It seems that only through
`computer science does HCI research dissemi-
`nate to products. Furthermore, without
`appropriate levels of funding of academic
`HCI research, there will be fewer doc-
`
`toral graduates in HCI to perform
`research in corporate labs, and
`fewer top—notch graduates in
`this area will be interested
`
`"IN EMBEDDED
`
`future will use gesture recognition,
`speech recognition, “intelligent agents,”
`adaptive interfaces, video, and many other
`technologies now being investigated by
`research groups at universities and corporate
`labs [38]. Therefore, it is imperative that uni-
`versity, corporate, and government-supported
`research continue and be well-supported, so
`that we can develop the science and technolo-
`gy needed for the user interfaces of the future.
`
`Acknowledgments
`
`I must thank a large number of people who
`responded to posts of earlier versions of this
`article on the announcememschi mailing list
`for their very generous help, and to Jim I-Iollan
`who helped write the short excerpt of this arti-
`cle. Some of the information in this article was
`
`supplied by (in alphabetical order): Stacey Ash-
`lund, Meera M. Blattner, Keith Butler, Stuart
`K. Card, Bill Curtis, David E. Damouth, Dan
`
`use and learn than they are
`today. As described by Stu
`Card [6, p. 162]:
`of
`fimding
`Government
`advanced human-computer interac-
`tion technologies huilt the intellectual capi-
`tal and trained the research teams for pioneer
`systems that, over a period of25 years, revolu-
`tionized how people interaet with computers.
`Industrial research laboratories at the corporate
`level in Xerox, IBM, ATé'7f and others played a
`strong role in developing this technology and
`hringing it into a firrm suitahlefor the commer-
`cial arena.
`
`
`
`Another important argument in favor
`of HCI research at universities is that
`
`computer science students need to
`know about user interface issues.
`
`User interfaces are likely to be
`one of the main value-added
`
`competitive advantages of the
`future as both hardware and basic
`software become commodities. If
`
`students do not know about user
`
`interfaces,
`
`they will not serve
`
`interactions.. .march + aprH
`
`I998
`
`Page 9 of 11
`
`hummwrouumnmmwnum
`GNHEMHTMMLGHNBIWK
`
`Kxnmuuauumuws
`mnmmvnmmnnmnnmnn
`Twuamsummnuuzm
`mmwmnmmnmonam
`
`Mnauuwnnmanmammm
`inmznwmumnmmwm
`mumnnmmnmmmm
`uonaiscwaunureownncisnv
`
`mmumwMMJmJbam
`mHumanoummm1omn
`
`tmmufimormnmmuwmno
`
`mmunanmmsnnmnum
`mmnmknnmn
`
`OACMIWQSDNWMMSIW
`
`and labs, often with gov-
`ernment funding. To
`illustrate this point,
`this article has listed
`
`the funding sources
`of some of the major
`advances. Without
`
`this research, many
`of the advances in
`the field of HCI
`
`would probably not
`have taken place, and
`as a consequence, the
`user interfaces of com-
`
`mercial products would
`be far more diflicult to
`
`MENUS. HIGHLIGHTED
`
`OR UNDERLINED
`
`WORDS OR PHASES
`
`WITHIN THE TEXT
`
`BECOME THE MENU
`
`ITEMS, AND ARE
`
`SELECTABLE USING THE
`
`COMMONLY USED
`
`TOUCH SCREEN.
`
`CURSOR AND MOUSE
`
`METHODS."
`
`— L. KOVED AND
`ESMWMMRMANONTHE
`HYPERflE$SY5THH[1&l!312L
`
`in being professors, so
`the needed user inter-
`face courses will not be
`
`offered.
`The conventional
`
`style of graphical
`user interfaces that
`
`use windows, icons,
`menus, and a mouse
`
`are in a phase of
`standardization,
`
`where almost every-
`one
`is using the
`same, standard tech-
`
`nology and making
`incremental
`small,
`
`changes. As computers
`perform faster, more of
`the processing power
`is
`being devoted to the user
`interface. The interfaces of the
`
`
`
`artlcle —
`
`Rob
`
`Rich McDaniel,
`
`Miller, Bernita Myers,
`Yoshihiro Tsujino, and
`the reviewers.
`
`A short
`from this
`
`excerpt
`article
`
`appeared as part of
`“Strategic Direc-
`tions in Human
`
`Inter-
`Computer
`action,” B. Myers, J. Hollan,
`I. Cruz (eds.). ACM Computing Sur-
`veys 28, 4 (Dec. 1996), PP- 794-309.
`This research was sponsored partly by
`NCCOSC under Contract No. N6600l—94-
`
`C-6037, ARPA Order No. B326, and partly
`by the National Science Foundation under
`Grant No. IRI-9319969. The views and con-
`clusions contained in this document are those
`
`of the author and should not be interpreted as
`representing the official policies, either
`expressed or implied, of NCCOSC or the
`U.S. Government.
`
`
`
`Diaper, Didt Duda, Tim T.K.
`Dudley, Steven Feiner, Harry
`Forsdick, Bjorn Freeman-
`Benson, John Gould,
`Wayne Gray, Mark
`Green, Fred Hansen,
`
`D.
`Hefley,
`Bill
`Austin Henderson,
`
`Jean-
`Jim Hollan,
`Marie Hullot, Rob Jacob,
`
`Sandy
`John,
`Bonnie
`Kobayashi, T.K. Lrndauer, John
`Leggett, Roger
`Mantei, Jim
`Miller, William Newman, Jakob Nielsen, Don
`
`Norman, Dan Olsen, Ramesh Patil, Gary Perl-
`man, Did: Pew, Ken Pier, Jim Rhyne, Ben
`Shneiderman, John Sibert, David C. Smith,
`Elliot Soloway, Richard Stallman, Ivan Suther-
`land, Dan Swinehart, John Thomas, Alex
`Waibel, Marceli Wein, Mark Weiser, Alan
`
`Wexelblat, and Terry Winogmd. Editorial
`comments were also provided by the forego-
`ing as well as by Jim Hollan, Ellen Borison,
`
`References
`
`I. Baecker, R., et al. “A Historical and Intel-
`
`lectual Perspective.” In Reading: in
`Human—Carnpuner Intemttian: Tnuarzl the Year
`2000. (R. Baecker et al., eds.). 2nd ed. Mor-
`
`gan Kaufmann Publkhers, Inc., San Francisco,
`I995, pp. 35-47.
`2. Brooks, F. "The computer ‘scientist’ as tool-
`smith—Studies in interactive computer graph-
`
`ics.” In Pmcevdings ofIFIP Carjérrrrce, I977.
`pp. 625-634.
`3. Burtnyk, N. and Wein, M. ‘Computer
`
`text(M. Rudisill et al., eds.). Morgan Kauf-
`mann Publishers, San Francisco, 1996. pp.
`I22-I69.
`
`7. (Eons, S. "An Outline of the Requirements
`for a (hmputer-Aided Design System.” In
`
`Prrrteedings ofAFIPS Springfoirrt Computer
`Conference 23 (1963). pp. 299-304.
`8. Engelbart, D. “The Augmented Knowledge
`
`Workshop." In A History afPerwnal W¢rrbta—
`tinrnr (A. Goldberg, ed.). Addison—Wesley Pub-
`lishing Company, New York, 1988, pp.
`187-232.
`
`generated key frame animation.” Imrrnal aftbe
`S0012-I] ofMation Picture and Teladsion Engi-
`neers 8, 3 (1971), pp. 149-153.
`4. Bush, V. “Aswe maythinlr." TbeAr‘Iantit
`
`9. Engelbart, D. and Fnglish, W “A Research
`Center for
`Hiunan Intellect.”
`Reprinted in ACIW SIGGRAPH Video Review
`I06(1994), video made in I968
`
`Monthly 176 (July 1945), pp. l0l—l08.
`Reprintedanddiscumedin inremaiorrs, 3,2
`(Mar. 1996). pp. 35-67.
`5. Buxmn, W. et al. "Towards a eomprd1en-
`sive user interface management system.” In
`
`Computer Graplria 17 (1983). PP- 3542. Pro-
`ceedings of SIGGRAPH’83, Detroit, MI.
`6. Card, S.K. "Pioneers and Settlers: Methods
`
`Used in Successful User Interface D5ign.' In
`
`Hunran—C0mpuIer Intzjhte Design: Success Sm-
`riu, Emerging Mnbodc, and Real-World Con-
`
`10. English, W.K., Engelbart, D.C., and
`Berman, M.I.. "Display selection techniques
`for tat manipulation.” IEEE Tmrrsauiorr: on
`Human Facmrs in Eltcrmrricr HFE-8, I (1967),
`
`pp. 5-15.
`
`ll. Goldberg, A., ed. A Hi.mrr_'y afPersanaI
`Workstations. Addison-Wesley Publishing
`Company, New York, 1988.
`I2. Goldberg, A. and Robson, D. "A
`Metaphor for User Interface Daign." In Pm-
`
`firmer an S_y.rtern Science: 1 (1979), pp.
`I48—I 57.
`
`13. Henderson Jr., D.A. ‘The Trillium user
`
`interface design environment.” In Pmceadingr
`afSIGCHI ’8& Human Factors in Computing
`S_y.1'tnrr.r, Boston, MA, 1986, pp. 221.227.
`14. Hudson, SJ‘. and Smith, I. ‘Ultra-
`
`Lightweight Constraints.” In Prrrcudirrgr vf
`ACM SIGGRAPH .5}-mpnsium on User Inter-
`fitcc Sofia/arr and Technology (UI5T '96), Seat-
`tle, WA, 1996. PP- I47-I 55. System available
`from
`
`http:I/www.ce.gatech.eduIgvuIuiIsub_arctic/
`15. Johnson, T. ‘Sketchpad III: Three
`Dimensional Graphical Communication with
`
`a Digital Computer.” In Pmmdingr afAFIPS
`Springjaint Campuur Canfirrncr 23 (1963),
`pp. 347-353.
`16. Kasik, D.J. ‘A user interface management
`system.” In Computer Graphics 160982),
`Boston, MA, pp. 99-106. Proceedings SIC-
`GRAPH’82.
`
`17. Kay, A. The Reactive Engine. Doctoral
`dissertation, Electrical Engineering and Com-
`puter Science, University of Utah, 1969.
`18. Kay, A. “Personal dynamic media." IEEE
`
`teedingr aftbe I2tIr Harmii Innenrazinnal Can-
`
`Computer 10, 3 (1977), pp. 51-42.
`
`interactions..
`
`.march 4- april
`
`I998
`
`Page 10 of 11
`
`
`
`I9. Koved, L. and Shneiderrnan, 3. "Embed-
`
`ded menus:
`
`items in context.’ Com-
`
`munieafiom ofthe ACM 4, 29 (1985), pp.
`3 12-3 I8.
`
`20. levinthal, C. "Molecular model-building
`by computer." Scientific American 214, 6
`(1966), pp. 42-52.
`
`21. Levy, S. Hackers: Hema ofdze Computer
`Revolution. Anchor PressIDoubleday, Garden
`City, NY, 1984.
`22. Licklider, ].C.R. and Taylor, KW. "The
`computer as oommunimtion device." 301'. Tech
`(April 1968), pp. 21-31.
`23. Iinton, M.A., Vlissides, ].M., and Calder,
`
`RR. “Composing user interfaces with Inter-
`
`Views." IEEE Computer 22, 2 (1989), pp.
`8-22.
`
`24.
`tive
`
`N. and Van Dam, A. “Interac-
`systems: Parts I and 2." A94
`
`Survey [4, 3 (I982), pp.
`
`321-352.
`
`interfaces.” IEEE Cornputer23. 1 I (1990). pp.
`7l—85.
`
`30. Myers, B.A. et al. "EasilyAddingAnima-
`tions to Interfaces Using Cbnstraints.” In Pm-
`
`eeadingr ofACM SIGGRAPH Symposium on
`Urer Inrnfiae Sofiwore and Terlrnology ((113T
`'96), Seattle, WA. 1996, pp. l19—l28.
`31. Nelson, T. "A File Structure for the Com-
`
`plex, the Changing, and the Indeterminate."
`
`In Proceeding: ofACM Natr'onal Corferrnte,
`1965, pp. 84-100.
`32. Newman, WM. “A graphical technique
`
`for numerical input." The Conrputerjoumol
`II, 1 (1953). PP. fi3-64.
`33. Newman, W.M. “A System for Interactive
`
`Graphical Programming.” In Proceedings of
`AFIPS Springfoint Computer Corjenrnae 28
`(1968)- pp. 47-54.
`
`34. Nielsen, ]., Alultirnedio ondHypertart: die
`Inmmtmrd Beyond. Academic Press Profes-
`sional, Boston, 1995.
`
`25. Myers, B.A. ‘The user interface for Sap-
`
`35. Palay, A.]. et al. “The Andrew Toolkit, An
`
`phire.” IEEE Computer
`lion: 4, I2 (1984), pp. 13-13.
`26. Myers, B.A. “A taxonomy of user inta-
`
`ondAppIim—
`
`faces for window managers.” IEEE Compukr
`Graphics and Applications 8, 5 (1988). PP.
`65-84.
`
`27. Myers, B.A. "All the Widgets.” SIG-
`GRAPH Video Review 57 (1990). Available at
`
`http:/Iwww.acm.org/sigdrilvideo.
`28. Myers, B.A. “User interface software
`
`tools.” ACM Tnxn.1'a¢1ion.r on Computer
`Human Interaction 2, l (1995). 64-103.
`
`29. Myers, B.A. et al. ‘Garnet:
`Comprehensive support
`for graphical, highly
`interactive user
`
`
`
`Overview.” In Proc