`Human—Computer
`Interaction Technology
`
`Brad A. Myers
`
`Page 1 of 11
`
`TRADING TECH EXHIBIT 2026
`
`TRADESTATION v TRADING TECH
`
`CBM2015-00172
`
`
`
`artlcle —
`
`
`
`Brad A. Myers
`
`Human-Computer
`Interaction Institute
`
`School of Computer
`Science
`
`Carnegie Mellon
`University
`
`Pittsburgh, PA 15213-
`3891
`
`bam@cs.cmu.edu
`
`Research in human-computer interac-
`
`tion (HCI) has been spectacularly suc-
`
`cessful and has fundamentally changed
`
`computing. One example is the ubiqui-
`
`tous
`
`graphical
`
`interface
`
`used
`
`by
`
`Microsoft Windows 95, which is based
`
`on the Macintosh, which is based on
`
`work at Xerox PARC, which in turn is
`
`based on early research at the Stanford
`
`Research Laboratory (now SRI) and at
`
`Massachusetts Institute of Technology.
`
`Another example is that virtually all soft-
`ware written today employs user interface
`toolkits and interface builders—concepts that
`were developed first at universities. Even the
`remarkable growth of the World Wide Web is
`a direct
`result of HCI research: applying
`hypertext technology to browsers allows one
`to traverse a link across the world with a click
`
`of the mouse. More than anything else,
`improvements to interfaces have triggered this
`explosive growth. Furthermore, the research
`that will lead to the user interfaces for the
`
`computers of tomorrow is being carried out
`today at universities and a few corporate
`research labs.
`
`This paper attempts to briefly summarize
`many of the important research developments
`in HCI technology, emphasizing the role of
`university research, which may not be widely
`recognized. By “research,” I mean exploratory
`work at universities and government and cor-
`porate research labs (such as Xerox PARC)
`that is not directly related to products. By
`“HCI technology,” I am referring to the com-
`puter side of HCI. A companion article on the
`history of the “human side,” discussing the
`contributions
`from psychology, design,
`human factors, and ergonomics would also be
`appropriate.
`
`Figure 1 shows the time span for some of
`the technologies discussed in this article.
`including when they were introduced. Of
`course, a deeper analysis would reveal signifi-
`cant interaction among the university, corpo-
`rate research, and commercial activity lines. It
`is important
`to appreciate that years of
`research are involved in creating and making
`these technologies ready for widespread use.
`The same will be true for the HCI technolo-
`
`gies currently being developed that will pro-
`vide the interfaces of tomorrow.
`
`Clearly it is impossible to list every system
`and source in a paper of this scope, but I have
`tried to represent the earliest and most influ-
`ential systems. Further information can be
`found in other surveys of HCI topics (see, for
`example,
`[1, ll, 36, 41]). Another usefirl
`resource is the video All 777: Wtdgm, which
`shows the historical progression of a number
`of user interface ideas [27].
`
`The technologies discussed in this paper
`include fundamental interaction styles such as
`direct manipulation,
`the mouse pointing
`device, and windows; several important kinds
`of application areas, such as drawing,
`text
`editing, and spreadsheets;
`the technologies
`that will likely have the biggest impact on
`interfaces of the future, such as gesture recog-
`nition, multimedia, and three-dimensionality;
`and the technologies used to create interfaces
`using the other technologies, such as user
`interface management systems, toolkits, and
`interface builders.
`
`Basic Interactions
`Duuacr MAN11>UumoN or-‘
`
`GRAPHIQAL Onjt-:crs
`
`The now ubiquitous direct manipulation inter-
`face, where visible objects on the screen are
`directly manipulated with a pointing device,
`was first demonstrated by Ivan Sutherland in
`Sketchpad [47], the thesis of his doctoral dis-
`sertation in 1963. Sketchpad supported the
`manipulation of objects using a light pen,
`including grabbing objecm, moving them,
`changing size, and using constraints. It con-
`tained the seeds of myriad important interface
`ideas. The system was built at Lincoln [abs
`with support from the U.S. Air Force and the
`National Science Foundation (NSF).
`
`interactions.
`
`..marcI|
`
`+ april
`
`1998
`
`Page 2 of 11
`
`
`
`KEY:
`
`_ University Research
`
`technologies discussed in this article.
`
`2 C°'P°"ate R°5ea"h
`2 Commercial Products
`
`Figure 1. Approximate time lines showing where
`and when work was performed on some major
`
`Direct Manipulation of Graphical Objects
`
`The Mouse
`
`interactions...march + april
`
`1998
`
`Page 3 of 11
`
`
`
`William Newman’s Reaction Handler [33],
`
`created at Imperial College, London during
`1966 and 1967, provided direct manipulation
`of graphics and introduced Light Handles
`[32], a form of graphical potentiometer that
`was probably the first “widget.” Another early
`system was AMBIT/G (implemented at the
`Massachusetts Institute of Technology’s
`(MIT) Lincoln Labs in 1968 and
`
`artlcle —
`
`NLS project (funding from ARPA, NASA,
`and Rome ADC) [10]. It was intended to be a
`
`cheap replacement for light pens, which had
`been used at least since 1954 [11, p. 68].
`Many of the current uses of the mouse were
`demonstrated by Doug Engelbart as part of
`NLS in a movie created in 1968 [9]. The
`mouse was then made famous as a
`
`input device by Xerox
`practical
`PARC in the 19705.
`It first
`
`funded by the Advanced Research
`Projects Agency (ARPA).
`It
`employed, among other inter-
`face techniques, iconic repre-
`sentations, gesture recog-
`nition, dynamic menus with
`items selected using a point-
`ing device, selection of icons
`by pointing, and moded
`and mode-free styles of
`interaction.
`
`"11-|E 5|(E1'cHpAD
`
`SYSTEM MAKES IT
`
`appeared commercially as part
`of the Xerox Star (1981), the
`
`POSSIBLE FOR A
`
`MAN AND A coM_
`
`PUTER TO CON-
`
`VERSE RAPIDLY
`
`THROUGH THE
`
`Three Rivers Computer
`Company’s PERQ (1981)
`[25],
`the Apple Lisa
`(1982), and the Apple
`Macintosh (1984).
`
`WINDOWS
`
`Multiple tiled windows
`were
`demonstrated
`in
`
`David Canfield Smith
`
`MEDIUM OF LINE
`
`coined the term “icons” in his
`1975 doctoral thesis on Pyg-
`malion [44] (funded by ARPA
`and National Institute of Mental
`
`DRAw|NG5_-v
`
`-""},',",,“,’,_",',‘2',"“""’
`
`Engelbart’s NLS in 1968
`[9]. Early research at Stan-
`ford on systems like com-
`LOT (1974) [49] and at MIT
`with the EMACS text editor (1974)
`[46] also demonstrated tiled windows.
`
`Alan Kay proposed the idea of overlapping
`windows in his 1969 doctoral thesis [17], and
`
`overlapping windows first appeared in 1974 in
`his Smalltalk system [12] at Xerox PARC, and
`soon afterward in the InterLisp system [50].
`Some of the first commercial uses of win-
`
`dows were on Lisp Machines Inc. (LMI) and
`Symbolics Lisp Machines (1979), which grew
`out of MIT Artificial Intelligence (AI) Lab pro-
`jects. The Cedar Window Manager from Xerox
`PARC was the first major tiled window manag-
`er (1981) [48], followed soon by the Andrew
`window manager [35] by Carnegie Mellon
`University’s
`(CMU)
`Information
`Technology Center (1983, funded
`by IBM). The main commercial
`systems popularizing windows
`were the Xerox Star (1981), the
`
`Apple Lisa (1982), and most
`important,
`the Apple Macin-
`tosh (1984). The early versions of
`the Star and Microsoft Windows
`
`were tiled, but eventually they sup-
`ported overlapping windows like the
`
`
`
`Health — NIMH). Smith later popu-
`larized icons as one of the chief designers
`of the Xerox Star [45]. Many of the interac-
`tion techniques popular in direct manipula-
`tion interfaces, such as how objects and text
`are selected, opened, and manipulated, were
`researched at Xerox PARC in the 1970s. In
`
`particular, the idea of “WYSIWYG” (what
`you see is what you get) originated there with
`systems such as the Bravo text editor and the
`Draw drawing program [1 1]
`The concept of direct manipulation inter-
`faces for everyone was envisioned by Alan Kay
`of Xerox PARC in a 1977 article about the
`
`Dynabook [18]. The first commercial systems
`to use direct manipulation extensively were the
`Xerox Star (1981) [45], the Apple Lisa (1982)
`[54], and the Macintosh (1984)
`[55]. Ben
`
`Shneiderrnan at the University of Maryland
`coined the tenn “direct manipulation” in 1982,
`identified the components, and gave psycholog-
`ical motivations for direct manipulation [43].
`
`MoUsI-:
`
`The mouse was developed at Stanford
`Research Laboratory in 1965 as part of the
`
`interactions...march + april
`
`1998
`
`Page 4 of 11
`
`
`
`Lisa and Macintosh. The X Window System, a
`current international standard, was developed
`at MIT in 1984 [42]. For a survey of window
`managers, see [26].
`
`based display editors that was widely used
`[51]. The Hypertext Editing System [53, p.
`108] from Brown University had screen edit-
`ing and formatting of arbitrary-sized strings
`with a light pen in 1967 (funding from
`IBM). NLS demonstrated mouse-
`
`Applications
`DRAWING PROGRAMS
`
`Much of the current technology
`for drawing programs was
`demonstrated in Sutherland’s
`
`I963 Sketchpad system
`[47]. The use ofa mouse to
`
`manipulate graphics was
`demonstrated in NLS
`
`In 1968 Ken
`(1965).
`and
`Grant
`Pulfer
`Bechthold at the Nation-
`
`al Research Council of
`Canada built a mouse out
`
`of wood patterned after
`Engelbart's mouse and
`used it with a key-frame
`animation system to draw
`all the frames of a movie. A
`
`"THE DIFFICULTY
`
`OF MANKIND'S
`
`PROBLEMS WAS
`
`INCREASING AT A
`
`based editing in 1968 (Figure 4).
`TECO from MIT was an early
`screen editor
`(1967), and
`
`EMACS [46] was developed
`from it
`in 1974. Xerox
`
`PARC’s Bravo [11, p. 284]
`was the first WYSIWYG
`
`GREATER RATE THAN
`
`editor-formatter (1974).
`
`OUR ABILITY TO
`
`COPE.
`
`(WE ARE IN
`
`TROUBLE.) [SO] I
`
`COMMITTED MY
`
`CAREER TO
`
`'AUGMENTING THE
`
`HUMAN INTELLECI'.’"
`
`— DOUG ENGELBART} ON THE
`NLS SYSTEM [8, P. 189]
`
`It was designed by Butler
`Lampson and Charles
`Simonyi, who had started
`working on these con-
`cepts about 1970 while at
`Berkeley. The flrst com-
`mercial WYSIWYG edi-
`
`tors were
`LisaWrite,
`
`the
`and,
`
`Star,
`later,
`
`MacWrite. For surveys of
`text editors, see [24] and [53] .
`
`SPREADSHEETS
`
`The initial spreadsheet was Visi-
`Calc which was developed by
`Frankston and Bricklin (1977-1978) for
`
`the Apple II while they were students at MIT
`and the Harvard Business School, respectively.
`The solver was based on a dependency-direct-
`ed backtracking algorithm by Sussman and
`Stallman at the MIT AI Lab.
`
`I'IYPI-IRTI-IXT
`
`The idea for hypertext (by which documents are
`linked to related documents) is credited to Van-
`nevar Bush’s famous MEMEX idea from 1945
`
`[4]. Ted Nelson coined the term “hypertext” in
`1955 I31]. Engelbart’s NIS system [9] at the
`Stanford Research laboratories in 1965 made
`
`extensive use of linking (firnding from ARPA,
`NASA, and Rome ADC). The “NLS Journal”
`
`[1 1, p. 212] was one of the first on-line journals
`and included full
`of articles (1970). The
`Hypertext Editing System, jointly designed by
`Andy van Dam, Ted Nelson, and two students
`at Brown University (firnding from IBM) was
`distributed artensively [52].
`
`subsequent movie in 1971,
`Hunger, won a number of
`awards and was drawn using a
`tablet instead of the mouse (firnd-
`
`ing by the National Film Board of
`Canada) [3]. William Newmans Markup
`(1975) was the first drawing program for
`Xerox PARC’s Alto,
`followed shortly by
`Patrick Baudelaire’s Draw, which added han-
`
`dling of lines and curves [11, p. 326]. The
`first computer painting program was probably
`Dick
`Shoup's
`Superpaint
`at
`PARC
`(1974-1975). The first widely used drawing
`programs were MacPaint and MacDraw on
`the Macintosh (1984).
`
`TEXT EDITING
`
`In 1962, at the Stanford Research
`
`Laboratory, Engelbart proposed,
`and later implemented, a word
`pror with automatic word
`wrap, search and replace, user-
`definable macros, scrolling text,
`and commands to move, copy,
`and delete characters, words, or
`blocks of text. Stanford's TVEdit
`
`(1965) was one of the first CRT-
`
`
`
`interactions...marclI 4 april
`
`I998
`
`Page 5 of 11
`
`
`
`artlcle —
`
`manufacturing
`aided
`(CAM) was probably
`General Motor’s DAC-
`
`I (about 1963).
`
`VIDEO GAMES
`
`The first graphi-
`cal video game
`was
`probably
`SpaceWar by Slug
`Russel of MIT in 1962 for
`
`the PDP-1 [21, p. 49], including the
`first computer joysticks. The early computer
`Adventure game was
`created by Will
`Crowther at Bolt, Beranek 8c Newman
`
`
`
`The University of Vermont’s
`PROMIS (1976) was the first
`
`hypertext system released to
`the user community.
`It
`was used to link patient
`and patient care infor-
`mation at the Univer-
`
`sity of Vermont’s
`medical
`center. The
`
`from
`(1977)
`ZOG project
`CMU was another early hypertext
`system and was funded by Office of
`Naval Research (ONR) and ARPA [39]. Ben
`
`Shneiderman’s Hyperties was the first system
`in which highlighted items in the text could
`be clicked on to go to other pages (1983, Uni-
`versity of Maryland) [19]. HyperCard from
`Apple (1988) significantly helped to
`bring the idea to a wide audience.
`Tim Berners-Lee used the
`
`(BBN), and Don Woods developed it into a
`more sophisticated Adventure game at Stan-
`ford in 1966 [21, p. 132]. Conway’s
`game of LIFE was developed on
`computers at MIT and Stan-
`ford in 1970. The first popu-
`Iar commercial game was
`Pong (about 1976).
`
`"MuL1'|p|_E
`
`WINDOWS ALLOW A
`DOCUMENT
`(COMPOSED OF TEX-I-I
`
`idea to create the
`hypertext
`World Wide Web in 1990 at
`the
`government-funded
`European Particle Physics
`Laboratory
`(CERN).
`Mosaic, the first popular
`hypertext browser for
`the World-Wide Web,
`
`the
`was developed at
`University of Illinois’
`National Center
`for
`Supercomputer Appli-
`cations (NCSA). For a
`more complete history of
`HyperText, see [34].
`
`COMPUTI-1R—AIDI-ID DESIGN
`The same 1963 International
`Federation of Information Pro-
`
`PICTURES. MUSICAL
`
`NOTATION) TO BE
`
`CREATED AND
`
`VIEWED
`SIMUL-I-ANEOUSLY AT
`
`SEVERAL LEVELS OF
`REFINEMENT."
`_ AM" KAY’
`on 5MM_Lm,_K.72
`"3: "- 341-
`
`Up-and-Coming Areas
`GI-ISTURE RECOGNITION
`The
`first
`pen-based
`input device, the Rand
`tablet, was funded by
`ARPA. Sketchpad used
`light-pen
`gestures
`(1963). Teitelman in
`1964 developed the first
`trainable gesture recog-
`nizer. A very early demon-
`stration
`of
`gesture
`recognition was Tom Elliss
`GRAIL system on the Rand
`tablet (1964, funded by ARPA).
`It was quite common in light-
`pen-based systems to include some
`gesture recognition,
`for example in the
`AMBIT/G system (1968, funded by ARPA). A
`gesture-based text editor using proofreading
`symbols was developed at CMU by Michael
`Coleman in 1969. Bill Buxton at the Universi-
`
`cessing Societies (IFIPS) confer-
`ence
`at which Sketchpad was
`presented also contained a number of com-
`puter-aided design (CAD) systems, includ-
`ing Doug Ross’s Computer-Aided Design
`Project at MIT in the Electronic Systems Lab
`[40] and Coons’s work at MIT with Sketch-
`
`Pad [7]. Timothy ]ohnson’s pioneering work
`on the interactive 3D CAD system Sketch-
`pad 3 [15] was his doctoral thesis in 1963
`(funded by the U.S. Air Force). The first sys-
`tem in industry using CAD and computer-
`
`ty of Toronto has been studying gesture-based
`interactions since 1980. Gesture recognition
`has been used in commercial CAD systems
`since the 19705 and came to universal notice
`
`with the Apple Newton in 1992.
`
`interactions.
`
`..marcII + april
`
`I998
`
`Page 6 of 11
`
`
`
`MULTIMEDIA
`
`The FRESS project at Brown University used
`multiple windows and integrated text and
`graphics (1968, funding from industry). The
`Interactive Graphical Documents project at
`Brown was the first hypermedia (as opposed
`to hypertext) system and used raster graphics
`and text but not video (1979-1983, funded
`
`by NASA, U.S. Navy, and others). Another
`important center of current research on 3-D
`interfaces is Fred Brooks’s lab at University of
`North Carolina at Chapel Hill (e.g., [2]).
`
`VIRTUAL REALITY AND “AUGMENTED REALITY”
`
`THE ‘MOUSE’ WAS
`
`The original work on virtual reality (VR) was
`performed by Ivan Sutherland when he was at
`Harvard (1965-1968, frmding by Air
`Force, Central Intelligence Agency,
`and Bell Laboratories). Very
`important early work was
`done by Tom Fumess when
`he was at \Wri.ght-Patter-
`son
`AFB. Myron
`Kruegers early work at
`the University of Con-
`necticut on interfaces
`
`by ONR and NSF). The Diamond
`project at BBN (starting in 1982,
`funded by ARPA) explored
`combining multimedia infor-
`mation (text, spreadsheets,
`graphics,
`speech). The
`Movie Manual at
`the
`Architecture Machine
`
`DEVELOPED BY SRI.
`
`IT
`
`IS CONSTRUCTED FROM
`
`Group (MIT) was one
`of the first to demon-
`strate mixed video
`
`and computer graph-
`ics iI1 1983 (funded
`
`by ARPA).
`
`THREE-
`DIMENSIONALITY
`
`The first 3D system
`was probably Timothy
`Johnson's 3D CAD sys-
`tem mentioned earlier
`
`(1963, funded by the U.S.
`Air Force)
`[15]. The Lin-
`
`TWO POTENTIOMETERS.
`
`MOUNTED ORTHOGONALLY.
`
`EACH OF WHICH HAS A
`
`WHEEL ATTACHED TO ITS
`
`SHAFT. THE MOUNTING
`
`FRAME
`
`IS ENCLOSED
`
`INAZINCHXZINCHX4
`
`INCH WOODEN CASE."
`
`— lIlI.K. ENGLISH, D. C. ENGELBAR1; AND
`M.L. HERMAN, on THE ms SYSTEM
`(10. P. 7]
`
`incorporate and
`that
`interpret a video of
`the user was influen-
`tial. Fred Brooks’s and
`
`Henry Fuch’s groups
`at UNC did a lot of
`
`early research, includ-
`ing the study of force
`feedback (1971,
`fi1nd-
`
`ing from U.S. Atomic
`Energy Commission and
`NSF). Much of the early
`research on head-mounted
`
`coln Wand by Larry Roberts
`was an ultrasonic 3D location
`
`displays and on the DataGlove
`was supported by NASA.
`
`sensing system developed at Lincoln
`Labs (1966, funded by ARPA). That system
`also had the first interactive 3D hidden line
`
`COMPUTER-SUPPORTED
`COOPERATIVE WORK
`
`Doug Engelbart’s 1968 demonstration of
`NLS [9] included the remote participation of
`multiple people at various sites (funding from
`ARPA, NASA, and Rome ADC). Licklider
`
`and Taylor predicted online interactive com-
`munities in a 1968 article [22] and speculat-
`ed about the problem of access being limited
`to the privileged. Electronic mail, still the
`most widespread multiuser software, was
`enabled by the ARPAnet, which became
`operational
`in 1969, and by the Ethernet
`from Xerox PARC in 1973. An early comput-
`er conferencing system was TurolT’s EIES sys-
`tem at the New Jersey Institute ofTechnology
`(1975).
`
`elimination. An early use was for molecular
`modeling [20]. The late 1960s and early
`1970s saw the flowering of 3D raster graph-
`ics research at the University of Utah with
`Dave Evans,
`Ivan Sutherland, G.W.
`
`Romney, Henri Gouraud, Bui-Tuong
`Phong, and G.S. Watkins, much
`of it government funded. Also,
`the military-industrial flight
`simulation work of the 1960s
`
`
`
`and 1970s led the way to mak-
`ing 3D interfaces operate in real
`time on commercial systems
`from General Electric, Evans 8:
`
`Sutherland, Singer/Link (funded
`
`
`
`interactions...marclI 4 april
`
`I998
`
`Page 7 of 11
`
`
`
`artlcle —
`
`NATURAL LANGUAGE AND SPEECH
`
`The fundamental research for speech and nat-
`ural language understanding and generation
`has been performed at CMU, MIT, SRI,
`BBN, IBM, AT&T Bell Labs and BellCore,
`
`much of it government funded. See, for exam-
`ple, [37] for a survey of the early work.
`
`Software Tools and Architectures
`
`The area of user interface (UI) software tools
`
`is quite active now, and many companies are
`selling tools. Most of today’s applications are
`implemented using various forms of software
`tools. For a more complete survey and discus-
`sion of U1 tools, see [28].
`
`USER INTERFACE MANAGEMENT SYSTEMS
`AND TooLK1Ts
`
`User interface management systems (UIMS)
`and toolkits are software libraries and tools
`
`allow interfaces composed of widgets such as
`buttons, menus, and scrollbars to be placed
`using a mouse. The Steamer project at BBN
`(1979-1985; funded by ONR) demonstrated
`many of the ideas later incorporated into
`interface builders and was probably the first
`object-oriented graphics system. Trillium [13]
`was developed at Xerox PARC in 1981.
`Another early interface builder was the
`MenuLay system [5] developed by Bill Buxton
`at the University of Toronto (1983, funded by
`the Canadian Government). The Macintosh
`(1984)
`included a Resource Editor that
`
`allowed widgets to be placed and edited. Jean-
`Marie Hullot created SOS Interface in Lisp
`for the Macintosh while working at The
`French National
`Institute for Research in
`
`(INRIA)
`Computer Science and Control
`(1984, funded by the French government)
`which was the first modern interface builder.
`
`that help to create user interfaces. The first
`UIMS was ‘William Newman’s Reaction Han-
`
`Hullot built this into a commercial product in
`1986 and then went to work for NeXT and
`
`created the NeXT Interface Builder (1988),
`
`which popularized this type of tool. Now
`there are literally hundreds of commercial
`interface builders.
`
`COMPONENT Ancurrr-:crUREs
`
`The idea of creating interfaces by connecting
`separately written components was first
`demonstrated in the Andrew project [35] by
`Carnegie Mellon University's Information
`Technology Center (1983, funded by IBM). It
`is now being widely popularized by Microsofts
`object linking and embedding (OLE), Apple
`and IBM's OpenDoc, and the java Beans
`architectures.
`
`dler [33] created at Imperial College, London
`(1966-1967 with SRC fimding). Most of the
`early work was done at universities (University
`of Toronto with Canadian government fund-
`ing; George Washington University with
`NASA, NSF, Department of Energy, and
`National Bureau of Standards (NBS) funding;
`Brigham Young University with industrial
`funding; and so on). The term “UIMS” was
`coined by David Kasik at Boeing (1982) [16].
`Early window managers such as Smalltalk
`(1974) and InterLisp, both from Xerox PARC,
`came with a few widgets, such as popup menus
`and scrollbars. The Xerox Star (1981) was the
`
`first commercial system to have a large collec-
`tion of widgets. The Apple Macintosh (1984)
`was the first to actively promote its toolkit for
`use by other developers to enforce a consistent
`interface. An early C++ toolkit was InterViews
`[23], developed at Stanford (1988, industrial
`funding). Much of the modern research is
`being performed at universities, for example,
`the Garnet (1988) [29] and Amulet (1994)
`
`[30] projects at CMU (funded by DARPA)
`and SubArctic at Georgia Tech (1996, firnded
`by Intel and NSF) [14].
`
`INTERFACE BUILDERS
`
`Interface builders are interactive tools that
`
`interactions.
`
`..march + april
`
`1998
`
`Page 8 of 11
`
`Discussion
`
`It is clear that all of the most impor-
`tant
`innovations in HCI have
`benefited from research at
`
`both corporate research labs
`and universities. One moti-
`vation for this article is to
`
`overcome the impressions
`some people may have that
`much of the important work
`in human—computer interac-
`tion occurred in industry and
`that
`if university research in
`
`
`
`
`
`HCI is not supported, then industry will just
`carry on anyway. This is simply not true. I
`hope that this paper has shown that many of
`the most famous HCI successes developed by
`companies are deeply rooted in univer-
`sity research. In fact, virtually all of
`today’s major interface styles
`and applications have been
`significantly influenced by
`research at universities
`
`industry needs. It seems that only through
`computer science does HCI research dissemi-
`nate to products. Furthermore, without
`appropriate levels of funding of academic
`HCI research, there will be fewer doc-
`
`toral graduates in HCI to perform
`research in corporate labs, and
`fewer top-notch graduates in
`this area will be interested
`
`"IN EMBEDDED
`
`future will use gesture recognition,
`speech recognition, “intelligent agents,”
`adaptive interfaces, video, and many other
`technologies now being investigated by
`research groups at universities and corporate
`labs [38]. Therefore, it is imperative that uni-
`versity, corporate, and government-supported
`research continue and be well-supported, so
`that we can develop the science and technolo-
`gy needed for the user interfaces of the fixture.
`
`Acknowledgments
`
`I must thank a large number of people who
`responded to posts of earlier versions of this
`article on the announcements.chi mailing list
`for their very generous help, and to Jim Hollan
`who helped write the short excerpt of this arti-
`cle. Some of the information in this article was
`
`supplied by (in alphabetical order): Stacey Ash-
`lund, Meera M. Blattner, Keith Butler, Stuart
`K. Card, Bill Curtis, David E. Damouth, Dan
`
`use and learn than they are
`today. As described by Stu
`Card [6, p. 162]:
`of
`funding
`Government
`advanced human-computer interac-
`tion technologies built the intellectual capi-
`tal and trained the research teams for pioneer
`systems that, over a period of25 years, revolu-
`tionized how people interact‘ with computers.
`Industrial research laboratories at the corporate
`level in Xerox, IBM, ATé'7f and others played a
`strong role in developing this technology and
`bringing it into a form suitablefor the commer-
`cial arena.
`
`
`
`Another important argument in favor
`of HCI research at universities is that
`
`computer science students need to
`know about user interface issues.
`
`User interfaces are likely to be
`one of the main value-added
`
`competitive advantages of the
`future as both hardware and basic
`software become commodities. If
`
`students do not know about user
`
`interfaces,
`
`they will not serve
`
`interactions. ..march 4 april
`
`I998
`
`Page 9 of 11
`
`PEIMNTOMAKEDIGITAI./HARD
`oorvoi=|mKro|uu.|.os11mwoILx
`Kn rursomtoauasnoouusns
`cnnnranvrmovrrrzmovinao
`
`nurconasuzuarwinsou
`DISl'RlIUI'FJ)FOIPl0HT0lG)M
`
`MElOAI.ADVAN'|'AGE.‘lHECDPYllCIl'l'
`No'n<:I=..1Hr.11'n.:or1n£ruu.xyi
`nosumnsmnnwummn
`NOTICE iscivlannurroorvmcnsn
`
`PEIMNOFACM. Inc. Tocorv
`ani£Iw|sr..Tou-:ruuLIsH.1orosr
`
`ousalVEI5.orrou:msrnInurE'ro
`
`iisrsnnqumrsmlonsrazrlcranms
`sionlmolonlirua.
`
`0 A34 I072 521019310300 $3.50
`
`and labs, often with gov-
`ernment funding. To
`illustrate this point,
`this article has listed
`
`the funding sources
`of some of the major
`advances. Without
`
`this research, many
`of the advances in
`the field of HCI
`
`would probably not
`have taken place, and
`as a consequence, the
`user interfaces of com-
`
`mercial products would
`be far more difficult to
`
`MENUS. HIGHLIGHTED
`
`OR UNDERLINED
`
`WORDS OR PHASES
`
`WITHIN THE TEXT
`
`BECOME THE MENU
`
`ITEMS. AND ARE
`
`SELECTABLE USING THE
`
`COMMONLY USED
`
`TOUCH SCREEN.
`
`CURSOR AND MOUSE
`
`METHODS."
`
`— L. KOVED AND
`B. SHNEIDERMAN ON THE
`HYPERTIES SYSTEM [19, P. 312].
`
`in being professors, so
`the needed user inter-
`face courses will not be
`
`oflered.
`The conventional
`
`style of graphical
`user interfaces that
`
`use windows, icons,
`menus, and a mouse
`
`are in a phase of
`standardization,
`
`where almost every-
`one
`is using the
`same, standard tech-
`
`nology and making
`incremental
`small,
`
`changes. As computers
`perform faster, more of
`the processing power
`is
`being devoted to the user
`interface. The interfaces of the
`
`
`
`artlcle —
`
`Rob
`
`Rich McDaniel,
`
`Miller, Bernita Myers,
`Yoshihiro Tsujino, and
`the reviewers.
`A short
`from
`this
`
`excerpt
`article
`
`appeared as part of
`“Strategic Direc-
`tions in Human
`
`Computer Inter-
`action,” B. Myers, J. Hollan,
`I. Cruz (eds.). ACM Computing Sur-
`veys 28, 4 (Dec. 1996). pp. 794-809.
`This research was sponsored partly by
`NCCOSC under Contract No. N6600l-94-
`
`C-6037, ARPA Order No. B326, and partly
`by the National Science Foundation under
`Grant No. IRI-9319969. The views and con-
`clusions contained in this document are those
`
`of the author and should not be interpreted as
`representing the official policies, either
`expressed or implied, of NCCOSC or the
`U.S. Government.
`
`
`
`Diaper, Dick Duda, Tim T.K.
`Dudley, Steven Feiner, Harry
`Forsdidc, Bjorn Freeman-
`Benson, John Gould,
`Wayne Gray, Mark
`Green, Fred Hansen,
`
`Hefley, D.
`Bill
`Austin Henderson,
`
`Jean-
`Jim Hollan,
`Marie Hullot, Rob Jacob,
`Bonnie
`John,
`Sandy
`Kobayashi, T.K. Iandauer, John
`Leggett, Roger Lighty,
`Mantei, Jim
`Miller, William Newman, Jakob Nielsen, Don
`
`Norman, Dan Olsen, Ramesh Patil, Gary Perl-
`man, Didt Pew, Ken Pier, Jim Rhyne, Ben
`Shneiderrnan, John Sibert, David C. Smith,
`Elliot Soloway, Richard Stallman, Ivan Suther-
`land, Dan Swinehart, John Thomas, Alex
`Waibel, Marceli Wein, Mark Weiser, Alan
`
`Wexelblat, and Terry Winograd. Editorial
`comments were also provided by the forego-
`ing as well as by Jim Hollan, Ellen Borison,
`
`References
`1. Baecker, R., et al. “A Historical and Intel-
`
`lectual Perspective.” In Reading: in
`Hu7nan—Computer Interaction: Toward fire Year
`2000. (R. Baecker et al., eds.). 2nd ed. Mor-
`
`gan Kaufrnann Publishers, Inc., San Francisco,
`1995. pp. 35-47.
`2. Brooks, F. "The computer ‘scientist’ as tool-
`smith—Studies in interactive computer graph-
`
`ics.” In Proceedings ofIFIP Confirrence. 1977.
`pp. 625-634.
`3. Burmyk, N. and Wein, M. "Computer
`
`tex1(lVl. Rudisill et al., eds.). Morgan Kauf-
`mann Publishers, San Francisco, 1996, pp.
`122-169.
`
`7. Coons, S. "An Outline of the Requirements
`for a Computer—Aided Design System.” In
`
`Proaeevlings ofAFIPS Springfoint Cornpum
`Cmfi-mm 23 (1963). pp. 299-304.
`8. Engelbart, D. “Tlre Augmented Knowledge
`
`Workshop.” In A History ofPersonal lWrrb1a-
`tion: (A. Goldberg, ed.). Addison-Wesley Pub-
`lishing Company, New York, 1988, pp.
`187-232.
`
`generated key frame animation.” Iournal ofthe
`Society ofMotion Picture and Television Engi-
`neers 8. 3 (1971), pp. 149-153.
`4. Bush, V. “Mwe maythink." Tbelitlantic
`
`9. Engelbart, D. and English, W. “A Research
`Center for
`Human Intellect."
`Reprinted in ACM SIGGRAPH Video Review
`I06(1994), video made in 1968
`
`Mondliy 176 (July 1945), pp. 101-108.
`Reprintedanddiscussedininura¢1ion.r.3,2
`(Mar. 1996). pp. 35-67.
`5. Buxtson, W. et al. "Towards a comprehen-
`sive user interface management system." In
`
`Computer Graphics 17 (1983). Pp. 3542. Pro-
`ceedings of SIGGRAPIT83, Detroit, MI.
`6. Card, S.K. "Pioneers and Settlers: Methods
`
`Used in Successful User Interface Design." In
`
`Hurnan—Cornpurer Intejizte Design: Success Sm-
`rier, Emerging Melirodr, and Real-lVorId Con-
`
`10. English, W.K., Engelbart, D.C., and
`Berman, M.L “Display selection techniques
`for tat manipulation.” IEEE Transaction: on
`Human Fm-ms in Electronics HFE-8, 1 (1967),
`
`pp. S—l5.
`
`11. Goldberg, A., ed. A Hirtory ofPersonaI
`Wmfirtations Addison—Wesley Publishing
`Company, New York, 1988.
`12. Goldberg, A. and Robson, D. ‘A
`Metaphor for User Interface Design." In Pm-
`
`firenee on Syrtern Science: I (1979), pp.
`148-157.
`
`13. Henderson Jr., D.A. “The Trillium user
`
`interface design environment.” In Proceatirrgr
`o_fSIGCIII '86 Human Factors in Computing
`.30-terns, Boston, MA, 1986, pp. 221-227.
`14. Hudson, S.E. and Smith, 1. “Ultra-
`
`Iightweight Constraints." In Proceedings of
`ACM SIGGRAPH .S_'yrnpo.n'ron on User Inter-
`face Sofiruare and Technology (U15T '98, Seat-
`tle, WA, 1996, pp. 147-155. System available
`from
`
`http:IIwww.cc.gatech.eduIgvuIuiIsub_aretid
`15. Johnson, T. “Sketchpad III: Three
`Dimensional Graphical Communication with
`
`Computer." In Proceeding: ofAFIPS
`a
`Springjoint Computer C'on_ferem:e 23 (1963),
`pp. 347-353.
`16. Kasik, D.J. "A user interface management
`
`system.” In Computer Graplnls I6 (1982),
`Boston, MA, pp. 99-106. Proceedings SIG-
`GRAPI-1’82.
`
`17. Kay, A. The Reactive Engine. Doctoral
`dissertation, Electrical Engineering and (hm-
`puter Science, University of Utah, 1969.
`18. Kay, A. “Personal dynamic media." IEEE
`
`teedingr ofthe 121!) I-Iawaii International Con-
`
`Computer 10, 3 (1977), pp. 31-42.
`
`interactions.
`
`..march + april
`
`1998
`
`Page l0 of 11
`
`
`
`19. Koved, L. and Shneiderman, B. "Embed-
`
`ded menus:
`
`items in context.” Com-
`
`munications afdre/1CM4, 29 (1986), pp.
`3 12-3 18.
`
`20. Levinthal, C. "Molecular model-building
`by computer." Scientific American 214, 6
`(1966), pp. 42-52.
`
`21. Levy, S. Hackers: Heme: ofdx Computer
`Revolution. Anchor PressIDoubleday, Garden
`City, NY, 1984.
`22. Liclrlider, ].C.R. and Taylor, R.W. "The
`computer as communimtion device." Sci. Tech
`(April 1968). PP- 21-31.
`23. Linton, M.A., Vlisides, ].M., and Calder,
`
`P.R. “Composing user interfaces with Inter-
`
`Views.” IEEE Computer 22, 2 (1989), pp.
`8-22.
`
`N. and Van Dam, A. "Interac-
`24.
`tive editing systems: Parts 1 and 2." ACM
`
`Computing Suwey I4. 3 (1982), pp.
`321-352.
`
`interfaces.” IEEE Computer 23. 11 (1990). pp.
`71-85.
`
`30. Myers, B.A. et al. ‘Easily Adding Anirna—
`tions to Interfaces Using (hnstraintsf In Pm-
`
`teadingr qf'ACM SIGGRAPH Symposium on
`User Interfau Sofiwure and Technology (UIST
`'96), Seattle, WA, 1996, pp. 119-128.
`31. Nelson, T. "A File Structure for the Com-
`
`plex, the
`
`and the Indeterminate.”
`
`In Proceedings o_fACM National Conference,
`1965, pp. 84-100.
`32. Newman, WM. “A graphical technique
`
`for numerial input." The Computerjoumal
`II. 1 (1968), pp. 63-64.
`33. Newman, WM. “A System for Interactive
`
`Graphical Programming.” In Proceedings of
`AFIPS Springfoint Compum Conference 28
`(1963): PP- 47-54.
`
`34. Nielsen, ]., Multimedia and the
`Internetund Bqond Academic Press Profes-
`sional, Boston, 1995.
`
`25. Myers, B.A. ‘The user interface for Sap-
`
`35. Palay, A.]. et al. ‘The Andrew Toolkit, An
`
`phire.” IHE Computer
`tions 4. 12 (1984), pp. 13-23.
`26. Myers, B.A. “A taxonomy of user inter-
`
`undApplim-
`
`faces for window managers." IEEE Computer
`Graphics andApplication: 8. 5 (1988): PP.
`65-84.
`
`27. Myers, BA "All the Widgets.” SIG-
`GRAPH Video Review 57 (1990). Available at
`