t
`. L
`I,
`'r>.
`' --' ~~
`
`WOODROW BARFIELD
`11' HO MA S C A LIDE L L
`/)
`
`META 1017
`META V. THALES
`
`

`

`The book cover depicts a mid-1980s embodiment of Dr. Steve Mann's
`"\Xlearable Computer" (WearComp) invention he created for his early
`experiments in computer mediated reality. The apparatus was used to
`generate a series of cybernetic photographs 1 using a collaborative process
`called "dusting", exhibii:ed in his solo show at Night Gallery (185 Rich(cid:173)
`mond St., Toronto) during the summer cf 1985. Cybernetic photography
`was an early application of collaborative n1ediated reality spaces, as de~
`scribed in Chapter 19 of this book.
`
`Copyright @ 200 l by L2\vrence Erlbaum r'\ssociatcs, Inc
`All right reserved. No part uf this book may be reproduced
`inany form, by phutostat, microfilm, retrieval systcn1, or
`anyothcr means: vvithout prLor vvritten permission of the
`publisher.
`
`Lmvrence Erlbaurn .Associates, Inc 1 Publisher~
`10 Industrial Avenue
`Mahwah, NJ 07430
`
`Cover design by Ka:::hryn Houghtaling~ Lacey
`
`Library of Congress Cata!ogi.ng-in-PubHcation Data
`
`Fundarnentab nf wectrablc comoutcrs and augumcntcd
`reality/
`,_
`~
`editors, \X/oodn.w,; Barfield, Thomas Caudell.
`
`bibliogrnphical references and index.
`ISBN 0-8058-2901-6 (clLJth: alk. pciper)
`ISBN 0-8058-2902-4
`: alk. paper)
`l. \Vearable computers.
`Ht:man-corr1..putcr Lnticracciun.
`!. Barfield. Woodrow.
`II. Caudell, Thomas.
`QA.76.592 .F86 2000
`004.16-dc2l
`
`00-03931 l
`
`by Lav\:rence Eribaurn Associates are printed
`paper, and their bindings ,uc chosen for strength
`ori
`and durability.
`
`Printed tn the United Stares of A.merica
`lO 9 8 7 6 5 4 3 2 1
`
`META 1017
`META V. THALES
`
`

`

`This book is dedicated to Jessica YiQi onc1
`to my parents A1an.J and Woodrmu Ba1jielc!.
`-Woodrow Barfield Jr.
`
`I dedicate this Luork with all my loue to my
`brown-eyed girl. Ann.
`
`- Thomas Caudell
`
`META 1017
`META V. THALES
`
`

`

`Contents
`
`Preface
`
`I
`INTRODUCTION
`1 Basic Concepts in Wearable Computers and Augmented Reality
`Woodrow Barfield and Thomas Caudell
`2 Augmented Reality: Approaches and Technical Challenges
`Ronald T. Azuma
`
`II
`TECHNOLOGY
`3 A Survey of Tracking Technologies for Virtual Environments
`Jannick P. Rolland, Larry D. Davis, and Yohan Baillot
`4 Optical versus Video See-Through Head-Mounted Displays
`Jannick P. Rolland and Henry Fuchs
`
`5 Augmenting Reality Using Affine Object Representations
`James Vallino and Kiriakos N. Kutulakos
`6 Registration Error Analysis for Augmented Reality Systems
`Richard L. Holloway
`7 Mathematical Theory for Mediated Reality and WearCam-Based
`Augmented Reality
`Steve Mann
`
`III AUGMENTED REALITY
`8 Studies of the Localization of Virtual Objects in the Near
`Visual Field
`Stephen R. Ellis and Brian M. Menges
`
`9 Fundamental Issues in Mediated Reality, WearComp, and
`Camera-Based Augmented Reality
`Steve Mann
`
`xi
`
`3
`
`27
`
`67
`
`113
`
`157
`
`183
`
`219
`
`263
`
`295
`
`vii
`
`META 1017
`META V. THALES
`
`

`

`viii
`
`CONTENTS
`
`10 STAR: Tracking for Object-Centric Augmented Reality
`Ulrich Neumann
`11 NaviCam: A Palmtop Device Approach to Augmented Reality
`Jun Rekimoto
`12 Augmented Reality for Exterior Construction Applications
`Gudrun Klinker; Didier Stricker; and Dirk Reiners
`13 GPS-Based Navigation Systems for the Visually Impaired
`Jack M. Loomis, Reginald G. Golledge, and
`Roberta L. Klatzky
`14 Boeing's Wire Bundle Assembly Project
`David Mizell
`
`IV WEARABLE COMPUTERS
`15 Computational Clothing and Accessories
`Woodrow Barfield, Steve Mann, Kevin Baird, Francine
`Gemperle, Chris Kasabach, John Stivoric, Malcolm
`Bauer; Richard Martin, and Gilsoo Cho
`16 Situation Aware Computing with Wearable Computers
`Bernt Schiele, Thad Starner; Brad Rhodes, Brian Clarkson, and
`Alex Pentland
`17 Collaboration with Wearable Computers
`Mark Billinghurst, Edward Miller; and Suzanne Weghorst
`18 Tactual Displays for Sensory Substitution and
`Wearable Computers
`Hong Z. Tan and Alex Pentland
`19 From "Painting with Lightvectors" to "Painting with Looks":
`PhotographicNideographic Applications of WearComp-Based
`Augmented/ Mediated Reality
`Steve Mann
`20 Military Applications of Wearable Computers and
`Augmented Reality
`C. C. Tappert, A. S. Ruocco, K. A. Langdorf, F. J. Mabry,
`K. J. Heineman, T A. Brick, D. M. Cross, S. V. Pellissier;
`and R. C. Kaste
`21 Medical Applications for Wearable Computing
`Richard M. Satava, and Shaun B. Jones
`
`329
`
`353
`
`379
`
`429
`
`447
`
`471
`
`511
`
`539
`
`579
`
`599
`
`625
`
`649
`
`META 1017
`META V. THALES
`
`

`

`CONTENTS
`
`22 Constructing Wearable Computers for Maintenance Applications
`Ler.. Bass, Dan Siewiorek, Malcolm Bauer, Randy Casciola,
`Chris Kasabach, Richard Martin, Jane Siegel, Asim Smailagic,
`and John Stivoric
`23 Ap?lications of Wearable Computers and Augmented Reality
`to Manufacturing
`WODdrow Barfield, Kevin Baird, John Skewchuk, and
`George loannou
`24 Computer Networks for Wearable Computing
`Rick LaRowe and Chip Elliott
`25 Computing under the Skin
`Dwight Holland, Dawn J. Roberson, and Woodrow Barfield
`
`Index
`
`ix
`
`663
`
`695
`
`715
`
`747
`
`795
`
`META 1017
`META V. THALES
`
`

`

`Preface
`
`The power of the machine imposes itself upon us and we can scarcely conceive of
`living bodies anymore without it; we are strangely moved by the rapid friction of
`beings and things and we accustom ourselves, without knowing it, to perceive the
`forces of the former in terms of the forces dominating the latter.
`
`-circa 1913, Raymond Duchamps-Villon 1876-1918
`
`The concept of blending humans with machines has been in the dreams and
`nightmares of people since long before the industrial revolution, finding its
`way into many fables and stories over the centuries. In recent times, this
`topic weaves in and out of the entire science fiction and fantasy genres,
`culminating in the quintessential human/machine merger, the Borg, in the
`Star Trek science fiction series. As futuristic as this seems, the blending
`of humans with machines is now becoming fact through advances in com(cid:173)
`puter, communications, and human-computer interface technologies. This
`book introduces the reader to the basic concepts, challenges, and the un(cid:173)
`derlying technologies that are making this happen, and through discussion
`of applications, answers the question "Why do this?"
`
`xi
`
`META 1017
`META V. THALES
`
`

`

`xii
`
`PREFACE
`
`This book presents a broad coverage of the technologies and interface
`design issues associated with wearable computers and augmented reality
`displays, both rapidly developing fields in computer science, engineering,
`and human interface design. As general descriptions, wearable computers
`are fully functional, self-powered, self-contained computers that are worn
`on the body to provide access to and interaction with information anywhere
`and at anytime. Closely related is the topic of "augmented reality," an ad(cid:173)
`vanced human-computer interface technology that attempts to blend or fuse
`computer-generated information with our sensations of the natural world.
`Because of the close association between wearable computers and aug(cid:173)
`mented reality, we refer to both types of computing technology generically
`as wearware. Throughout this book the various chapter authors describe
`the integration of head-mounted displays, digital technology, auditory dis(cid:173)
`plays, and body tracking technologies to create wearable computer and aug(cid:173)
`mented reality systems. This technology has the potential of improving the
`efficiency and quality of human labors, particularly in their performance of
`engineering, manufacturing, construction, diagnostic, maintenance, moni(cid:173)
`toring, health care delivery, and transactional activities.
`One of the main applications for the technologies described in this book
`is in the fields of medicine and health care. Using immersive head-mounted
`displays, physicians are able to examine and interact with virtual represen(cid:173)
`tations of patients for the purposes of enhanced training for general medical
`education, and to enhance specialized surgical, anesthesia, or other crisis
`and procedural training skills. In addition, augmented reality technologies
`are enabling researchers and physicians to project medical information di(cid:173)
`rectly on or into the patient as a supporting informational aid to enhance
`case-specific decision making. One clear example is the ability to visualize
`a tumor's location relative to the patient's surrounding anatomy for better
`surgical outcomes. In the area of mobile computing, wearable computer
`technology is just beginning to be used in medical settings, allowing physi(cid:173)
`cians the capability to access medical information whenever and wherever
`they are. On the patient side of wearable computer technology, nascent
`wearable computers are already being utilized by some of the diabetic pop(cid:173)
`ulation in the form of Insulin Pump Therapy, with other wearable computer
`technologies on the immediate horizon to treat a host of other conditions.
`Much of the current research directions on the topics of wearable com(cid:173)
`puters and augmented reality can be traced to Ivan Sutherland's seminal
`dissertation work at MIT, and to more recent ideas associated with the topic
`of ubiquitous computing. Sutherland, often thought to be the "father of vir(cid:173)
`tual reality," developed a see-through display in which computer-generated
`
`META 1017
`META V. THALES
`
`

`

`PREFACE
`
`Xiii
`
`graphics were superimposed over the real world. This aspect of interface
`design, superimposing graphics or text over the real world, is an important
`feature of wearable computer and augmented reality systems. The ability
`to project or merge information on objects in the real world has led to what
`Steve Feiner of Columbia University terms "knowledge enhancement" of
`the world, that is, projecting information of interest (i.e., knowledge) on
`objects in the world. Extensions of this idea have led to the development
`of "smart spaces, rooms or environment."
`The ideas associated with ubiquitous computing have also contributed
`to developments in wearable computers and augmented reality. What better
`way to access computing resources anywhere and at any time than to be
`wearing them on your body? In this regard, advances in microelectronics
`and wireless networking are making truly ubiquitous computing a reality.
`However, before the general public will be seen wearing computers, com(cid:173)
`ponents of the wearable computer ( the CPU housing unit, input and output
`devices) will have to look far more like clothing or clothing accessories
`than the commercial wearable computer systems available now. Thus, a new
`field is rapidly emerging, that of computational clothing (a chapter in this
`book). Furthermore, based on developments in microelectronics, sensor
`technology, and medicine, there is an emerging trend to apply computing
`resources under the surface of the skin and in some cases to integrate digi(cid:173)
`tal technology with the user's physiological systems. Such capabilities will
`allow computing technology to monitor and control various physiological
`processes, or to act as a sensory and motor prosthesis.
`Since this topic has important implications for human use of technology
`and the potential for further integration of human physiological systems
`with digital technology, we close the book with a chapter on computing
`under the skin, briefly discussing some of the ideas associated with the
`concept of cyberevolution.
`This book is a collection of twenty-five chapters that each address an
`important aspect of wearable computers and augmented reality, either from
`the conceptual or from an application framework. Given the wide coverage
`of topics on issues related to the display of computer-generated images
`in the environment, this book can be used as a text for computer science,
`computer engineering, and interface design courses. The book is orga(cid:173)
`nized around four topic areas. The first main topic covered in the book
`contains introductory material and consists of two chapters, "Basic Con(cid:173)
`cepts in Wearable Computers and Augmented Reality" and "Augmented
`Reality: Approaches and Technical Challenges." The next part of the book,
`containing five chapters, focuses on the technologies associated with the
`
`META 1017
`META V. THALES
`
`

`

`xiv
`
`PREFACE
`
`design of augmented reality and wearable computer displays. One of the
`issues discussed within several of the chapters presented here is the issue
`of image registration. In many applications it is necessary to accurately
`register computer-generated images with objects or locations within the
`real environment. This important topic receives broad coverage from the
`chapters in Part II of the book: "A Survey of Tracking Technologies for Vir(cid:173)
`tual Environments," "Optical versus Video See-Through Head-Mounted
`Displays," "Augmenting Reality Using Affine Object Representations,"
`"Registration Error Analysis for Augmented Reality Systems," and "Math(cid:173)
`ematical Theory for Mediated Reality and WearCam-Based Augmented
`Reality."
`The third part of the book is on the topic of "augmented reality," which
`include chapters on the technology as well as on applications. The chap(cid:173)
`ters in this section include: "Studies of the Localization of Virtual Ob(cid:173)
`jects in the Near Visual Field," "Fundamental Issues in Mediated Reality,
`WearComp, and Camera-Based Augmented Reality," "STAR: Tracking
`for Object-Centric Augmented Reality," "NaviCam: A Palmtop Device
`Approach to Augmented Reality," "Augmented Reality for Exterior Con(cid:173)
`struction Applications," "GPS-Based Navigation System for the Visually
`Impaired," and "Boeing's Wire Bundle Assembly Project."
`Finally, the last part in the book is on "wearable computers," again with
`the technology discussed and with numerous applications provided. The
`chapters in Part IV include: "Computational Clothing and Accessories,"
`"Situation Aware Computing with Wearable Computers," "Collaboration
`with Wearable Computers," "Tactual Displays for Sensory Substitution
`and Wearable Computers," "From 'Painting with Lightvectors' to 'Paint(cid:173)
`ing with Looks': PhotographicNideographic Applications of WearComp(cid:173)
`Based Augmented/Mediated Reality," "Military Applications of Wearable
`Computers and Augmented Reality," "Medical Applications for Wearable
`Computing," "Constructing Wearable Computers for Maintenance Appli(cid:173)
`cations," "Applications of Wearable Computers and Augmented Reality
`to Manufacturing," "Computer Networks for Wearable Computers," and
`"Computing Under the Skin."
`In summary, this book presents concepts related to the use and underly(cid:173)
`ing technologies of augmented reality and wearable computer systems. As
`shown in this book, there are many application areas for this technology
`such as medicine, manufacturing, training, clothing, and recreation. Wear(cid:173)
`able computers will allow a much closer association of information with the
`user than is possible with traditional desktop computers. Future extensions
`of wearable computers will contain sensors that will allow the wearable
`
`META 1017
`META V. THALES
`
`

`

`PREFACE
`
`xv
`
`device to see what the user sees, hear what the user hears, sense the user's
`physical state, and analyze what the user is typing. Combining sensors with
`an intelligent agent will result in a system that will be able to analyze what
`the user is doing and thus predict the resources he or she will need next
`or in the near future. We expect to see significant advances in wearware,
`providing humans tools that we have dreamed about having for centuries.
`We hope this book helps to stimulate further advances in this field.
`We would like to thank those individuals that have either directly con(cid:173)
`tributed to the book or have provided motivation for the project. At
`Lawrence Erlbaum Associates, Ray O'Connell and Lane Akers were sup(cid:173)
`portive in the initial phases of discussions associated with the theme of
`the book and Anne Duffy from Lawrence Erlbaum Associates was instru(cid:173)
`mental in serving as Senior Editor for the project. Corin Huff, also from
`Lawrence Erlbaum Associates, provided help in obtaining needed material
`from chapter authors. Most importantly, we would like to thank the chapter
`authors for providing interesting and stimulating material.
`
`Woodrow Barfield
`Thomas Caudell
`
`META 1017
`META V. THALES
`
`

`

`1
`
`Basic Concepts
`in Wearable Computers
`and Augmented Reality
`
`Woodrow Barfield
`Virginia Tech
`
`Thomas Caudell
`Uniuersity of New Mexico
`
`l .
`
`INTRODUCTION
`
`This chapter introduces the joint fields of wearable computers and aug(cid:173)
`mented reality, both of which represent the theme of this book-a dis(cid:173)
`cussion of information technologies that allow user's to access information
`anywhere and at any time. In many ways, the design of wearable computers
`and augmented reality systems has been motivated by two primary goals.
`The first is driven by the need for people to access information, especially as
`they move around the environment; the second is motivated by the need for
`people to better manage information. Until just recently, if a user needed to
`access computational resources, the user had to go to where the computer
`resources were located, typically a desktop PC or a mainframe computer.
`Once the user left the terminal, the flow of information stopped. Now net(cid:173)
`worked wearable computers along with other digital devices allow the user
`to access information at any time, and at any location. However, the ability
`to access large amounts of information may not always be beneficial-too
`much information presented too fast may result in information overload.
`For this reason, the issue of information management is also important. In
`
`3
`
`META 1017
`META V. THALES
`
`

`

`4
`
`BARFIELD AND CAUDELL
`
`this regard, wearable computers and augmented reality systems along with
`software acting as an intelligent agent can act as a filter between the user
`and the information. Intelligent agents will allow only the relevant infor(cid:173)
`mation for a given situation to be projected on a head-mounted display, a
`hand-held computer, or an auditory display.
`Wearable computer systems will likely be a component of other ad(cid:173)
`vanced information technology initiatives as well. For example, in the area
`of "smart spaces," by embedding sensors and microprocessors into every(cid:173)
`day things, wearable computer and augmented reality systems will be able
`to respond to and communicate with objects in the environment (Pentland,
`1996). In addition, more and more, wearable computing medical devices
`will be implanted under the skin to regulate physiological parameters, or to
`serve as cognitive or sensory prosthesis. One prototype wearable device in
`this area consists of an electrode that is implanted in the motor cortex of the
`brain to allow speech-incapable patients to communicate via a computer
`(Bakay and Kennedy in Siuru, 1999). Gold recording wires pick up electri(cid:173)
`cal signals of the brain and transmit the signals through the skin to a receiver
`and amplifier outside the scalp. The system is powered by an inductive coil
`placed over the scalp so that wires for powering the device do not have to
`pass through the skull. Signal processors are used to separate individual
`signals from the multiple signals that are recorded from inside the conical
`electrode tip. In the current implementation of the system, these signals
`are used to drive a cursor on a computer monitor. Such a device may prove
`beneficial to the 700,000 Americans who suffer from stroke each year and
`the tens of thousands more who suffer spinal cord injuries and diseases
`such as Lou Gehrig's disease. To conclude, there are many design issues
`that must be addressed before wearable computer systems reach their full
`potential and gain widespread use from the general public. To this aim,
`how wearable computer and augmented reality technology is designed, as
`well as application areas for wearable computers and augmented reality
`systems, is the topic of this chapter.
`
`2. SENSORY PROSTHESIS
`
`Over the past several thousand years, nature has provided humans the sen(cid:173)
`sory systems that allow them to detect and respond to visual, auditory,
`olfactory, haptic, and gustatory information. Nature has also provided hu(cid:173)
`mans well-developed cognitive abilities that allow decisions to be made
`under conditions of uncertainty, patterns to be detected that are embedded
`
`META 1017
`META V. THALES
`
`

`

`I . WEARABLE COMPUTERS AND AUGMENTED REALITY
`
`5
`
`in noise, and common sense judgments to be made. However, even though
`we can sense a broad range of stimuli, often over a wide range of en(cid:173)
`ergy values, our senses are still limited in many ways (Barfield, Hendrix,
`Bjomeseth, Kaczmarek, and Lotens, 1995). For example, we see images
`that represent only part of the electromagnetic spectrum, detect tactile and
`force feedback sensations across a limited range of energy values, and have
`marked decrements in spatializing images when sound is the primary cue.
`Due to these limitations, several types of prosthesis have been developed to
`extend our sensory, motor, and information processing abilities. For exam(cid:173)
`ple, to extend the visual modality, we have invented glasses, microscopes,
`and telescopes; and to extend the auditory modality we have invented mi(cid:173)
`crophones, hearing aids, and telephones. Furthermore, to extend the haptic
`modality we have invented sensors that detect forces, which then transmit
`the force information back to the human.
`We have also invented other types of sensory and motor prosthesis as
`well. Some of these include artificial hearts and kidneys and artificial arms
`and legs. However, until only recently, the prostheses that have been de(cid:173)
`veloped were designed primarily to enhance the detection capability of
`our sensory systems or to assist our motor capabilities, and to a far lesser
`extent, to enhance our cognitive capabilities. With the invention of the
`computer and developments in digital technology, microelectronics, and
`wireless networking, this is beginning to change. We are now able to wear
`digital devices that contain considerable computational resources; these
`devices clearly enhance our decision-making capabilities. Further, these
`digital devices may be worn "on the skin" (or body), as is the case with
`a wearable computer, or "under the skin," as is the case with medical de(cid:173)
`vices. Digital devices worn on the skin have led to exciting developments
`in the area of computational clothing and digital accessories. In this area
`of research, computer scientists and interface design specialists are work(cid:173)
`ing with clothing designers as well as experts in textiles and fabrics to
`build wearable computers that look more like clothing and less like com(cid:173)
`puters. Advances in computing under the skin have also led to exciting
`developments in information technology. In fact, in the near future we
`may be able to integrate computer chips directly into the nervous sys(cid:173)
`tem. Along these lines, researchers at Johns Hopkins University and North
`Carolina State University have developed a computer microchip that is
`connected to the retina (Liu et al., 1999). The chip is designed to send
`light impulses to the brain. Thus far the device has allowed fifteen test sub(cid:173)
`jects with blindness resulting from retinal damage to see shapes and detect
`movement.
`
`META 1017
`META V. THALES
`
`

`

`6
`
`BARFIELD AND CAUDELL
`
`3. DESCRIPTION OF SYSTEMS
`
`We describe a wearable computer as a fully functional, self-powered, self(cid:173)
`contained computer that is worn on the body. As noted earlier, a wearable
`computer provides access to information, and interaction with information,
`anywhere and at anytime. Closely related is the topic of "augmented real(cid:173)
`ity," which can be thought of as an advanced human-computer interface
`technology that attempts to blend or fuse computer-generated information
`with our sensations of the natural world. For example, using a see-through
`head-mounted display (HMD), one may project computer-generated graph(cid:173)
`ics into the environment surrounding the user to enhance the visual aspects
`of the environment. The main differences between what researcher's term "a
`wearable computer" versus "augmented reality" is twofold: (1) augmented
`reality is primarily a technology used to augment our senses and (2) wear(cid:173)
`able computers are far more mobile than augmented reality systems. With
`augmented reality, the range of mobility is dependent on the length of the
`cable connecting the wearable computer system to the computing platform.
`However, with a wearable computer, the computer and output devices are
`actually worn on the human's body, allowing a much broader range of
`mobility. Because of the close association between wearable computers
`and augmented reality, one can refer to both types of computing technol(cid:173)
`ogy generically as "wearware" or simply as wearable computer systems.
`Throughout this book we will describe the integration ofHMD, digital tech(cid:173)
`nology, auditory displays, and body tracking technologies with augmented
`reality and wearable computing hardware and software. These technologies
`have the potential of improving the efficiency and quality of human labors,
`particularly in their performance of engineering, manufacturing, construc(cid:173)
`tion, diagnostic, maintenance, monitoring, and transactional activities.
`One of the primary display or output devices for wearable computer
`systems is a head-mounted display. Currently, there are three main appli(cid:173)
`cation areas for HMDs, these include virtual reality, augmented reality, and
`wearable computers. These three application areas are summarized below.
`
`Virtual reality: With virtual reality, a participant uses an HMD to experi(cid:173)
`ence an immersive representation of a computer-generated simulation
`of a virtual world. In this case, the user does not view the real world
`and is connected to the computer rendering the scene with a cable,
`typically allowing about 3-4 meters of movement.
`Augmented reality: With augmented reality, a participant wears a see(cid:173)
`through display ( or views video of the real world with an opaque
`HMD) that allows graphics or text to be projected in the real world
`
`META 1017
`META V. THALES
`
`

`

`I . WEARABLE COMPUTERS AND AUGMENTED REALITY
`
`7
`
`graphics display_
`
`occular
`
`, beam splitter
`I
`A - - - -+ - - - 1
`
`real image
`
`FIG. I. I. Schematic diagram of an optical-based head-mounted
`display system for use with augmented reality and wearable com(cid:173)
`puter systems.
`
`(Caudell and Mizell, 1992; Lion, Rosenberg, and Barfield, 1993;
`Barfield, Rosenberg, and Lotens, 1995) (Figure 1.1). As with the
`virtual reality experience, the user is connected to the computer ren(cid:173)
`dering the graphics or text with a cable, again allowing about 3-4
`meters of movement.
`Wearable computers: With wearable computers, the user actually wears
`the computer and, as in virtual or augmented reality, wears the visual
`display (hand-held or head-mounted) (Figure 1.2). The wearable com(cid:173)
`puter may be wirelessly connected to a LAN or WAN, thus allowing
`information to be accessed whenever and wherever the user is in the
`environment.
`
`As noted earlier, wearable computers allow hands-free manipulation of
`real objects as does augmented reality displays. However, because virtual
`reality displays are completely immersive, the user cannot directly see his
`hands, which makes manipulation of real objects difficult. Of course, with
`appropriate input devices, manipulation of virtual objects can occur. The
`see-through display capability that allows text or graphics to be projected
`within the real world is unique to augmented reality and wearable computer
`systems. However, all three application areas, virtual reality, augmented
`reality, and wearable computers, can use non-see-through or opaque dis(cid:173)
`plays. With augmented reality and wearable computers, an opaque HMD
`(monocular or binocular) can show live video with computer-generated text
`or graphics overlaid over the video.
`Wearable computer systems can also be thought of as personal informa(cid:173)
`tion devices (Starner, Mann, Rhodes, Levine, Healey, Kirsch, Picard, and
`Pentland, 1997; Rhodes, 1997). With a wearable computer, the user expects
`his interface to be accessible continually and unchanging, unless specified
`otherwise. With experience, the user personalizes his system to ensure
`appropriate responses to everyday tasks. As a result, the user's wearable
`
`META 1017
`META V. THALES
`
`

`

`8
`
`BARFIELD AND CAUDELL
`
`1
`I
`
`FIG. 1.2. User with wearable computer accessing map of a
`University campus via wireless networked system.
`
`computer system becomes a mediator for other computers and interfaces,
`providing a familiar, dependable interface and set of tools complementing
`the abilities the wearable computer infrastructure provides (more processor
`power, additional sensing, etc.). With sophisticated user models and corre(cid:173)
`sponding software agents, such an interface can be extended to recognize
`and predict the resources needed by the user (Stamer et. al., 1997).
`There are several dimensions by which wearable computers and aug(cid:173)
`mented reality systems can be evaluated, two of which include the level of
`mobility provided by the computing system and the level of scene fidelity
`afforded by the rendering platform. The level of scene fidelity refers to the
`quality of the image provided by either the virtual reality simulation, real
`world scene, or augmented world. One of the primary differences between
`virtual reality and augmented reality is in the complexity of the projected
`graphical objects. In basic augmented reality systems, only simple wire
`frames, template outlines, designators, and text are displayed and animated.
`An immediate result of this difference is that augmented reality systems can
`be driven by microprocessors either worn on the body or integrated into the
`
`META 1017
`META V. THALES
`
`

`

`I . WEARABLE COMPUTERS AND AUGMENTED REALITY
`
`9
`
`work place. Today's processors have the computational power to transform
`and plot complex graphics in real time. Unlike full virtual reality systems,
`augmented reality systems are generally not attempting to generate a com(cid:173)
`plete "world" or realistic scene. Instead, augmented reality systems tend to
`rely on reality to simulate reality, only superimposing the necessary graph(cid:173)
`ical objects necessary to perform the task at hand. By doing so, many of the
`human factors issues found in full virtual reality systems such as vertigo
`and simulation sickness are avoided. The person continues to receive all
`their orientation cues from the physical visual scene. Display technology,
`input and output devices, computer architectures, network communication,
`power supplies, and image registration and calibration techniques are all
`important aspects of augmented reality systems.
`Human's carry their sensors with them as they move around the envi(cid:173)
`ronment, and they experience the world with the resolution provided by
`these biological sensors. Wearable computers also allow a high degree of
`mobility, but not quite that associated with our biological sensors. For ex(cid:173)
`ample, currently we cannot swim with wearable computers although our
`biological sensors easily allow this range of mobility. Furthermore, the
`weight of wearable computers and augmented reality systems further adds
`to their lack of mobility. Wearable computers allow a high degree of scene
`fidelity because the real world is viewed either directly with see-through
`optics or via live video. However, the level of scene fidelity may be less
`than that associated with augmented reality because wearable computers
`currently do not have the rendering capability of workstations that are often
`used (e.g., SGis) to render graphics for augmented reality environments.
`Finally, virtual reality in its present form is often low on scene fidelity and
`very low on mobility within the real world.
`
`4. DESIGN ISSUES
`
`Display technology, input and output devices, power supplies, and image re(cid:173)
`gistration techniques are important aspects of wearable computers and aug(cid:173)
`mented reality systems. The following sections briefly discus