Declaration of Rachel J. Watters on Authentication of Publication
`
`I, Rachel J. Watters, am a librarian, and the Head of Resource Sharing for the
`
`General Library System, Memorial Library, located at 728 State Street, Madison,
`
`Wisconsin, 53706. Part of my job responsibilities include oversight of Wisconsin
`
`TechSearch ("WTS"), an interlibrary loan department at the University of Wisconsin(cid:173)
`
`Madison. I have worked as a librarian at the University of Wisconsin library system
`
`since 1998, starting as a graduate student employee in the Kurt F. Wendt Engineering
`
`Library and WTS, then as a librarian in Interlibrary Loan at Memorial Library. I began
`
`professional employment at WTS in 2002 and became WTS Director in 2011. In 2019,
`
`I became of Head of Resource Sharing for UW-Madison's General Library System. I ·
`
`have a master's degree in Library and Information Studies from the University of
`
`Wisconsin-Madison. Through the course of my studies and employment, I have
`
`become well informed about the operations of the University of Wisconsin library
`
`system, which follows standard library practices.
`
`This Declaration relates to the dates of receipt and availability of the following:
`
`Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., and
`Colucci, D. (February 2001). High-performance wide-area
`optical tracking: the HiBall Tracking System. Presence:
`Teleoperators and Virtual Environments, 10(1), 1-21.
`
`Standard operating procedures for materials at the University of Wisconsin(cid:173)
`
`Madison Libraries. When an issue was received by the Library, it would be checked in,
`
`stamped with the date of receipt, added to library holdings records, and made available
`
`1
`
`META 1018
`META V. THALES
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`to readers as soon after its an-ival as possible. The procedure normally took a few days
`
`or at most 2 to 3 weeks.
`
`Exhibit A to this Declaration is a true and accurate copy of the title page with
`
`library date stamp of Presence: Teleoperators and Virtual Environments (February
`
`2001 ), from the University of Wisconsin-Madison Library collection. Exhibit A also
`
`includes an excerpt of pages 1 to 21 of that issue, showing the article entitled High(cid:173)
`
`performance wide-area optical tracking: the HiBall Tracking System (2001 ). Based on
`
`this information, the date stamp on the journal issue title page indicates High(cid:173)
`
`performance wide-area optical tracking: the HiBall Tracking System (2001) was
`
`received by the Kurt F. Wendt Library, University of Wisconsin-Madison on May 28,
`
`2001.
`
`Based on the information in Exhibit A, it is clear that the journal issue was
`
`received by the library on or before May 28, 2001, catalogued and available to library
`
`patrons within a few days or at most 2 to 3 weeks after May 28, 2001.
`
`Members of the interested public could locate the Presence: Teleoperators and
`
`Virtual Environments (February 2001) publication after it was cataloged by searching
`
`the public library catalog or requesting a search through WTS. The search could be
`
`done by title and/or subject key words. Members of the interested public could access
`
`the publication by locating it on the library's shelves or requesting it from WTS.
`
`2
`
`META 1018
`META V. THALES
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`I declare that all statements made herein of my own knowledge are true and that
`
`all statements made on information and belief are believed to be true; and further that
`
`these statements were made with the knowledge that willful false statements and the like
`
`so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18
`
`of the United States Code.
`
`Date: December 13, 2021
`
`Memorial Library
`728 State Street
`Madison, Wisconsin 53706
`
`Head of Resource Sharing
`
`3
`
`META 1018
`META V. THALES
`
`

`

`
`
`
`
`
`
`
`
`Exhibit A
`Exhibit A
`
`TA 1018
`META V. THALES
`
`META 1018
`META V. THALES
`
`

`

`This resource is also available
`on the WWW.
`Use MADCA T to launch.
`
`VOLUME 10 / NUMBER I / FEBRUARY 2001
`
`MIT PRESS
`
`META 1018
`META V. THALES
`
`

`

`Teleoperators and Virtual Environments,
`Frve Cambridge Center, Cambridge,
`MA 02142-1407. Periodicals postage
`paid at Boston, MA and at additional
`post offices.
`
`Advertising and Mailing Ust
`Rental
`Inquiries may be addressed to the
`Marketing Dept, MIT Press Journals,
`Frve Cambridge Center, Cambridge,
`MA 02142-1407. (617) 253-2866; fax
`(617) 258-5028; e-mail: joumals(cid:173)
`infc@"nitedu.
`
`Copyright Information
`Permission to photocopy artcles for
`internal or personal use, or the in(cid:173)
`ternal or personal use of specific client,
`is granted by the copyright owner for
`users registered with the Copyright
`Clearance Center (CCq Transactional
`Reportng Service, provided that the
`fee of $8.00 per artcle-copy is paid
`directly to CCC, 222 Rosewood Drive,
`Danvers, MA O 1923. The fee code for
`users of the Transactional Reportng
`Service is I 054-7 460/0 I $8.00. For
`those organizatons that have been
`granted a photocopy license with
`CCC, a separate system of payment
`has been ananged. Address all other
`inquiries to Subsidiary Rights Manager,
`MIT Press Journals, Frve Cambridge
`Center, Cambridge, MA 02142-1407;
`e-mail: joumals-right,@nitedu.
`
`PRESENCE: Teleoperators and
`Virtual Environments is available on
`microfilm from University Microfilms,
`Inc., 300 N. Zeeb Road, Ann Arbor,
`Ml 48106.
`
`© 200 I by the Massachusetts
`Institute ofT echnology.
`
`PRESENCE
`Teleoperators and
`Virtual Environments
`
`Volume I 0, Number I
`February 200 I
`ISSN I 054-7460
`
`Editor-in-Chief
`Nathaniel I. Durlach
`Virtual Environment and
`T eleoperator Research Consortium
`Research Laboratory of Electronics
`MIT
`
`Managing Editor
`Rebecca Lee Garnett
`MIT
`
`Director of Corporate
`Contributions
`Thomas A. Furness Ill
`Human Interface Technology
`Laboratory
`Univer-sity of Washington
`
`Editorial Address
`Editor-in-Chief, PRESENCE
`MIT
`77 Massachusetts Avenue
`Room 36-709
`Cambridge, MA 021 39
`presence@mit.edu
`
`Individuals wishing to submit
`manuscripts should follow the
`guidelines provided in this issue.
`
`Abstracting and Indexing
`PRESENCE: T eleoperators and Virtual
`Environments is included in Computer
`Abstracts, Ergonomics Abstracts,
`Information Science Abstracts,
`International Aerospace Abstracts, and
`Multi-Index to Cyber-Space, Virtual,
`and Artificial Reality. CompuMath
`Citation Index, Current Contents/
`Engineering Computing and
`Technology, Research Alert, and
`SciSearch.
`
`Sponsorship
`
`PRESENCE is endebted to the
`following organizations for their
`generous support:
`Air Force Office of Scientific
`Research
`Department of Biomedical
`Engineering, Boston University
`Human Int erface Technology
`Laboratory at the University of
`Washington
`Office of Naval Research
`Research Laboratory of Electronics,
`MIT
`Visual Computing Department,
`Hewlett Packard Laboratories
`Senior Editors
`Woodrow Barfield
`Virginia Polytechnic Institute
`Gary Bishop
`University of North Carolina
`
`Lanry Hodges
`Georgia Institute of Technology
`John M. Hollerbach
`University of Utah
`Randy Pausch
`Carnegie Mellon University
`Thomas B. Sheridan
`MIT
`Mel Slater
`University College London
`Kay Stanney
`University of Central Florida
`Elizabeth Wenzel
`NASA Ames Research Center
`David Zeltzer
`Fraunhofer Center for Research
`in Computer Graphics
`Michael Zyda
`Naval Postgraduate School
`
`Editorial Board
`Bernard Adelstein
`NASA Ames Research Center
`Tenry Allard
`NASA Ames Research Center
`Walter A. Aviles
`T eneo Computing, LLC
`Frank Biocca
`Michigan State University
`Jens Blauert
`Ruhr-University Bochum,
`Germany
`Grigore Burdea
`Rutgers University
`H . Steven Colburn
`Boston University
`Rudy Darken
`Naval Postgraduate School
`Paul Dizio
`Brandeis University
`Stephen Ellis
`NASA Ames Research Center
`Scott S. Fisher
`T elepres\.nce Research
`Richard M. Held
`MIT
`Kenneth 0. Johnson
`Johns Hopkins University
`Lynette A. Jones
`MIT
`James R. Lackner
`Brandeis University
`Jaron Lanier
`Columbia University
`Susan Lederman
`Queen's University, Canada
`Jack M. Loomis
`University of California
`Michael Macedonia
`STRICOM
`Michael Moshel l
`University of Central Florida
`Michael Naimark
`Interval Research Corporation
`Albert "Skip" Rizzo
`University of Southern California
`Warren Robinett
`University of North Carolina
`Jannick Rolland
`University of Central Florida
`Roy Ruddle
`University of Leeds
`
`Karun· Shimoga
`CMU Robotics Institute
`Barbara Shinn-Cunningham
`· Boston University
`Gurminder Singh
`National University of Singapore
`Robert J. Stone
`VR SolutionsNirtual Presence
`Ltd. UK
`Martin Stytz
`Air Force Institute of
`Technology
`Susumu T achi
`Univer-sity of Tokyo
`James Templeman
`Naval Research Laboratory
`Geb Thomas
`University of Iowa
`Colin Ware
`University of New Hampshire
`Richard C Waters
`Mitsubishi Electric Research
`Laboratory
`Suzanne Weghor-st
`University of Washington
`Janet Weisenberger
`Ohio State University
`Robert B. Welch
`NASA Ames Research Center
`Thomas E. von Wiegand
`MIT
`Business Offices and
`Subscription Rates
`PRESENCE: Teleoperators and Virtual
`Environments is published bimonthly
`(February, April, June, August,
`October, December) by The MIT
`Press, Cambridge, MA 02142-1407.
`Subscriptions and address changes
`should be addressed to MIT Press
`Journals, Five Cambridge Center,
`Cambridge, MA 02142-1407; (6 17)
`253-2889; fax (617) 577-1545;
`e-mail joumals-orders@mit.edu.
`An electronic, full-text version of
`PRESENCE is available from the MIT
`Press. Subscriptions are on a volume(cid:173)
`year basis. Subscription rates:
`Individuals
`Electronic only-
`$45.00, Students/retired $43.00,
`Institutions $342.00. Canadians add
`the 7% GST Print and
`Electronic- Individuals $80.00,
`Students/retred $48.00, Institutions
`$380.00. Outsde the U.S. and Canada
`add $30.00 for postage and handling.
`Current issues are $ 15.00. Back issue
`rates: Individuals $32.00, Institutions
`$64.00. Outside the U.S. and Canada
`add $5.00 per issue for postage and
`handling. Canadians add 7% GST
`Claims may be e-mailed to: joumals(cid:173)
`claim@nit edu. Claims for missing
`issues will be honored free of charge if
`made within three months after the
`publication date of the issue. Prices
`subject to change without notice.
`http//mitpressmitedu/PRES
`
`Postmaster
`Send address changes to PRESENCE:
`
`META 1018
`META V. THALES
`
`

`

`PRESENCE
`
`TELEOPERATORS AND VIRTUAL ENYIRONtlENTS
`I , F E B R U A R Y 2 0 0 I
`
`I 0, N U M B E R
`
`V O L U M E
`
`Editorial Notes
`Guest Editors' Introduction: VRST'99 Special Issue
`
`ARTICLES S High-Performance Wide-Area Optical Tracking: The
`HiBall Tracking System
`Greg Welch, Gary Bishop, Leandra Vicci, Stephen Brumback,
`Kurtis Keller, and D)nardo Colucci
`S GNU /MA VERIK: A Microkernel for Large-Scale Virtual
`Environments
`Roger Hubbold, Jon Cook, Martin Keates, Simon Gibson,
`Toby Howard, Alan Murta, Adrian West,
`and Steve Pettifer
`S Patterns of Network and User Activity in an Inhabited
`Television Event
`Chris Greenhalgh, Steve Benford, and Mike. Craven
`S Components for Distributed Yirtual Environments
`Manuel Oliveira, Jon Crow croft, and Mel Slater
`S An Adaptive Multiresolution Method for Progressive
`Model Transmission
`Danny To, Rynson W. H. Lau, and Mark Green
`S Testbed Evaluation of Virtual Environment Interaction
`Techniques
`DougA. Bowman, Donald B. Johnson, and Larry F. Hodges
`S An Introduction to 3-D User Interface Design
`Doug A. Bowman, Ernst Kruijff, Joseph J La Viola, Jr.,
`and Ivan Poupyrev
`An Overview of the COVEN Platform
`Emmanuel Frecon, Gareth Smith, Anthony Steed,
`Marten Stenius, and Olov Stahl
`
`lll
`
`lV
`
`1
`
`22
`
`35
`
`51
`
`62
`
`75
`
`96
`
`109
`
`FORUM
`
`WHAT'S HAPPENING
`
`META 1018
`META V. THALES
`
`

`

`High-Performance Wide-Area
`Optical Tracking
`The HiBall Tracking System
`
`Greg Welch
`welch@cs.unc.edu
`
`Gary Bishop
`gb@cs.unc.edu
`
`Leandra Vicci
`vicci@cs.unc.edu
`
`Stephen Brumback
`brumback@cs.unc.edu
`
`Abstract
`
`Kurtis Keller
`keller@cs.unc.edu
`Department of Computer Science
`University of North Carolina at
`Chapel Hill
`
`D'nardo Colucci
`colucci@virtual-reality.com
`Alternate Realities Corporation
`
`Since the early 1980s, the Tracker Project at the University of North Carolina at
`Chapel Hill has been working on wide-area head tracking for virtual and augmented
`environments. Our long-term goal has been to achieve the high performance re(cid:173)
`quired for accurate visual simulation throughout our entire laboratory, beyond into
`the hallways, and eventually even outdoors.
`
`In this article, we present results and a complete description of our most recent
`electro-optical system, the Hi Ball Tracking System. In particular, we discuss motiva(cid:173)
`tion for the geometric configuration and describe the novel optical, mechanical,
`electronic, and algorithmic aspects that enable unprecedented speed, resolution,
`accuracy, robustness, and flexibility
`
`Introduction
`
`Systems for head tracking for interactive computer graphics have been
`explored for more than thirty years (Sutherland, 1968). As illustrated in
`figure 1, the authors have been working on the problem for more than twenty
`years (Azuma, 1993, 1995; Azuma & Bishop, 1994a, 19946; Azuma & Ward,
`1991; Bishop, 1984; Gottschalk & Hughes, 1993; UNC Tracker Project,
`2000; Wang, 1990; Wang et al., 1990; Ward, Azuma, Bennett, Gottschalk, &
`Fuchs, 1992; Welch, 1995, 1996; Welch & Bishop, 1997; Welch et al., 1999 ).
`From the beginning, our efforts have been targeted at wide-area applications
`in particular. This focus was originally motivated by applications for which we
`believed that actually walking around the environment would be superior to
`virtually "flying ." For example, we wanted to interact with room-filling virtual
`molecular models, and to naturally explore life-sized virtual architectural mod(cid:173)
`els. Today, we believe that a wide-area system with high performance every(cid:173)
`where in our laboratory provides increased flexibility for all of our graphics,
`vision, and interaction research .
`
`1.1 Previous Work
`
`In the early 1960s, Ivan Sutherland implemented both mechanical and
`ultrasonic (carrier phase) head-tracking systems as part of his pioneering work
`in virtual environments. He describes these systems in his seminal paper "A
`Head-Mounted Three Dimensional Display" (Sutherland, 1968). In the
`
`Welch et al.
`
`I
`
`Presence, Vol. 10. No. I, February 2001, 1-2 1
`© 200 I by lhe Massachusetts Institute of Technology
`
`META 1018
`META V. THALES
`
`

`

`2 PRESENCE: VOLUME I 0, NUMBER I
`
`Initial wide-area
`opto-electronic idea
`
`Simpler LED panels
`and off-line calibration
`
`SCAAT and
`autocalibration
`
`• 1991
`
`1993
`
`1995
`
`1997
`
`1999
`
`Bishop's VLSI
`Self-Tracker
`
`Figure I.
`
`Original system
`(SIGGRAPH 91)
`
`The HiBall
`
`The HiBall system
`
`ensuing years, commercial and research teams have ex~
`plored mechanical, magnetic, acoustic, inertial, and op(cid:173)
`tical technologies. Complete surveys include Bhatnagar
`(1993 ); Burdea & Coiffet (1994); Meyer, Applewhite,
`& Biocca (1992); and Mulder (1994a, 19946, 1998).
`Commercial magnetic tracking systems fQr example
`(Ascension, 2000; Polhemus, 2000) have enjoyed popu(cid:173)
`larity as a result of a small user-worn component and
`relative ease of use . Recently, inertial hybrid systems
`(Foxlin, Harrington, & Pfeifer, 1998; Intersense, 2000)
`have been gaining popularity for similar reasons, with
`the added benefit of reduced high-frequency noise and
`direct measurements of derivatives.
`An early example of an optical system for tracking or
`motion capture is the Twinkle Box by Burton (Burton,
`1973; Burton & Sutherland, 1974). This system mea(cid:173)
`sured the positions of user-worn flashing lights with
`optical sensors mounted in the environment behind ro(cid:173)
`tating slotted disks. The Selspot system (Woltring, 1974)
`used fixed, camera-like, photodiode sensors and target(cid:173)
`mounted infrared light-emitting diodes that could be
`tracked in a one-cubic-meter volume. Beyond the
`HiBall Tracking System, examples of current optical
`tracking and motion-capture systems include the Flash-
`
`Point and Pixsys systems by Image Guided Technologies
`(IGT, 2000), the laserBIRD system by Ascension Tech(cid:173)
`nology (Ascension, 2000), and the CODA Motion Cap(cid:173)
`ture System by B & L Engineering (BL, 2000 ). These
`systems employ analog optical-sensor systems to achieve
`relatively high sample rates for a moderate number of
`targets. Digital cameras (two-dimensional, image-forming
`optical devices ) are used in motion -capture systems such
`as the HiRes 3D Motion Capture System by the Motion
`Analysis Corporation (Kadaba & Stine, 2000 ; MAC,
`2000) to track a relatively large number ohargets, al(cid:173)
`beit at a relatively low rate because of the need for 2-D
`image processi ng.
`
`1.l Previous Work at UNC-Chapel Hill
`
`As part of his 1984 dissertation on Self-Tracker,
`Bishop put forward the idea of outward-looking track(cid:173)
`ing systems based on user-mounted sensors that esti(cid:173)
`mate user pose 1 by observing landmarks in the environ(cid:173)
`ment (Bishop, 1984 ). He described two kinds of
`
`1. We use the word pose to indicate both position and orientation
`( six degrees of freedom ).
`
`META 1018
`META V. THALES
`
`

`

`Welch et al. 3
`
`Figure 2.
`
`landmarks: high signal-to-noise-ratio beacons such as
`light-emitting diodes (LEDs) and low signal-to-noise(cid:173)
`ratio landmarks such as naturally occurring features.
`Bishop designed and demonstrated custom VLSI chips
`( figure 2) that combined image sensing and processing
`on a single chip (Bishop & Fuchs, 1984). The idea was
`to combine multiple instances of these chips into an
`outward-looking cluster that estimated cluster motion
`by observing natural features in the unmodified environ(cid:173)
`ment. Integrating the resulting motion to estimate pose
`is prone to accumulating error, so further development
`required a complementary system based on easily de (cid:173)
`tectable landmarks (LEDs) at known locations. This
`LED-based system was the subject of a 1990 disserta(cid:173)
`tion by Jih-Fang Wang (Wang, 1990).
`In 1991, we demonstrated a working, scalable, elec(cid:173)
`tro-optical head-tracking system in the Tom orrmv )s R e(cid:173)
`alities gallery at that year's ACM SIGGRAPH confer(cid:173)
`ence (Wang et al. , 1990; Wang, Chi, & Fuchs, 1990;
`Ward et al., 1992). The system (figure 3) used four,
`head-worn, lateral-effect photodiodes that looked up (cid:173)
`ward at a regular array of infrared LEDs installed in pre(cid:173)
`cisely machined ceiling panels. A user-worn backpack
`contained electronics that digitized and communicated
`the photo~coordinates of the sighted LEDs . Photo(cid:173)
`grammetric techniques were used to compute a user's
`head pose using the known LED positions and the cor(cid:173)
`responding measured photo-coordinates from each
`LEPD sensor (Azuma & Ward, 1991 ). The system was
`ground-breaking in that it was unaffected by ferromag-
`
`Figure 3.
`
`netic and conductive materials in the environment, and
`the working volume of the system was determined
`solely by the number of ceiling panels . (See figure 3,
`top.)
`
`META 1018
`META V. THALES
`
`

`

`4 PRESENCE: VOLUME I 0, NUMBER I
`
`than 0.5 mm and 0 .03 deg. of absolute error and noise,
`everywhere in a 4 .5 m X 8.5 m room (with more than
`two meters of height variation). The area can be ex(cid:173)
`panded by adding more panels, or by using checker(cid:173)
`board configurations that spread panels over a larger
`area. The weight of the user-worn HiBall is approxi(cid:173)
`mately 300 grams, making it lighter than one optical
`sensor in the 1991 system. Multiple HiBall units can be
`daisy-chained together for head or hand tracking, pose(cid:173)
`aware input devices, or precise 3-D point digitization
`throughout the entire working volume.
`
`l
`
`Design Considerations
`
`In all of the optical systems we have developed
`(see section 1.2), we have chosen what we call an inside-
`. looking-out configuration, in which the optical sensors
`are on tl1e (moving) user and the landmarks (for in(cid:173)
`stance, the LEDs ) are fixed in the laboratory. The corre(cid:173)
`~ponding outside-looliing-in alternative would be to
`place the landmarks on the user ai1,d to fix tl1e optical
`sensors in the laboratory. (One can think about similar
`outside-in and inside-out distinctions for acoustic and
`magnetic technologies .) The two configurations are de(cid:173)
`picted in fi gure 5.
`There are some disadvantages to the inside-looking(cid:173)
`out approach. For small or medium-sized working vol(cid:173)
`umes, mounting the sensors on the user is more chal(cid:173)
`lenging than mounting them in the environment. It is
`difficult to make user-worn sensor packaging small, and
`communication from the moving sensors to the rest of
`the system is m ore complex. In contrast, there are fewer
`mechanical considerations when mounting sensors in
`the environment for an outside-looking-in configura(cid:173)
`tion. Because landmarks can be relatively simple, small,
`and cheap, tl1ey can often be located in numerous places
`on the user, and communication from the user to the
`rest of the system can be relatively simple or even un (cid:173)
`necessary. This is particularly attractive for full-body
`motion capture (BL, 2000; MAC, 2000).
`However, there are some significant advantages to the
`inside-looking-out approach for head tracking. By
`operating with sensors on the user rather than in the
`
`Figure 4.
`
`1.3 The HiBall Tracking System
`
`In this article, we describe a new and vastly im(cid:173)
`proved version of the 1991 system. We call the new sys(cid:173)
`tem the H iBall Tracliing System. Thanks to significant
`improvements in hardware and software, this HiBall
`system offers unprecedented speed, resolution, accuracy,
`robustness, and flexibility. The bulky and heavy sensors
`and backpack of the previous system have been replaced
`by a small HiBall unit (figure 4, bottom). In addition,
`the precisely machined LED ceiling panels of the previ(cid:173)
`ous system have been replaced by looser-tolerance pan(cid:173)
`els that are relatively inexpensive to make and simple to
`install (figure 4, top; figure 10). Finally, we are using an
`unusual Kalman-filter-based algorithm that generates
`very accurate pose estimates at a high rate with low la(cid:173)
`tency, and that simultaneously self-calibrates the system.
`As a result of these improvements, the HiBall Track(cid:173)
`ing System can generate more than 2,000 pose esti(cid:173)
`mates per second, with less than 1 ms oflatency, better
`
`META 1018
`META V. THALES
`
`

`

`lab-mounted (fixed)
`optical sensor
`
`. . .
`rl
`
`head-mounted landmarks
`
`·=·•..
`.. ..........
`.. .. ..
`..
`
`.. ...
`
`Welch et al. 5
`
`a
`
`a
`
`"··
`a.
`.
`.
`. .
`. .
`' ....
`.. .. .. .. ..
`.
`. .
`· · .-
`
`f
`
`lab-mounted
`(fixed) landmarks
`
`Outside-Looking-In
`
`~ Inside-Looking-Out
`
`Figure S.
`
`environment, the system can be scaled indefinitely. The .
`system can evolve from using dense active landmarks to
`fewer, lower signal-to-noise ratio, passive, and some day
`natural features for a Self-Tracker that operates entirely
`without explicit landmark infrastructure (Bishop, 1984;
`Bishop & Fuchs, 1984; Welch, 1995 ).
`The inside-looking-out configuration is also moti(cid:173)
`vated by a desire to maximize sensitivity to changes in
`user pose. In particular, a significant problem with an
`outside-looking-in configuration is that only position
`estimates can be made directly, and so orientation must
`be inferred from position estimates of multiple fixed
`landmarks. The result is that orientation sensitivity is a
`function of both the distance to the landmarks from the
`sensor and the baseline between the landmarks on the
`user. In particular, as the distance to the user increases
`or the baseline between the landmarks decreases, the
`sensitivity goes down . For sufficient orientation sensitiv(cid:173)
`ity, one would likely need a baseline that is considerably
`larger than the user's head. This would be undesirable
`from an ergonomic standpoint and could actually re(cid:173)
`strict the user's motion.
`
`With respect to translation, the change in measured
`photo-coordinates is tl1e same for an environment(cid:173)
`mounted (fixed) sensor and user-mounted (moving)
`landmark as it is for a user-mounted sensor and an envi(cid:173)
`ronment-mounted landmark. In other words, the trans(cid:173)
`lation and corresponding sensitivity are the same for
`either case.
`
`3
`
`System Overview
`
`The HiBall Tracking System consists of three
`main components (figure 6). An outward-looking
`sensing unit we call the HiBall is fixed to each user to
`be tracked. The HiBall unit observes a subsystem of
`fixed-location infrared LEDs we call the Ceiling.2
`Communication and synchronization between the
`host computer and these subsystems is coordinated
`
`2. At the present time, th e LEDs are in fact entirely located in the
`ceiling of our laboratory (hence the subsystem name Ceiling), but
`LEDs co uld as well be located on walls or other fixed locations.
`
`META 1018
`META V. THALES
`
`

`

`6 PRESENCE: VOLUME I 0, NUMBER I
`
`I
`
`4.5 X 8.5 m
`Ceiling (with LED's)
`
`Ceiling-HiBall Interface
`Board (CIB)
`
`Figure 6.
`
`by the Ceiling-HiBall Inte1face Boa_,rd (CIB). In sec(cid:173)
`tion 4, we describe these components in more detail.
`Each HiBall observes LEDs through multiple sen(cid:173)
`sor-lens views that are distributed over a large solid
`angle. LEDs are sequentially flashed ( one at a time )
`such that they are seen via a diverse set of views for
`each HiBall. Initial acquisition is performed using a
`brute -force search through LED space, but, once ini(cid:173)
`tial lock is made , the selection of LEDs to flash is tai (cid:173)
`lored to the views of the active HiBall units . Pose es(cid:173)
`timates are maintained using a Kalman-filter-based
`prediction -correction approach known as single(cid:173)
`constraint-at-a-time (SCAAT) tracking . This tech(cid:173)
`nique has been extended to provide self-calibration of
`the ceiling, concurrent with HiBall tracking. In sec(cid:173)
`tion 5, we describe the methods we employ, includ(cid:173)
`ing the initial acquisition process and the SCAAT ap (cid:173)
`proach to pose estimation, with the autocalibration
`extens ion .
`
`4
`
`System Components
`
`4.1 The HiBall
`
`The original electro-optical tracker (figure 3, bot(cid:173)
`tom ) used independently housed lateral -effect photo(cid:173)
`diode units (LEPDs) attached to a lightweight tubular
`framework . As it turns out, the mechanical framework
`would flex (distort) during use, contributing to estima(cid:173)
`tion errors. In part to address this problem, the HiBall
`sensor unit was designed as a single, rigid, hollow ball
`having dodecahedral symmetry, with lenses in the upper
`six faces and LEPDs on the insides of the opposing six
`lower faces (figure 7) . T his immediately gives six pri(cid:173)
`ma1y "camera" views uniformly spaced by 57 deg. The
`views efficiently share the same internal air space and are
`rigid witl1 respect to each other. In addition, light enter(cid:173)
`ing any lens sufficiently off-axis can be seen by a neigh(cid:173)
`boring LEPD, giving rise to five secondaiy views through
`the top or central lens, and three seconda1y views
`
`META 1018
`META V. THALES
`
`

`

`Welch et al.
`
`7
`
`~
`~ ~ t:if'/2;} '
`\\Q ~
`.. ~
`.. ,,,,
`. '";-·
`-r,
`. ' .·
`·""o ·
`
`~ I ~C•~~•~
`. :,i~:~.~~I ,1~~ , \
`-:-:1~-i~:·:~)[-f~ ti li~~
`,o,. .
`:::'I.-;
`• . ".'in,'' .·,.,,._.![";!.~ ~ • \ -.. ~ il.\.' ~. ~
`~i~~::;t··@~
`... -"I:!.$
`. .! :~n'
`;<,? :-
`.~-.
`
`•·='···
`
`'··~. . . •
`
`'·.c;; ,
`
`>UUM•;.,,.,,,
`
`'I
`
`O < :.
`
`•
`
`!
`
`'
`
`,
`
`.
`
`'
`
`.
`·-•-,,,.
`
`Figure 7.
`
`through the five other lenses. Overall, this provides 26
`fields of view that are used to sense widely separated
`groups of LEDs in the environment. Although the extra
`views complicate the initialization of the Kalman filter as
`described in section 5.5, they turn out to be of great
`benefit during steady-state tracking by effectively in(cid:173)
`creasing the overall HiBall field of view without sacrific(cid:173)
`ing optical-sensor resolution.
`The lenses are simple piano-convex fixed-focus lenses .
`Infrared (IR) filtering is provided by fabricating the
`lenses themselves from RG-780 Schott glass filter mate(cid:173)
`rial which is opaque to better than 0.001% for all visible
`wavelengths and transmissive to better than 99% for IR
`wavelengths longer than 830 nm. The longv.ave filter(cid:173)
`ing limit is provided by the DLS-4 LEPD silicon photo(cid:173)
`detector (DDT Sensors, Inc.) with peak responsivity at
`950 nm but essentially blind above 1150 nm.
`The LEPDs themselves are not imaging devices;
`rather, they detect the centroid of the luminous flux
`incident on the detector. The x-position of the centroid
`determines the ratio of two output currents, and the
`
`Figure 8.
`
`y-position determines the ratio of two other output cur(cid:173)
`rents . The~total output current of each pair are com(cid:173)
`mensurate and are proportional to the total incident
`flux. Consequently, focus is not an issue, so the simple
`fixed-focus lenses work well over a range of LED dis(cid:173)
`tances from about half a meter to infinity. The LEPDs
`and associated electronic components are mounted on a
`custom rigid-flex printed circuitboard (figure 8). This
`arrangement makes efficient use of the internal HiBall
`volume while maintaining isolation between analog and
`digital circuitry, and increasing reliability by alleviating
`the need for intercomponent mechanical connectors.
`Figure 9 shows the physical arrangement of the
`folded electronics in the HiBall. Each LEPD has four
`transimpedance amplifiers (shown together as one
`"Amp" in figure 9), tl1e analog outputs of which are
`multiplexed with those of the other LEPDs, then sam(cid:173)
`pled, held, and converted by four 16-bit Delta-Sigma
`analog-to-digital (A/D) converters. Multiple samples
`are integrated via an accumulator. The digitized LEPD
`data are organized into packets for communication back
`to the CIB. The packets also contain information to
`assist in error detection. The communication protocol is
`simple, and, while presently implemented by wire, the
`modulation scheme is amenable to a wireless implemen(cid:173)
`tation. The present wired implementation allows multi(cid:173)
`ple HiBall units to be daisy-chained, so a single cable
`can support a user witl1 multiple HiBall units.
`
`META 1018
`META V. THALES
`
`

`

`£ LEPO ..... LEPO Sensors
`
`6-1 Multiplexer
`.....
`~
`~
`AID Converter
`---- -- - --
`
`Amp ..... Amp Amplifiers
`-- --- - 'I
`-
`•
`-
`cu
`-
`...
`0
`....,
`C:
`0
`u
`
`I
`
`I
`I
`
`I
`.1
`I
`I
`I
`I
`I
`i
`
`6 Sensors
`and
`Amplifiers
`
`Analog
`
`Digital
`
`Base and
`Connector
`
`Figure 9.
`
`·-
`
`I
`I
`I
`
`FPGA
`
`4.1 The Ceiling
`
`As presently implemented, the infrared LEDs are
`packaged in 61 cm square panels to fit a standard false(cid:173)
`ceiling grid ( figure 10, top). Each panel uses five printed
`circuit boards: a main controller board and four identi(cid:173)
`cal transverse -mounted strips (bottom). Each strip is
`populated with eight LEDs for a total of 32 LEDs per
`panel. We mount the assembly on top of a metal panel
`such that the LEDs protrude through 32 corresponding
`holes. The design results in a ceiling with a rectangular
`LED pattern with periods of7.6 cm and 15.2 cm. This
`spacing is used for the initial estimates of the LED posi(cid:173)
`tions in the lab; then, during normal operation, the
`SCAAT algorithm continually refines the LED position
`
`estimates (section 5.4). The SCAAT autocalibration not
`only relaxes design and installation constraints, but pro(cid:173)
`vides greater precision in the face of initial and ongoing
`uncertainty in the ceiling structure.
`We currently have enough panels to cover an area
`approximately 5.5 m by 8.5 m with a total of approxi(cid:173)
`mately 3,000 LEDs. 3 The panels are daisy-chained to
`each other, and panel-selection encoding is position
`(rather than device) dependent. Operational commands
`are presented to the first panel of the daisy chain. At
`each panel, if the panel-select code is zero, the
`
`3. The area is actualJy L-shaped; a small storage room occupies one
`comer.
`
`META 1018
`META V. THALES
`
`

`

`W elch et al. 9
`
`Figure 11.
`
`ceiling bandwidth. (The ceiling bandwidth is inherently
`limited by LED power resu-ictions as described in sec(cid:173)
`tion 4.2, but tl1is can be increased by spatially multiplex(cid:173)
`ing the cei