`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DECLARATION OF GORDON MACPHERSON
`
`I, Gordon MacPherson, am over twenty-one (21) years of age. I have never been
`convicted of a felony, and I am fully competent to make this declaration. I declare the following
`to be true to the best of my knowledge, information and belief:
`
`1. I am Director Board Governance & IP Operations of The Institute of Electrical and
`Electronics Engineers, Incorporated (“IEEE”).
`
`2. IEEE is a neutral third party in this dispute.
`
`3. I am not being compensated for this declaration and IEEE is only being reimbursed
`for the cost of the article I am certifying.
`
`4. Among my responsibilities as Director Board Governance & IP Operations, I act as a
`custodian of certain records for IEEE.
`
`5. I make this declaration based on my personal knowledge and information contained
`in the business records of IEEE.
`
`6. As part of its ordinary course of business, IEEE publishes and makes available
`technical articles and standards. These publications are made available for public
`download through the IEEE digital library, IEEE Xplore.
`
`7. It is the regular practice of IEEE to publish articles and other writings including
`article abstracts and make them available to the public through IEEE Xplore. IEEE
`maintains copies of publications in the ordinary course of its regularly conducted
`activities.
`
`8. The article below has been attached as Exhibit A to this declaration:
`
`
`A. D. Zetu, et al, “Extended-range hybrid tracker and applications to motion
`and camera tracking in manufacturing systems”, IEEE Transactions on
`Robotics and Automation, Vol. 16, Issue 3, June 2000.
`
`
`
`
`
`9. I obtained a copy of Exhibit A through IEEE Xplore, where it is maintained in the
`ordinary course of IEEE’s business. Exhibit A is a true and correct copy of the
`Exhibit, as it existed on or about December 29, 2021.
`
`10. The article and abstract from IEEE Xplore show the date of publication. IEEE
`Xplore populates this information using the metadata associated with the publication.
`
`445 Hoes Lane Piscataway, NJ 08854
`
`DocuSign Envelope ID: 7FCDEB04-9D8A-4D7A-9401-7811BCC4CFCA
`
` 1
`
`META 1029
`META V. THALES
`
`

`

`
`11. D. Zetu, et al, “Extended-range hybrid tracker and applications to motion and camera
`tracking in manufacturing systems” was published IEEE Transactions on Robotics
`and Automation, Vol. 16, Issue 3. IEEE Transactions on Robotics and Automation,
`Vol. 16, Issue 3 was published in June 2000. Copies of this publication was made
`available no later than the last day of the publication month. The article is currently
`available for public download from the IEEE digital library, IEEE Xplore.
`
`12. I hereby declare that all statements made herein of my own knowledge are true and
`that all statements made on information and belief are believed to be true, and further
`that these statements were made with the knowledge that willful false statements and
`the like are punishable by fine or imprisonment, or both, under 18 U.S.C. § 1001.
`
`I declare under penalty of perjury that the foregoing statements are true and correct.
`
`
`
`
`Executed on:
`
`
`
`
`
`
`
`
`
`
`DocuSign Envelope ID: 7FCDEB04-9D8A-4D7A-9401-7811BCC4CFCA
`
`1/6/2022
`
` 2
`
`META 1029
`META V. THALES
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT A
`
`DocuSign Envelope ID: 7FCDEB04-9D8A-4D7A-9401-7811BCC4CFCA
`
` 3
`
`META 1029
`META V. THALES
`
`

`

`IEEE.org
`
`IEEE Xplore
`
`IEEE-SA
`
`IEEE Spectrum
`
`More Sites
`
`Create
`Account
`
`Personal
`Sign In
`
`Cart
`
`
`
`
`
`Access provided by:
`Everything Demo User
`
`Sign Out
`
`Browse  My Settings  Help 
`
`Access provided by:
`Everything Demo User
`
`Sign Out
`
`All
`
`
`
`
`
`ADVANCED SEARCH
`
`Journals & Magazines > IEEE Transactions on Robotics... > Volume: 16 Issue: 3 
`
`Extended-range hybrid tracker and applications to motion
`and camera tracking in manufacturing systems
`Publisher: IEEE
`
`Cite This
`
` PDF
`
`  
`Alerts
`
`Manage Content
`
`Alerts
`Add to Citation
`
`Alerts
`
`D. Zetu ; P. Banerjee ; D. Thompson All Authors
`
`193
`Full
`Text Views
`
`8P
`
`aper
`Citations
`
` D
`
`ownl
`PDF
`
`Abstract:Extended- or long-range tracking effectiveness is crucial for the
`incorporation of mobile robots into manufacturing systems. In this paper, we
`conceptualize and develop a... View more
`
` Metadata
`Abstract:
`Extended- or long-range tracking effectiveness is crucial for the incorporation of
`mobile robots into manufacturing systems. In this paper, we conceptualize and
`develop a prototype long-range hybrid tracker based on a combination of a
`laser tracker and a magnetic tracker and apply the concept to the following two
`applications: (1) extended-range human motion tracking on factory floors and
`(2) factory floor object reconstruction from camera images. The easily portable
`system not only utilizes the strengths of a laser tracker in tracking mobile
`objects over long ranges in large environments, such as a manufacturing shop
`floor and the strength of a magnetic tracker to compensate for violation of line-
`of-sight constraint, but it also reduces the overall cost by reducing the number
`of expensive beacons required by the laser tracker. The hybrid tracker assists
`in the development of two concepts: (1) real-time synchronization of human
`
`Abstract
`
`Authors
`
`References
`
`Citations
`
`Keywords
`
`Metrics
`
`More Like This
`
`
`
` 4
`
`META 1029
`META V. THALES
`
`More
`Like
`This
`Performance Evaluation of Real-Time System
`for Vision-Based Navigation of Small
`Autonomous Mobile Robots
`2019 10th International Conference on
`Dependable Systems, Services and
`Technologies (DESSERT)
`Published: 2019
`Simulation of real-time systems: an object-
`oriented approach supported by a virtual
`reality-based tool
`38th Annual Simulation Symposium
`Published: 2005
`Show
`More
`

`

`head and hand motion in a manufacturing environment with those of an avatar
`in a virtual manufacturing environment and (2) a mathematically simpler and
`practical camera self-calibration technique for the creation of three-dimensional
`objects in a virtual environment from camera images.
`
`Published in: IEEE Transactions on Robotics and Automation ( Volume: 16 ,
`Issue: 3, Jun 2000)
`
`Page(s): 281 - 293
`
`INSPEC Accession Number: 6661504
`
`Date of Publication: Jun 2000 
`
`DOI: 10.1109/70.850646
`
` ISSN Information:
`
`Publisher: IEEE
`
`Authors
`
`References
`
`Citations
`
`Keywords
`
`Metrics
`
`
`
`
`
`
`
`
`
`
`
`IEEE Personal Account
`
`Purchase Details
`
`Profile Information
`
`Need Help?
`
`CHANGE USERNAME/PASSWORD
`
`PAYMENT OPTIONS
`
`COMMUNICATIONS PREFERENCES
`
`US & CANADA: +1 800 678 4333
`
`VIEW PURCHASED DOCUMENTS
`
`PROFESSION AND EDUCATION
`
`WORLDWIDE: +1 732 981 0060
`
`TECHNICAL INTERESTS
`
`CONTACT & SUPPORT
`
`Follow
`
`  
`
`About IEEE Xplore | Contact Us | Help | Accessibility | Terms of Use | Nondiscrimination Policy | IEEE Ethics Reporting  | Sitemap | Privacy & Opting Out of Cookies
`A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
`
`© Copyright 2021 IEEE - All rights reserved.
`
`IEEE Account
`
`Purchase Details
`
`Profile Information
`
`Need Help?
`
`» Change Username/Password
`» Update Address
`
`» Payment Options
`» Order History
`» View Purchased Documents
`
`» Communications Preferences
`» Profession and Education
`» Technical Interests
`
`» US & Canada: +1 800 678 4333
`» Worldwide: +1 732 981 0060
`» Contact & Support
`
`About IEEE Xplore Contact Us
`
`
`|
`
`|
`
`Help
`
`
`|
`
`Accessibility
`
`
`|
`
`Terms of Use
`
`
`|
`
`Nondiscrimination Policy
`
`
`|
`
`Sitemap
`
`
`|
`
`Privacy & Opting Out of Cookies
`
`A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
`© Copyright 2021 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.
`
` 5
`
`META 1029
`META V. THALES
`
`

`

`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`281
`
`Extended-Range Hybrid Tracker and Applications
`to Motion and Camera Tracking in
`Manufacturing Systems
`
`Dan Zetu, Pat Banerjee, and Darren Thompson
`
`Abstract—Extended- or long-range tracking effectiveness is cru-
`cial for the automation of manufacturing systems. In this paper, we
`conceptualize and develop a prototype long-range hybrid tracker
`based on a combination of a laser tracker and a magnetic tracker
`and apply the concept to the following two applications: 1) ex-
`tended-range human motion tracking on factory floors and 2) fac-
`tory floor object reconstruction from camera images. The easily
`portable system not only utilizes the strengths of a laser tracker
`in tracking mobile objects over long ranges in large environments,
`such as a manufacturing shop floor and the strength of a magnetic
`tracker to compensate for violation of line-of-sight constraint, but
`it also reduces the overall cost by reducing the number of expensive
`beacons required by the laser tracker. The hybrid tracker assists
`in the development of two concepts: 1) real-time synchronization
`of human head and hand motion in a manufacturing environment
`with those of an avatar in a virtual manufacturing environment
`and 2) a mathematically simpler and practical camera self-cali-
`bration technique for the creation of three-dimensional objects in
`a virtual environment from camera images.
`
`tracking, magnetic
`laser
`Index Terms—Hybrid tracker,
`tracking, stereo reconstruction, virtual reality applications.
`
`I. INTRODUCTION
`
`A UTONOMOUS navigation of mobile robots and material
`
`handling equipment (such as automated guided vehicles or
`forklifts) is often a prerequisite for automation of manufacturing
`systems. In order to achieve autonomous navigation of such
`components of manufacturing systems, they need to know at any
`time where they are located within the environment, with respect
`to a global coordinate system [20]. Similarly, in virtual reality
`(VR)-aided manufacturing systems design and maintenance ap-
`plications, it is necessary to capture the motion of human par-
`ticipants in order to replicate it on avatars within virtual envi-
`ronments (VE’s) representing specific manufacturing systems.
`This motion has to be often captured over a longer range than
`the ranges of current tracking systems for VR applications. A
`survey of the existing position trackers used in VR can be found
`in [12] and [25].
`Currently, extended-range trackers are employed for tracking
`mobile robots [7], [23], [13]. The most widely used long-range
`tracking systems in robotics are active beacon systems. The
`biggest disadvantage of active beacon navigation systems is the
`
`Manuscript received April 26, 1999; revised December 15, 1999. This paper
`was recommended for publication by Editor P. Luh upon evaluation of the re-
`viewers’ comments. This research was supported in part by the National Science
`Foundation under Grant DMI 9500396.
`The authors are with Department of Mechanical Engineering, University of
`Illinois at Chicago, Chicago, IL 60607 USA.
`Publisher Item Identifier S 1042-296X(00)04761-3.
`
`line-of-sight constraint (LOS) [7]. There may be instances when
`tracking a certain part (such as the end-effector) of a robot is nec-
`essary, and this part cannot be visible to the tracking system, un-
`less multiple beacons are placed within the motion environment,
`thus increasing substantially the cost of the tracking system.
`In VR applications, the task of a tracker is to report the posi-
`tion and orientation of a user’s head and hand. Accordingly, the
`VR system updates the perspective display to make it consistent
`with the user’s viewpoint. There are multiple types of tracking
`systems used in VR: magnetic, optical, mechanical, acoustic,
`and inertial. Each of these trackers has advantages and disad-
`vantages. For example, magnetic trackers have no LOS con-
`straints, but their accuracy decreases dramatically with increase
`in distance from transmitter and is also influenced by metallic
`objects in the neighborhood. Optical trackers are very fast and
`accurate, and they are also immune to magnetic interference,
`but their use is restricted by the LOS constraint. Mechanical
`trackers, based on linkages, are very accurate, but their work
`is severely restricted within a small-range volume (determined
`by the geometry of linkages). Acoustic (ultrasonic) trackers are
`relatively cheap and accurate, but they also have limited range
`and LOS restriction.
`The above considerations suggest combining some of the ad-
`vantages offered by individual tracking systems to design a hy-
`brid tracker for autonomous navigation in real manufacturing
`environments and human motion in VE’s. In this paper, a hy-
`brid tracker, based on a combination between a laser tracker and
`a magnetic tracker, is described. The laser tracker used is an ac-
`tive beacon system1 [36], and the magnetic tracker employed
`is called MotionStar2 [37]. The laser tracker has the advantage
`of enabling accurate tracking of position and orientation over
`long ranges (the system we use has a maximum range of 100 m,
`but through serialization of multiple such systems, unlimited
`range can be obtained). The magnetic tracker enables tracking
`of multiple parts of an object without problems due to occlu-
`sion. Moreover, when the LOS constraint for the laser tracker is
`temporarily violated, the magnetic tracker can compensate for
`it. This is an important advantage since it reduces the number of
`beacons or landmarks that have to be placed within the environ-
`ment, thus reducing substantially the cost of tracking systems.
`This paper is organized as follows. Related work in hybrid
`tracking is discussed in Section II. Section III describes the
`geometry of the hybrid (laser and magnetic) tracking system.
`
`1CONAC by MTI Research, Westford, MA 01886 USA.
`2Manufactured by Ascension Technologies Corporation, Burlington, VT
`05402 USA.
`
`1042–296X/00$10.00 © 2000 IEEE
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1029
`META V. THALES
`
`

`

`282
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Section IV describes the application of the proposed hybrid
`tracker to motion tracking during real-time synchronization
`of real environments and VE’s. Section V covers three-di-
`mensional (3-D) object reconstruction from camera images
`and describes our camera self-calibration procedure using
`extended-range tracking. Section VI is devoted to conclusions
`and future directions.
`
`II. RELATED WORK
`
`Due to their contribution to the end-to-end latency of VR
`systems, tracking systems have received a great deal of atten-
`tion among VR research community. Hybrid tracking has been
`explored mostly in the area of augmented reality (AR), where
`accurate registration between real environment and virtual ob-
`jects superimposed on it is critical. In [2], the need for hybrid
`tracking in AR is stressed, especially for outdoors applications.
`Most of the tracking in AR is being performed with the aid of
`video cameras, by tracking fiducial marks (placed at known lo-
`cations in the environment) using computer vision techniques.
`Despite the accuracy of these techniques, they are slow (due
`to the necessity of searching the marks by scanning the image
`pixel by pixel), their range is limited by the placement of the
`fiducial marks and are not robust to occlusions of the fiducial
`marks. In [26], a hybrid tracker is described, which combines
`computer vision-based tracking with inertial tracking. Since it is
`well known that inertial trackers exhibit drift with time (their er-
`rors increase over time [3]), their output is corrected by using vi-
`sion-based tracking. Another hybrid system has been proposed
`in [17], where it has been proven that by combining two types of
`vision-based tracking, called “inside-out” (camera(s) mounted
`on the head of the user and fiducials mounted at known loca-
`tions in the environment) and “outside-in” (cameras mounted at
`known locations in the environment and fiducials mounted on
`the user’s head), the uncertainty in head pose (position and ori-
`entation) estimation is considerably decreased. In [28], the ac-
`curacy of a magnetic tracker is improved by augmenting it with
`a passive image-based system that observes known fiduciary
`marks in the real world. At the same time, the magnetic tracker
`measurements help in reducing the search area of the fiducials
`in two-dimensional (2-D) images captured by head-mounted
`cameras, thus reducing the latency of the hybrid tracker. Other
`hybrid systems have been previously proposed in [5], [9], and
`[14]. For example, in [5] and [9], combinations between iner-
`tial and optical technologies are described in terms of accuracy
`and end-to-end latency. In [14], an inertial system is aided by
`angular position sensors. None of these applications address the
`problem of tracking motion in large environments, such as a fac-
`tory floor. For this kind of application, active beacon systems are
`very suitable, due to their accuracy and extended range but, due
`to the LOS constraint, usually a large number of beacons has to
`be mounted on the factory floor. We overcome this disadvantage
`by using a magnetic tracker in combination with a laser tracker.
`
`III. DESCRIPTION OF THE HYBRID TRACKING SYSTEM AND
`GENERIC METHODOLOGY FOR MOTION TRACKING
`
`As mentioned in Sections I and II, our hybrid tracker for mo-
`tion tracking is a combination of a laser tracker and a magnetic
`
`one. The laser tracker provides high accuracy and update rate
`for high ranges (0–100 m), but its use is restricted by the LOS
`constraint. On the other hand, the magnetic tracker does not re-
`quire LOS, but it is accurate only within small working vol-
`umes. The laser tracker is based on triangulation of laser signals
`emitted by two beacons and received by one or more position
`transponders (PT’s), attached to the moving object. One PT can
`report only the position with respect to one beacon so, in order
`to retrieve the orientation, one has to employ three PT’s rigidly
`mounted on a special fixture. The advantage of using a mag-
`netic tracker is its suitability to applications with frequent oc-
`clusions between transmitter and receiver. The magnetic tracker
`employed in our tracker uses pulsed direct current (dc) magnetic
`fields instead of alternate current (ac) magnetic fields (which are
`being used by Polhemus, Inc. magnetic trackers and older ver-
`sions of Ascension Technologies trackers). DC fields are signifi-
`cantly less susceptible to metallic distortion than ac fields. How-
`ever, dc-based magnetic trackers are susceptible to interference
`with magnetic fields generated by ferromagnetic objects (such
`as computer monitors or dc motors, see [27]). Even though it
`is hard to estimate up front the probability of encountering such
`objects during a motion sequence, it is reasonable to assume that
`in most cases the wearer of a magnetic tracker will not be in the
`immediate proximity of ferromagnetic objects that would cat-
`astrophically affect the tracker’s output. Overall, by weighing
`its advantages, the dc-based magnetic tracker remains a reliable
`magnetic tracker for motion capture in manufacturing environ-
`ments. By using a Kalman filter [19] to minimize the external
`effects on its performance, reasonable results can be obtained,
`as will be seen later in this paper. The advantages of incorpo-
`rating a magnetic tracker into our hybrid tracker are as follows.
`It can track multiple targets without worrying about occlu-
`sions between transmitter and receivers.
`Since its behavior is not influenced by an LOS constraint,
`the magnetic tracker can be used as a backup, when the LOS
`between laser tracker’s beacons and PT’s is temporarily
`occluded. This enables reduction in the number of beacons
`(landmarks) used for motion tracking, thus reducing the cost
`of the tracking system. Details are provided below.
`Typically, magnetic tracker’s receivers are placed on compo-
`nents whose motion trajectories have to be captured and cannot
`be “seen” all the time by beacons of the laser tracker. The PT’s
`of the laser tracker are mounted on the moving objects, in a loca-
`tion that is always visible to the transmitting beacons. The posi-
`tions of the tracked components (i.e., the components equipped
`with a magnetic receiver) are reported either with respect to a
`beacon’s coordinate system or to a coordinate system attached
`to the motion environment, termed world coordinate system
`(WCS) (in this case, the transformation between the WCS and
`beacon coordinate systems is known a priori). For this purpose,
`one of the magnetic receivers is rigidly attached to the PT’s, so
`the transformation between this receiver and PT’s is invariant
`as the tracked object moves. The magnetic transmitter is also
`placed on the moving object.
`The described hybrid tracker has the advantage of being
`easily portable, unlike other trackers currently in use, such as
`UNC HiBall [31], which requires a large number of beacons
`(LED’s) mounted on the ceiling and whose range is limited
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1029
`META V. THALES
`
`

`

`ZETU et al.: EXTENDED-RANGE HYBRID TRACKER AND APPLICATIONS TO MOTION AND CAMERA TRACKING
`
`283
`
`TABLE I
`EXPLANATION OF NOTATIONS IN FIG. 1
`
`Fig. 1. Geometry of the hybrid tracker.
`
`by the number of such beacons. The tracker described in [31]
`has the advantage that, by mounting a large number of closely
`located LED’s on the ceiling, one will have less problems with
`LOS but, in order to increase the tracking range, the cost of
`the system rises significantly, and the system becomes less
`portable. By being equipped with only two stationary beacons
`(at least for now), our tracking system is expected to be more
`susceptible to LOS problems. By mounting the beacons in
`optimal locations (to minimize the likelihood of violating the
`LOS constraint) and by using also the magnetic tracker to
`aid the laser tracker temporarily (when the LOS constraint
`is violated), we expect the impact of LOS problems to be
`minimal. The following two questions arise. 1) How accurate
`are the tracker outputs when the magnetic tracker aids in cir-
`cumventing LOS problems? 2) For how long can the violation
`of the LOS constraint be tolerated so that the position estimates
`fall within acceptable accuracy limits? These questions will be
`addressed through an example in Section IV.
`The generic geometry of our hybrid tracker is depicted in
`Fig. 1. In this figure, the case when the positions of the tracked
`components are reported with respect to a WCS is illustrated.
`Note that in Fig. 1, only one beacon (B) of the laser tracking
`system is shown. In reality, the laser tracker has two beacons,
`but the position is reported with respect to a coordinate system
`associated with one of the beacons, so in order to simplify the
`figure, only this beacon is shown. The notations employed in
`Fig. 1 are summarized in Table I.
`In Fig. 1, only one tracked component (denoted Ri) is shown,
`for the purpose of clarity. Our hybrid tracker can track as many
`components as the magnetic tracker allows (up to 40 targets).
`The position of the tracked component Ri, w.r.t. WCS, is rep-
`resented by the vector
`, and w.r.t. the magnetic
`transmitter is given by
`As can be seen
`from Fig. 1,
`and
`can be related by the following equation:
`
`(1)
`
`is measured by the receiver Ri w.r.t. the mag-
`The vector
`netic transmitter. So (1) is the basic equation for tracking a
`component within WCS. For tracking the object globally (as
`a whole), only PT is used, therefore the magnetic tracker is
`
`not needed (unless the LOS constraint is violated). The hybrid
`tracker described here can track an unlimited number of moving
`objects. For each object, a distinct set of PT’s and a separate
`magnetic tracker is needed. The examples provided in this paper
`consider only a single tracked object, without loss of any gen-
`erality.
`
`A. Hybrid Tracker Precalibration
`
`In order to compute the position of a tracked component with
`respect to WCS, one needs the transformation between PT co-
`ordinate system and the coordinate system associated with the
`magnetic receiver that is rigidly attached to the PT’s (labeled
`RT in Fig. 1). This transformation is labeled
`in Fig. 1 and
`is invariant as the PT-RT ensemble moves. In order to compute
`, precalibration of the hybrid tracker is performed before
`starting the motion tracking process. The geometry associated
`with precalibration is depicted in Fig. 2.
`The notations used in Fig. 2 are summarized in Table II, for
`a generic case, as well as for two applications that demonstrate
`the use of our hybrid tracker (human motion capture and camera
`tracking for object reconstruction—applications described in
`Sections IV and V, respectively). In Fig. 2, the coordinate
`transformations and coordinate systems specific only to (or
`at least closely related to) camera tracking are written with a
`different font and are represented by dashed lines.
`Hybrid tracker precalibration is performed as follows. The
`tracked object is placed in two arbitrary locations within the en-
`vironment (care must be taken so that no ferromagnetic objects
`are located in the neighborhood to ensure that magnetic readings
`are not distorted), from where readings from PT’s and RT are
`collected with the object stationary. The two consecutive posi-
`tions are denoted by indices
`and in Fig. 2. The relative trans-
`formations between positions
`and
`can be written as in (2)
`and (3) below (for both PT and RT).
`Let
`
`(2)
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1029
`META V. THALES
`
`

`

`284
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Fig. 2. Hybrid tracker precalibration setup.
`
`and let
`
`From Fig. 2, the following equation can be written:
`
`(3)
`
`(4)
`
`Equation (4) follows from the fact that the transformations
`involved form a closed loop. From (4), it follows that
`
`(5)
`
`is computed. Equation (5) is an equation of the
`from which
`form AX = XB, typically encountered in hand-eye calibration in
`robotics applications. To solve (5), we use the method proposed
`in [29].
`
`B. Violation of the LOS Constraint
`In order to track all components with respect to WCS, the
`transformation between beacon (B) and PT coordinate system
`has to be known and is given by the laser tracker. In order to
`recover this transformation, all three PT’s have to be visible at
`any time by both beacons of the laser tracker. In tracking mo-
`bile robots on the factory floor by using active beacon systems,
`usually beacons are placed at optimal locations throughout the
`environment [7]. This can be easily done when the paths are pre-
`defined or are expected to take place in well-known areas, but
`also increases the cost of tracking systems.
`When a tracker is used to capture unpredictable motion (such
`as human motion), one cannot design a priori an optimal con-
`figuration of beacons to prevent violation of the LOS constraint.
`To get around this problem, we can use the magnetic tracker
`(specifically the receiver attached to PT’s - RT in Fig. 1) to
`back-up the system when the LOS of the laser tracker is tem-
`porarily occluded. Consider again Fig. 2. Let us assume that the
`tracked component moves from position
`to position . In po-
`sition , all three PT’s are visible, and therefore the transforma-
`tion
`is correctly reported. In position , at least one PT is
`occluded. In this case, the transformation
`can be recov-
`ered from the previous estimate of the PT’s position and orien-
`
`tation
`and the relative motion undertaken by magnetic
`receiver RT (denoted as
`), by the following equation:
`
`(6)
`
`is measured w.r.t. the magnetic transmitter.
`In (6),
`Equation (6) is valid when the magnetic transmitter remains
`fixed relative to WCS or its motion w.r.t. WCS is negligible
`by comparison of RT motion w.r.t. WCS. For example, when
`tracking human motion, the magnetic transmitter is placed on
`the back of the human and RT on the user’s head. When the
`human operator bends (and thus PT’s are not visible from the
`beacons), the magnetic transmitter remains relatively fixed.
`The potential violation of this assumption is considered while
`designing the Kalman filter that deals with LOS constraint
`violations (described in Section IV), by scaling up the mea-
`surement noise uncertainty.
`
`C. Operating the Hybrid Tracker
`When retrieving position and orientation information by
`fusing data provided by two or more sensors, typically the
`assumption that measurements are available simultaneously
`from all sensors is made. In reality, this is almost never the
`case, due to different update rates of the various sensors. In our
`case, measurements from the laser and magnetic trackers are
`fed to a 300-MHz Pentium PC via serial cables and from there
`to an SGI workstation that performs all the calculations for
`position and orientation estimates. Since tracking is initiated
`only when the first data packet arrives from both sensors,
`communication overhead is not relevant for the time increment
`between two consecutive measurements. The interval between
`two measurements of the laser tracker is 22 ms. When using a
`single receiver, the update rate of the magnetic tracker is 5 [ms]
`and increases by the same amount as a new receiver is added.
`The temporal diagram shown in Fig. 3 depicts the succession
`of measurement packets as those arrive to the SGI workstation
`when using three receivers of the magnetic tracker (in this case
`the update rate is 15 [ms]).
`As can be seen from Fig. 3, the measurements arriving from
`the laser and the magnetic trackers are not synchronous. This
`introduces an error in estimating the true position of a tracked
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1029
`META V. THALES
`
`

`

`ZETU et al.: EXTENDED-RANGE HYBRID TRACKER AND APPLICATIONS TO MOTION AND CAMERA TRACKING
`
`285
`
`TABLE II
`EXPLANATION OF NOTATIONS IN FIG. 2
`
`Fig. 3. Temporal diagram of the hybrid tracker measurements.
`
`component, since it is not possible to collect a measurement
`from both trackers at exactly the same moment in time. The fact
`that there is no constant offset between readings complicates the
`problem. Due to the small temporal difference between mea-
`surements collected from the two sensors and due to the fact
`that the expected number of magnetic sensors typically used in
`
`our applications is between 2–6, the errors are not expected to
`be significant in comparison to the errors inflicted by the noise
`in the measurements. Consider the case shown in Fig. 3. In the
`current stage of our hybrid tracker, if in between two successive
`readings from the laser tracker there is only one reading from
`the magnetic tracker, this one is considered in the calculations.
`If two or more readings appear, these are first averaged to obtain
`a more realistic estimate.
`The position of a tracked component is estimated through a
`Kalman filter [19]. When using Kalman filters in tracking appli-
`cations, the following steps have to be performed before process
`initiation [8], [24]:
`• identification of state variables and measurement param-
`eters;
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1029
`META V. THALES
`
`

`

`286
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Fig. 4. Generic methodology for motion tracking.
`
`• choice of a dynamic model (the dynamic model depends
`on the particular application and on the nature of motion
`being captured);
`• model the process and measurement noise;
`• initialize state variables and error covariance.
`The steps mentioned above are the same for time-varying sys-
`tems (measurements are collected when the tracked object is
`in motion) and time-invariant systems (measurements are col-
`lected when the tracked object is stationary). Our hybrid tracker
`can be used for both types of systems.
`In our case, two Kalman filters are alternately used, de-
`pending on whether or not the LOS constraint of the laser
`tracker is violated. When there is occlusion between the beacon
`and any PT, the laser tracker stops sending data to the interface
`module (each PT has its own interface module in order to
`increase the update rate). This case is tested by monitoring the
`time interval
`elapsed from previous measurement. If
`exceeds 26 [ms], the LOS constraint is considered violated
`(recall that the update rate of the laser tracker is 22 [ms] and
`we allow 4 [ms] for possible communication glitches). In this
`case, the system switches to the a