IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`281
`
`Extended-Range Hybrid Tracker and Applications
`to Motion and Camera Tracking in
`Manufacturing Systems
`
`Dan Zetu, Pat Banerjee, and Darren Thompson
`
`Abstract—Extended- or long-range tracking effectiveness is cru-
`cial for the automation of manufacturing systems. In this paper, we
`conceptualize and develop a prototype long-range hybrid tracker
`based on a combination of a laser tracker and a magnetic tracker
`and apply the concept to the following two applications: 1) ex-
`tended-range human motion tracking on factory floors and 2) fac-
`tory floor object reconstruction from camera images. The easily
`portable system not only utilizes the strengths of a laser tracker
`in tracking mobile objects over long ranges in large environments,
`such as a manufacturing shop floor and the strength of a magnetic
`tracker to compensate for violation of line-of-sight constraint, but
`it also reduces the overall cost by reducing the number of expensive
`beacons required by the laser tracker. The hybrid tracker assists
`in the development of two concepts: 1) real-time synchronization
`of human head and hand motion in a manufacturing environment
`with those of an avatar in a virtual manufacturing environment
`and 2) a mathematically simpler and practical camera self-cali-
`bration technique for the creation of three-dimensional objects in
`a virtual environment from camera images.
`
`tracking, magnetic
`laser
`Index Terms—Hybrid tracker,
`tracking, stereo reconstruction, virtual reality applications.
`
`I. INTRODUCTION
`
`A UTONOMOUS navigation of mobile robots and material
`
`handling equipment (such as automated guided vehicles or
`forklifts) is often a prerequisite for automation of manufacturing
`systems. In order to achieve autonomous navigation of such
`components of manufacturing systems, they need to know at any
`time where they are located within the environment, with respect
`to a global coordinate system [20]. Similarly, in virtual reality
`(VR)-aided manufacturing systems design and maintenance ap-
`plications, it is necessary to capture the motion of human par-
`ticipants in order to replicate it on avatars within virtual envi-
`ronments (VE’s) representing specific manufacturing systems.
`This motion has to be often captured over a longer range than
`the ranges of current tracking systems for VR applications. A
`survey of the existing position trackers used in VR can be found
`in [12] and [25].
`Currently, extended-range trackers are employed for tracking
`mobile robots [7], [23], [13]. The most widely used long-range
`tracking systems in robotics are active beacon systems. The
`biggest disadvantage of active beacon navigation systems is the
`
`Manuscript received April 26, 1999; revised December 15, 1999. This paper
`was recommended for publication by Editor P. Luh upon evaluation of the re-
`viewers’ comments. This research was supported in part by the National Science
`Foundation under Grant DMI 9500396.
`The authors are with Department of Mechanical Engineering, University of
`Illinois at Chicago, Chicago, IL 60607 USA.
`Publisher Item Identifier S 1042-296X(00)04761-3.
`
`line-of-sight constraint (LOS) [7]. There may be instances when
`tracking a certain part (such as the end-effector) of a robot is nec-
`essary, and this part cannot be visible to the tracking system, un-
`less multiple beacons are placed within the motion environment,
`thus increasing substantially the cost of the tracking system.
`In VR applications, the task of a tracker is to report the posi-
`tion and orientation of a user’s head and hand. Accordingly, the
`VR system updates the perspective display to make it consistent
`with the user’s viewpoint. There are multiple types of tracking
`systems used in VR: magnetic, optical, mechanical, acoustic,
`and inertial. Each of these trackers has advantages and disad-
`vantages. For example, magnetic trackers have no LOS con-
`straints, but their accuracy decreases dramatically with increase
`in distance from transmitter and is also influenced by metallic
`objects in the neighborhood. Optical trackers are very fast and
`accurate, and they are also immune to magnetic interference,
`but their use is restricted by the LOS constraint. Mechanical
`trackers, based on linkages, are very accurate, but their work
`is severely restricted within a small-range volume (determined
`by the geometry of linkages). Acoustic (ultrasonic) trackers are
`relatively cheap and accurate, but they also have limited range
`and LOS restriction.
`The above considerations suggest combining some of the ad-
`vantages offered by individual tracking systems to design a hy-
`brid tracker for autonomous navigation in real manufacturing
`environments and human motion in VE’s. In this paper, a hy-
`brid tracker, based on a combination between a laser tracker and
`a magnetic tracker, is described. The laser tracker used is an ac-
`tive beacon system1 [36], and the magnetic tracker employed
`is called MotionStar2 [37]. The laser tracker has the advantage
`of enabling accurate tracking of position and orientation over
`long ranges (the system we use has a maximum range of 100 m,
`but through serialization of multiple such systems, unlimited
`range can be obtained). The magnetic tracker enables tracking
`of multiple parts of an object without problems due to occlu-
`sion. Moreover, when the LOS constraint for the laser tracker is
`temporarily violated, the magnetic tracker can compensate for
`it. This is an important advantage since it reduces the number of
`beacons or landmarks that have to be placed within the environ-
`ment, thus reducing substantially the cost of tracking systems.
`This paper is organized as follows. Related work in hybrid
`tracking is discussed in Section II. Section III describes the
`geometry of the hybrid (laser and magnetic) tracking system.
`
`1CONAC by MTI Research, Westford, MA 01886 USA.
`2Manufactured by Ascension Technologies Corporation, Burlington, VT
`05402 USA.
`
`1042–296X/00$10.00 © 2000 IEEE
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`282
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Section IV describes the application of the proposed hybrid
`tracker to motion tracking during real-time synchronization
`of real environments and VE’s. Section V covers three-di-
`mensional (3-D) object reconstruction from camera images
`and describes our camera self-calibration procedure using
`extended-range tracking. Section VI is devoted to conclusions
`and future directions.
`
`II. RELATED WORK
`
`Due to their contribution to the end-to-end latency of VR
`systems, tracking systems have received a great deal of atten-
`tion among VR research community. Hybrid tracking has been
`explored mostly in the area of augmented reality (AR), where
`accurate registration between real environment and virtual ob-
`jects superimposed on it is critical. In [2], the need for hybrid
`tracking in AR is stressed, especially for outdoors applications.
`Most of the tracking in AR is being performed with the aid of
`video cameras, by tracking fiducial marks (placed at known lo-
`cations in the environment) using computer vision techniques.
`Despite the accuracy of these techniques, they are slow (due
`to the necessity of searching the marks by scanning the image
`pixel by pixel), their range is limited by the placement of the
`fiducial marks and are not robust to occlusions of the fiducial
`marks. In [26], a hybrid tracker is described, which combines
`computer vision-based tracking with inertial tracking. Since it is
`well known that inertial trackers exhibit drift with time (their er-
`rors increase over time [3]), their output is corrected by using vi-
`sion-based tracking. Another hybrid system has been proposed
`in [17], where it has been proven that by combining two types of
`vision-based tracking, called “inside-out” (camera(s) mounted
`on the head of the user and fiducials mounted at known loca-
`tions in the environment) and “outside-in” (cameras mounted at
`known locations in the environment and fiducials mounted on
`the user’s head), the uncertainty in head pose (position and ori-
`entation) estimation is considerably decreased. In [28], the ac-
`curacy of a magnetic tracker is improved by augmenting it with
`a passive image-based system that observes known fiduciary
`marks in the real world. At the same time, the magnetic tracker
`measurements help in reducing the search area of the fiducials
`in two-dimensional (2-D) images captured by head-mounted
`cameras, thus reducing the latency of the hybrid tracker. Other
`hybrid systems have been previously proposed in [5], [9], and
`[14]. For example, in [5] and [9], combinations between iner-
`tial and optical technologies are described in terms of accuracy
`and end-to-end latency. In [14], an inertial system is aided by
`angular position sensors. None of these applications address the
`problem of tracking motion in large environments, such as a fac-
`tory floor. For this kind of application, active beacon systems are
`very suitable, due to their accuracy and extended range but, due
`to the LOS constraint, usually a large number of beacons has to
`be mounted on the factory floor. We overcome this disadvantage
`by using a magnetic tracker in combination with a laser tracker.
`
`III. DESCRIPTION OF THE HYBRID TRACKING SYSTEM AND
`GENERIC METHODOLOGY FOR MOTION TRACKING
`
`As mentioned in Sections I and II, our hybrid tracker for mo-
`tion tracking is a combination of a laser tracker and a magnetic
`
`one. The laser tracker provides high accuracy and update rate
`for high ranges (0–100 m), but its use is restricted by the LOS
`constraint. On the other hand, the magnetic tracker does not re-
`quire LOS, but it is accurate only within small working vol-
`umes. The laser tracker is based on triangulation of laser signals
`emitted by two beacons and received by one or more position
`transponders (PT’s), attached to the moving object. One PT can
`report only the position with respect to one beacon so, in order
`to retrieve the orientation, one has to employ three PT’s rigidly
`mounted on a special fixture. The advantage of using a mag-
`netic tracker is its suitability to applications with frequent oc-
`clusions between transmitter and receiver. The magnetic tracker
`employed in our tracker uses pulsed direct current (dc) magnetic
`fields instead of alternate current (ac) magnetic fields (which are
`being used by Polhemus, Inc. magnetic trackers and older ver-
`sions of Ascension Technologies trackers). DC fields are signifi-
`cantly less susceptible to metallic distortion than ac fields. How-
`ever, dc-based magnetic trackers are susceptible to interference
`with magnetic fields generated by ferromagnetic objects (such
`as computer monitors or dc motors, see [27]). Even though it
`is hard to estimate up front the probability of encountering such
`objects during a motion sequence, it is reasonable to assume that
`in most cases the wearer of a magnetic tracker will not be in the
`immediate proximity of ferromagnetic objects that would cat-
`astrophically affect the tracker’s output. Overall, by weighing
`its advantages, the dc-based magnetic tracker remains a reliable
`magnetic tracker for motion capture in manufacturing environ-
`ments. By using a Kalman filter [19] to minimize the external
`effects on its performance, reasonable results can be obtained,
`as will be seen later in this paper. The advantages of incorpo-
`rating a magnetic tracker into our hybrid tracker are as follows.
`It can track multiple targets without worrying about occlu-
`sions between transmitter and receivers.
`Since its behavior is not influenced by an LOS constraint,
`the magnetic tracker can be used as a backup, when the LOS
`between laser tracker’s beacons and PT’s is temporarily
`occluded. This enables reduction in the number of beacons
`(landmarks) used for motion tracking, thus reducing the cost
`of the tracking system. Details are provided below.
`Typically, magnetic tracker’s receivers are placed on compo-
`nents whose motion trajectories have to be captured and cannot
`be “seen” all the time by beacons of the laser tracker. The PT’s
`of the laser tracker are mounted on the moving objects, in a loca-
`tion that is always visible to the transmitting beacons. The posi-
`tions of the tracked components (i.e., the components equipped
`with a magnetic receiver) are reported either with respect to a
`beacon’s coordinate system or to a coordinate system attached
`to the motion environment, termed world coordinate system
`(WCS) (in this case, the transformation between the WCS and
`beacon coordinate systems is known a priori). For this purpose,
`one of the magnetic receivers is rigidly attached to the PT’s, so
`the transformation between this receiver and PT’s is invariant
`as the tracked object moves. The magnetic transmitter is also
`placed on the moving object.
`The described hybrid tracker has the advantage of being
`easily portable, unlike other trackers currently in use, such as
`UNC HiBall [31], which requires a large number of beacons
`(LED’s) mounted on the ceiling and whose range is limited
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`ZETU et al.: EXTENDED-RANGE HYBRID TRACKER AND APPLICATIONS TO MOTION AND CAMERA TRACKING
`
`283
`
`TABLE I
`EXPLANATION OF NOTATIONS IN FIG. 1
`
`Fig. 1. Geometry of the hybrid tracker.
`
`by the number of such beacons. The tracker described in [31]
`has the advantage that, by mounting a large number of closely
`located LED’s on the ceiling, one will have less problems with
`LOS but, in order to increase the tracking range, the cost of
`the system rises significantly, and the system becomes less
`portable. By being equipped with only two stationary beacons
`(at least for now), our tracking system is expected to be more
`susceptible to LOS problems. By mounting the beacons in
`optimal locations (to minimize the likelihood of violating the
`LOS constraint) and by using also the magnetic tracker to
`aid the laser tracker temporarily (when the LOS constraint
`is violated), we expect the impact of LOS problems to be
`minimal. The following two questions arise. 1) How accurate
`are the tracker outputs when the magnetic tracker aids in cir-
`cumventing LOS problems? 2) For how long can the violation
`of the LOS constraint be tolerated so that the position estimates
`fall within acceptable accuracy limits? These questions will be
`addressed through an example in Section IV.
`The generic geometry of our hybrid tracker is depicted in
`Fig. 1. In this figure, the case when the positions of the tracked
`components are reported with respect to a WCS is illustrated.
`Note that in Fig. 1, only one beacon (B) of the laser tracking
`system is shown. In reality, the laser tracker has two beacons,
`but the position is reported with respect to a coordinate system
`associated with one of the beacons, so in order to simplify the
`figure, only this beacon is shown. The notations employed in
`Fig. 1 are summarized in Table I.
`In Fig. 1, only one tracked component (denoted Ri) is shown,
`for the purpose of clarity. Our hybrid tracker can track as many
`components as the magnetic tracker allows (up to 40 targets).
`The position of the tracked component Ri, w.r.t. WCS, is rep-
`resented by the vector
`, and w.r.t. the magnetic
`transmitter is given by
`As can be seen
`from Fig. 1,
`and
`can be related by the following equation:
`
`(1)
`
`is measured by the receiver Ri w.r.t. the mag-
`The vector
`netic transmitter. So (1) is the basic equation for tracking a
`component within WCS. For tracking the object globally (as
`a whole), only PT is used, therefore the magnetic tracker is
`
`not needed (unless the LOS constraint is violated). The hybrid
`tracker described here can track an unlimited number of moving
`objects. For each object, a distinct set of PT’s and a separate
`magnetic tracker is needed. The examples provided in this paper
`consider only a single tracked object, without loss of any gen-
`erality.
`
`A. Hybrid Tracker Precalibration
`
`In order to compute the position of a tracked component with
`respect to WCS, one needs the transformation between PT co-
`ordinate system and the coordinate system associated with the
`magnetic receiver that is rigidly attached to the PT’s (labeled
`RT in Fig. 1). This transformation is labeled
`in Fig. 1 and
`is invariant as the PT-RT ensemble moves. In order to compute
`, precalibration of the hybrid tracker is performed before
`starting the motion tracking process. The geometry associated
`with precalibration is depicted in Fig. 2.
`The notations used in Fig. 2 are summarized in Table II, for
`a generic case, as well as for two applications that demonstrate
`the use of our hybrid tracker (human motion capture and camera
`tracking for object reconstruction—applications described in
`Sections IV and V, respectively). In Fig. 2, the coordinate
`transformations and coordinate systems specific only to (or
`at least closely related to) camera tracking are written with a
`different font and are represented by dashed lines.
`Hybrid tracker precalibration is performed as follows. The
`tracked object is placed in two arbitrary locations within the en-
`vironment (care must be taken so that no ferromagnetic objects
`are located in the neighborhood to ensure that magnetic readings
`are not distorted), from where readings from PT’s and RT are
`collected with the object stationary. The two consecutive posi-
`tions are denoted by indices
`and in Fig. 2. The relative trans-
`formations between positions
`and
`can be written as in (2)
`and (3) below (for both PT and RT).
`Let
`
`(2)
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`284
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Fig. 2. Hybrid tracker precalibration setup.
`
`and let
`
`From Fig. 2, the following equation can be written:
`
`(3)
`
`(4)
`
`Equation (4) follows from the fact that the transformations
`involved form a closed loop. From (4), it follows that
`
`(5)
`
`is computed. Equation (5) is an equation of the
`from which
`form AX = XB, typically encountered in hand-eye calibration in
`robotics applications. To solve (5), we use the method proposed
`in [29].
`
`B. Violation of the LOS Constraint
`In order to track all components with respect to WCS, the
`transformation between beacon (B) and PT coordinate system
`has to be known and is given by the laser tracker. In order to
`recover this transformation, all three PT’s have to be visible at
`any time by both beacons of the laser tracker. In tracking mo-
`bile robots on the factory floor by using active beacon systems,
`usually beacons are placed at optimal locations throughout the
`environment [7]. This can be easily done when the paths are pre-
`defined or are expected to take place in well-known areas, but
`also increases the cost of tracking systems.
`When a tracker is used to capture unpredictable motion (such
`as human motion), one cannot design a priori an optimal con-
`figuration of beacons to prevent violation of the LOS constraint.
`To get around this problem, we can use the magnetic tracker
`(specifically the receiver attached to PT’s - RT in Fig. 1) to
`back-up the system when the LOS of the laser tracker is tem-
`porarily occluded. Consider again Fig. 2. Let us assume that the
`tracked component moves from position
`to position . In po-
`sition , all three PT’s are visible, and therefore the transforma-
`tion
`is correctly reported. In position , at least one PT is
`occluded. In this case, the transformation
`can be recov-
`ered from the previous estimate of the PT’s position and orien-
`
`tation
`and the relative motion undertaken by magnetic
`receiver RT (denoted as
`), by the following equation:
`
`(6)
`
`is measured w.r.t. the magnetic transmitter.
`In (6),
`Equation (6) is valid when the magnetic transmitter remains
`fixed relative to WCS or its motion w.r.t. WCS is negligible
`by comparison of RT motion w.r.t. WCS. For example, when
`tracking human motion, the magnetic transmitter is placed on
`the back of the human and RT on the user’s head. When the
`human operator bends (and thus PT’s are not visible from the
`beacons), the magnetic transmitter remains relatively fixed.
`The potential violation of this assumption is considered while
`designing the Kalman filter that deals with LOS constraint
`violations (described in Section IV), by scaling up the mea-
`surement noise uncertainty.
`
`C. Operating the Hybrid Tracker
`When retrieving position and orientation information by
`fusing data provided by two or more sensors, typically the
`assumption that measurements are available simultaneously
`from all sensors is made. In reality, this is almost never the
`case, due to different update rates of the various sensors. In our
`case, measurements from the laser and magnetic trackers are
`fed to a 300-MHz Pentium PC via serial cables and from there
`to an SGI workstation that performs all the calculations for
`position and orientation estimates. Since tracking is initiated
`only when the first data packet arrives from both sensors,
`communication overhead is not relevant for the time increment
`between two consecutive measurements. The interval between
`two measurements of the laser tracker is 22 ms. When using a
`single receiver, the update rate of the magnetic tracker is 5 [ms]
`and increases by the same amount as a new receiver is added.
`The temporal diagram shown in Fig. 3 depicts the succession
`of measurement packets as those arrive to the SGI workstation
`when using three receivers of the magnetic tracker (in this case
`the update rate is 15 [ms]).
`As can be seen from Fig. 3, the measurements arriving from
`the laser and the magnetic trackers are not synchronous. This
`introduces an error in estimating the true position of a tracked
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`ZETU et al.: EXTENDED-RANGE HYBRID TRACKER AND APPLICATIONS TO MOTION AND CAMERA TRACKING
`
`285
`
`TABLE II
`EXPLANATION OF NOTATIONS IN FIG. 2
`
`Fig. 3. Temporal diagram of the hybrid tracker measurements.
`
`component, since it is not possible to collect a measurement
`from both trackers at exactly the same moment in time. The fact
`that there is no constant offset between readings complicates the
`problem. Due to the small temporal difference between mea-
`surements collected from the two sensors and due to the fact
`that the expected number of magnetic sensors typically used in
`
`our applications is between 2–6, the errors are not expected to
`be significant in comparison to the errors inflicted by the noise
`in the measurements. Consider the case shown in Fig. 3. In the
`current stage of our hybrid tracker, if in between two successive
`readings from the laser tracker there is only one reading from
`the magnetic tracker, this one is considered in the calculations.
`If two or more readings appear, these are first averaged to obtain
`a more realistic estimate.
`The position of a tracked component is estimated through a
`Kalman filter [19]. When using Kalman filters in tracking appli-
`cations, the following steps have to be performed before process
`initiation [8], [24]:
`• identification of state variables and measurement param-
`eters;
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`286
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`Fig. 4. Generic methodology for motion tracking.
`
`• choice of a dynamic model (the dynamic model depends
`on the particular application and on the nature of motion
`being captured);
`• model the process and measurement noise;
`• initialize state variables and error covariance.
`The steps mentioned above are the same for time-varying sys-
`tems (measurements are collected when the tracked object is
`in motion) and time-invariant systems (measurements are col-
`lected when the tracked object is stationary). Our hybrid tracker
`can be used for both types of systems.
`In our case, two Kalman filters are alternately used, de-
`pending on whether or not the LOS constraint of the laser
`tracker is violated. When there is occlusion between the beacon
`and any PT, the laser tracker stops sending data to the interface
`module (each PT has its own interface module in order to
`increase the update rate). This case is tested by monitoring the
`time interval
`elapsed from previous measurement. If
`exceeds 26 [ms], the LOS constraint is considered violated
`(recall that the update rate of the laser tracker is 22 [ms] and
`we allow 4 [ms] for possible communication glitches). In this
`case, the system switches to the alternate Kalman filter that
`uses the same state and measurement models, but has a larger
`initial error covariance and different measurement noise model
`due to the fact that the accuracy of the magnetic receiver RT
`(that backs up the laser tracker) is expected to be lower than the
`one of the laser tracker. When the LOS between beacons and
`PT’s is free, the laser tracker starts outputting measurements
`automatically and a switch to the regular Kalman filter is per-
`formed. Filtering is resumed with the predicted state variables
`and error covariance given by the back-up filter, instead of the
`same values before occlusion of the LOS (we found out that this
`approach is more appropriate because the motion estimation is
`smoother and the amount of jitter is reduced). The only change
`is that the direct laser tracker measurement is used instead of
`(6). The generic methodology of operating the hybrid tracker
`when capturing motion can be summarized as in the diagram
`shown in Fig. 4.
`
`IV. APPLICATION TO HUMAN MOTION
`
`Human motion is an example of using the hybrid tracker
`with time-varying systems. Capturing human motion in man-
`ufacturing environments in order to be replicated in VE’s is a
`challenging task. In VR applications, typically head and hand
`of a user are tracked in order to update the perspective. In order
`to achieve realistic human motion in VE’s, more components of
`a human body have to be tracked (such as torso and joints). In
`order to illustrate the application of our hybrid tracker to human
`motion capture, we limit ourselves to tracking only the head and
`hand of a human on a factory floor, replicated by an avatar in a
`VE representing the real factory floor. VE is a priori registered
`with the real environment.
`The three PT’s of the laser tracker and one receiver of the
`magnetic tracker (RT in Fig. 1) are rigidly mounted on a fixture
`with the shape of a hat, mounted on the user’s head. The mag-
`netic transmitter is placed in a backpack, located on the back
`of the user or, when motion takes places within a small volume
`(but at a large distance from a reference point, thus requiring
`laser tracking as well), it can be placed in a fixed position, close
`to the human operator.
`The user’s hand is tracked by means of a magnetic receiver,
`attached to the wrist, based on which the hand position with
`respect to WCS can be computed. The laser tracker gives the
`head position.
`The measurements performed by the tracker over time are
`noisy. It is reasonable to assume [8], [32], [4] that the process
`of position estimation is driven by normally distributed noise. In
`order to optimally estimate the coordinates of hand position with
`respect to WCS in a noisy environment, we design a Kalman
`filter, as mentioned in Section III. The precise dynamic model
`of hand and head motion is unknown, but the Kalman filter can
`provide very good results even for this kind of application [32].
`Also, the Kalman filter has been found ([4]) to perform well
`even when the assumptions of normal distribution of noise rep-
`resenting the uncertainties in the measurements in the model are
`violated.
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`ZETU et al.: EXTENDED-RANGE HYBRID TRACKER AND APPLICATIONS TO MOTION AND CAMERA TRACKING
`
`287
`
`Fig. 5. Kalman filter predictions (x and y directions of motion) versus real tracker output. First sequence.
`
`Fig. 6. Kalman filter predictions (x and y directions of motion) versus real tracker output. Second sequence.
`
`The use of a Kalman filter for motion capture requires a mo-
`tion model. Unfortunately, it is almost impossible to obtain an
`accurate model for the hand and/or head motion. To get around
`this problem, researchers have used different models to approx-
`imate head and/or hand motion. In [32], the position-velocity
`model (defined in [8]) has been used, which assumes motion
`takes place at constant velocity and models acceleration as white
`noise. In [21], it is assumed that head rotations are infrequent
`and that angular speed and angular acceleration are nonzero
`only during infrequent change in viewing direction. These as-
`sumptions led to the choice of an integrated Gauss–Markov
`process to model the head movement. In [15], a hand motion
`model with constant acceleration has been used. All these ap-
`proximations provide satisfactory results, with occasional over-
`shoot when sudden change of direction or velocity occurs. We
`have used the acceleration model [15], [6] to approximate hand
`and head motion. The Kalman filter for hand and head tracking
`is briefly described in Appendix A.
`In order to determine the performance of the hybrid tracker,
`the magnetic receiver that records the hand position is posi-
`tioned initially at some known world locations, in order to deter-
`mine the process and measurement noise covariance matrices
`and
`, respectively. Measurements are collected and the filter
`is run offline. The error is the difference between the estimated
`
`and actual positions of the tracker. A cost function is defined
`as the sum of the squared errors at each time step. Through the
`minimization of the cost function, matrices
`and
`are com-
`puted. The Kalman filter error covariance matrix
`is assumed
`to be diagonal. The diagonal elements of
`are initialized to
`some large values (2 for the elements corresponding to posi-
`tion, 50 for velocity, and 60 for acceleration). The elements cor-
`responding to velocity and acceleration are initialized to higher
`values than the ones corresponding to position because motion
`tracking starts with the user being stationary.
`When tracking human motion, one does not have available
`ground truth data, since it is impossible to predict exactly the
`movement path. To assess the accuracy and consistency of our
`Kalman filter model, we have captured motion sequences with
`frequent changes of direction and monitor the difference be-
`tween Kalman filter predictions and real tracker measurements.
`Metallic objects were present close to the movement path to il-
`lustrate a relatively insignificant impact on our model. The re-
`sults are shown in Figs. 5 and 6.
`Figs. 5 and 6 show that there is a reasonable consistency
`between Kalman filter predictions and real tracker measure-
`ments, with slight overshoot or undershoot when a sudden
`change of direction occurs. The type of motion depicted in
`Figs. 5 and 6 takes place in very unfavorable conditions for
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on December 29,2021 at 14:38:18 UTC from IEEE Xplore. Restrictions apply.
`
`META 1026
`META V. THALES
`
`

`

`288
`
`IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 16, NO. 3, JUNE 2000
`
`our motion model. Typically, we do not expect such frequent
`changes of direction, and therefore the overshoot or under-
`shoot will be reduced. We performed multiple experiments
`and all provided similar results. The maximum overshoot
`encountered was about 7 [cm]. Fig. 6 depicts a special case,
`when the laser tracker is occluded for approximately 9 s. This
`occlusion happens to coincide with a change in direction of
`motion. When such situations occur, the measurement noise
`level
`is scaled up to reflect
`the additional uncertainty in
`position estimation when using the relative motion of the
`magnetic tracker [(6)] instead of the laser tracker [(1)]. As
`can be seen, even though it is still at an acceptable level, the
`amount of overshoot in this case is larger than the typical
`overshoot when changing direction of motion, as shown in
`Fig. 5.
`The results presented in this section show the consistency
`of our Kalman filter and show that the motion model we have
`chosen gives sufficiently accurate results even in unfavorable
`cases encountered in human motion. Also, as was shown in
`Fi