`Foxlin
`
`54) INERTIAL ORIENTATION TRACKER
`APPARATUS METHOD HAVING
`AUTOMATIC DRIFT COMPENSATION FOR
`TRACKING HUMAN HEAD AND OTHER
`SIMILARLY SIZED BODY
`
`75 Inventor: Eric M. Foxlin, Cambridge, Mass.
`73 Assignee: Massachusetts Institute of
`Technology, Cambridge, Mass.
`
`21 Appl. No.: 882,650
`22 Filed:
`Jun. 25, 1997
`Related U.S. Application Data
`
`62 Division of Ser. No. 261,364, Jun. 16, 1994, Pat. No.
`5,645,077.
`(51) Int. Cl. ............................................... A61B 5/103
`52 U.S. C. ...
`600/595; 600/587; 128/897
`58 Field of Search ..................................... 600/595, 587,
`600/27,592, 594; 128/898; 73/488, 510;
`364/453
`
`56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`1/1974 Horner et al..
`3,786,458
`4,197.855 4/1980 Lewin.
`4,800,897
`1/1989 Nolsson .................................. 600/595
`4,928,709 5/1990 Allison et al. .......................... 600/595
`5,181,181
`1/1993 Glynn.
`5,192.254 3/1993 Young ..................................... 500/595
`5,203,346 4/1993 Fuhr et al..
`5,373,857 12/1994 Travers et al..
`5,373,858 12/1994 Rose et al..
`5,425,750 6/1995 Moberg.
`5,474,088 12/1995 Zaharkin et al. ....................... 600/595
`5,513,651
`5/1996 Cusimano et al.
`... 600/595
`5,647.375
`7/1997 Farfan De Los Godos ........... 600/595
`OTHER PUBLICATIONS
`Watson Industries, Inc., Attitude and Heading Reference
`System #AHRS-C300A, Owner's Manual.
`Gyration, Incorporated: An Overview of the Company,
`Technology, and Products.
`
`USOO5807284A
`Patent Number:
`11
`(45) Date of Patent:
`
`5,807,284
`Sep. 15, 1998
`
`K. Meyer, H.L. Applewhite and F.A. Biocca, “A Survey of
`Position Trackers.” Presence, vol. 1, No. 22, Spring 1992,
`pp. 173–200.
`A. Lawrence, Modern Inertial Technology, Springer-Verlag,
`1992, pp. 1–23.
`F.J. Ferrin, Survey of Helmet Tracking Technologies, SPIE
`vol. 1456, Large-Screen-Projection, Avionic, and Helmet
`Mounted Displays, 1991, pp. 86-94.
`M. Friedman, T. Starner, A. Pentland, “Synchronization in
`Virtual Realities,” MIT Media Lab Vision and Modeling
`Group Technical Report No. 157, Jan. 1991.
`
`(List continued on next page.)
`
`Primary Examiner Richard J. Apley
`ASSistant Examiner Justine R. Yu
`Attorney, Agent, or Firm-Steven J. Weissburg
`57
`ABSTRACT
`A Self contained Sensor apparatus generates a Signal that
`corresponds to at least two of the three orientational aspects
`of yaw, pitch and roll of a human-Scale body, relative to an
`external reference frame. A Sensor generates first Sensor
`Signals that correspond to rotational accelerations or rates of
`the body about certain body axes. The sensor may be
`mounted to the body. Coupled to the Sensor is a signal
`processor for generating orientation signals relative to the
`external reference frame that correspond to the angular rate
`or acceleration signals. The first Sensor Signals are imper
`vious to interference from electromagnetic, acoustic, optical
`and mechanical Sources. The Sensors may be rate Sensors.
`An integrator may integrate the rate Signal over time. A drift
`compensator is coupled to the rate Sensors and the integrator.
`The drift compensator may include a gravitational tilt Sensor
`or a magnetic field Sensor or both. A verifier periodically
`measures the orientation of the body by a means different
`from the drift sensitive rate sensors. The verifier may take
`into account characteristic features of human motion, Such
`as Stillness periods. The drift compensator may be, in part,
`a Kalman filter, which may utilize Statistical data about
`human head motion.
`
`13 Claims, 14 Drawing Sheets
`
`12
`
`- -
`
`DRIFT
`SENSTIVE
`ANG AR
`RATE
`SENSORS
`
`
`
`Continuous
`
`orientation
`with drift
`
`NTEGRATOR
`(by Euler angles)
`
`ORF
`COMPENSATOR
`
`Continuous
`
`GREA
`
`COMPENSATING
`ANGUAR
`POSITION
`SENSORS
`
`
`
`
`
`
`
`I Orientation
`| without drift
`
`META 1022
`META V. THALES
`
`
`
`5,807,284
`Page 2
`
`OTHER PUBLICATIONS
`
`D. K. Bhatnagar, “Position Trackers for Head Mounted
`Display Systems: A Survey,” Mar. 29th, 1993.
`M. Koifman and S.J. Merhav, “Autonomously Aided Strap
`down Attitude Reference System,” Journal of Guidance and
`Control, vol. 14, No. 6, 1991, pp. 1164-1172.
`
`J. Liang, C. Shaw and M. Green, “On Temporal-Spatial
`Realism in the Virtual Reality Environment,” Proceedings of
`the ACM Symposium on User Interface Software and Tech
`nology, Nov. 13-11, 1991, pp. 19-25.
`U.H. List, “Nonlinear Prediction of Head Movements for
`Helmet-Mounted Displays.” Air Force Human Resources
`Laboratory, Technical Paper 83-45, Dec. 1983.
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 1 of 14
`
`5,807,284
`
`80||
`
`SHOSNES
`
`SHOSNES
`
`Z || ||
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 2 of 14
`
`5,807,284
`
`
`
`FIG. 2A
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 3 of 14
`
`5,807,284
`
`
`
`FIG. 2 B
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 4 of 14
`
`5,807,284
`
`
`
`Ø || 9
`
`
`
`HOSSE OOHg
`
`
`
`HE LHE ANOO
`
`C]/\?
`
`SHETIVOS
`
`SHOSNES
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet S of 14
`
`5,807,284
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`O BAIE OE}}
`
`OBEZ O 1 SET15DN\f
`
`
`
`HET][]E LESEH
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 6 of 14
`
`5,807,284
`
`BEGIN
`
`430
`
`PLACE SENSOR ASSEMBLY
`ON SUPPORT IN
`REFERENCE ORIENTATION
`
`MEASURE AND STORE
`ROUGH SENSORBASES
`
`NITALIZE EULER ANGLE
`VARABLES TO ZERO
`
`TERATE THROUGH MAN
`OPERATING STEPS USING
`ROUGH BASES
`
`
`
`
`
`
`
`
`
`432
`
`434
`
`436
`
`440
`
`yeS
`
`446
`
`DETERMINE RESIDUAL BASES BY DIVIDING
`EULER ANGLES BY NAND STORE BASES
`
`8
`44
`
`MEASURE AND STORE OUTPUT VALUES OF INCLINOMETER
`AND COMPASS FOR REFERENCE ORIENTATION
`
`END INITIALIZATION
`
`45O
`1.
`
`FIG. 4 B
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 7 of 14
`
`5,807,284
`
`46O
`
`462
`
`BEGIN
`
`GENERATE ANGULAR RATE SIGNALS IN BODY
`COOFRD WITH DRF SENSTIVE SENSORS
`
`464
`
`ERROR COMPENSATE
`
`466
`
`CONVERT TO ANGULAR RATES
`RELATIVE TO GROUND
`
`DETERMINE TIME SINCE LAST
`NCREMENT AND MULTIPLY BY ANGULAR
`RATES TO GENERATE INCREMENTAL
`ANGULAR DIFFERENCE SIGNALS
`
`468
`
`47O
`
`GENERATE UPDATED ANGLESIGNALS USING
`PREVIOUS SIGNALS AND NCREMENTS
`
`478
`
`MEASURE ANGLES
`RELATIVE TO GROUND
`USING COMPENSATING
`SENSORS
`
`ERROR COMPENSATE
`
`482
`
`COMPARE COMPENSATING
`SENSOR SIGNALS TO
`NTEGRATED DRFTING
`SIGNALS TO GENERATE
`ERROR SIGNALS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`472
`
`
`
`
`
`
`
`
`
`AS BODY BEE
`STILL LONG
`ENOUGHT
`
`REPLACE CONTENTS OF ERROR
`SIGNAL BUFFER WITH NEW
`ERROR SIGNALS
`
`474
`
`488
`
`GENERATE FRACTIONAL ERROR
`CORRECTION SIGNALS
`
`CHECK EULER ANGLES IN FRANGE
`
`490
`
`
`
`yes
`
`494
`
`486
`
`
`
`DECREMENT ERROR SIGNAL BUFFER
`BY FRACTIONAL ERROR SIGNAL
`476
`NCREMENT/DECREMENT UPDATED
`ANGLE SIGNALS BY FRACTIONAL
`ERROR SIGNAL
`- 492
`GO TO PARSE COMMANDS
`
`O
`
`OUTPUT ANGE SIGNALS
`
`496
`
`FIG 4 C
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 8 of 14
`
`5,807,284
`
`2
`
`PTCHERROR
`
`o
`L
`
`CD
`
`9 O
`
`-
`SP -
`2
`C
`
`-2
`O
`
`4O
`2O
`TIME (SECONDS)
`
`6O
`
`
`
`(?)
`ul
`L
`1.
`
`SB
`O
`
`-
`CD
`2
`C
`
`-3
`O
`
`ROLLERROR
`
`-
`4O
`2O
`TIME (SECONDS)
`
`6O
`
`FIG. A
`
`FIG. 5B
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 9 of 14
`
`5,807,284
`
`ORIGINAL NOSE
`
`(f)
`
`O.
`H
`or O
`(D
`
`C)
`-O.
`O
`
`5
`
`l
`O
`
`5
`SECONDS
`
`2O
`
`25
`
`3O
`
`FIG. 6A
`
`O. -
`
`LOW-FREQUENCY COMPONENT
`-
`
`O
`
`Of)
`
`I or
`
`CD
`
`C
`
`-O.
`
`O.
`
`
`
`(f)
`Lud
`U
`S O
`u
`C)
`-O.
`O
`
`5
`
`O
`
`5
`SECONDS
`
`2O
`
`25
`
`3O
`
`FIG. 6B
`
`NOSE WITH LOW FREQUENCY COMPONENT REMOVED
`--
`
`5
`
`O
`
`L
`5
`SECONDS
`
`2O
`
`25
`
`3O
`
`FIG. 6 C
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 10 of 14
`
`5,807,284
`
`I
`
`I
`
`5
`
`a o
`H
`2
`3 O 5
`is
`
`O
`O
`
`O
`
`2O
`
`5O
`4O
`3O
`FREQUENCY (HZ)
`
`FIG 7
`
`6O
`
`7O
`
`8O
`
`20O
`
`IOO
`
`O
`
`(?
`
`CD
`S
`
`CO
`C - IOO
`O
`
`-2OO
`O
`
`O
`
`2O
`
`1-
`5O
`4O
`3O
`FREQUENCY (Hz)
`
`FIG 7
`
`I
`
`III
`
`-
`
`6O
`
`I
`7O
`
`8O
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 11 of 14
`
`5,807,284
`
`—srO
` HOLOIGSudF—+-46||TA
`
`
`
`
`YVINDNY
`
`ALVY
`
`SYOSNSS
`
`gzel
`
`Zeek
`
`
`NOILYSNSdWODJSIHG
`
`YOLVSNAdWODONIGVSH
`
`cotl
`
`HOYHSpaidSixy2
`
`YHOLVSNAdWOD
`
`(SYHSLSWOYS1S00V
`
`HO)HSLSWONIIONI
`
`ELELaOLEl
`
`
`
`HOHHSHSHLO
`
`HOOILSNOVAW
`
`
`
`
`
`NVW1v'6'8)HOLVYWILSS
`
`(WIL
`
`YOSN3S
`
`META 1022
`META V. THALES
`
`META 1022
`META V. THALES
`
`
`
`
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 12 of 14
`
`5,807,284
`
`
`
`FIG 9
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 13 of 14
`
`5,807,284
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`O2
`
`SENSOR
`ASSEMBLY
`
`TRANSMTER
`UNIT
`
`1 O22
`
`-signal
`conditioning
`-anti-aliasing
`filters
`-6 channel A/D
`-encoder
`-RF transmitter
`
`BASE
`ELECTRONICS
`UNIT
`
`-RF receiver
`-decoder
`-processing
`(orientation
`angles)
`-RS 232 interface
`
`
`
`RS-232 Connection
`
`WE HOST
`COMPUTER
`
`O3O
`
`FIG. IO
`
`1 112
`
`6(t + At)
`1 22 (b (t + At)
`p? (t + At)
`
`ORIENTATION
`PREDICTOR
`
`HEAD
`ORIENTATION
`TRACKER
`
`
`
`
`
`POSITION
`PREDICTOR
`
`META 1022
`META V. THALES
`
`
`
`U.S. Patent
`
`Sep. 15, 1998
`
`Sheet 14 of 14
`
`5,807,284
`
`
`
`multiply by
`ar. -
`
`META 1022
`META V. THALES
`
`
`
`5,807,284
`
`1
`INERTIAL ORIENTATION TRACKER
`APPARATUS METHOD HAVING
`AUTOMATIC DRIFT COMPENSATION FOR
`TRACKING HUMAN HEAD AND OTHER
`SIMILARLY SIZED BODY
`
`This application is a divisional application of U.S. patent
`application Ser. No. 08/261,364, filed Jun. 16, 1994, now
`U.S. Pat. No. 5,645,077.
`
`GOVERNMENT RIGHTS
`The U.S. Government has certain rights in this invention
`pursuant to Contract No. AFOSR-90-0020-B, awarded by
`the Air Force Office Scientific Research and Contract No.
`NASA NCC 2-771, awarded by the National Aeronautics
`and Space Administration.
`BACKGROUND
`In connection with many aspects of man and machine
`interaction, it is important to track the motions of parts of a
`human body. For instance, in virtual reality (“VR”)
`applications, the problem of making a fast, accurate, and
`economical head-tracker that operates throughout a large
`WorkSpace is crucial. It is also important for other head
`mounted display (“HMD') applications. Extensive research
`has been devoted to the development of optical, magnetic,
`acoustic and mechanical tracking Systems, but head-trackers
`are still one of the weakest links in existing virtual
`environment Systems. The fastest and potentially most accu
`rate trackers are mechanical, but these are typically clumsy
`and range-restrictive. The largest tracking range has been
`achieved at the University of North Carolina by optoelec
`tronic methods, but this type of System is extremely expen
`Sive and difficult to install, calibrate, and maintain. This type
`of optical tracker is Sometimes referred to as an “inside-out”
`tracker, because a camera that is mounted on the user is
`aimed out toward light Sources mounted on the ceiling.
`UltraSonic trackers are inexpensive, but must Sacrifice Speed
`to achieve reasonable range and are Sensitive to acoustical
`interference, reflections, and obstructions. Magnetic trackers
`are the most popular because of their convenience of opera
`tion (they don’t even require line of sight), but the maximum
`range is a few feet and distortions caused by metallic objects
`can be problematic. For reviews of the existing four head
`tracker technologies, See: Meyer, K., Applewhite, H. and
`Biocca, F., “A survey of position trackers.” Presence, 1(2)
`:173–200, Spring 1992; Ferrin, F., “Survey of helmet track
`ing technologies,” SPIE, 1456, Large-Screen-Projection,
`Avionic, and Helmet-Mounted Displays: 86-94, 1991; and
`Bhatnagar, D., Position trackers for head mounted display
`Systems. A Survey, technical report, University of North
`Carolina at Chapel Hill, March 1993, all three of which are
`incorporated herein by reference.
`All of the known trackers (magnetic, optical, mechanical
`and acoustical) require interaction with another component
`of the apparatus that is located a distance from the body
`being tracked. With magnetic trackers, a magnetic field
`generator is provided, Spaced from the tracked body. With an
`optical or acoustical tracker, light or Sound Sources are
`provided at known locations. Mechanical trackers are con
`nected to a reference through an arm-like device. Thus, none
`provide a Self-contained apparatus for mounting on the body
`to be tracked, which apparatus can track the orientation of
`the body without interaction with radiation or energy from
`any other apparatus. Such a Self contained tracking appara
`tus is desirable. AS used herein, a “self-contained' tracking
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`apparatus is one that can track the orientation of a body to
`which it is mounted, without interaction with radiation,
`energy, Signals, or physical connections from any other
`apparatuS.
`Another drawback with acoustic and outside-in optical
`trackers is that they fundamentally only measure position.
`Orientation is then computed from the positions of three
`fixed points on the head. (By “orientation” it is meant herein
`the rotational alignment relative to an external reference
`frame.) Therefore, the angular resolution is limited by the
`uncertainty in the position measurements as well as the
`distance between the three fixed points on the head. With
`100 mm spacing between the fixed points, a positional jitter
`of t1.0-mm causes an orientational jitter of up to t1.1.
`Additionally, Since the position tracker is essentially part of
`the angular orientation tracker, it is not possible to meet
`independent Specifications for the orientation tracker rela
`tive to the Specifications for the position tracker.
`It is also important to track other body members for other
`aspects of man and machine interaction. Most machines
`require a user instruction input device, typically actuated by
`the users hand. The head, feet, torSo and other body parts
`may also provide input instructions. Persons with Special
`needs, Such as paralysis of certain limbs, often use head and
`leg motions to complete tasks more typically conducted by
`hand motions.
`Inertial navigation Systems (INS) using accelerometers
`and rate gyroscopes have been used for decades for ships,
`planes, missiles and Spacecraft. Typically, Such apparatus
`have been rather large, at least on the order of 8-10 cm in
`diameter and twice that in length, weighing on the order of
`10 kg. An inertial navigation System is a type of Self
`contained tracking apparatus, as that term is used herein. By
`“inertial apparatus”, it is meant an apparatus that measures
`its own motion relative to an inertial reference frame
`through the measurement of acceleration.
`Abasic type of INS is called Strapdown INS, and consists
`of three orthogonal accelerometers and three orthogonal rate
`gyros fixed to the object being tracked. The orientation of the
`object is computed by jointly integrating the outputs of the
`rate gyros (or angular rate Sensors), whose outputs are
`proportional to angular velocity about each axis. The posi
`tion can then be computed by double integrating the outputs
`of the accelerometers using their known orientations. If the
`actual acceleration is C. and the acceleration of gravity is
`-> g, then the acceleration measured by the triaxial acceler
`->
`ometers will be C.
`= C.+g. To obtain the position it is
`neasured
`necessary to know the direction and magnitude of g relative
`to the tracked object at all times in order to double integrate
`C = C.-g. Detailed information about inertial navi
`gation Systems is available in the literature, Such as
`Broxmeyer, C., Inertial Navigation Systems, McGraw-Hill,
`New York, (1964); Parvin, R., Inertial Navigation, Van
`Nostrand, Princeton, N.J. (1962); and Britting, K., Inertial
`Navigation Systems Analysis, Wiley-Interscience, New York
`(1971), which are incorporated herein by reference.
`A difficulty with using gyroscopes for head-orientation
`tracking is drift. Drift arises from integrating over time, a
`Signal that is noisy, or has a bias. Drift would make the
`Virtual world appear to gradually rotate about the user's head
`even when the user is not moving. By measuring the output
`of an angular rate Sensor while it is at rest, it is possible to
`know its output bias and subtract the bias from all Subse
`quent measurements. However, there is inevitably Some
`
`->
`
`-> -->
`
`-> -->
`
`->
`
`META 1022
`META V. THALES
`
`
`
`5,807,284
`
`3
`random noise produced by the Sensor in addition to its bias.
`In the Short term, the angular drift is a random walk with
`RMS value growing proportional to Vt. However, the small
`bias that remains even in a well-calibrated System leads to a
`drift error that grows as t, which will eventually exceed the
`Brownian Motion error that grows as Vt.
`U.S. Pat. No. 5,181,181, issued on Jan. 19, 1993, to
`Glynn, for “Computer Apparatus Input Device for Three
`Dimensional Information,” discloses a computer input
`mouse, which Senses Six degrees of motion using three
`accelerometers for Sensing linear translation and three angu
`lar rate Sensors for Sensing angular rotation about three axes.
`The disclosure does not acknowledge or address the problem
`of drift.
`Complete virtual environment Systems also Suffer from a
`problem that is not directly related to the problem of tracking
`body member motions and orientations. A great deal of
`graphical rendering is required to present to the user a visual
`image of the environment being Simulated. The view to be
`presented depends on the user's orientation and position.
`The rendering requires significant computation, which is
`time consuming. Typically, the computation can not begin
`until the orientation is known. Thus, the Speed of informa
`tion acquisition is extremely important. It would also
`shorten the overall system latency if a reliable method of
`predicting the user's orientation in advance existed.
`OBJECTS
`Thus the several objects of the invention include to track
`the angular orientation of the head (or other body member)
`with undiminished performance over an unlimited range or
`working volume. Another object is to track body member
`orientation with low latency, thus preserving the illusion of
`presence and avoiding Simulator Sickness. Another object is
`to track body member orientation with low output Signal
`noise, So that it won’t be necessary to reduce jitter through
`use of delay-ridden filters. Another object is to track body
`orientation without interference problems. Interference
`Sources to be avoided include acoustical, optical, mechanical
`and electromagnetic. Another object of the invention is to
`predict what the orientation of the body member will be a
`Short time in the future, to reduce System delays when the
`orientation tracker is used with other delay-inducing
`apparatus, Such as a graphics rendering engine in a Virtual
`environment Simulator. An additional object is to resolve
`body member orientation without limitation due to the
`quality or absence of a position Sensing device. Yet another
`object is to facilitate a modular head tracking apparatus,
`using independent orientation and position tracking
`modules, thus permitting tailoring each to independent
`Specifications. Another object of the invention is to track the
`orientation of a body using a Self-contained Sensing
`apparatus, So that an unlimited number of trackers may be
`used in the same area Simultaneously without performance
`degradation.
`
`SUMMARY
`In a preferred embodiment, the invention is a Self con
`tained Sensor apparatus for generating a signal that corre
`sponds to at least two of the three orientational aspects of
`yaw, pitch and roll of a human-Scale body, relative to an
`external reference frame. The apparatus comprises: a Self
`contained Sensor for generating first Sensor Signals that
`correspond to rotational accelerations or rates of the body
`about certain axes relative to Said body; a mechanism for
`mounting the Sensor to the body and, coupled to the Sensor,
`
`4
`a signal processor for generating orientation signals relative
`to the external reference frame that correspond to the angu
`lar rate or acceleration Signals, wherein the first Sensor
`Signals are impervious to interference from electromagnetic,
`acoustic, optical and mechanical Sources.
`In a preferred embodiment that uses rate Sensors, the
`Signal processor also includes an integrator to integrate the
`rate Signal over time. The rate Sensors may be vibrating
`piezoelectric devices, Silicon micro-machined devices,
`magneto-hydrodynamic devices or optical devices, to name
`Several.
`Another preferred embodiment of the invention further
`includes a drift compensator, coupled to the angular rate
`Sensor and the integrator, for compensating for any drift with
`respect to time in the rotational orientation Signal.
`According to one preferred embodiment, the drift com
`pensator may include a gravitational tilt Sensor, or a mag
`netic field Sensor, or both.
`The drift compensator, according to another preferred
`embodiment, includes a verifier that periodically measures
`the orientation of the body by a means different from using
`the rotational rate Signal and generates an orientation drift
`compensation signal based on the verification measurement
`to reduce the effect of drift.
`The verifier may take into account characteristic features
`of human motion, Such as the existence of Stillness periods.
`The drift compensator may be implemented using, in part, a
`Kalman filter, which may utilize Statistical data about human
`head motion.
`The apparatus of the invention may also include an
`orientation predictor, that predicts the orientation in which
`the body will be a short time in the future.
`According to yet another preferred embodiment, the
`invention is an apparatus for generating a Signal that corre
`sponds to the orientation of a human-Scale body, relative to
`a reference frame. The apparatus comprises a Self contained
`first Sensor for generating a drift Sensitive orientation Signal
`that corresponds to the rotational orientation with respect to
`at least two degrees of freedom of the body and is imper
`vious to interference from electromagnetic, acoustic, optical
`and mechanical Sources and is Subject to drift over time. The
`apparatus also includes a Self contained Second Sensor for
`generating a drift compensating orientation Signal that cor
`responds to the rotational orientation with respect to the at
`least two degrees of freedom of the body and is impervious
`to interference from electromagnetic, acoustic, optical and
`mechanical Sources and which is relatively impervious to
`drift over time, and a mounting mechanism for mounting the
`first Sensor and the Second Sensor to the body. Coupled to
`Said first Sensor and Said Second Sensor, a signal corrector
`means for generating a corrected rotational orientation Sig
`nal based on Said drift Sensitive and drift compensating
`orientation signals.
`Another preferred aspect of the invention is a method for
`generating a signal that corresponds to the orientation of a
`human-Scale body, relative to a reference frame. The method
`comprises the Step of using a first Self contained Sensor
`physically coupled to the body, generating a drift Sensitive
`orientation signal that corresponds to the rotational orienta
`tion with respect to at least two degrees of freedom of the
`body and that is impervious to interference from
`electromagnetic, acoustic, optical and mechanical Sources
`and is subject to drift over time. The method also includes
`the Steps of: using a Second Self contained Sensor physically
`coupled to the body, generating a drift compensating orien
`tation Signal that corresponds to the rotational orientation
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`META 1022
`META V. THALES
`
`
`
`5,807,284
`
`S
`with respect to Said at least two degrees of freedom of the
`body and that is impervious to interference from
`electromagnetic, acoustic, optical and mechanical Sources
`and which is relatively impervious to drift over time; and
`generating a corrected rotational orientation Signal based on
`the drift Sensitive and drift compensating orientation signals.
`According to another preferred embodiment, the method
`also includes, when generating the corrected rotational ori
`entation Signal, the Step of taking into account characteristic
`aspects of human motion, Such as the occurrence of periods
`of Stillness, or Statistics about head motions in particular
`applications.
`Still another preferred embodiment of the invention is an
`apparatus for Simulating a virtual environment that is dis
`played to a user. The apparatus comprises a Self contained
`orientation Sensor for generating an orientation signal, Such
`as described above, and is impervious to interference from
`electromagnetic, acoustic, optical and mechanical Sources, a
`position Sensor, a mechanism for mounting the position and
`orientation Sensors to the body member, a virtual environ
`ment generating means a means for displaying virtual envi
`ronment Signals to the user and means for coupling the
`position Sensor and the orientation Sensor to the virtual
`environment means.
`
`6
`FIG. 6C is a graphical representation of the noise Signal
`shown in FIG. 6A, with the low frequency component
`removed;
`FIG. 7A is a graphical representation of the magnitude of
`the transfer function for the pitch axis of a preferred embodi
`ment of the apparatus of the invention;
`FIG. 7B is a graphical representation of the phase of the
`transfer function for the pitch axis of a preferred embodi
`ment of the apparatus of the invention;
`FIG. 8 is a more detailed view of a preferred embodiment
`of the apparatus of the invention;
`FIG. 9 is a schematic illustration of an embodiment of the
`apparatus of the invention mounted upon a user's hand;
`FIG. 10 is a block diagram illustration of basic compo
`nents of a wireleSS embodiment of the apparatus of the
`invention;
`FIG. 11 is a block diagram illustration of basic compo
`nents of an embodiment of the invention using orientation
`and position prediction modules,
`FIG. 12 is a block diagram illustration of an orientation
`predictor.
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS OF THE INVENTION
`In its most basic form, a preferred embodiment of the
`invention is an inertial angular orientation tracking appara
`tus. (The invention does not produce position information,
`only angular orientation information.) Drift sensitive
`Sensors, Such as angular rate Sensors, produce a signal that
`is integrated to give a signal that represents angular position.
`The angular position signal may drift, due to integration of
`a bias or noise in the output of the rate Sensors. To correct
`this drift, compensating Sensors, Such as gravimetric tilt
`Sensors and Sometimes also geomagnetic heading Sensor(s)
`periodically measure the angular position, and this directly
`measured position Signal is used to correct the drift of the
`integrated position signal. The direct angular position Sen
`Sors cannot be used alone for dynamic applications because
`the gravitational Sensors are also affected by non
`gravitational accelerations, and therefore only accurately
`reflect angular position when under the influence of no
`non-gravitational accelerations. Some Suitable compensat
`ing Sensors, Such as pendulous inclinometers, also take
`longer to Settle than would be desirable for a Stand alone
`Sensor to deliver continuous orientation signals.
`Typically, the drift Sensitive Sensors are angular rate
`Sensors, (these include: rate gyroscopes and vibrating
`piezoelectric, magneto-hydrodynamic, optical and micro
`machined silicon devices) the output from which are inte
`grated once. However, other Suitable drift Sensitive Sensors
`include linear accelerometers used to Sense angular rate,
`gyroscopic angular position sensors (with no need to
`integrate) and angular accelerometers (integrated twice).
`Similarly, typically the compensating Sensors are incli
`nometers and compasses. However, other Suitable compen
`Sating Sensors include accelerometers. For purposes of Sim
`plifying the following discussion, the embodiment discussed
`uses angular rate Sensors as the drift Sensitive Sensors and a
`Z-axis inclinometer and a compass as the compensating
`Sensors. However, the other types of Sensors mentioned are
`also intended to be included as alternate embodiments of the
`invention.
`The present invention may be applied toward tracking the
`head and other body parts with appropriate modifications.
`However, for Simplicity, this discussion focuses on tracking
`the head and Suitable apparatus therefore.
`
`15
`
`25
`
`35
`
`40
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`These and other features, aspects, and advantages of the
`present invention will become better understood with regard
`to the following description, appended claims and accom
`panying drawings, where:
`FIG. 1 is a schematic block diagram overview of a
`preferred embodiment of the apparatus of the invention;
`FIG. 2A is a schematic illustration of an embodiment of
`the apparatus of the invention mounted upon a user's head,
`along with a head mounted display;
`FIG. 2B is a schematic illustration of another, more
`compact embodiment of the apparatus of the invention,
`mounted on a user's head;
`FIG. 3 is a schematic block diagram illustration of a
`preferred embodiment of the apparatus of the invention
`partially embodied by a computer with a memory and an
`input output device;
`FIG. 4A is a schematic flow chart illustration of an
`Overview of the Steps of a computer program for controlling
`a computer, that may constitute a portion of a preferred
`embodiment of the apparatus of the invention;
`FIG. 4B is a flow chart representation of the steps of a
`computer program for controlling a computer to perform
`initialization Steps;
`FIG. 4C is a flowchart representation of the steps of a
`computer program for controlling a computer to perform the
`main operating Steps of measuring Euler angles and com
`pensating for drift,
`55
`FIG. 5A is a graphical representation of the error in the
`pitch angle as determined by a preferred embodiment of the
`apparatus of the invention;
`FIG. 5B is a graphical representation of the error in the
`roll angle as determined by a preferred embodiment of the
`apparatus of the invention;
`FIG. 6A is a graphical representation of the noise Signal
`generated by an embodiment of the invention attached to a
`Stationary body;
`FIG. 6B is a graphical representation of the low frequency
`component of the noise Signal shown in FIG. 6A,
`
`45
`
`50
`
`60
`
`65
`
`META 1022
`META V. THALES
`
`
`
`5,807,284
`
`7
`A Schematic configuration for a generic embodiment of
`the invention is shown in FIG. 1. A set of drift sensitive
`angular rate Sensors 104 continuously generate Signals F that
`correspond to the rate of change of the angular orientation of
`the individual Sensors. (The Sensors themselves are dis
`cussed in more detail below.) The rate sensors 104 are
`connected to an integrator module 106, which jointly inte
`grates the Signal F that represents the Set of angular rates
`using a Suitable method, Such as the Euler method or a
`quaternion method or using Direction Cosine Matrices. The
`output of the integration module is a signal D, which
`represents a set of orientation angles, which correspond
`roughly to the orientation of the body to which the rate
`Sensors are attached, as explained below. This orientation
`Signal is passed to a drift compensation module 108.
`A set of drift compensating angular position Sensors 110
`is also provided. This Set of position Sensors generates a
`Signal S that is related to the angular orientation of the body
`to which the rate Sensors are attached, as explained below.
`This signal S is also provided to the drift compensation
`module 108. The drift compensation module uses the angu
`lar position signal S to generate a signal C that represents the
`angular orientation corrected for errors in the angular ori
`entation signal D output from the integrator 106, which
`errors may have arisen due to drift or other causes. The
`corrected angular orientation Signal C is fed back to the
`integration module 106 for transforming the coordinates of
`the angular velocities. The corrected angular orientation
`Signal C is also provided as an output to whatever equipment
`is connected to the angular orientation apparatus, requiring
`this information. Such equipment can comprise a virtual
`reality System, a computer, a teleoperator, etc.
`Aschematic representation of how the apparatus shown in
`FIG. 1 would be mounted upon a human body member, such
`as a