United States Patent
`US 8,441,438 B2
`(10) Patent No.:
`(12)
`Ye et al.
`(45) Date of Patent:
`May 14, 2013
`
`
`US008441438B2
`
`8/2008 Liberty
`7,414,611 B2
`2/2009 Liberty et al.
`7,489,298 B2
`5/2009 Liberty et al.
`7,535,456 B2
`8/2010 Satoetal. woe 702/127
`7,774,155 B2*
`-.. 345/158
`7,817,134 B2* 10/2010 Huangetal.
`
`
`4/2011 Ohta wees
`w 345/157
`7,924,264 B2*
`.............. 702/141
`8,010,313 B2*
`8/2011 Mathewset al.
`
`4/2008 Mondesir et al... 463/31
`2008/0096654 Al*
`
`... 345/158
`....
`2009/0262074 Al* 10/2009 Nasiri et al.
`5011/0307173 Al* 12/2011 Riley 701/220
`OTHER PUBLICATIONS
`
`Azuma, Ronald et al. Improving Static and Dynamic Registration in
`an Optical See-Through HMD. Proceedings of SIGGRAPH ’94
`(Orlando, Fla., Jul. 24 29, 1994), Computer Graphics, Annual Con-
`ference Series, 1994, 197 204.*
`
`* cited by examiner
`
`Primary Examiner — William Boddie
`Assistant Examiner — Bryan E Earles
`(74) Attorney, Agent, or Firm — Ding Yu Tan
`
`ABSTRACT
`(57)
`A three-dimensional (3D) pointing device capable of accu-
`rately outputting a deviation including yaw, pitch androll
`angles in a 3D reference frame and preferably in an absolute
`manner is provided. Said 3D pointing device comprises a
`six-axis motion sensor module including a rotation sensor
`and an accelerometer, and a processing andtransmitting mod-
`ule. The six-axis motion sensor module generatesa first sig-
`nal set comprising angular velocities and a second signal set
`comprising axial accelerations associated with said move-
`ments and rotations of the 3D pointing device in the 3D
`reference frame. The processing and transmitting module
`utilizes a comparison method to compare thefirst signal set
`with the second signal set to obtain an updated state of the
`six-axis motion sensor module based on a current state and a
`
`measuredstate thereof in order to output the resulting devia-
`tion in the 3D reference frame andpreferably in an absolute
`manner.
`
`19 Claims, 7 Drawing Sheets
`
`—
`
`Output 3rd quaternion
`to Ist quaternion
`
`Obtain resultant
`deviation including yaw,
`pitch and roll angles
`
`Obtain display data and
`translate the resultant
`angles to movement
`pattern in the display
`
`740
`
`FAS
`
`70
`
`GOOGLE 1046
`GOOGLE1046
`
`
`
`™“ velocities ot T aSennen
`
`
`
`teference frame
`
`710
`
`15
`
`720-_| Obtain a current state
`(2nd quaternion} at T
`aObtain “measured axial
`725—_| accelerations” of a
`measured state at T
`—_—_—_¥
`730-_| Calculate “predicted axial
`accelerations” of a
`
`measured state at T
`
`—_—_—_—_*
`Obtain an updated state
`735-\_| (3rd quaternion) by
`comparing current state
`with measured state
`
`(54)
`
`(75)
`
`3D POINTING DEVICE AND METHOD FOR
`COMPENSATING MOVEMENT THEREOF
`:
`Inventors: ZhouYe, Foster City, CA (US):
`Chin-LungLi, Taoyuan County (TW);
`Shun-Nan Liou, Kaohsiung (TW)
`:
`.
`mt
`(73) Assignee: Cywee Group Limited, Tortola (VG)
`(*) Notice:
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 218 days.
`
`(21) Appl. No.: 12/943,934
`
`(22)
`
`Filed:
`
`Nov. 11, 2010
`
`(65)
`
`Prior Publication Data
`US 2011/0163950 Al
`Tul. 7.2011
`,
`Related U.S. Application Data
`
`(51)
`
`(60) iouisional application No. 61/292,558, filed on Jan.
`,
`,
`Int. Cl.
`G09G 5/00
`(52) US.CL
`USPC oie ccc ceseeseseeeseeesescnesenseceesensenee 345/156
`(58) Field of Classification Search ...........0.00008 None
`See applicationfile for complete search history.
`
`(2006.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,138,154 A
`5,440,326 A
`5,898,421 A
`6,061,611 A *
`6,993,451 B2*
`7,158,118 B2
`7,236,156 B2
`7,239,301 B2
`7,262,760 B2
`
`8/1992 Hotelling
`8/1995 Quinn
`4/1999 Quinn
`5/2000 Whitmore .......ce 70/4
`1/2006 Changetal. ....... 702/153
`1/2007 Liberty
`6/2007 Liberty etal.
`7/2007 Liberty et al.
`8/2007 Liberty
`
`705—_|
`
`—
`Initialize on initial-value
`set
`
`Obtain a previous state
`™ (1st quaternion) at T-14
`——<—<_<—_—__
`Obtain measured angular
`
`
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 1 of 7
`
`US 8,441,438 B2
`
`120
`
`122
`
`120
`
`122
`
`Z°
`
`Xp
`
`Yp
`
`Xp
`113
`
`Yp
`
`112
`
`(RELATED ART)
`ty
`
`Xp
`
`110
`
`111
`
`FIG.
`
`Zp
`1
`
`Yp
`
`110
`
`112
`
`111
`
`Y
`
`D
`
`Xp
`
`113
`
`Zp
`
`FIG. 2 (RELATED ART)
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 2 of 7
`
`US 8,441,438 B2
`
`530
`
`FIG. 3
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 3 of 7
`
`US 8,441,438 B2
`
`542
`
`Rotation
`
`
`
`Data
`Transmitting
`Unit
`
`Computing
`Processor
`
`3544
`
`FIG
`
`4eo
`
`502ae
`544
`
`56) —~,
`
`Y
`
`bsY
`
`ONY
`WRAAYWHAJiY
`NANANAN:ieLOGVA
`AXX)S
`NAAN
`
`046
`
`540
`
`922 590
`
`LOO
`LOOAy
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 4 of 7
`
`US 8,441,438 B2
`
`620
`
`630
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 5 of 7
`
`US 8,441,438 B2
`
`Initialize an initial—value
`set
`
`velocities at T
`
` 705
`
`
`
`
` 710 Obtain a previous state
`(1st quaternion) at T-1
`
`
` Obtain measured angular
`
`
`
`715
`
`720
`
`Obtain a current state
`
`
`Output 3rd quaternion
`(2nd quaternion) at T
`
`
`
`740
`to 1st quaternion
`
`
`
`
`
`Obtain "measured axial
`
`
`
`Obtain resultant
`accelerations
`of a
`
`deviation including yaw,
` 745
`
`measured state at T
`
`pitch and roll angles
`
`
`725
`
` 730—_|
`
`Calculate "predicted axial
`accelerations” of a
`
`measured state at T Obtain an updated state
`
`with measured state
`
`(3rd quaternion) by
`comparing current state
`
`735-_|
`
`FIG. 7
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 6 of 7
`
`US 8,441,438 B2
`
`705
`
`Initialize an initial—value
`set
`
`710
`
`715
`
`720
`
`725
`
`730
`
`735
`
`Obtain a previous state
`
`(1st quaternion) at T-1
`
`
`Obtain measured angular
`velocities at T
`
`
`Obtain a current state
`Output 3rd quaternion
`(2nd quaternion) at T
`
`to 1st quaternion
`740
`
`
`
` Obtain “measured axial
`measured state at T
`measured state at T Obtain an updated state
`
`
`(3rd quaternion) by
`comparing current state
`
`with measured state
`
`Obtain resultant
`deviation including yaw,
`pitch and roll angles
`
`
`
`
`Obtain display data and
`translate the resultant
`
` angles to movement
`
`pattern in the display
`
`reference frame
`
`745
`
`750
`
`accelerations of a
`
` Calculate “predicted axial
`
`accelerations” of a
`
`FIG. 8
`
`

`

`U.S. Patent
`
`May14, 2013
`
`Sheet 7 of 7
`
`US 8,441,438 B2
`
`Prax
`
`FIG. 9
`
`

`

`US 8,441,438 B2
`
`1
`3D POINTING DEVICE AND METHOD FOR
`COMPENSATING MOVEMENT THEREOF
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application claims priority benefits of U.S. Patent
`Provisional Application No. 61/292,558, filed on Jan. 6,
`2010. The entirety of the above-mentioned patent applica-
`tions is hereby incorporated by reference herein and made a
`part of this specification.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention generally relates to a three-dimen-
`sional (3D) pointing device utilizing a motion sensor module
`and method of compensating and mapping signals of the
`motion sensor module subject to movements androtations of
`said 3D pointing device. Moreparticularly, the present inven-
`tion relates to a 3D pointing deviceutilizing a six-axis motion
`sensor module with an enhanced comparison to calculate and
`compensate accumulated errors associated with the motion
`sensor module andto obtain actual resulting deviation angles
`in spatial reference frame and under dynamic environments.
`2. Description of the Related Art
`FIG. 1 is a schematic diagram showing a user using a
`handheld 3D pointing device 110 to point at a point on the
`screen 122 of a 2D display device 120. If the pointing device
`110 emits a light beam,the point would be the location where
`the light beam hits the screen 122. For example, the pointing
`device 110 may be a mouseof a computer or a pad of a video
`game console. The display device 120 may bea part of the
`computeror the video game console. There are two reference
`frames, such as the spatial pointer reference frame and the
`display frame, associated with the pointing device 110 and
`the display device 120, respectively. Thefirst reference frame
`or spatial pointer reference frame associated withthe pointing
`device 110 is defined by the coordinate axes X,, Y, and Z,as
`shown in FIG. 1. The secondreference frameor display frame
`associated with the display device 120 is defined by the coor-
`dinate axes Xp, Yp and Z,, as shown in FIG. 1. The screen 122
`of the display device 120 is a subset ofthe X,,Y, plane ofthe
`reference frame X,Y,Z,p associated with the display device
`120. Therefore, the X,Y, plane is also knownasthe display
`plane associated with the display device 120.
`Auser may perform control actions and movementsutiliz-
`ing the pointing device for certain purposes including enter-
`tainmentsuch as playing a video game,on the display device
`120 through the aforementioned pointer on the screen 122.
`For proper interaction with the use of the pointing device,
`when the user movesthe pointing device 110, the pointer on
`the screen 122 is expected to move along with theorientation,
`direction and distance travelled by the pointing device 110
`and the display 120 shall display such movement of the
`pointer to a new location on the screen 122 ofthe display 120.
`Theorientation ofthe pointing device 110 may be represented
`by three deviation angles of the 3D pointing device 110 with
`respect to the reference frame X,Y,Z,, namely, the yaw
`angle 111, the pitch angle 112 and the roll angle 113. The yaw,
`pitch and roll angles 111, 112, 113 may be best understood in
`relation to the universal standard definition of spatial angles
`related to commercial vehicles or transportation such as ships
`and airplanes. Conventionally, the yaw angle 111 mayrepre-
`sent the rotation of the pointing device 110 about the Z, axis;
`the pitch angle 112 mayrepresentthe rotation of the pointing
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`device 110 about theY, axis; the roll angle 113 may represent
`the rotation of the pointing device 110 about the X,, axis.
`In a knownrelated art as shown in FIG. 1, when the yaw
`angle 111 of the pointing device 110 changes, the aforemen-
`tioned pointer on the screen 122 must move horizontally or in
`a horizontal direction with reference to the ground in
`response to the change of the yaw angle 111. FIG. 2 shows
`what happens whenthe user rotates the pointing device 110
`counterclockwise by a degree such as a 90-degree about the
`X;p axis.
`In another knownrelated art as shown in FIG. 2, when the
`yaw angle 111 changes, the aforementioned pointer on the
`screen 122 is expected to move vertically in response. The
`change ofthe yaw angle 111 can be detected by a gyro-sensor
`which detects the angular velocity w,, of the pointing device
`110 about the X, axis. FIG. 1 and FIG.2 show that the same
`change of the yaw angle 111 may be mapped to different
`movements ofthe point on the screen 122. Therefore, a proper
`compensation mechanism for the orientation of the pointing
`device 110 is required such that corresponding mapping of
`the pointer on the screen 122 of the display 120 may be
`obtained correctly and desirably. The term compensation of
`the prior arts by Liberty (U.S. Pat. No. 7,158,118, U.S. Pat.
`No. 7,262,760 and U.S. Pat. No. 7,414,611) refers to the
`correction and compensation of signals subject to gravity
`effects or extra rotations about the axis related to “roll”. The
`
`term of “comparison”of the present invention may generally
`refer to the calculating and obtaining ofthe actual deviation
`angles of the 3D pointing device 110 with respectto the first
`reference frame or spatial pointing frame X,Y,Z, utilizing
`signals generated by motion sensors while reducing or elimi-
`nating noises associated with said motion sensors; whereas
`the term mapping mayrefer to the calculating andtranslating
`of said deviation angles in the sptatial pointing frame
`X,Y>,Zp onto the aforementioned pointer on the display
`plane associated with the 2D display device 120 of a second
`reference frame or display frame X,YpZp.-
`It is knownthat a pointing device utilizing 5-axis motion
`sensors, namely, Ax, Ay, Az, o,and w, may be compensated.
`For example, U.S. Pat. No. 7,158,118 by Liberty, U.S. Pat.
`No. 7,262,760 by Liberty and U.S. Pat. No. 7,414,611 by
`Liberty provide such pointing device having a 5-axis motion
`sensor and discloses a compensation using two gyro-sensors
`@y and w, to detect rotation about the Yp and Zp axes, and
`accelerometers Ax, Ay and Az to detect the acceleration ofthe
`pointing device along the three axes of the reference frame
`X,YpZp. The pointing device by Liberty utilizing a 5-axis
`motion sensor maynot output deviation angles ofthe pointing
`device in, for example, a 3D reference frame; in other words,
`due to due to the limitation of the 5-axis motion sensor of
`
`accelerometers and gyro-sensors utilized therein, the point-
`ing device by Liberty cannot output deviation angles readily
`in 3D reference frame but rather a 2D reference frame only
`and the output of such device having 5-axis motion sensorsis
`aplanar pattern in 2D reference frameonly.In addition, it has
`been found that the pointing device and compensation dis-
`closed therein cannot accurately or properly calculate or
`obtain movements, angles and directions of the pointing
`device while being subject to unexpected dynamic movement
`during the obtaining of the signals generated by the motion
`sensors, in particular, during unexpected drifting movements
`and/or accelerations along with the direction of gravity. In
`other words, it has been found that dynamic actions or extra
`accelerations including additional accelerations, in particular
`the one acted upon the direction substantially parallel to or
`along with the gravity imposed on the pointing device with
`the compensation methodsprovided by Liberty, said pointing
`
`

`

`US 8,441,438 B2
`
`3
`device by Liberty cannot properly or accurately output the
`actual yaw, pitch androll angles in the spatial reference frame
`X,Y>,Z, and following which, consequently, the mapping of
`the spatial angles onto any 2D display reference frame such as
`XpYpZp may begreatly affected and erred. To be more
`specific, as the 5-axis compensation by Liberty cannot detect
`or compensate rotation about the X, axis directly or accu-
`rately, the rotation about the X;, axis hasto be derived from the
`gravitational acceleration detected by the accelerometer. Fur-
`thermore, the reading of the accelerometer may be accurate
`only when the pointing device is static since dueto the limi-
`tation on known accelerometers that these sensors may not
`distinguish the gravitational acceleration from the accelera-
`tion ofthe forces including centrifugal forces or other types of
`additional accelerations imposedor exerted by the user.
`Furthermore, it has been found that knownprior arts may
`only be able to outputa “relative” movement pattern in a 2D
`reference frame based on the result calculated from the sig-
`nals of motion sensors. For example, the abovementioned
`prior arts by Liberty may only output a 2D movementpattern
`in arelative manneranda pointer on a display screen to show
`such corresponding 2D relative movement pattern. To be
`more specific, the pointer moves from a first location to a
`second new locationrelative to said first location only. Such
`relative movement from the previous location to the next
`location with respect to time cannot accurately determine
`and/or output the next location, particularly in situations
`wherethe previous location may have been an erred location
`or have been faultily determined as an incorrect reference
`point for the next location that is to be calculated therefrom
`and obtained based ontheirrelative relationship adapted. One
`illustration of such defect of known prior arts adapting a
`relative relationship in obtaining a movementpattern may be
`clearly illustrated by an example showingthe faultily output-
`ted movements of a pointer intended to move out of a bound-
`ary or an edge of display screen. It has been foundthat as the
`pointer of knownprior arts reaches the edge of a display and
`continues to move out of the boundary or edge at a certain
`extra extent beyond said boundary, the pointer fails to dem-
`onstrate a correct or “absolute” pattern as it moves to a new
`location either within the display or remaining outside of the
`boundary; in other words, instead of returning to a new loca-
`tion by taking into accountsaid certain extra extend beyond
`the boundary made earlier in an “absolute” manner,
`the
`pointer of known arts discards such virtual distance of the
`extra extend beyond the boundary already made and an erred
`next position is faultily outputted dueto therelative relation-
`ship adapted andutilized by the pointer. may be never calcu-
`lated or processed dueto the faultily obtained location at the
`edge or boundary of the display as well as the relative rela-
`tionship adapted to obtain its next location therefrom.
`Therefore,it is clear that an improved pointing device with
`enhancedcalculating or comparison method capable of accu-
`rately obtaining and calculating actual deviation angles in the
`spatial pointer frame as well as mapping of such angles onto
`a pointer on the display frame in dynamic environments and
`conditions is needed.In addition, as the trend of 3D technol-
`ogy advances and is applicable to various fields including
`displays and interactive systems, there is a significant need for
`a3D pointing device capable ofaccurately outputting a devia-
`tion of such device readily useful in a 3D or spatial reference
`frame. Furthermore, there is a need to provide an enhanced
`comparison methodapplicable to the processing of signals of
`motion sensors such that errors and/or noises associated with
`
`such signals or fusion of signals from the motions sensors
`maybe corrected or eliminated. In addition, according to the
`field of application, such output of deviation in 3D reference
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`frame may too be further mappedortranslated to a pattern
`useful in a 2D reference frame.
`
`SUMMARY OF THE INVENTION
`
`According to one aspect of an example embodimentof the
`present invention, a 3D pointing device utilizing a six-axis
`motion sensor module is provided. The 3D pointing device
`comprises an accelerometer to measureor detect axial accel-
`erations Ax, Az, Ay and a rotation sensor to measure or detect
`angular velocities ,, ,, @, such that resulting deviation
`including resultant angles comprising yaw, pitch and roll
`angles in a spatial pointer frame of the 3D pointing device
`subject to movements androtations in dynamic environments
`maybe obtained and such thatsaid resulting deviation includ-
`ing said resultant angles may be obtained and outputted in an
`absolute manner reflecting or associating with the actual
`movements and rotations of the 3D pointer device of the
`present invention in said spatial pointer reference frame.
`According to another aspect of the present invention, the
`present invention provides an enhanced comparison method
`to eliminate the accumulated errors as well as noises over
`time associated with signals generated by a combination of
`motion sensors, including the ones generated by accelerom-
`eters A,, A,, A, and the ones generated by gyroscopes ,, ©,,,
`@, in dynamic environments. In other words, accumulated
`errors associated with a fusion of signals from a motions
`sensor module comprising a plurality of motion sensors to
`detect movements on androtations about different axes of a
`reference frame may be eliminated or corrected.
`Accordingto still another aspect of the present invention,
`the present
`invention provides an enhanced comparison
`method to correctly calculating and outputting a resulting
`deviation comprising a set of resultant angles including yaw,
`pitch and roll angles in a spatial pointer frame, preferably
`about each of three orthogonal coordinate axes ofthe spatial
`pointer reference frame, by comparing signals of rotation
`sensorrelated to angular velocities or rates with the ones of
`accelerometer related to axial accelerations such that these
`angles may be accurately outputted and obtained, which may
`too be further mapping to another reference frame different
`from said spatial pointer frame.
`Accordingto still another aspect of the present invention,
`the present invention provides a mapping of the abovemen-
`tioned resultant angles, preferably about each of three
`orthogonal coordinate axes of the spatial pointer reference
`frame, including yaw, pitch and roll angles in a spatial pointer
`reference frame onto a display frame such that a movement
`pattern in a display frame different from the spatial pointer
`reference frame may be obtained according to the mapping or
`translation of the resultant angles of the resultant deviation
`onto said movementpattern.
`According to another example embodimentof the present
`invention, a 3D pointing device utilizing a six-axis motion
`sensor module with an enhanced comparison method for
`eliminating accumulatederrors of said six-axis motion sensor
`module to obtain deviation angles corresponding to move-
`ments and rotations of said 3D pointing device in a spatial
`pointer reference frame is provided. The 3D pointing device
`and the comparison methodprovided by the present invention
`by comparing signals from the abovementioned six-axis
`motion sensor module capable of detecting rotation rates or
`angular velocities of the 3D pointing device aboutall of the
`X>p, Yp and Z, axes as well as axial accelerations of the 3D
`pointing device alongall of the X,, Y, and Z, axes. In other
`words, the present invention is capable of accurately output-
`ting the abovementioned deviation angles including yaw,
`
`

`

`US 8,441,438 B2
`
`5
`pitch and roll angles in a 3D spatial pointer reference frame of
`the 3D pointing device to eliminate or reduce accumulated
`errors and noises generated over time in a dynamic environ-
`mentincluding conditions such as being subject to a combi-
`nation of continuous movements, rotations, external gravity
`forces and additional extra accelerations in multiple direc-
`tions or movementand rotations that are continuously non-
`linear with respect to time; and furthermore, based on the
`deviation angles being compensated and accurately outputted
`in 3D spatial pointer reference frame may be further mapped
`onto or translated into another reference frame such as the
`abovementioned display frame, for example a reference in
`two-dimension (2D).
`According to another example embodimentofthe present
`invention, a 3D pointing device utilizing a six-axis motion
`sensor module is provided; wherein the six-axis motion sen-
`sor module of the 3D pointing device comprises at least one
`gyroscope and at least one accelerometer. In one preferred
`embodiment of the present invention, the six-axis motion
`sensor module comprises a rotation sensor capable of detect-
`ing and generating angular velocities of w,, ,, 0, and an
`accelerometer capable of detecting and generating axial
`accelerations of Ax, Ay, Az. It can be understood that in
`another preferred embodiment, the abovementioned rotation
`sensor may comprise three gyroscopes corresponding to each
`of the said angular velocities of w,, @,, m, in a 3D spatial
`pointer reference frame of the 3D pointing device; whereas
`the abovementioned accelerometer may comprise three
`accelerometers corresponding to each of the said axial accel-
`erations Ax, Ay, Az in a 3D spatial pointer reference frame of
`the 3D pointing device. The rotation sensor detects the rota-
`tion of the 3D pointing device with respect to a reference
`frame associated with the 3D pointing device and provides a
`rotation rate or angular velocity output. The angular velocity
`output includes three components correspondingto the rota-
`tion rate or angular velocities m,, m,, w, of the 3D pointing
`device about the first axis, the second axis andthe third axis of
`the reference frame, namely, Xp, Yp and Zp of the 3D spatial
`pointer frame. The accelerometer detects the axial accelera-
`tions of the 3D pointing device with respect to the spatial
`pointer reference frame such as a 3D-pointer reference frame
`and provides an acceleration output. The acceleration output
`includes three components corresponding to the accelera-
`tions, Ax, Az, Ay ofthe 3D pointing device alongthefirst axis,
`the second axis and the third axis of the reference frame,
`namely, Xp, Yp and Zp ofthe 3D spatial pointer frame.It can,
`however, be understoodthat the axes of Xp, Yp and Zp of the
`3D spatial pointer frame may too be represented simply by the
`denotation of X,Y and Z.
`According to another example embodimentofthe present
`invention, a method for compensating accumulatederrors of
`signals ofthe abovementioned six-axis motion sensor module
`in dynamic environments associated in a spatial pointer ref-
`erence frame is provided. In one embodiment, the method
`may be performed or handled by a hardware processor. The
`processoris capable of compensating the accumulatederrors
`associated with the resultant deviation in relation to the sig-
`nals of the above-mentioned six-axis motion sensor module
`ofthe 3D pointing device subject to movements and rotations
`in a spatial pointer reference frame and in a dynamic envi-
`ronment by performing a data comparison to compare signals
`ofrotation sensorrelated to angular velocities with the ones of
`accelerometer related to axial accelerations such that the
`
`resultant deviation corresponding to the movements androta-
`tions ofthe 3D pointing devicein the 3D spatial pointer frame
`maybe obtained accurately over time in the dynamic envi-
`ronments.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`According to another embodimentofthe present invention,
`a methodfor obtaining a resulting deviation including result-
`ant angles in a spatial pointer reference frame of a three-
`dimensional (3D) pointing device utilizing a six-axis motion
`sensor module therein and subject to movements and rota-
`tions in dynamic environments in said spatial pointer refer-
`ence frameis provided. Said method comprisesthe steps of:
`obtaining a previous state associated with previous angular
`velocities ,, ,, ®, gained from the motion sensorsignals of
`the six-axis motion sensor module at a previous time T-1;
`obtaining a currentstate ofthe six-axis motion sensor module
`by obtaining measured angularvelocities w,, ,, w, gained
`from the motion sensor signals at a current time T; obtaining
`a measured state of the six-axis motion sensor module by
`obtaining measured axial accelerations Ax, Ay, Az gained
`from the motion sensor signals at the current time T and
`calculating predicted axial accelerations Ax', Ay', Az' based
`on the measured angular velocities w,, ,, ©, of the current
`state; obtaining an updatedstate ofthe six-axis motion sensor
`module by comparing the current state with the measured
`state of the six-axis motion sensor module; and calculating
`and converting the updated state ofthe six axis motion sensor
`moduleto said resulting deviation comprising said resultant
`angles in said spatial pointer reference frameofthe 3D point-
`ing device.
`According to another aspect of the present invention, a
`method for mapping deviation angles associated with move-
`ments and rotations of a 3D pointing device in a spatial
`pointer reference frame onto a display frame of a display
`having a predetermined screen size is provided. In one
`embodiment, the method for mappingortranslating deviation
`angles including yaw,pitch androll angles in a spatial pointer
`reference frame to an pointing object, such as a pointer, hav-
`ing movements in a display frame, preferably a 2D reference
`frame, comprises the steps of obtaining boundary informa-
`tion of the display frame by calculating a predefined sensitiv-
`ity associated with the display frame and performing angle
`and distance translation in the display frame based on said
`deviation angles and boundary information.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings are included to provide a
`further understanding of the invention, and are incorporated
`in and constitute a part of this specification. The drawings
`illustrate embodiments ofthe invention and, together with the
`description, serve to explain the principles of the invention.
`FIG. 1 shows a knownrelated art having a 5-axis motion
`sensor in 2D reference frame.
`
`FIG. 2 showsthe knownrelated art having a 5-axis motion
`sensor as shown in FIG. 1 being rotated or rolled about Xp
`axis andis subject to further dynamic interactions or environ-
`ment.
`
`FIG. 3 is an exploded diagram showing a 3D pointing
`device utilizing a six-axis motion sensor module according to
`one embodiment of the present invention in a 3D spatial
`pointer reference frame.
`FIG.4 is a schematic block diagram illustrating hardware
`components of a 3D pointing device according to one
`embodimentof the present invention.
`FIG. 5 is a schematic diagram showing a 3D pointing
`device utilizing a six-axis motion sensor module according to
`anther embodimentof the present invention in a 3D spatial
`pointer reference frame.
`
`

`

`US 8,441,438 B2
`
`7
`FIG. 6 is an exploded diagram showing a 3D pointing
`device utilizing a six-axis motion sensor module according to
`anther embodimentof the present invention in a 3D spatial
`pointer reference frame.
`FIG.7 is a flow chart illustrating a method for compensat-
`ing deviation angles of a 3D pointing device having move-
`ments and rotations in a 3D spatial pointer reference frame
`and in a dynamic environmentaccording to an embodimentof
`the present invention.
`FIG. 8 showsa flow chart illustrating a method ofmapping
`deviation angles of a 3D pointing device having movements
`and rotations in a 3D spatial pointer reference frame and ina
`dynamic environmentonto a display reference frame accord-
`ing to another embodimentof the present invention.
`FIG. 9 is a schematic diagram showing the mappingofthe
`resultant angles of the resultant deviation of a 3D pointing
`device according to an embodimentofthe present invention.
`
`DESCRIPTION OF THE EMBODIMENTS
`
`to the present
`Reference will now be made in detail
`embodiments of the invention, examples of which areillus-
`trated in the accompanying drawings. Whereverpossible, the
`same reference numbers are used in the drawings and the
`description to refer to the sameorlike parts.
`FIG. 3 is an exploded diagram showing a 3D pointing
`device 300 according to an embodimentofthe present inven-
`tion. The 3D pointing device 300 is subject to movements and
`rotations in dynamic environments in a 3D spatial pointer
`reference frame. Thespatial pointer reference frame is analo-
`gousto the reference frame X,YpZp in FIG. 1 and FIG. 2. The
`movements androtations ofthe 3D pointing device 300 in the
`aforementioned dynamic environments in the spatial pointer
`reference frame may be continuously nonlinear with respect
`to time.
`
`The 3D pointing device 300 includes a top cover 310, a
`printed circuit board (PCB) 340, a rotation sensor 342, an
`accelerometer 344, a data transmitting unit 346, a computing
`processor 348, a bottom cover 320, and a battery pack 322.
`The top cover 310 may include a few control buttons 312 for
`a user to issue predefined commands for remote control. In
`one embodiment, the housing 330 may comprise the top
`cover 310 and the bottom cover 320. The housing 330 may
`moveandrotate in the spatial pointer reference frame accord-
`ing to user manipulation or any external forces in any direc-
`tion and/or under the abovementioned dynamic environ-
`ments. As shown in the FIG. 3, in one embodiment, the
`rotation sensor 342, the accelerometer 344, the data transmit-
`ting unit 346, and the computing processor 348 maybeall
`attached to the PCB 340. The PCB 340 is enclosed by the
`housing 330. The PCB 340 includes at least one substrate
`having a longitudinal side configured to be substantially par-
`allel to the longitudinal surface of the housing 330. An addi-
`tional battery pack 322 provides electrical power for the
`entire 3D pointing device 300.
`FIG.4 is a schematic block diagram illustrating hardware
`components of the 3D pointing device 300. The 3D pointing
`device 300 includes a six-axis motion sensor module 302 and
`a processing and transmitting module 304. The six-axis
`motion sensor module 302 includes the rotation sensor 342
`and the accelerometer 344. The processing and transmitting
`module 304 includes the data transmitting unit 346 and the
`computing processor 348.
`The rotation sensor 342 of the six-motion sensor module
`
`302 detects and generatesthefirst signal set including angular
`velocities ,, ,, ®, associated with the movementsandrota-
`tions of the 3D pointing device 300 about each of three
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`55
`
`8
`orthogonal coordinate axes X,Y,Z, of the spatial pointer
`reference frame. The angular velocities w,, w,, and , are
`corresponding to the coordinate axes Xp, Y, and Zp respec-
`tively. The accelerometer 344 detects and generates the sec-
`ondsignal set including axial accelerations Ax, Ay, Az asso-
`ciated with the movements androtations of the 3D pointing
`device 300 along each ofthe three orthogonal coordinate axes
`XpY>pZp of the spatial pointer reference frame. The axial
`accelerations Ax, Ay and Az are correspondingto the coordi-
`nate axes Xp, Yp and Z, respectively. The term