US 8,441,438 132
`(10) Patent No.:
`(12) United States Patent
`Ye et al.
`(45) Date of Patent:
`May 14, 2013
`
`
`USOO844143 8B2
`
`(54)
`
`(75)
`
`3D POINTING DEVICE AND METHOD FOR
`COMPENSATING MOVEMENT THEREOF
`.
`InVemorS: th’“ Ye, F051“ Cltys CA (Us);
`Chln-Lllllg L1, Taoyuan County (TW);
`Shun-Nan Liou, Kaohsiung (TW)
`-
`-
`.
`~
`(73) ASSIgnee~ Cywee Group lelteds T011018 (VG)
`( * ) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 15403) by 218 days.
`
`21
`
`PP
`s
`A 1. No.2 12/943 934
`
`(22)
`
`Filed:
`
`Nov. 11, 2010
`
`(65)
`
`Prior Publication Data
`
`US 2011/0163950 A1
`
`Jul 7 2011
`’
`Related US. Application Data
`
`(60)
`
`(51)
`
`16’r02\(/)i1s(i)onal application No. 61/292,558, filed on Jan.
`,
`i
`Int. Cl,
`G09G 5/00
`(52) US. Cl.
`USPC .......................................................... 345/156
`(58) Field of Classification Search ........................ None
`See application file for complete search history.
`
`(2006.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,138,154 A
`8/1992 Hotelling
`5,440,326 A
`8/1995 Quinn
`5,898,421 A
`4/1999 Quinn
`6961511 A :
`5/2000 Whitmore ~~~~~~~~~~~~~~~~~~~~~~~~~ 70 1/4
`gfiggfilié E;
`3388? 3:32:32?th 31' """"""""" 702/153
`7,236,156 B2
`6/2007 Liberty et a1.
`7,239,301 B2
`7/2007 Liberty et a1.
`7,262,760 B2
`8/2007 Liberty
`
`705—\’
`
`8/2008 Liberty
`7,414,611 B2
`7,489,298 132 MW Liberty et al~
`7,535,456 B2
`5/2009 leerty et a1.
`7,774,155 B2*
`8/2010 Sato et a1.
`..................... 702/127
`
`7,817,134 B2* 10/2010 Huang et a1.
`345/158
`4/2011 Ohta ...............
`7,924,264 132*
`345/157
`
`8,010,313 B2 *
`8/2011 Mathews et a1.
`702/141
`
`..... 463/31
`2008/0096654 A1*
`4/2008 Mondesir et a1.
`2009/0262074 A1* 10/2009 Nasiri et a1.
`345/158
`
`12/2011 Riley ............................ 701/220
`2011/0307173 A1,,
`OTHER PUBLICATIONS
`
`Azuma, Ronald et a1. Improving Static and Dynamic Registration in
`an Optical See-Through HMD. Proceedings of SIGGRAPH ’94
`(Orlando, Fla., Jul. 24 29, 1994), Computer Graphics, Annual Con-
`ference Series, 1994, 197 204*
`
`* cited by examiner
`
`Primary Examiner 7 William Boddie
`Assistant Examiner 7 Bryan E Earles
`(74) Attorney, Agent, or Firm 7 Ding Yu Tan
`
`ABSTRACT
`(57)
`A three-dimensional (3D) pointing device capable of accu-
`rately outputting a deviation including yaw, pitch and roll
`angles in a 3D reference frame and preferably in an absolute
`manner is provided. Said 3D pointing device comprises a
`six-axis 111011011 sensor module including a rotation sensor
`andanaccelerometer, andaprocessing andtransmitting mod-
`ule. The six-axis motion sensor module generates a first sig-
`.
`.
`.
`.
`.
`nal set compr1s1ng angular veloc1t1es and a second s1gnal set
`comprising axial accelerations associated with said move-
`ments and rotations of the 3D pointing device in the 3D
`reference frame. The processing and transmitting module
`utilizes a comparison method to compare the first signal set
`with the second signal set to obtain an updated state of the
`six-axis motion sensor module based on a current state and a
`measured state thereof in order to output the resulting devia-
`tion in the 3D reference frame and preferably in an absolute
`manna
`
`19 Claims, 7 Drawing Sheets
`
`<————
`
`
`
`,
`Output 3rd quaternion
`to is! quatemion
`.
`Obtain resultant
`WW0" including yaw.
`pitch and roll angles
`
`Obtain display data and
`WWW the resultant
`angles to movement
`pattern in the display
`reference frame
`
`
`
`
`
`740
`
`745
`
`75°
`
`GOOGLE 1046
`
`GOOGLE 1046
`
`~—
`Initialize an initial—value
`set
`
`710x Obtain a previous state
`
`(lsl: quatemion) at T—l__2
`—V—
`Obtain measured angular
`
`
`
`’\, velocities at T
`
`715
`
`,—V____
`7201, Obtain a current state
`-
`(2nd quatemian) at T
`Obtain 'measured axial
`72514 aocelerations' of a
`measured state at T
`—v—
`
`730—1, Eggflfiioyrgdgffi: axral
`measured state at T
`—V—
`Obtain an updated state
`7351, (3rd quatemion) by
`comparing current state
`with measured state
`
`
`
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 1 017
`
`US 8,441,438 132
`
`
`
`110
`
`112
`
`XP
`
`111
`
`FIG. 2 (RELATED ART)
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 2 of7
`
`US 8,441,438 132
`
`330
`
`310
`
`320
`
`322
`
`3_og
`
`FIG. 3
`
`

`

`U.S. Patent
`
`May 14, 2013
`
`Sheet 3 of 7
`
`US 8,441,438 B2
`
`342
`
`Rotation
`
`
`
`Data
`
`Transmitting
`Unit
`
`Computing
`Processor
`
`344
`
`FIG
`
`40
`
`/
`
`1.45/99
`
`<<<<Q?».Q4%?
`«owawfxfi
`6%???
`Qavwwwv
`.ovsvsvuov
`,sss~
`ax???Eat
`fixa>>\
`3.6%
`3%.?
`,~99@«x?
`
`546
`
`540
`
`522 520
`
`
`
`
`
`
`
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 4 of7
`
`US 8,441,438 132
`
`620
`
`630
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 5 of7
`
`US 8,441,438 132
`
`Initialize an initial—value
`set
`
` 705
`
`
`
`
` 710 Obtain a previous state
`(1st quaternion) at T—i
`
` 715
`
`o
`o
`Obtain measured angular
`veloc:ties at T
`
`
`
`720
`
`725
`
`730
`
`735
`
`
`Obtain a current state
`
`
`Output 3rd quaternion
`(2nd quaternion) at T
`
`
`
`to lst quaternion
`
`
`
`
`Obtain "measured axial
`
`
`
`Obtain resultant
`accelerations
`of a
`
`deviation including yaw,
`
`
`measured state at T
`
`pitch and roll angles
`
`
`740
`
`745
`
`
`
`
`
`
`Calculate ”predicted axial
`accelerations" of a
`measured state at T
`
`
`
`Obtain an updated state
`(3rd quaternion) by
`comparing current state
`with measured state
`
`FIG. 7
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 6 of7
`
`US 8,441,438 B2
`
`705
`
`Initialize an initial—value
`
`set
`
`Obtain a previous state
`(lst quaternion) at T-1
`
`Obtain measured angular
`velocities at T
`
`Obtain a current state
`(2nd quaternion) at T
`
`Obtain "measured axial
`accelerations” of a
`measured state at T
`
`710
`
`715
`
`720
`
`725
`
`75°
`
`735
`
`
`
`740
`
`
`
`
`
`
`
`Obtain display data and
`figgglgfigiwgedgfimg‘ "Ml
`
`
`measured state at T
`translate the resultant
`angles to movement
`
`
`pattern in the display
`
`
`reference frame
`
`
`
`
`Obtain an updated state
`(5rd quaternion) by
`
`
`comparing current state
`
`with measured state
`
`
`Output 3rd quaternion
`to lst quaternion
`
`Obtain resultant
`,
`.
`,
`,
`deVIation Including yaw,
`pitch and roll angles
`
`745
`
`
`
`750
`
`FIG. 8
`
`

`

`US. Patent
`
`May 14, 2013
`
`Sheet 7 of7
`
`US 8,441,438 132
`
`Pmax
`
`FIG. 9
`
`

`

`US 8,441,438 B2
`
`1
`3D POINTING DEVICE AND METHOD FOR
`COMPENSATING MOVEMENT THEREOF
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application claims priority benefits of U.S. Patent
`Provisional Application No. 61/292,558, filed on Jan. 6,
`2010. The entirety of the above-mentioned patent applica-
`tions is hereby incorporated by reference herein and made a
`part of this specification.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention generally relates to a three-dimen-
`sional (3D) pointing device utilizing a motion sensor module
`and method of compensating and mapping signals of the
`motion sensor module subject to movements and rotations of
`said 3D pointing device. More particularly, the present inven-
`tion relates to a 3D pointing device utilizing a six-axis motion
`sensor module with an enhanced comparison to calculate and
`compensate accumulated errors associated with the motion
`sensor module and to obtain actual resulting deviation angles
`in spatial reference frame and under dynamic environments.
`2. Description of the Related Art
`FIG. 1 is a schematic diagram showing a user using a
`handheld 3D pointing device 110 to point at a point on the
`screen 122 of a 2D display device 120. If the pointing device
`110 emits a light beam, the point would be the location where
`the light beam hits the screen 122. For example, the pointing
`device 110 may be a mouse of a computer or a pad of a video
`game console. The display device 120 may be a part of the
`computer or the video game console. There are two reference
`frames, such as the spatial pointer reference frame and the
`display frame, associated with the pointing device 110 and
`the display device 120, respectively. The first reference frame
`or spatial pointer reference frame associated with the pointing
`device 110 is defined by the coordinate axes X1,, Y1, and Z1, as
`shown in FIG. 1. The second reference frame or display frame
`associated with the display device 120 is defined by the coor-
`dinate axes XD, YD and ZD as shown in FIG. 1. The screen 122
`ofthe display device 120 is a subset ofthe XDYD plane ofthe
`reference frame XDYDZD associated with the display device
`120. Therefore, the XDYD plane is also known as the display
`plane associated with the display device 120.
`A user may perform control actions and movements utiliz-
`ing the pointing device for certain purposes including enter-
`tainment such as playing a video game, on the display device
`120 through the aforementioned pointer on the screen 122.
`For proper interaction with the use of the pointing device,
`when the user moves the pointing device 110, the pointer on
`the screen 122 is expected to move along with the orientation,
`direction and distance travelled by the pointing device 110
`and the display 120 shall display such movement of the
`pointer to a new location on the screen 122 ofthe display 120.
`The orientation ofthe pointing device 1 10 may be represented
`by three deviation angles of the 3D pointing device 110 with
`respect to the reference frame XPYPZP, namely, the yaw
`angle 1 11, the pitch angle 1 12 and the roll angle 1 13. The yaw,
`pitch and roll angles 111, 112, 113 may be best understood in
`relation to the universal standard definition of spatial angles
`related to commercial vehicles or transportation such as ships
`and airplanes. Conventionally, the yaw angle 111 may repre-
`sent the rotation of the pointing device 110 about the ZP axis;
`the pitch angle 112 may represent the rotation of the pointing
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`device 110 about theYP axis; the roll angle 113 may represent
`the rotation of the pointing device 110 about the X], axis.
`In a known related art as shown in FIG. 1, when the yaw
`angle 111 of the pointing device 110 changes, the aforemen-
`tioned pointer on the screen 122 must move horizontally or in
`a horizontal direction with reference to the ground in
`response to the change of the yaw angle 111. FIG. 2 shows
`what happens when the user rotates the pointing device 110
`counterclockwise by a degree such as a 90-degree about the
`XP axis.
`In another known related art as shown in FIG. 2, when the
`yaw angle 111 changes, the aforementioned pointer on the
`screen 122 is expected to move vertically in response. The
`change ofthe yaw angle 111 can be detected by a gyro-sensor
`which detects the angular velocity 00X of the pointing device
`110 about the X], axis. FIG. 1 and FIG. 2 show that the same
`change of the yaw angle 111 may be mapped to different
`movements ofthe point on the screen 122. Therefore, a proper
`compensation mechanism for the orientation of the pointing
`device 110 is required such that corresponding mapping of
`the pointer on the screen 122 of the display 120 may be
`obtained correctly and desirably. The term compensation of
`the prior arts by Liberty (U.S. Pat. No. 7,158,118, U.S. Pat.
`No. 7,262,760 and U.S. Pat. No. 7,414,611) refers to the
`correction and compensation of signals subject to gravity
`effects or extra rotations about the axis related to “roll”. The
`
`term of “comparison” of the present invention may generally
`refer to the calculating and obtaining of the actual deviation
`angles of the 3D pointing device 110 with respect to the first
`reference frame or spatial pointing frame XPYPZP utilizing
`signals generated by motion sensors while reducing or elimi-
`nating noises associated with said motion sensors; whereas
`the term mapping may refer to the calculating and translating
`of said deviation angles in the sptatial pointing frame
`XPYPZP onto the aforementioned pointer on the display
`plane associated with the 2D display device 120 of a second
`reference frame or display frame XDYDZD.
`It is known that a pointing device utilizing 5-axis motion
`sensors, namely, Ax, Ay, Az, (nYand (DZ may be compensated.
`For example, U.S. Pat. No. 7,158,118 by Liberty, U.S. Pat.
`No. 7,262,760 by Liberty and U.S. Pat. No. 7,414,611 by
`Liberty provide such pointing device having a 5-axis motion
`sensor and discloses a compensation using two gyro-sensors
`(DY and (1)210 detect rotation about the Yp and Zp axes, and
`accelerometers Ax, Ay and Az to detect the acceleration ofthe
`pointing device along the three axes of the reference frame
`XPYPZP. The pointing device by Liberty utilizing a 5-axis
`motion sensor may not output deviation angles ofthe pointing
`device in, for example, a 3D reference frame; in other words,
`due to due to the limitation of the 5-axis motion sensor of
`
`accelerometers and gyro-sensors utilized therein, the point-
`ing device by Liberty cannot output deviation angles readily
`in 3D reference frame but rather a 2D reference frame only
`and the output of such device having 5-axis motion sensors is
`a planar pattern in 2D reference frame only. In addition, it has
`been found that the pointing device and compensation dis-
`closed therein cannot accurately or properly calculate or
`obtain movements, angles and directions of the pointing
`device while being subject to unexpected dynamic movement
`during the obtaining of the signals generated by the motion
`sensors, in particular, during unexpected drifting movements
`and/or accelerations along with the direction of gravity. In
`other words, it has been found that dynamic actions or extra
`accelerations including additional accelerations, in particular
`the one acted upon the direction substantially parallel to or
`along with the gravity imposed on the pointing device with
`the compensation methods provided by Liberty, said pointing
`
`

`

`US 8,441,438 B2
`
`3
`device by Liberty cannot properly or accurately output the
`actual yaw, pitch and roll angles in the spatial reference frame
`XPYPZP and following which, consequently, the mapping of
`the spatial angles onto any 2D display reference frame such as
`XDYDZD may be greatly affected and erred. To be more
`specific, as the 5-axis compensation by Liberty cannot detect
`or compensate rotation about the X], axis directly or accu-
`rately, the rotation about the XP axis has to be derived from the
`gravitational acceleration detected by the accelerometer. Fur-
`thermore, the reading of the accelerometer may be accurate
`only when the pointing device is static since due to the limi-
`tation on known accelerometers that these sensors may not
`distinguish the gravitational acceleration from the accelera-
`tion ofthe forces including centrifugal forces or other types of
`additional accelerations imposed or exerted by the user.
`Furthermore, it has been found that known prior arts may
`only be able to output a “relative” movement pattern in a 2D
`reference frame based on the result calculated from the sig-
`nals of motion sensors. For example, the abovementioned
`prior arts by Liberty may only output a 2D movement pattern
`in a relative manner and a pointer on a display screen to show
`such corresponding 2D relative movement pattern. To be
`more specific, the pointer moves from a first location to a
`second new location relative to said first location only. Such
`relative movement from the previous location to the next
`location with respect to time cannot accurately determine
`and/or output the next location, particularly in situations
`where the previous location may have been an erred location
`or have been faultily determined as an incorrect reference
`point for the next location that is to be calculated therefrom
`and obtained based on their relative relationship adapted. One
`illustration of such defect of known prior arts adapting a
`relative relationship in obtaining a movement pattern may be
`clearly illustrated by an example showing the faultily output-
`ted movements of a pointer intended to move out of a bound-
`ary or an edge of display screen. It has been found that as the
`pointer of known prior arts reaches the edge of a display and
`continues to move out of the boundary or edge at a certain
`extra extent beyond said boundary, the pointer fails to dem-
`onstrate a correct or “absolute” pattern as it moves to a new
`location either within the display or remaining outside of the
`boundary; in other words, instead of returning to a new loca-
`tion by taking into account said certain extra extend beyond
`the boundary made earlier in an “absolute” manner,
`the
`pointer of known arts discards such virtual distance of the
`extra extend beyond the boundary already made and an erred
`next position is faultily outputted due to the relative relation-
`ship adapted and utilized by the pointer. may be never calcu-
`lated or processed due to the faultily obtained location at the
`edge or boundary of the display as well as the relative rela-
`tionship adapted to obtain its next location therefrom.
`Therefore, it is clear that an improved pointing device with
`enhanced calculating or comparison method capable of accu-
`rately obtaining and calculating actual deviation angles in the
`spatial pointer frame as well as mapping of such angles onto
`a pointer on the display frame in dynamic environments and
`conditions is needed. In addition, as the trend of 3D technol-
`ogy advances and is applicable to various fields including
`displays and interactive systems, there is a significant need for
`a 3D pointing device capable ofaccurately outputting a devia-
`tion of such device readily useful in a 3D or spatial reference
`frame. Furthermore, there is a need to provide an enhanced
`comparison method applicable to the processing of signals of
`motion sensors such that errors and/or noises associated with
`
`such signals or fusion of signals from the motions sensors
`may be corrected or eliminated. In addition, according to the
`field of application, such output of deviation in 3D reference
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`frame may too be further mapped or translated to a pattern
`useful in a 2D reference frame.
`
`4
`
`SUMMARY OF THE INVENTION
`
`According to one aspect of an example embodiment of the
`present invention, a 3D pointing device utilizing a six-axis
`motion sensor module is provided. The 3D pointing device
`comprises an accelerometer to measure or detect axial accel-
`erations Ax, Az, Ay and a rotation sensor to measure or detect
`angular velocities 00X, my, 002 such that resulting deviation
`including resultant angles comprising yaw, pitch and roll
`angles in a spatial pointer frame of the 3D pointing device
`subject to movements and rotations in dynamic environments
`may be obtained and such that said resulting deviation includ-
`ing said resultant angles may be obtained and outputted in an
`absolute manner reflecting or associating with the actual
`movements and rotations of the 3D pointer device of the
`present invention in said spatial pointer reference frame.
`According to another aspect of the present invention, the
`present invention provides an enhanced comparison method
`to eliminate the accumulated errors as well as noises over
`
`time associated with signals generated by a combination of
`motion sensors, including the ones generated by accelerom-
`eters AX, Ay, AZ and the ones generated by gyroscopes 00X, my,
`002 in dynamic environments. In other words, accumulated
`errors associated with a fusion of signals from a motions
`sensor module comprising a plurality of motion sensors to
`detect movements on and rotations about different axes of a
`
`reference frame may be eliminated or corrected.
`According to still another aspect of the present invention,
`the present
`invention provides an enhanced comparison
`method to correctly calculating and outputting a resulting
`deviation comprising a set of resultant angles including yaw,
`pitch and roll angles in a spatial pointer frame, preferably
`about each of three orthogonal coordinate axes of the spatial
`pointer reference frame, by comparing signals of rotation
`sensor related to angular velocities or rates with the ones of
`accelerometer related to axial accelerations such that these
`
`angles may be accurately outputted and obtained, which may
`too be further mapping to another reference frame different
`from said spatial pointer frame.
`According to still another aspect of the present invention,
`the present invention provides a mapping of the abovemen-
`tioned resultant angles, preferably about each of three
`orthogonal coordinate axes of the spatial pointer reference
`frame, including yaw, pitch and roll angles in a spatial pointer
`reference frame onto a display frame such that a movement
`pattern in a display frame different from the spatial pointer
`reference frame may be obtained according to the mapping or
`translation of the resultant angles of the resultant deviation
`onto said movement pattern.
`According to another example embodiment of the present
`invention, a 3D pointing device utilizing a six-axis motion
`sensor module with an enhanced comparison method for
`eliminating accumulated errors of said six-axis motion sensor
`module to obtain deviation angles corresponding to move-
`ments and rotations of said 3D pointing device in a spatial
`pointer reference frame is provided. The 3D pointing device
`and the comparison method provided by the present invention
`by comparing signals from the abovementioned six-axis
`motion sensor module capable of detecting rotation rates or
`angular velocities of the 3D pointing device about all of the
`XP, YP and Z1, axes as well as axial accelerations of the 3D
`pointing device along all of the XP, YP and ZP axes. In other
`words, the present invention is capable of accurately output-
`ting the abovementioned deviation angles including yaw,
`
`

`

`US 8,441,438 B2
`
`5
`pitch and roll angles in a 3D spatial pointer reference frame of
`the 3D pointing device to eliminate or reduce accumulated
`errors and noises generated over time in a dynamic environ-
`ment including conditions such as being subject to a combi-
`nation of continuous movements, rotations, external gravity
`forces and additional extra accelerations in multiple direc-
`tions or movement and rotations that are continuously non-
`linear with respect to time; and furthermore, based on the
`deviation angles being compensated and accurately outputted
`in 3D spatial pointer reference frame may be further mapped
`onto or translated into another reference frame such as the
`
`abovementioned display frame, for example a reference in
`two-dimension (2D).
`According to another example embodiment of the present
`invention, a 3D pointing device utilizing a six-axis motion
`sensor module is provided; wherein the six-axis motion sen-
`sor module of the 3D pointing device comprises at least one
`gyroscope and at least one accelerometer. In one preferred
`embodiment of the present invention, the six-axis motion
`sensor module comprises a rotation sensor capable of detect-
`ing and generating angular velocities of 00X, my, 002 and an
`accelerometer capable of detecting and generating axial
`accelerations of Ax, Ay, AZ. It can be understood that in
`another preferred embodiment, the abovementioned rotation
`sensor may comprise three gyroscopes corresponding to each
`of the said angular velocities of 00X, my, 002 in a 3D spatial
`pointer reference frame of the 3D pointing device; whereas
`the abovementioned accelerometer may comprise three
`accelerometers corresponding to each of the said axial accel-
`erations Ax, Ay, AZ in a 3D spatial pointer reference frame of
`the 3D pointing device. The rotation sensor detects the rota-
`tion of the 3D pointing device with respect to a reference
`frame associated with the 3D pointing device and provides a
`rotation rate or angular velocity output. The angular velocity
`output includes three components corresponding to the rota-
`tion rate or angular velocities 00X, my, 002 of the 3D pointing
`device about the first axis, the second axis and the third axis of
`the reference frame, namely, Xp, Yp and Zp of the 3D spatial
`pointer frame. The accelerometer detects the axial accelera-
`tions of the 3D pointing device with respect to the spatial
`pointer reference frame such as a 3D-pointer reference frame
`and provides an acceleration output. The acceleration output
`includes three components corresponding to the accelera-
`tions, Ax, AZ, Ay ofthe 3D pointing device along the first axis,
`the second axis and the third axis of the reference frame,
`namely, Xp,Yp and Zp ofthe 3D spatial pointer frame. It can,
`however, be understood that the axes of Xp, Yp and Zp of the
`3D spatial pointer frame may too be represented simply by the
`denotation of X, Y and Z.
`According to another example embodiment of the present
`invention, a method for compensating accumulated errors of
`signals ofthe abovementioned six-axis motion sensor module
`in dynamic environments associated in a spatial pointer ref-
`erence frame is provided. In one embodiment, the method
`may be performed or handled by a hardware processor. The
`processor is capable of compensating the accumulated errors
`associated with the resultant deviation in relation to the sig-
`nals of the above-mentioned six-axis motion sensor module
`
`ofthe 3D pointing device subject to movements and rotations
`in a spatial pointer reference frame and in a dynamic envi-
`ronment by performing a data comparison to compare signals
`ofrotation sensor related to angular velocities with the ones of
`accelerometer related to axial accelerations such that the
`
`resultant deviation corresponding to the movements and rota-
`tions ofthe 3D pointing device in the 3D spatial pointer frame
`may be obtained accurately over time in the dynamic envi-
`ronments.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`According to another embodiment ofthe present invention,
`a method for obtaining a resulting deviation including result-
`ant angles in a spatial pointer reference frame of a three-
`dimensional (3D) pointing device utiliZing a six-axis motion
`sensor module therein and subject to movements and rota-
`tions in dynamic environments in said spatial pointer refer-
`ence frame is provided. Said method comprises the steps of:
`obtaining a previous state associated with previous angular
`velocities 00X, my, 002 gained from the motion sensor signals of
`the six-axis motion sensor module at a previous time T—l;
`obtaining a current state ofthe six-axis motion sensor module
`by obtaining measured angular velocities 00X, my, 002 gained
`from the motion sensor signals at a current time T; obtaining
`a measured state of the six-axis motion sensor module by
`obtaining measured axial accelerations Ax, Ay, AZ gained
`from the motion sensor signals at the current time T and
`calculating predicted axial accelerations Ax', Ay', AZ' based
`on the measured angular velocities 00X, my, 002 of the current
`state; obtaining an updated state ofthe six-axis motion sensor
`module by comparing the current state with the measured
`state of the six-axis motion sensor module; and calculating
`and converting the updated state ofthe six axis motion sensor
`module to said resulting deviation comprising said resultant
`angles in said spatial pointer reference frame ofthe 3D point-
`ing device.
`According to another aspect of the present invention, a
`method for mapping deviation angles associated with move-
`ments and rotations of a 3D pointing device in a spatial
`pointer reference frame onto a display frame of a display
`having a predetermined screen size is provided. In one
`embodiment, the method for mapping or translating deviation
`angles including yaw, pitch and roll angles in a spatial pointer
`reference frame to an pointing object, such as a pointer, hav-
`ing movements in a display frame, preferably a 2D reference
`frame, comprises the steps of obtaining boundary informa-
`tion of the display frame by calculating a predefined sensitiv-
`ity associated with the display frame and performing angle
`and distance translation in the display frame based on said
`deviation angles and boundary information.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings are included to provide a
`further understanding of the invention, and are incorporated
`in and constitute a part of this specification. The drawings
`illustrate embodiments ofthe invention and, together with the
`description, serve to explain the principles of the invention.
`FIG. 1 shows a known related art having a 5-axis motion
`sensor in 2D reference frame.
`
`FIG. 2 shows the known related art having a 5-axis motion
`sensor as shown in FIG. 1 being rotated or rolled about Xp
`axis and is subject to further dynamic interactions or environ-
`ment.
`
`FIG. 3 is an exploded diagram showing a 3D pointing
`device utiliZing a six-axis motion sensor module according to
`one embodiment of the present invention in a 3D spatial
`pointer reference frame.
`FIG. 4 is a schematic block diagram illustrating hardware
`components of a 3D pointing device according to one
`embodiment of the present invention.
`FIG. 5 is a schematic diagram showing a 3D pointing
`device utiliZing a six-axis motion sensor module according to
`anther embodiment of the present invention in a 3D spatial
`pointer reference frame.
`
`

`

`US 8,441,438 B2
`
`7
`FIG. 6 is an exploded diagram showing a 3D pointing
`device utilizing a six-axis motion sensor module according to
`anther embodiment of the present invention in a 3D spatial
`pointer reference frame.
`FIG. 7 is a flow chart illustrating a method for compensat-
`ing deviation angles of a 3D pointing device having move-
`ments and rotations in a 3D spatial pointer reference frame
`and in a dynamic environment according to an embodiment of
`the present invention.
`FIG. 8 shows a flow chart illustrating a method ofmapping
`deviation angles of a 3D pointing device having movements
`and rotations in a 3D spatial pointer reference frame and in a
`dynamic environment onto a display reference frame accord-
`ing to another embodiment of the present invention.
`FIG. 9 is a schematic diagram showing the mapping of the
`resultant angles of the resultant deviation of a 3D pointing
`device according to an embodiment of the present invention.
`
`DESCRIPTION OF THE EMBODIMENTS
`
`to the present
`Reference will now be made in detail
`embodiments of the invention, examples of which are illus-
`trated in the accompanying drawings. Wherever possible, the
`same reference numbers are used in the drawings and the
`description to refer to the same or like parts.
`FIG. 3 is an exploded diagram showing a 3D pointing
`device 300 according to an embodiment of the present inven-
`tion. The 3D pointing device 300 is subject to movements and
`rotations in dynamic environments in a 3D spatial pointer
`reference frame. The spatial pointer reference frame is analo-
`gous to the reference frame XPYPZP in FIG. 1 and FIG. 2. The
`movements and rotations ofthe 3D pointing device 300 in the
`aforementioned dynamic environments in the spatial pointer
`reference frame may be continuously nonlinear with respect
`to time.
`
`The 3D pointing device 300 includes a top cover 310, a
`printed circuit board (PCB) 340, a rotation sensor 342, an
`accelerometer 344, a data transmitting unit 346, a computing
`processor 348, a bottom cover 320, and a battery pack 322.
`The top cover 310 may include a few control buttons 312 for
`a user to issue predefined commands for remote control. In
`one embodiment, the housing 330 may comprise the top
`cover 310 and the bottom cover 320. The housing 330 may
`move and rotate in the spatial pointer reference frame accord-
`ing to user manipulation or any external forces in any direc-
`tion and/or under the abovementioned dynamic environ-
`ments. As shown in the FIG. 3, in one embodiment, the
`rotation sensor 342, the accelerometer 344, the data transmit-
`ting unit 346, and the computing processor 348 may be all
`attached to the PCB 340. The PCB 340 is enclosed by the
`housing 330. The PCB 340 includes at least one substrate
`having a longitudinal side configured to be substantially par-
`allel to the longitudinal surface of the housing 330. An addi-
`tional battery pack 322 provides electrical power for the
`entire 3D pointing device 300.
`FIG. 4 is a schematic block diagram illustrating hardware
`components of the 3D pointing device 300. The 3D pointing
`device 300 includes a six-axis motion sensor module 302 and
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`a processing and transmitting module 304. The six-axis
`motion sensor module 302 includes the rotation sensor 342
`
`60
`
`and the accelerometer 344. The processing and transmitting
`module 304 includes the data transmitting unit 346 and the
`computing processor 348.
`The rotation sensor 342 of the six-motion sensor module
`
`302 detects and generates the first signal set including angular
`velocities 00X, my, 002 associated with the movements and rota-
`tions of the 3D pointing device 300 about each of three
`
`65
`
`8
`orthogonal coordinate axes XPYPZP of the spatial pointer
`reference frame. The angular velocities 00X, my and 002 are
`corresponding to the coordinate axes XP, YP and ZP respec-
`tively. The accelerometer 344 detects and generates the sec-
`ond signal set including axial accelerations Ax, Ay, Az asso-
`ciated with the movements and rotations of the 3D pointing
`device 300 along