`
`
`
`
`
`
`
`
`
`I IIIII IIIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111
`
`
`
`
`
`
`
`
`
`
`
`US008552978B2
`
`02)United States Patent
`
`
`Ye et al.
`
`(10)Patent No.: US 8,552,978 B2
`
`(45)Date of Patent:
`Oct. 8, 2013
`
`(54)30 POINTING DEVICE AND METHOD FOR
`
`
`COMPENSATING ROTATIONS OF THE 3D
`POINTING DEVICE THEREOF
`
`(56)
`
`References Cited
`
`(75)Inventors:
`Zhou Ye, Foster City, CA (US);
`
`
`
`Chin-Lung LI, Taoyuan Comity (TW);
`
`
`Shun-Nan Lion, Kaohsiung (TW)
`
`(73)Assignee:
`
`
`
`
`Cywee Group Limited, Tortola (VG)
`
`U.S. PATENT DOCUMENTS
`8/ 1992 Hotelling
`
`5,138,154 A
`8/1995
`
`5,440,326 A
`Quinn
`5,898,421 A
`4/1999 Quinn
`1/2007
`
`7,158,118 B2
`Liberty
`7,236,156
`6/2007 Liberty et al.
`B2
`
`7,239,301 B2
`7/2007
`Liberty et al.
`
`7,262,760 B2
`8/2007 Liberty
`
`7,414,611 82
`8/2008 Liberty
`the term of this ( *) Notice: Subject to any disclaimer,
`
`
`
`2009/0262074 Al *
`
`
`
`
`
`
`
`patent is e11.1ended or adjusted under 35
`* cited by examiner
`
`U.S.C. 154(b) by 144 days.
`
`10/2009 Nasiri et al. ................. 345/158
`
`(21)Appl. No.: 13/176,771
`
`(22)Filed:Jul. 6, 2011
`
`
`
`Prior Publication Data
`
`Primary Examiner- Lun-Yi Lao
`
`
`
`Assistant Examiner - Insa Sadio
`
`
`(74)Attorney, Agent, or Finn - Ding Yu Tan
`
`(57)
`
`ABSTRACT
`
`(65)
`
`
`US 2011/0260968 A l Oct. 27,201 I
`
`A 3D pointing device utilizing an orientation sensor, capable
`
`
`
`
`
`
`
`
`
`of accurately transforming rotations and movements of the
`
`
`
`3D pointing device into a movement pattern in the display
`(63)Continuation-in-part of application No. 13/072,794,
`
`
`
`
`
`
`plane of a display device is provided. The 3D pointing device
`filed on Mar. 28,201 I, which is a continuation-in-part
`
`
`
`
`
`includes the orientation sensor, a rotation sensor, and a com­
`
`
`of application No. 12/943,934, filed on Nov. 1I,2010.
`
`
`
`
`
`puting processor. The orientation sensor generates an orien­
`
`
`
`filed on Jan.(60)Provisional application No. 61/292,558,
`
`
`
`
`tation output associated with the orientation of the 30 point­
`6, 2010.
`
`
`
`ing device associated with three coordinate axes of a global
`
`
`
`
`reference frame associated with the Earth. The rotation sensor
`
`
`
`
`
`generates a rotation output associated with the rotation of the
`
`
`
`
`3D pointing device associated with three coordinate axes of a
`
`
`
`spatial reference frame associated with the 30 pointing
`G09G5!08
`(52)U.S. Cl.
`
`
`
`
`device itself The computing processor uses the orientation
`USPC ........... 345/157; 345/156; 345/158; 345/173;
`
`
`
`
`
`
`
`
`
`output and the rotation output to generate a transformed out­
`
`
`178/18.01; 178/18.03; 178/19.01
`
`
`
`put associated with a fixed reference frame associated with
`(58)Field of Classification
`
`
`
`
`the display device above. The transformed output represents
`Search
`USPC ... 345/156-168, 173-183;
`178/18.01-18.04,
`
`
`a segment of the movement pattern.
`178/19.01-19.04
`
`
`See application file for complete search history.
`
`
`
`Related U.S. Application Data
`
`
`
`(51)Int. Cl.
`G06F 31033
`
`(2013.01)
`(2006.01)
`
`
`
`
`
`18 Claims, 12 Drawing Sheets
`
`342
`r--------1 r-302
`348 304
`-�-�r"'
`346
`/
`I
`I r----------------
`_ _L __ ,
`1 I
`I
`I Data
`I
`I
`T ronsmtting
`I
`Una
`I
`I
`I
`
`Computing
`Processor
`
`I
`
`926
`
`'-{
`
`L----------------------J
`
`345
`
`922
`
`d
`
`Magnetometer
`
`910
`
`L---------
`
`001
`
`
`
` LG 1001
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 1 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`110
`
`
`
`
`112
`
`XP
`
`
`
`111
`
`
`
`
`
`FIG. 2 (RELATED ART)
`
`
`
`002
`
`002
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 2 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`330
`
`
`
`310
`
`320
`
`
`FIG. 3
`
`003
`
`003
`
`

`

`U.S. Patent
`
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 3 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`T
`
`I1_4__w1__g1“.mwht_Wm_M_m0_3_0r__CP___~______~___9__.m_ton.“a_mm._MDn_m_
`_|IIIIIII
`
`
`
`
`
`
`
`
`
`
`
`
`__
`
`
`
`
`
`
`
`
`
`_llllllllllll||||___
`
`
`
`H
`
`
`
`
`
`
`
`
`
`
`M.nr"3_"mm_HmmH_RS_____rlllllllllllllllllllllllL
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 4
`
`/‘>
`
`iv
`
`
`\%,~.59.?
`13%6.}?6,,
`r’3‘?
`$st
`\«6505959
`gag
`a~a-~¢
`ax???
`“Nag?
`«kg?
`ax???
`s “gag
`
`546
`
`
`
`540
`
`
`
`
`
`
`
`004
`
`004
`
`
`
`
`
`
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 4 of 12
`
`
`
`US 8,552,978 B2
`
`630
`
`
`
`005
`
`005
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 5 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`Initialize an initial—value
`
`set
`
`
`
`
`
`
`
`
`
`
`
`
`705
`
`
`710
`
`715
`
`720
`
`725
`
`730
`
`735
`
`
`
`
`
`
`
`Output 3rd quaternion
`
`
`
`to ist quaternion
`
`
`
`
`
`
`
`740
`
`
`
`745
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`
`(lst quaternion) at T—1
`
`
`
`
`
`
`Obtain measured angular
`
`
`veloc:ties at T
`
`
`
`Obtain a current state
`
`
`
`
`
`(2nd quaternion) at T
`
`
`
`
`
` Obtain ”measnured axial
`
`
`
`
`
`
`accelerations of a
`
`
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`
`
`Calculate ”predicted axial
`
`
`
`accelerations" based on
`
`
`
`
`current state at T
`
`
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`Obtain resultant
`
`
`
`
`
`deviation including yaw,
`
`
`
`
`
`
`
`pitch and roll angles
`
`
`FIG. 7
`
`006
`
`006
`
`

`

`
`
`U.S. Patent
`
`
`Oct. 8, 2013
`
`
`
`Sheet 6 of 12
`
`
`
`
`US 8,552,978 B2
`
`705
`
`
`
`
`
`
`Initialize an initial—value
`
`set
`
`
`
`
`Obtain a previous state
`
`
`
`
`(ist quatemion) at T-l
`
`
`
`
`
`
`Obtain measured angular
`
`
`velomties at T
`
`
`
`
`
`
`
`710
`
`
`
`715
`
`720
`
`725
`
`
`
`Output 3rd quatemion
`
`
`
`to let quatemion
`
`
`
`740
`
`
`
`
`
`Obtain resultant
`
`
`deviation including yaw,
`
`
`
`
`pitch and roll angles
`
`
`
`
`
`
`
`
`
`
`
`
`Obtain display data and
`
`
`
`translate the resultant
`
`
`
`angles to movement
`
`
`
`
`pattern in the display
`
`
`
`reference frame
`
`745
`
`
`
`750
`
`
`
`
`
`
`Obtain a current state
`
`
`
`
`(2nd quaternion) at T
`
` Obtain ”measured axial
`
`
`
`
`
`
`
`
`accelerations of a
`
`
`
`measured state at T
`
` Calculate "predicted axial
`
`
`
`
`
`
`
`
`
`accelerations" based on
`
`
`
`current state at T
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`
`
`730
`
`
`
`735
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG.
`
`
`8
`
`007
`
`007
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 7 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 9
`
`008
`
`008
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 8 of 12
`
`
`
`US 8,552,978 B2
`
`1005
`
`
`
`
`Initialize an initial-value set
`
`
`
`1010
`
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`(lst quaternion) at T—1
`
`
`
`
`
`
`
`1015
`
`
`
`
`
`Obtain measured angular
`
`
`velocities at T
`
`1020
`
`1025
`
`1030
`
`1035
`
`1040
`
`1045
`
`Output 5rd quaternion
`
`
`
`
`
`to 1st quaternion
`
`
`
`
`
`Obtain resultant
`
`
`deviation including yaw,
`
`
`
`
`
`
`
`
`
`1060
`
`
`
`
`(2nd quaternion) at T
`
`
`
`
`
`Obtain a current state
`
`
`
`
` Obtain "measured axial
`
`pitch and roll angles
`
`
`
`
`
`
`accelerations" of a
`
`
`
`measured state at T
`
`
`
`Calculate "predicted axial
`
`
`
`
`accelerations" based on
`
`
`
`current state at T
`
`
`
`
`
`
`
`
`
`
`
`Obtain "measured
`
`magnetism" of a
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`
`
` Calculate "predicted
`
`
`
`magnetism" based on
`
`
`
`
`
`
`current state at T
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`
`
`
`
`FIG. 10
`
`
`
`
`
`009
`
`009
`
`

`

`
`
`US. Patent
`
`Oct. 8, 2013
`
`
`
`
`Sheet 9 of 12
`
`
`
`
`US 8,552,978 B2
`
`“05
`
`
`
`
`Initialize an initial-value set
`
`
`
`1110
`
`1115
`
`1120
`
`1125
`
`1130
`
`“35
`
`1140
`
`1145
`
`1 150
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Output 4th quaternion
`
`
`
`to lst quaternion
`
`
`
`
`
`Obtain resultant
`deviation including yaw,
`
`
`
`
`
`
`pitch and roll angles
`
`
`
`,
`,
`-
`
`
`
`Obtain display data and
`
`
`
`translate the resultant
`
`
`
`angles to movement
`
`
`
`
`pattern in the display
`
`
`reference frame
`
`
`
`
`
`
`
`
`“60
`
`
`
`1165
`
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`
`(ist quaternion) at T—1
`
`
`
`
`
`
`
`Obtain measured angular
`
`
`velocuties at T
`
`
`
`
`
`
`Obtain a current state
`
`
`
`(2nd quaternion) at T
`
`
`
`
`Obtain "measured axial
`n
`o
`
`
`accelerations of a
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`Calculate "predicted axial
`
`
`
`
`accelerations" based on
`
`current state at T
`
`
`
`
`
`
`
`
`
`Obtain a 1st updated
`
`
`
`
`
`
`
`state (3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`
`
`with measured state
`
`
`
`
`Obtain and calculate
`
`
`
`
`
`”measured yaw angle" of
`
`
`
`
`
`a measured state at T
`
`
`
`Calculate ”predicted
`
`
`
`
`
`
`
`yaw angle" based on 1st
`
`
`
`updated state at T
`
`
`
`
`
`
`Obtain a 2nd updated
`
`
`
`
`state (4th quaternion) by
`
`
`
`comparing current state
`with measured state
`
`
`
`
`
`
`
`
`010
`
`.
`
`
`FIG.1 1
`
`
`
`010
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 10 of 12
`
`
`
`US 8,552,978 B2
`
`1210
`
`1220
`
`1225
`
`
`
`1255
`
`
`
`
`
`
`
`
`Perform 2nc
`
`data association
`
`
`
`
`
`
`
`and determine whether
`
`
`
`
`
`the result of comparison
`
`
`
`
`falls within a
`
`predetermined
`
`value
`
`1260
`
`
`
`
`
`
`
`
`Obtain a 2nd updated
`
`state
`
`
`
`
`
`
`
`
`
`Obtain a previous state at T—1
`
`
`
`whicwiefltimizatj
`T
`tobtain measured angular
`
`
`' velocities at T
`'
`L ______________ J
`
`
`
`
`
`
`
`
`
`Obtain 1st measured state at T
`T__7_____"_T___T
`
`
`
`
`
`
`
`
`
`
`l obtain measured axnal
`r
`
`
`
`
`L accelerations at T
`
`J1
`
`
`
`
`
`
`Calculate lst predicted
`
`measurement
`
`
`(predicted axial accelerations)
`
`
`
`
`
`
`1235
`
`
`
`1265
`
`
`
`
`Output updated state
`
`
`
`to previous state
`
`
`
`
`
`
`
`
`
`lst data association
`
`
`
`
`
`
`
`and determine whether the
`
`
`
`
`
`
`
`result of comparison falls withi
`value
`
`
`a predetermined
`
`spatial reference frame Obtain a lst updated state
`
`1240
`
`
`
`
`1 270
`
`
`
`
`Obtain resultant
`
`
`deviation including
`
`
`deviation angles in
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 245
`
`
`
`
`
`
`
`Obtain 2nd measured state at T
`
`
`
`“1
`.
`.
`T
`
`
`
`[obtain measured magnetism;
`
`
`
`
`
`'Land yaw angle at T
`
`1 250
`
`
`
`
`redicted
`Calculate 2nd
`
`
`
`measurement predicted yaw angle)
`
`
`
`
`
`
`FIG. 12
`
`
`
`OH
`
`011
`
`

`

`U.S. Patent
`
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 11 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`device associated with Earth
`
`
`
`
`
`Generate an orientation output
`
`
`
`
`
`assOciated with the 3D pointing
`
`
`
`
`
`
`
`
`
`Generate a rotation output
`
`
`
`
`associated with the 3D
`
`
`pointing device
`
`
`
`
`1 320
`
`
`
`
`
`Obtain the orientation of
`
`the display device
`associated with Earth
`
`
`
`
`
`
`
`Generate a transformed
`
`rotation associated with the
`display device
`
`Generate the transformed
`
`1368
`
`output based on the
`transformed rotation
`
`
`
`
`
`
`
`012
`
`device
`
`
`
`Obtain the orientation of the
`3D pointing device
`
`associated with the display
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1366
`
`012
`
`

`

`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 12 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 420
`
`Computing
`Processor
`
`
`
`
`
`
`1 420
`s/’*151O
`
`
`
`
`
`
`
`
`
`
`
`
`
`Computing
`
`
`
`Processor
`
`
`
`
`
`
`
`345
`
`344
`
`
`Orientation Sensor
`
`
`
`
`FIG. 15
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 420
`Computing
`
`Processor
`
`345
`
`
`
`
`
`
`
`
`
`
`
`
`
`013
`
`

`

`
`
`US 8,552,978 B2
`
`1
`
`3D POINTING DEVICE AND METHOD FOR
`
`
`
`
`
`
`COMPENSATING ROTATIONS OF THE 3D
`
`
`
`POINTING DEVICE THEREOF
`
`
`
`
`CROSS-REFERENCE TO RELATED
`
`
`APPLICATION
`
`
`This application is a continuation in part application of and
`
`
`
`
`
`
`claims the priority benefit ofU. S. application Ser. No. 13/072,
`
`
`
`
`
`
`
`
`794, filed on Mar. 28, 201 1, now pending. The prior applica-
`
`
`
`
`
`
`
`
`
`
`tion Ser. No. 13/072,794 is a continuation in part application
`
`
`
`
`
`
`
`of and claims the priority benefit of U.S. application Ser. No.
`
`
`
`
`
`
`
`
`
`
`12/943,934, filed on Nov. 11, 2010, now pending, which
`
`
`
`
`
`
`
`
`
`claims the priority benefit ofU.S. provisional application Ser.
`
`
`
`
`
`
`
`
`No. 61/292,558, filed on Jan. 6, 2010. The entirety of each of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the above-mentioned patent applications is hereby incorpo-
`rated by reference herein and made a part ofthis specification.
`
`
`
`
`
`
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`1. Field of the Invention
`
`
`
`The present invention generally relates to a 3D pointing
`
`
`
`
`
`
`
`device, more particularly to a 3D pointing device for use in
`
`
`
`
`
`
`
`computers, motion detection or navigation utilizing a orien-
`
`
`
`
`
`
`tation sensor and a method for compensating signals of the
`
`
`
`
`
`
`
`orientation sensor subject to movements and rotations of said
`
`
`
`
`
`
`
`3D pointing device.
`
`
`
`2. Description of the Related Art
`
`
`
`
`FIG. 1 is a schematic diagram showing a user using a
`
`
`
`
`
`
`portable electronic device 110, such as a 3D pointing device
`
`
`
`
`
`
`
`or computer mouse, for detecting motions of the device and
`
`
`
`
`
`
`
`
`
`translating the detected motions to a cursor display such as a
`
`
`
`
`
`
`
`cursor pointing on the screen 122 of a 2D display device 120.
`
`
`
`
`
`
`
`Ifthe pointing device 110 emits a light beam, the correspond-
`
`
`
`
`
`
`
`
`
`
`ing point would be the location where the light beam hits the
`
`
`
`
`
`
`
`
`
`
`screen 122. For example, the pointing device 110 may be a
`
`
`
`
`
`
`
`
`
`mouse of a computer or a pad of a video game console. The
`
`
`
`
`
`
`
`display device 120 may be a part ofthe computer or the video
`
`
`
`
`
`
`
`
`game console. There are two reference frames, such as the
`
`
`
`
`
`
`
`
`
`spatial pointer reference frame and the display frame, asso-
`
`
`
`
`
`
`
`
`
`ciated with the pointing device 110 and the display device
`
`
`
`
`
`
`
`
`
`
`120, respectively. The first reference frame or spatial pointer
`
`
`
`
`
`
`
`
`reference frame associated with the pointing device 110 is
`
`
`
`
`
`
`
`
`defined by the coordinate axes XP, YP and Z1, as shown in
`
`
`
`
`
`
`FIG. 1. The second reference frame or display frame associ-
`
`
`
`
`
`
`
`
`ated with the display device 120 is defined by the coordinate
`
`
`
`
`
`
`
`
`axes XD,YD and ZD as shown in FIG. 1. The screen 122 ofthe
`
`
`
`
`
`
`
`
`display device 120 is a subset of the XDYD plane of the
`
`
`
`
`
`
`
`
`
`
`reference frame XDYDZD associated with the display device
`
`
`
`
`
`
`
`
`120. Therefore, the XDYD plane is also known as the display
`
`
`
`
`
`
`
`
`plane associated with the display device 120.
`
`
`
`
`
`
`
`A user may perform control actions and movements utiliz-
`
`
`
`
`
`
`
`
`ing the pointing device for certain purposes including enter-
`
`
`
`
`
`
`
`
`
`tainment such as playing a video game, on the display device
`
`
`
`
`
`
`
`120 through the aforementioned pointer on the screen 122.
`
`
`
`
`
`
`
`
`For proper interaction with the use of the pointing device,
`
`
`
`
`
`
`
`
`
`when the user moves the pointing device 110, the pointer on
`
`
`
`
`
`
`
`
`the screen 122 is expected to move along with the orientation,
`
`
`
`
`
`
`
`
`
`direction and distance travelled by the pointing device 110
`
`
`
`
`
`
`
`
`and the display 120 shall display such movement of the
`
`
`
`
`
`
`
`
`
`pointer to a new location on the screen 122 ofthe display 120.
`
`
`
`
`
`
`
`The orientation ofthe pointing device 1 10 may be represented
`
`
`
`
`
`
`by three deviation angles of the 3D pointing device 110 with
`
`
`
`
`
`
`
`
`respect to the reference frame XPYPZP, namely, the yaw
`
`
`
`
`
`
`
`angle 111, the pitch angle 112 and the roll angle 113. The yaw,
`
`
`
`
`
`
`
`
`
`
`
`
`pitch and roll angles 111, 112, 113 may be best understood in
`
`
`
`
`
`
`
`
`
`
`relation to the universal standard definition of spatial angles
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`60
`
`
`
`65
`
`
`2
`
`
`related to commercial vehicles or transportation such as ships
`
`
`
`
`
`
`and airplanes. Conventionally, the yaw angle 111 may repre-
`
`
`
`
`
`
`
`
`
`sent the rotation of the pointing device 110 about the ZP axis;
`
`
`
`
`
`
`
`
`
`the pitch angle 112 may represent the rotation of the pointing
`
`
`
`
`
`
`
`
`
`device 110 about theYP axis; the roll angle 113 may represent
`
`
`
`
`
`
`
`
`
`
`
`the rotation of the pointing device 110 about the XP axis.
`
`
`
`
`
`
`
`
`In a known related art as shown in FIG. 1, when the yaw
`
`
`
`
`
`
`
`
`angle 111 of the pointing device 110 changes, the aforemen-
`
`
`
`
`
`
`
`
`tioned pointer on the screen 122 must move horizontally or in
`
`
`
`
`
`
`
`a horizontal direction with reference to the ground in
`
`
`
`
`
`
`
`
`response to the change of the yaw angle 111. FIG. 2 shows
`
`
`
`
`
`
`
`
`
`what happens when the user rotates the pointing device 110
`
`
`
`
`
`
`
`
`
`counterclockwise by a degree such as a 90-degree about the
`
`
`
`
`
`
`X], axis. In another known related art as shown in FIG. 2,
`
`
`
`
`
`
`
`
`
`
`when the yaw angle 111 changes, the aforementioned pointer
`
`
`
`
`
`
`
`
`on the screen 122 is expected to move vertically in response.
`
`
`
`
`
`
`
`The change of the yaw angle 111 can be detected by a gyro-
`
`
`
`
`
`
`
`
`
`sensor which detects the angular velocity 00X of the pointing
`
`
`
`
`
`
`
`
`
`device 110 about the XP axis. FIG. 1 and FIG. 2 show that the
`
`
`
`
`
`
`
`
`
`
`
`same change ofthe yaw angle 111 may be mapped to different
`
`
`
`
`
`
`
`
`movements ofthe point on the screen 122. Therefore, a proper
`
`
`
`
`
`
`compensation mechanism for the orientation of the pointing
`
`
`
`
`
`
`device 110 is required such that corresponding mapping of
`
`
`
`
`
`
`
`the pointer on the screen 122 of the display 120 may be
`
`
`
`
`
`
`
`
`
`
`
`obtained correctly and desirably. The term compensation of
`
`
`
`
`
`
`
`the prior arts by Liberty (U.S. Pat. Nos. 7,158,118, 7,262,760
`
`
`
`
`
`
`
`
`
`and 7,414,611) refers to the correction and compensation of
`
`
`
`
`
`
`
`signals subject to gravity effects or extra rotations about the
`
`
`
`
`
`
`
`
`axis related to “roll”. The term of “comparison” ofthe present
`
`
`
`
`
`
`
`invention may generally refer to the calculating and obtaining
`
`
`
`
`
`
`
`
`of the actual deviation angles of the 3D pointing device 110
`
`
`
`
`
`
`
`
`
`with respect to the first reference frame or spatial pointing
`
`
`
`
`
`
`
`
`frame XPYPZP utilizing signals generated by motion sensors
`
`
`
`
`
`
`
`while reducing or eliminating noises associated with said
`
`
`
`
`
`
`
`motion sensors; whereas the term mapping may refer to the
`
`
`
`
`
`
`
`
`
`calculating and translating of said deviation angles in the
`
`
`
`
`
`
`
`
`spatial pointing frame XPYPZP onto the aforementioned
`
`
`
`
`
`
`
`pointer on the display plane associated with the 2D display
`
`
`
`
`
`
`
`
`device 120 of a second reference frame or display frame
`
`
`
`
`
`
`
`XDYDZD.
`
`
`It is known that a pointing device utilizing 5-axis motion
`
`
`
`
`
`
`
`
`
`sensors, namely, Ax, Ay, Az, (nYand (DZ may be compensated.
`
`
`
`
`
`
`
`
`For example, U.S. Pat. No. 7,158,118 by Liberty, U.S. Pat.
`
`
`
`
`
`
`
`
`
`No. 7,262,760 by Liberty and U.S. Pat. No. 7,414,611 by
`
`
`
`
`
`
`
`
`
`Liberty provide such pointing device having a 5-axis motion
`
`
`
`
`
`
`
`
`sensor and discloses a compensation using two gyro-sensors
`
`
`
`
`
`
`
`(DY and (1)210 detect rotation about the Yp and Zp axes, and
`
`
`
`
`
`
`
`
`
`accelerometers Ax, Ay and Az to detect the acceleration ofthe
`
`
`
`
`
`pointing device along the three axes of the reference frame
`
`
`
`
`
`
`
`
`
`XPYPZP. The pointing device by Liberty utilizing a 5-axis
`
`
`
`
`
`
`
`motion sensor may not output deviation angles ofthe pointing
`
`
`
`
`
`
`
`device in, for example, a 3D reference frame; in other words,
`
`
`
`
`
`
`
`due to due to the limitation of the 5-axis motion sensor of
`
`
`
`
`
`
`
`
`accelerometers and gyro-sensors utilized therein, the point-
`
`
`
`
`
`
`
`ing device by Liberty cannot output deviation angles readily
`
`
`
`
`
`
`
`
`in 3D reference frame but rather a 2D reference frame only
`
`
`
`
`
`
`
`
`and the output of such device having 5-axis motion sensors is
`
`
`
`
`
`
`
`
`a planar pattern in 2D reference frame only. In addition, it has
`
`
`
`
`
`
`
`
`been found that the pointing device and compensation dis-
`
`
`
`
`
`
`
`
`
`closed therein cannot accurately or properly calculate or
`
`
`
`
`
`
`
`
`obtain movements, angles and directions of the pointing
`
`
`
`
`
`
`
`
`device while being subject
`to undesirable interferences,
`
`
`
`
`
`
`
`external or internal, in the dynamic environment during the
`
`
`
`
`
`
`
`
`obtaining of the signals generated by the motion sensors, in
`
`
`
`
`
`
`
`particular, during unexpected drifting movements and/or
`
`
`
`
`
`
`accelerations along with the direction of gravity. In other
`
`
`
`
`
`
`
`
`
`words, it has been found that dynamic actions or extra accel-
`
`
`
`
`
`
`
`
`
`
`O14
`
`014
`
`

`

`
`
`US 8,552,978 B2
`
`3
`
`erations including additional accelerations, in particular the
`
`
`
`
`
`
`one acted upon the direction substantially parallel to or along
`
`
`
`
`
`
`
`with the gravity imposed on the pointing device with the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`compensation methods provided by Liberty, said pointing
`device by Liberty cannot properly or accurately output the
`
`
`
`
`
`
`
`actual yaw, pitch and roll angles in the spatial reference frame
`
`
`
`
`
`
`
`
`
`
`XPYPZP and following which, consequently, the mapping of
`
`
`
`
`
`
`
`
`the spatial angles onto any 2D display reference frame such as
`
`
`
`
`
`
`
`
`
`XDYDZD may be greatly affected and erred. To be more
`
`
`
`
`
`
`
`
`
`specific, as the 5-axis compensation by Liberty cannot detect
`
`
`
`
`
`
`
`or compensate rotation about the XP axis directly or accu-
`
`
`
`
`
`
`
`
`rately, the rotation about the XP axis has to be derived from the
`
`
`
`
`
`
`
`
`
`gravitational acceleration detected by the accelerometer. Fur-
`
`
`
`
`
`thermore, the reading of the accelerometer may be accurate
`
`
`
`
`
`
`
`only when the pointing device is static since due to the limi-
`
`
`
`
`
`
`
`
`
`tation on known accelerometers that these sensors may not
`
`
`
`
`
`
`
`distinguish the gravitational acceleration from the accelera-
`
`
`
`
`
`
`
`tion ofthe forces including centrifugal forces or other types of
`
`
`
`
`
`
`
`additional accelerations imposed or exerted by the user.
`
`
`
`
`
`Furthermore, it has been found that known prior arts may
`
`
`
`
`
`
`
`
`
`only be able to output a “relative” movement pattern in a 2D
`
`
`
`
`
`
`reference frame based on the result calculated from the sig-
`
`
`
`
`
`
`
`
`nals of motion sensors. For example, the abovementioned
`
`
`
`
`
`
`
`
`prior arts by Liberty may only output a 2D movement pattern
`
`
`
`
`
`
`
`
`in a relative manner and a pointer on a display screen to show
`
`
`
`
`
`
`
`
`such corresponding 2D relative movement pattern. To be
`
`
`
`
`
`
`
`more specific, the pointer moves from a first location to a
`
`
`
`
`
`
`
`
`second new location relative to said first location only. Such
`
`
`
`
`
`
`
`
`
`relative movement from the previous location to the next
`
`
`
`
`
`
`
`
`location with respect to time cannot accurately determine
`
`
`
`
`
`
`
`and/or output the next location, particularly in situations
`
`
`
`
`
`
`
`
`where the previous location may have been an erred location
`
`
`
`
`
`
`
`
`or have been faultily determined as an incorrect reference
`
`
`
`
`
`
`
`
`point for the next location that is to be calculated therefrom
`
`
`
`
`
`
`
`
`
`and obtained based on their relative relationship adapted. One
`
`
`
`
`
`
`
`
`illustration of such defect of known prior arts adapting a
`
`
`
`
`
`
`
`
`
`relative relationship in obtaining a movement pattern may be
`
`
`
`
`
`
`clearly illustrated by an example showing the faultily output-
`
`
`
`
`
`
`
`ted movements of a pointer intended to move out of a bound-
`
`
`
`
`
`
`ary or an edge of display screen. It has been found that as the
`
`
`
`
`
`
`
`
`
`pointer of known prior arts reaches the edge of a display and
`
`
`
`
`
`
`
`
`
`continues to move out of the boundary or edge at a certain
`
`
`
`
`
`
`extra extent beyond said boundary, the pointer fails to dem-
`
`
`
`
`
`
`
`
`
`onstrate a correct or “absolute” pattern as it moves to a new
`
`
`
`
`
`
`location either within the display or remaining outside of the
`
`
`
`
`
`
`
`boundary; in other words, instead of returning to a new loca-
`
`
`
`
`
`
`
`tion by taking into account said certain extra extend beyond
`
`
`
`
`
`
`
`
`
`the boundary made earlier in an “absolute” manner,
`the
`
`
`
`
`
`
`
`
`
`pointer of known arts discards such virtual distance of the
`
`
`
`
`
`
`
`
`extra extend beyond the boundary already made and an erred
`
`
`
`
`
`
`
`
`next position is faultily outputted due to the relative relation-
`
`
`
`
`
`
`
`
`ship adapted and utilized by the pointer.
`
`
`
`
`
`
`Therefore, it is clear that an improved device for use in for
`
`
`
`
`
`
`
`
`example motion detection, computers or navigation with
`
`
`
`
`
`
`enhanced calculating or comparison method capable of accu-
`
`
`
`
`
`
`rately obtaining and calculating actual deviation angles in the
`
`
`
`
`
`
`
`
`spatial pointer frame is needed. For applications of naviga-
`
`
`
`
`
`
`
`tions or computers
`including portable communication
`
`
`
`
`
`
`devices integrated with displays therein, the electronic device
`
`
`
`
`
`
`
`
`may too include the mapping of such actual angles onto a
`
`
`
`
`
`
`
`
`cursor, pointer or position information on the display frame in
`
`
`
`
`
`
`dynamic environments and conditions including undesirable
`
`
`
`
`
`
`external interferences. In addition, as the trend of 3D tech-
`
`
`
`
`
`
`
`
`nology advances and is applicable to various fields including
`
`
`
`
`
`
`
`displays, interactive systems and navigation, there is a sig-
`
`
`
`
`
`
`
`nificant need for an electronic device, including for example
`
`
`
`
`
`
`
`a motion detector, a 3D pointing device, a navigation equip-
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`60
`
`
`
`65
`
`
`4
`
`
`ment, or a communication device integrated with motion
`
`
`
`
`
`
`
`sensors therein, capable of accurately outputting a deviation
`
`
`
`
`
`
`of such device readily useful in a 3D or spatial reference
`
`
`
`
`
`
`
`
`
`frame. Furthermore, there is a need to provide an enhanced
`
`
`
`
`
`
`comparison method and/or model applicable to the process-
`
`
`
`
`
`
`
`ing of signals ofmotion sensors such that errors and/or noises
`
`
`
`
`
`
`
`
`
`associated with such signals or fusion of signals from the
`
`
`
`
`
`
`
`
`
`motions sensors may be corrected or eliminated. In addition,
`
`
`
`
`
`
`according to the field of application, such output of deviation
`
`
`
`
`
`
`
`in 3D reference frame may too be further mapped or trans-
`
`
`
`
`
`
`
`lated to a pattern useful in a 2D reference frame.
`
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`
`
`According to one aspect of an exemplary embodiment of
`
`
`
`
`
`the present invention, an electronic device utilizing a nine-
`
`
`
`
`
`
`
`
`axis motion sensor module for use in for example computers,
`
`
`
`
`
`
`
`
`motion detection or navigation is provided. The electronic
`
`
`
`
`
`
`device comprises an accelerometer to measure or detect axial
`
`
`
`
`
`
`accelerations Ax, Ay, AZ, a magnetometer to measure or
`
`
`
`
`
`
`detect magnetism Mx, My, MZ and a rotation sensor to mea-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sure or detect angular velocities 00X, my, 002 such that resulting
`
`
`
`
`
`
`
`deviation including resultant angles comprising yaw, pitch
`and roll angles in a spatial pointer frame of the electronic
`
`
`
`
`
`
`
`
`device subject to movements and rotations in dynamic envi-
`
`
`
`
`
`
`
`ronments may be obtained and such that said resulting devia-
`
`
`
`
`
`
`
`
`
`tion including said resultant angles may be obtained and
`
`
`
`
`
`
`
`
`outputted in an ab solute manner reflecting or associating with
`
`
`
`
`
`
`the actual movements and rotations ofthe electronic device of
`
`
`
`
`
`
`
`the present invention in said spatial pointer reference frame
`
`
`
`
`
`
`
`
`and preferably excluding undesirable external interferences
`
`
`
`
`
`
`in the dynamic environments.
`
`
`
`
`According to another aspect of the present invention, the
`
`
`
`
`
`
`
`present invention provides an enhanced comparison method
`
`
`
`
`
`
`and/or model to eliminate the accumulated errors as well as
`
`
`
`
`
`
`
`noises over time associated with signals generated by a com-
`
`
`
`
`
`
`
`
`bination of motion sensors, including the ones generated by
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`accelerometers AX, Ay, AZ, the ones generated by magnetom-
`
`
`
`
`
`
`
`
`
`eters Mx, My, M2 and the ones generated by gyroscopes 00X,
`
`
`
`
`
`
`
`my, 00X in dynamic environments. In other words, accumulated
`errors associated with a fusion of signals from a motions
`
`
`
`
`
`
`
`
`sensor module comprising a plurality of motion sensors to
`
`
`
`
`
`
`
`detect movements on and rotations about different axes of a
`
`
`
`
`
`
`
`reference frame may be eliminated or corrected.
`
`
`
`
`
`According to still another aspect of the present invention,
`
`
`
`
`
`
`
`invention provides an enhanced comparison
`the present
`
`
`
`
`
`
`
`method to correctly calculating and outputting a resulting
`
`
`
`
`
`
`deviation comprising a set of resultant angles including yaw,
`
`
`
`
`
`
`
`pitch and roll angles in a spatial pointer frame, preferably
`
`
`
`
`
`
`
`
`about each of three orthogonal coordinate axes of the spatial
`
`
`
`
`
`
`
`
`pointer reference frame, by comparing signals of rotation
`
`
`
`
`
`
`
`
`sensor related to angular velocities or rates with the ones of
`
`
`
`
`
`
`
`
`accelerometer related to axial accelerations and the ones of
`
`
`
`
`
`
`
`magnetometer related to magnetism such that these angles
`
`
`
`
`
`
`
`may be accurately outputted and obtained, which may too be
`
`
`
`
`
`
`
`further mapping to another reference frame different from
`
`
`
`
`
`
`
`said spatial pointer frame.
`
`
`
`
`In the event of interferences including external interfer-
`
`
`
`
`
`
`
`ences introduced by either the device user or the surrounding
`
`
`
`
`
`
`
`
`environment, such as external electromagnetic fields, accord-
`
`
`
`
`
`
`ing to still another aspect of the present invention, the present
`
`
`
`
`
`
`
`
`invention provides a unique update program comprising a
`
`
`
`
`
`
`data association model
`to intelligently process signals
`
`
`
`
`
`
`received from a motion sensor module to output a resultant
`
`
`
`
`
`
`
`deviation preferably in 3D reference frame such that the
`
`
`
`
`
`
`
`
`
`adverse effects caused by the interferences may be advanta-
`
`
`
`
`
`
`
`geously reduced or compensated.
`
`
`
`
`O15
`
`015
`
`

`

`
`
`US 8,552,978 B2
`
`5
`
`According to still another aspect of the present invention,
`
`
`
`
`
`
`
`invention further provides a mapping of the
`the present
`
`
`
`
`
`
`
`abovementioned resultant angles, preferably about each of
`
`
`
`
`
`
`three orthogonal coordinate axes of the spatial pointer refer-
`
`
`
`
`
`
`
`ence frame, including yaw, pitch and roll angles in a spatial
`
`
`
`
`
`
`
`
`
`pointer reference frame onto a display frame either external to
`
`
`
`
`
`
`
`
`the device of the present invention or integrated therein such
`
`
`
`
`
`
`
`that a movement pattern in a display frame different from the
`
`
`
`
`
`
`
`spatial pointer reference frame may be obtained according to
`
`
`
`
`
`
`
`the mapping or translation ofthe resultant angles ofthe result-
`
`
`
`
`
`
`ant deviation onto said movement pattern.
`
`
`
`
`
`
`According to another example embodiment of the present
`
`
`
`
`
`
`invention, an electronic device capable of generating 3D
`
`
`
`
`
`
`
`
`deviation angles and for use in for example computers,
`
`
`
`
`
`
`
`
`
`motion detection or navigation is provided. The electronic
`
`
`
`
`
`
`device may utilize a nine-axis motion sensor module with an
`
`
`
`
`
`
`
`
`enhanced comparison method or model for eliminating accu-
`
`
`
`
`
`
`mulated errors of said nine-axis motion sensor module to
`
`
`
`
`
`
`
`
`obtain deviation angles corresponding to movements and
`
`
`
`
`
`
`rota