`Ye et al.
`
`(10) Patent No.:
`
`(45) Date of Patent:
`
`US 8,552,978 B2
`Oct. 8, 2013
`
`U8008552978B2
`
`3D POIN'I‘ING DEVICE AND METHOD FOR
`COMPENSATING ROTATIONS OF THE 3D
`POINTING DEVICE THEREOF
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5.138.154 A
`5.440.326 A
`5.898.421 A
`7.158.118 B2
`7.236.156 B2
`7.239.301 B2
`7.262.760 32
`7.414.611 B2
`200910262074 Al “
`
`8.51992 Hotelling
`8/1995 Quinn
`431999 Quinn
`lr"2007 Liberty
`652007 Libeny eta].
`71'2007 Liberty eta],
`852007 Liberty
`852008 Liberty
`10.52009 Nasiri et al.
`
`.................. 3453158
`
`“ cited by examiner
`
`Primary Examiner 7 Lun-YI Lao
`Assistant lira/Miner 7 lnsa Sadio
`
`(74) Attorney, Agent, or Firm 7 Ding Yu Tan
`
`(57)
`
`ABSTRACT
`
`A 3D pointing device utilizing an orientation sensor. capable
`of accurately transfomiing rotations and movements of the
`3D pointing device into a movement pattem in the display
`plane of a display device is provided. The 3D pointing device
`includes the orientation sensor. a rotation sensor. and a com-
`puting processor. The orientation sensor generates an orien—
`tation output associated with the orientation] ol'the 3D point-
`ing device associated with three coordinate axes of a global
`reference frame associated with the Earth. The rotation sensor
`
`generates a rotation output associated with the rotation ofthe
`3D pointing device associated with three coordinate axes of a
`spatial reference frame associated with the 3D pointing
`device itself The computing processor uses the orientation
`output and the rotation output to generate a transformed out-
`put associated with a fixed reference frame associated with
`the display device above. The transfomied output represents
`a segment of the movement pattern.
`
`(54)
`
`(75)
`
`Inventors: Zhou Ye. Foster City. CA (US):
`Chin-Lung L1. 'l'aoyuan C01mty (TW):
`Shun-Nan Liou. Kaohsiung (TW)
`
`(73) Assignee: Cywee Group Limited. Tortola (VG)
`
`( * ) Notice:
`
`Subject to any disclaimer. the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 144 days.
`
`(21) Appl.No.: 13/176,771
`
`(22)
`
`Filed:
`
`Jul. 6, 2011
`
`(65)
`
`Prior Publication Data
`US 2011/0260968A1
`Oct. 27. 2011
`
`Related US. Application Data
`
`(63) Continuation-in-part of application No. 13/072794.
`filed on Mar. 28. 20] 1. which is a continuation-in-part
`ofapplication No. 12/941934. filed on Nov. 11. 2010.
`
`(60) Provisional application No. 61/292558. tiled 011 Jan.
`6.2010.
`
`(51)
`
`(2013.01 )
`(2006.01)
`
`1111.0.
`6an 3/033
`6096 5/08
`(52) vs. C].
`USPC ........... 345/157; 345/156; 345/158: 345/173;
`178/1801: 178/1803: 178/1901
`(58) Field of Classification Search
`USPC
`345/1567168. 1737183: 178/180171804.
`178/1901—1904
`
`See application file for complete search history.
`
`18 Claims. 12 Drawing Sheets
`
`
`
`P110!
`
`001
`
`Google 1001
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 1 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`110
`
`
`
`
`112
`
`XP
`
`
`
`111
`
`
`
`
`
`FIG. 2 (RELATED ART)
`
`
`
`002
`
`002
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 2 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`330
`
`
`
`310
`
`320
`
`
`FIG. 3
`
`003
`
`003
`
`
`
`U.S. Patent
`
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 3 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`T
`
`I1_4__w1__g1“.mwht_Wm_M_m0_3_0r__CP___~______~___9__.m_ton.“a_mm._MDn_m_
`_|IIIIIII
`
`
`
`
`
`
`
`
`
`
`
`
`__
`
`
`
`
`
`
`
`
`
`_llllllllllll||||___
`
`
`
`H
`
`
`
`
`
`
`
`
`
`
`M.nr"3_"mm_HmmH_RS_____rlllllllllllllllllllllllL
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 4
`
`/‘>
`
`iv
`
`
`\%,~.59.?
`13%6.}?6,,
`r’3‘?
`$st
`\«6505959
`gag
`a~a-~¢
`ax???
`“Nag?
`«kg?
`ax???
`s “gag
`
`546
`
`
`
`540
`
`
`
`
`
`
`
`004
`
`004
`
`
`
`
`
`
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 4 of 12
`
`
`
`US 8,552,978 B2
`
`630
`
`
`
`005
`
`005
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 5 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`Initialize an initial—value
`
`set
`
`
`
`
`
`
`
`
`
`
`
`
`705
`
`
`710
`
`715
`
`720
`
`725
`
`730
`
`735
`
`
`
`
`
`
`
`Output 3rd quaternion
`
`
`
`to ist quaternion
`
`
`
`
`
`
`
`740
`
`
`
`745
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`
`(lst quaternion) at T—1
`
`
`
`
`
`
`Obtain measured angular
`
`
`veloc:ties at T
`
`
`
`Obtain a current state
`
`
`
`
`
`(2nd quaternion) at T
`
`
`
`
`
` Obtain ”measnured axial
`
`
`
`
`
`
`accelerations of a
`
`
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`
`
`Calculate ”predicted axial
`
`
`
`accelerations" based on
`
`
`
`
`current state at T
`
`
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`Obtain resultant
`
`
`
`
`
`deviation including yaw,
`
`
`
`
`
`
`
`pitch and roll angles
`
`
`FIG. 7
`
`006
`
`006
`
`
`
`
`
`U.S. Patent
`
`
`Oct. 8, 2013
`
`
`
`Sheet 6 of 12
`
`
`
`
`US 8,552,978 B2
`
`705
`
`
`
`
`
`
`Initialize an initial—value
`
`set
`
`
`
`
`Obtain a previous state
`
`
`
`
`(ist quatemion) at T-l
`
`
`
`
`
`
`Obtain measured angular
`
`
`velomties at T
`
`
`
`
`
`
`
`710
`
`
`
`715
`
`720
`
`725
`
`
`
`Output 3rd quatemion
`
`
`
`to let quatemion
`
`
`
`740
`
`
`
`
`
`Obtain resultant
`
`
`deviation including yaw,
`
`
`
`
`pitch and roll angles
`
`
`
`
`
`
`
`
`
`
`
`
`Obtain display data and
`
`
`
`translate the resultant
`
`
`
`angles to movement
`
`
`
`
`pattern in the display
`
`
`
`reference frame
`
`745
`
`
`
`750
`
`
`
`
`
`
`Obtain a current state
`
`
`
`
`(2nd quaternion) at T
`
` Obtain ”measured axial
`
`
`
`
`
`
`
`
`accelerations of a
`
`
`
`measured state at T
`
` Calculate "predicted axial
`
`
`
`
`
`
`
`
`
`accelerations" based on
`
`
`
`current state at T
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`
`
`730
`
`
`
`735
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG.
`
`
`8
`
`007
`
`007
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 7 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 9
`
`008
`
`008
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 8 of 12
`
`
`
`US 8,552,978 B2
`
`1005
`
`
`
`
`Initialize an initial-value set
`
`
`
`1010
`
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`(lst quaternion) at T—1
`
`
`
`
`
`
`
`1015
`
`
`
`
`
`Obtain measured angular
`
`
`velocities at T
`
`1020
`
`1025
`
`1030
`
`1035
`
`1040
`
`1045
`
`Output 5rd quaternion
`
`
`
`
`
`to 1st quaternion
`
`
`
`
`
`Obtain resultant
`
`
`deviation including yaw,
`
`
`
`
`
`
`
`
`
`1060
`
`
`
`
`(2nd quaternion) at T
`
`
`
`
`
`Obtain a current state
`
`
`
`
` Obtain "measured axial
`
`pitch and roll angles
`
`
`
`
`
`
`accelerations" of a
`
`
`
`measured state at T
`
`
`
`Calculate "predicted axial
`
`
`
`
`accelerations" based on
`
`
`
`current state at T
`
`
`
`
`
`
`
`
`
`
`
`Obtain "measured
`
`magnetism" of a
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`
`
` Calculate "predicted
`
`
`
`magnetism" based on
`
`
`
`
`
`
`current state at T
`
`
`
`
`
`
`Obtain an updated state
`
`
`
`
`(3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`with measured state
`
`
`
`
`
`
`
`FIG. 10
`
`
`
`
`
`009
`
`009
`
`
`
`
`
`US. Patent
`
`Oct. 8, 2013
`
`
`
`
`Sheet 9 of 12
`
`
`
`
`US 8,552,978 B2
`
`“05
`
`
`
`
`Initialize an initial-value set
`
`
`
`1110
`
`1115
`
`1120
`
`1125
`
`1130
`
`“35
`
`1140
`
`1145
`
`1 150
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Output 4th quaternion
`
`
`
`to lst quaternion
`
`
`
`
`
`Obtain resultant
`deviation including yaw,
`
`
`
`
`
`
`pitch and roll angles
`
`
`
`,
`,
`-
`
`
`
`Obtain display data and
`
`
`
`translate the resultant
`
`
`
`angles to movement
`
`
`
`
`pattern in the display
`
`
`reference frame
`
`
`
`
`
`
`
`
`“60
`
`
`
`1165
`
`
`
`
`
`
`
`Obtain a previous state
`
`
`
`
`
`(ist quaternion) at T—1
`
`
`
`
`
`
`
`Obtain measured angular
`
`
`velocuties at T
`
`
`
`
`
`
`Obtain a current state
`
`
`
`(2nd quaternion) at T
`
`
`
`
`Obtain "measured axial
`n
`o
`
`
`accelerations of a
`
`
`
`measured state at T
`
`
`
`
`
`
`
`
`Calculate "predicted axial
`
`
`
`
`accelerations" based on
`
`current state at T
`
`
`
`
`
`
`
`
`
`Obtain a 1st updated
`
`
`
`
`
`
`
`state (3rd quaternion) by
`
`
`
`comparing current state
`
`
`
`
`
`with measured state
`
`
`
`
`Obtain and calculate
`
`
`
`
`
`”measured yaw angle" of
`
`
`
`
`
`a measured state at T
`
`
`
`Calculate ”predicted
`
`
`
`
`
`
`
`yaw angle" based on 1st
`
`
`
`updated state at T
`
`
`
`
`
`
`Obtain a 2nd updated
`
`
`
`
`state (4th quaternion) by
`
`
`
`comparing current state
`with measured state
`
`
`
`
`
`
`
`
`010
`
`.
`
`
`FIG.1 1
`
`
`
`010
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 10 of 12
`
`
`
`US 8,552,978 B2
`
`1210
`
`1220
`
`1225
`
`
`
`1255
`
`
`
`
`
`
`
`
`Perform 2nc
`
`data association
`
`
`
`
`
`
`
`and determine whether
`
`
`
`
`
`the result of comparison
`
`
`
`
`falls within a
`
`predetermined
`
`value
`
`1260
`
`
`
`
`
`
`
`
`Obtain a 2nd updated
`
`state
`
`
`
`
`
`
`
`
`
`Obtain a previous state at T—1
`
`
`
`whicwiefltimizatj
`T
`tobtain measured angular
`
`
`' velocities at T
`'
`L ______________ J
`
`
`
`
`
`
`
`
`
`Obtain 1st measured state at T
`T__7_____"_T___T
`
`
`
`
`
`
`
`
`
`
`l obtain measured axnal
`r
`
`
`
`
`L accelerations at T
`
`J1
`
`
`
`
`
`
`Calculate lst predicted
`
`measurement
`
`
`(predicted axial accelerations)
`
`
`
`
`
`
`1235
`
`
`
`1265
`
`
`
`
`Output updated state
`
`
`
`to previous state
`
`
`
`
`
`
`
`
`
`lst data association
`
`
`
`
`
`
`
`and determine whether the
`
`
`
`
`
`
`
`result of comparison falls withi
`value
`
`
`a predetermined
`
`spatial reference frame Obtain a lst updated state
`
`1240
`
`
`
`
`1 270
`
`
`
`
`Obtain resultant
`
`
`deviation including
`
`
`deviation angles in
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 245
`
`
`
`
`
`
`
`Obtain 2nd measured state at T
`
`
`
`“1
`.
`.
`T
`
`
`
`[obtain measured magnetism;
`
`
`
`
`
`'Land yaw angle at T
`
`1 250
`
`
`
`
`redicted
`Calculate 2nd
`
`
`
`measurement predicted yaw angle)
`
`
`
`
`
`
`FIG. 12
`
`
`
`OH
`
`011
`
`
`
`U.S. Patent
`
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 11 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`device associated with Earth
`
`
`
`
`
`Generate an orientation output
`
`
`
`
`
`assOciated with the 3D pointing
`
`
`
`
`
`
`
`
`
`Generate a rotation output
`
`
`
`
`associated with the 3D
`
`
`pointing device
`
`
`
`
`1 320
`
`
`
`
`
`Obtain the orientation of
`
`the display device
`associated with Earth
`
`
`
`
`
`
`
`Generate a transformed
`
`rotation associated with the
`display device
`
`Generate the transformed
`
`1368
`
`output based on the
`transformed rotation
`
`
`
`
`
`
`
`012
`
`device
`
`
`
`Obtain the orientation of the
`3D pointing device
`
`associated with the display
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1366
`
`012
`
`
`
`
`US. Patent
`
`
`
`
`Oct. 8, 2013
`
`
`
`
`Sheet 12 of 12
`
`
`
`US 8,552,978 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 420
`
`Computing
`Processor
`
`
`
`
`
`
`1 420
`s/’*151O
`
`
`
`
`
`
`
`
`
`
`
`
`
`Computing
`
`
`
`Processor
`
`
`
`
`
`
`
`345
`
`344
`
`
`Orientation Sensor
`
`
`
`
`FIG. 15
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1 420
`Computing
`
`Processor
`
`345
`
`
`
`
`
`
`
`
`
`
`
`
`
`013
`
`
`
`
`
`US 8,552,978 B2
`
`1
`
`3D POINTING DEVICE AND METHOD FOR
`
`
`
`
`
`
`COMPENSATING ROTATIONS OF THE 3D
`
`
`
`POINTING DEVICE THEREOF
`
`
`
`
`CROSS-REFERENCE TO RELATED
`
`
`APPLICATION
`
`
`This application is a continuation in part application of and
`
`
`
`
`
`
`claims the priority benefit ofU. S. application Ser. No. 13/072,
`
`
`
`
`
`
`
`
`794, filed on Mar. 28, 201 1, now pending. The prior applica-
`
`
`
`
`
`
`
`
`
`
`tion Ser. No. 13/072,794 is a continuation in part application
`
`
`
`
`
`
`
`of and claims the priority benefit of U.S. application Ser. No.
`
`
`
`
`
`
`
`
`
`
`12/943,934, filed on Nov. 11, 2010, now pending, which
`
`
`
`
`
`
`
`
`
`claims the priority benefit ofU.S. provisional application Ser.
`
`
`
`
`
`
`
`
`No. 61/292,558, filed on Jan. 6, 2010. The entirety of each of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the above-mentioned patent applications is hereby incorpo-
`rated by reference herein and made a part ofthis specification.
`
`
`
`
`
`
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`1. Field of the Invention
`
`
`
`The present invention generally relates to a 3D pointing
`
`
`
`
`
`
`
`device, more particularly to a 3D pointing device for use in
`
`
`
`
`
`
`
`computers, motion detection or navigation utilizing a orien-
`
`
`
`
`
`
`tation sensor and a method for compensating signals of the
`
`
`
`
`
`
`
`orientation sensor subject to movements and rotations of said
`
`
`
`
`
`
`
`3D pointing device.
`
`
`
`2. Description of the Related Art
`
`
`
`
`FIG. 1 is a schematic diagram showing a user using a
`
`
`
`
`
`
`portable electronic device 110, such as a 3D pointing device
`
`
`
`
`
`
`
`or computer mouse, for detecting motions of the device and
`
`
`
`
`
`
`
`
`
`translating the detected motions to a cursor display such as a
`
`
`
`
`
`
`
`cursor pointing on the screen 122 of a 2D display device 120.
`
`
`
`
`
`
`
`Ifthe pointing device 110 emits a light beam, the correspond-
`
`
`
`
`
`
`
`
`
`
`ing point would be the location where the light beam hits the
`
`
`
`
`
`
`
`
`
`
`screen 122. For example, the pointing device 110 may be a
`
`
`
`
`
`
`
`
`
`mouse of a computer or a pad of a video game console. The
`
`
`
`
`
`
`
`display device 120 may be a part ofthe computer or the video
`
`
`
`
`
`
`
`
`game console. There are two reference frames, such as the
`
`
`
`
`
`
`
`
`
`spatial pointer reference frame and the display frame, asso-
`
`
`
`
`
`
`
`
`
`ciated with the pointing device 110 and the display device
`
`
`
`
`
`
`
`
`
`
`120, respectively. The first reference frame or spatial pointer
`
`
`
`
`
`
`
`
`reference frame associated with the pointing device 110 is
`
`
`
`
`
`
`
`
`defined by the coordinate axes XP, YP and Z1, as shown in
`
`
`
`
`
`
`FIG. 1. The second reference frame or display frame associ-
`
`
`
`
`
`
`
`
`ated with the display device 120 is defined by the coordinate
`
`
`
`
`
`
`
`
`axes XD,YD and ZD as shown in FIG. 1. The screen 122 ofthe
`
`
`
`
`
`
`
`
`display device 120 is a subset of the XDYD plane of the
`
`
`
`
`
`
`
`
`
`
`reference frame XDYDZD associated with the display device
`
`
`
`
`
`
`
`
`120. Therefore, the XDYD plane is also known as the display
`
`
`
`
`
`
`
`
`plane associated with the display device 120.
`
`
`
`
`
`
`
`A user may perform control actions and movements utiliz-
`
`
`
`
`
`
`
`
`ing the pointing device for certain purposes including enter-
`
`
`
`
`
`
`
`
`
`tainment such as playing a video game, on the display device
`
`
`
`
`
`
`
`120 through the aforementioned pointer on the screen 122.
`
`
`
`
`
`
`
`
`For proper interaction with the use of the pointing device,
`
`
`
`
`
`
`
`
`
`when the user moves the pointing device 110, the pointer on
`
`
`
`
`
`
`
`
`the screen 122 is expected to move along with the orientation,
`
`
`
`
`
`
`
`
`
`direction and distance travelled by the pointing device 110
`
`
`
`
`
`
`
`
`and the display 120 shall display such movement of the
`
`
`
`
`
`
`
`
`
`pointer to a new location on the screen 122 ofthe display 120.
`
`
`
`
`
`
`
`The orientation ofthe pointing device 1 10 may be represented
`
`
`
`
`
`
`by three deviation angles of the 3D pointing device 110 with
`
`
`
`
`
`
`
`
`respect to the reference frame XPYPZP, namely, the yaw
`
`
`
`
`
`
`
`angle 111, the pitch angle 112 and the roll angle 113. The yaw,
`
`
`
`
`
`
`
`
`
`
`
`
`pitch and roll angles 111, 112, 113 may be best understood in
`
`
`
`
`
`
`
`
`
`
`relation to the universal standard definition of spatial angles
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`60
`
`
`
`65
`
`
`2
`
`
`related to commercial vehicles or transportation such as ships
`
`
`
`
`
`
`and airplanes. Conventionally, the yaw angle 111 may repre-
`
`
`
`
`
`
`
`
`
`sent the rotation of the pointing device 110 about the ZP axis;
`
`
`
`
`
`
`
`
`
`the pitch angle 112 may represent the rotation of the pointing
`
`
`
`
`
`
`
`
`
`device 110 about theYP axis; the roll angle 113 may represent
`
`
`
`
`
`
`
`
`
`
`
`the rotation of the pointing device 110 about the XP axis.
`
`
`
`
`
`
`
`
`In a known related art as shown in FIG. 1, when the yaw
`
`
`
`
`
`
`
`
`angle 111 of the pointing device 110 changes, the aforemen-
`
`
`
`
`
`
`
`
`tioned pointer on the screen 122 must move horizontally or in
`
`
`
`
`
`
`
`a horizontal direction with reference to the ground in
`
`
`
`
`
`
`
`
`response to the change of the yaw angle 111. FIG. 2 shows
`
`
`
`
`
`
`
`
`
`what happens when the user rotates the pointing device 110
`
`
`
`
`
`
`
`
`
`counterclockwise by a degree such as a 90-degree about the
`
`
`
`
`
`
`X], axis. In another known related art as shown in FIG. 2,
`
`
`
`
`
`
`
`
`
`
`when the yaw angle 111 changes, the aforementioned pointer
`
`
`
`
`
`
`
`
`on the screen 122 is expected to move vertically in response.
`
`
`
`
`
`
`
`The change of the yaw angle 111 can be detected by a gyro-
`
`
`
`
`
`
`
`
`
`sensor which detects the angular velocity 00X of the pointing
`
`
`
`
`
`
`
`
`
`device 110 about the XP axis. FIG. 1 and FIG. 2 show that the
`
`
`
`
`
`
`
`
`
`
`
`same change ofthe yaw angle 111 may be mapped to different
`
`
`
`
`
`
`
`
`movements ofthe point on the screen 122. Therefore, a proper
`
`
`
`
`
`
`compensation mechanism for the orientation of the pointing
`
`
`
`
`
`
`device 110 is required such that corresponding mapping of
`
`
`
`
`
`
`
`the pointer on the screen 122 of the display 120 may be
`
`
`
`
`
`
`
`
`
`
`
`obtained correctly and desirably. The term compensation of
`
`
`
`
`
`
`
`the prior arts by Liberty (U.S. Pat. Nos. 7,158,118, 7,262,760
`
`
`
`
`
`
`
`
`
`and 7,414,611) refers to the correction and compensation of
`
`
`
`
`
`
`
`signals subject to gravity effects or extra rotations about the
`
`
`
`
`
`
`
`
`axis related to “roll”. The term of “comparison” ofthe present
`
`
`
`
`
`
`
`invention may generally refer to the calculating and obtaining
`
`
`
`
`
`
`
`
`of the actual deviation angles of the 3D pointing device 110
`
`
`
`
`
`
`
`
`
`with respect to the first reference frame or spatial pointing
`
`
`
`
`
`
`
`
`frame XPYPZP utilizing signals generated by motion sensors
`
`
`
`
`
`
`
`while reducing or eliminating noises associated with said
`
`
`
`
`
`
`
`motion sensors; whereas the term mapping may refer to the
`
`
`
`
`
`
`
`
`
`calculating and translating of said deviation angles in the
`
`
`
`
`
`
`
`
`spatial pointing frame XPYPZP onto the aforementioned
`
`
`
`
`
`
`
`pointer on the display plane associated with the 2D display
`
`
`
`
`
`
`
`
`device 120 of a second reference frame or display frame
`
`
`
`
`
`
`
`XDYDZD.
`
`
`It is known that a pointing device utilizing 5-axis motion
`
`
`
`
`
`
`
`
`
`sensors, namely, Ax, Ay, Az, (nYand (DZ may be compensated.
`
`
`
`
`
`
`
`
`For example, U.S. Pat. No. 7,158,118 by Liberty, U.S. Pat.
`
`
`
`
`
`
`
`
`
`No. 7,262,760 by Liberty and U.S. Pat. No. 7,414,611 by
`
`
`
`
`
`
`
`
`
`Liberty provide such pointing device having a 5-axis motion
`
`
`
`
`
`
`
`
`sensor and discloses a compensation using two gyro-sensors
`
`
`
`
`
`
`
`(DY and (1)210 detect rotation about the Yp and Zp axes, and
`
`
`
`
`
`
`
`
`
`accelerometers Ax, Ay and Az to detect the acceleration ofthe
`
`
`
`
`
`pointing device along the three axes of the reference frame
`
`
`
`
`
`
`
`
`
`XPYPZP. The pointing device by Liberty utilizing a 5-axis
`
`
`
`
`
`
`
`motion sensor may not output deviation angles ofthe pointing
`
`
`
`
`
`
`
`device in, for example, a 3D reference frame; in other words,
`
`
`
`
`
`
`
`due to due to the limitation of the 5-axis motion sensor of
`
`
`
`
`
`
`
`
`accelerometers and gyro-sensors utilized therein, the point-
`
`
`
`
`
`
`
`ing device by Liberty cannot output deviation angles readily
`
`
`
`
`
`
`
`
`in 3D reference frame but rather a 2D reference frame only
`
`
`
`
`
`
`
`
`and the output of such device having 5-axis motion sensors is
`
`
`
`
`
`
`
`
`a planar pattern in 2D reference frame only. In addition, it has
`
`
`
`
`
`
`
`
`been found that the pointing device and compensation dis-
`
`
`
`
`
`
`
`
`
`closed therein cannot accurately or properly calculate or
`
`
`
`
`
`
`
`
`obtain movements, angles and directions of the pointing
`
`
`
`
`
`
`
`
`device while being subject
`to undesirable interferences,
`
`
`
`
`
`
`
`external or internal, in the dynamic environment during the
`
`
`
`
`
`
`
`
`obtaining of the signals generated by the motion sensors, in
`
`
`
`
`
`
`
`particular, during unexpected drifting movements and/or
`
`
`
`
`
`
`accelerations along with the direction of gravity. In other
`
`
`
`
`
`
`
`
`
`words, it has been found that dynamic actions or extra accel-
`
`
`
`
`
`
`
`
`
`
`014
`
`014
`
`
`
`
`
`US 8,552,978 B2
`
`3
`
`erations including additional accelerations, in particular the
`
`
`
`
`
`
`one acted upon the direction substantially parallel to or along
`
`
`
`
`
`
`
`with the gravity imposed on the pointing device with the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`compensation methods provided by Liberty, said pointing
`device by Liberty cannot properly or accurately output the
`
`
`
`
`
`
`
`actual yaw, pitch and roll angles in the spatial reference frame
`
`
`
`
`
`
`
`
`
`
`XPYPZP and following which, consequently, the mapping of
`
`
`
`
`
`
`
`
`the spatial angles onto any 2D display reference frame such as
`
`
`
`
`
`
`
`
`
`XDYDZD may be greatly affected and erred. To be more
`
`
`
`
`
`
`
`
`
`specific, as the 5-axis compensation by Liberty cannot detect
`
`
`
`
`
`
`
`or compensate rotation about the XP axis directly or accu-
`
`
`
`
`
`
`
`
`rately, the rotation about the XP axis has to be derived from the
`
`
`
`
`
`
`
`
`
`gravitational acceleration detected by the accelerometer. Fur-
`
`
`
`
`
`thermore, the reading of the accelerometer may be accurate
`
`
`
`
`
`
`
`only when the pointing device is static since due to the limi-
`
`
`
`
`
`
`
`
`
`tation on known accelerometers that these sensors may not
`
`
`
`
`
`
`
`distinguish the gravitational acceleration from the accelera-
`
`
`
`
`
`
`
`tion ofthe forces including centrifugal forces or other types of
`
`
`
`
`
`
`
`additional accelerations imposed or exerted by the user.
`
`
`
`
`
`Furthermore, it has been found that known prior arts may
`
`
`
`
`
`
`
`
`
`only be able to output a “relative” movement pattern in a 2D
`
`
`
`
`
`
`reference frame based on the result calculated from the sig-
`
`
`
`
`
`
`
`
`nals of motion sensors. For example, the abovementioned
`
`
`
`
`
`
`
`
`prior arts by Liberty may only output a 2D movement pattern
`
`
`
`
`
`
`
`
`in a relative manner and a pointer on a display screen to show
`
`
`
`
`
`
`
`
`such corresponding 2D relative movement pattern. To be
`
`
`
`
`
`
`
`more specific, the pointer moves from a first location to a
`
`
`
`
`
`
`
`
`second new location relative to said first location only. Such
`
`
`
`
`
`
`
`
`
`relative movement from the previous location to the next
`
`
`
`
`
`
`
`
`location with respect to time cannot accurately determine
`
`
`
`
`
`
`
`and/or output the next location, particularly in situations
`
`
`
`
`
`
`
`
`where the previous location may have been an erred location
`
`
`
`
`
`
`
`
`or have been faultily determined as an incorrect reference
`
`
`
`
`
`
`
`
`point for the next location that is to be calculated therefrom
`
`
`
`
`
`
`
`
`
`and obtained based on their relative relationship adapted. One
`
`
`
`
`
`
`
`
`illustration of such defect of known prior arts adapting a
`
`
`
`
`
`
`
`
`
`relative relationship in obtaining a movement pattern may be
`
`
`
`
`
`
`clearly illustrated by an example showing the faultily output-
`
`
`
`
`
`
`
`ted movements of a pointer intended to move out of a bound-
`
`
`
`
`
`
`ary or an edge of display screen. It has been found that as the
`
`
`
`
`
`
`
`
`
`pointer of known prior arts reaches the edge of a display and
`
`
`
`
`
`
`
`
`
`continues to move out of the boundary or edge at a certain
`
`
`
`
`
`
`extra extent beyond said boundary, the pointer fails to dem-
`
`
`
`
`
`
`
`
`
`onstrate a correct or “absolute” pattern as it moves to a new
`
`
`
`
`
`
`location either within the display or remaining outside of the
`
`
`
`
`
`
`
`boundary; in other words, instead of returning to a new loca-
`
`
`
`
`
`
`
`tion by taking into account said certain extra extend beyond
`
`
`
`
`
`
`
`
`
`the boundary made earlier in an “absolute” manner,
`the
`
`
`
`
`
`
`
`
`
`pointer of known arts discards such virtual distance of the
`
`
`
`
`
`
`
`
`extra extend beyond the boundary already made and an erred
`
`
`
`
`
`
`
`
`next position is faultily outputted due to the relative relation-
`
`
`
`
`
`
`
`
`ship adapted and utilized by the pointer.
`
`
`
`
`
`
`Therefore, it is clear that an improved device for use in for
`
`
`
`
`
`
`
`
`example motion detection, computers or navigation with
`
`
`
`
`
`
`enhanced calculating or comparison method capable of accu-
`
`
`
`
`
`
`rately obtaining and calculating actual deviation angles in the
`
`
`
`
`
`
`
`
`spatial pointer frame is needed. For applications of naviga-
`
`
`
`
`
`
`
`tions or computers
`including portable communication
`
`
`
`
`
`
`devices integrated with displays therein, the electronic device
`
`
`
`
`
`
`
`
`may too include the mapping of such actual angles onto a
`
`
`
`
`
`
`
`
`cursor, pointer or position information on the display frame in
`
`
`
`
`
`
`dynamic environments and conditions including undesirable
`
`
`
`
`
`
`external interferences. In addition, as the trend of 3D tech-
`
`
`
`
`
`
`
`
`nology advances and is applicable to various fields including
`
`
`
`
`
`
`
`displays, interactive systems and navigation, there is a sig-
`
`
`
`
`
`
`
`nificant need for an electronic device, including for example
`
`
`
`
`
`
`
`a motion detector, a 3D pointing device, a navigation equip-
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`60
`
`
`
`65
`
`
`4
`
`
`ment, or a communication device integrated with motion
`
`
`
`
`
`
`
`sensors therein, capable of accurately outputting a deviation
`
`
`
`
`
`
`of such device readily useful in a 3D or spatial reference
`
`
`
`
`
`
`
`
`
`frame. Furthermore, there is a need to provide an enhanced
`
`
`
`
`
`
`comparison method and/or model applicable to the process-
`
`
`
`
`
`
`
`ing of signals ofmotion sensors such that errors and/or noises
`
`
`
`
`
`
`
`
`
`associated with such signals or fusion of signals from the
`
`
`
`
`
`
`
`
`
`motions sensors may be corrected or eliminated. In addition,
`
`
`
`
`
`
`according to the field of application, such output of deviation
`
`
`
`
`
`
`
`in 3D reference frame may too be further mapped or trans-
`
`
`
`
`
`
`
`lated to a pattern useful in a 2D reference frame.
`
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`
`
`According to one aspect of an exemplary embodiment of
`
`
`
`
`
`the present invention, an electronic device utilizing a nine-
`
`
`
`
`
`
`
`
`axis motion sensor module for use in for example computers,
`
`
`
`
`
`
`
`
`motion detection or navigation is provided. The electronic
`
`
`
`
`
`
`device comprises an accelerometer to measure or detect axial
`
`
`
`
`
`
`accelerations Ax, Ay, AZ, a magnetometer to measure or
`
`
`
`
`
`
`detect magnetism Mx, My, MZ and a rotation sensor to mea-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sure or detect angular velocities 00X, my, 002 such that resulting
`
`
`
`
`
`
`
`deviation including resultant angles comprising yaw, pitch
`and roll angles in a spatial pointer frame of the electronic
`
`
`
`
`
`
`
`
`device subject to movements and rotations in dynamic envi-
`
`
`
`
`
`
`
`ronments may be obtained and such that said resulting devia-
`
`
`
`
`
`
`
`
`
`tion including said resultant angles may be obtained and
`
`
`
`
`
`
`
`
`outputted in an ab solute manner reflecting or associating with
`
`
`
`
`
`
`the actual movements and rotations ofthe electronic device of
`
`
`
`
`
`
`
`the present invention in said spatial pointer reference frame
`
`
`
`
`
`
`
`
`and preferably excluding undesirable external interferences
`
`
`
`
`
`
`in the dynamic environments.
`
`
`
`
`According to another aspect of the present invention, the
`
`
`
`
`
`
`
`present invention provides an enhanced comparison method
`
`
`
`
`
`
`and/or model to eliminate the accumulated errors as well as
`
`
`
`
`
`
`
`noises over time associated with signals generated by a com-
`
`
`
`
`
`
`
`
`bination of motion sensors, including the ones generated by
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`accelerometers AX, Ay, AZ, the ones generated by magnetom-
`
`
`
`
`
`
`
`
`
`eters Mx, My, M2 and the ones generated by gyroscopes 00X,
`
`
`
`
`
`
`
`my, 00X in dynamic environments. In other words, accumulated
`errors associated with a fusion of signals from a motions
`
`
`
`
`
`
`
`
`sensor module comprising a plurality of motion sensors to
`
`
`
`
`
`
`
`detect movements on and rotations about different axes of a
`
`
`
`
`
`
`
`reference frame may be eliminated or corrected.
`
`
`
`
`
`According to still another aspect of the present invention,
`
`
`
`
`
`
`
`invention provides an enhanced comparison
`the present
`
`
`
`
`
`
`
`method to correctly calculating and outputting a resulting
`
`
`
`
`
`
`deviation comprising a set of resultant angles including yaw,
`
`
`
`
`
`
`
`pitch and roll angles in a spatial pointer frame, preferably
`
`
`
`
`
`
`
`
`about each of three orthogonal coordinate axes of the spatial
`
`
`
`
`
`
`
`
`pointer reference frame, by comparing signals of rotation
`
`
`
`
`
`
`
`
`sensor related to angular velocities or rates with the ones of
`
`
`
`
`
`
`
`
`accelerometer related to axial accelerations and the ones of
`
`
`
`
`
`
`
`magnetometer related to magnetism such that these angles
`
`
`
`
`
`
`
`may be accurately outputted and obtained, which may too be
`
`
`
`
`
`
`
`further mapping to another reference frame different from
`
`
`
`
`
`
`
`said spatial pointer frame.
`
`
`
`
`In the event of interferences including external interfer-
`
`
`
`
`
`
`
`ences introduced by either the device user or the surrounding
`
`
`
`
`
`
`
`
`environment, such as external electromagnetic fields, accord-
`
`
`
`
`
`
`ing to still another aspect of the present invention, the present
`
`
`
`
`
`
`
`
`invention provides a unique update program comprising a
`
`
`
`
`
`
`data association model
`to intelligently process signals
`
`
`
`
`
`
`received from a motion sensor module to output a resultant
`
`
`
`
`
`
`
`deviation preferably in 3D reference frame such that the
`
`
`
`
`
`
`
`
`
`adverse effects caused by the interferences may be advanta-
`
`
`
`
`
`
`
`geously reduced or compensated.
`
`
`
`
`015
`
`015
`
`
`
`
`
`US 8,552,978 B2
`
`5
`
`According to still another aspect of the present invention,
`
`
`
`
`
`
`
`invention further provides a mapping of the
`the present
`
`
`
`
`
`
`
`abovementioned resultant angles, preferably about each of
`
`
`
`
`
`
`three orthogonal coordinate axes of the spatial pointer refer-
`
`
`
`
`
`
`
`ence frame, including yaw, pitch and roll angles in a spatial
`
`
`
`
`
`
`
`
`
`pointer reference frame onto a display frame either external to
`
`
`
`
`
`
`
`
`the device of the present invention or integrated therein such
`
`
`
`
`
`
`
`that a movement pattern in a display frame different from the
`
`
`
`
`
`
`
`spatial pointer reference frame may be obtained according to
`
`
`
`
`
`
`
`the mapping or translation ofthe resultant angles ofthe result-
`
`
`
`
`
`
`ant deviation onto said movement pattern.
`
`
`
`
`
`
`According to another example embodiment of the present
`
`
`
`
`
`
`invention, an electronic device capable of generating 3D
`
`
`
`
`
`
`
`
`deviation angles and for use in for example computers,
`
`
`
`
`
`
`
`
`
`motion detection or navigation is provided. The electronic
`
`
`
`
`
`
`device may utilize a nine-axis motion sensor module with an
`
`
`
`
`
`
`
`
`enhanced comparison method or model for eliminating accu-
`
`
`
`
`
`
`mulated errors of said nine-axis motion sensor module to
`
`
`
`
`
`
`
`
`obtain deviation angles corresponding to movements and
`
`
`
`
`
`
`rotations of said electronic device in a spatial pointer refer-
`
`
`
`
`
`
`
`ence frame. The comparison method or model may be advan-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tageously provided by comparing signals from the above-
`mentioned nine-axis motion sensor module capable of
`
`
`
`
`
`
`
`detecting rotation rates or angular velocities of the electronic
`
`
`
`
`
`
`device about all of the XP, YP and Z1, axes as well as axial
`
`
`
`
`
`
`
`
`accelerations and ambient magnetism including such as
`
`
`
`
`
`
`Earth’ s magnetic field