(12) United States Patent
`Ye et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,552,978 B2
`Oct. 8, 2013
`
`(54)
`
`(75)
`
`(73)
`
`( . )
`
`31) I'OLN Tl NG 1)f;VI C EANI) METI-IOI) fUR
`COJ\lPt:NSATING ROTATIONS OF TIlE: 3D
`I)()INTLNG DEVICE TUEREOF
`
`Inventors: Zhou Y<" Fosier City, CA (US);
`Chln-L.ung Li. Taoyuan County (TW);
`Shun-Na n Lin", Kaohsiuug (rW)
`
`Assigncc: Cywf.'e Group Limited, Tonola (VG)
`
`Notice:
`
`Subject to any disclaimer. the term of this
`patcilt is extended or adjusted under 35
`V.S.C. 154(b) by 144 days _
`
`(21)
`
`App!. No_: 13/176;77 1
`
`(22)
`
`Filed:
`
`Jut 6, 20lJ
`
`(65)
`
`Prior I'ublicatlon Data
`
`US 2011 /0260968 AI
`
`Oct. 27, 20 11
`
`Rt'latl' d U.s. Appikatiolllhla
`
`(63)
`
`Continuation-ill-pan of application No. 13/072,794,
`filed on Mar. 28. 2011. which is a cOlllinuutioll-in-part
`of application No. 12/943,934, filed on Nov. 11 , 2010.
`
`(60)
`
`Provisional application No. 6 11292,558, fikd on Jan.
`6.2010.
`
`Inl. C I.
`G06F 11033
`G09G .i/08
`U.S. C I.
`USPC .....
`
`(51 )
`
`(52)
`
`(58)
`
`(20 13.01)
`(2006.01)
`
`345/ 157; 345/156; 345/158; 345/ 173;
`178/18.0 1: 178/18.03: 178/19.01
`!<' il'ld of C b ssificlllion Sl'arch
`USPC ... 345/156- 168, \73- 183; \781\8.01-18.04,
`178119.0\ - \9.04
`See application file for complete search history.
`
`(56)
`
`Re(el"('nCl.'~ CIted
`
`U.S. PATENT DOCUMENTS
`SlI992 HOlclling
`5.138.1S4 A
`5.440.]26 A
`Sl1995 Quinn
`5.898.421 A
`411999 Quinn
`7.158,118 132
`112007 Liberty
`612007 Liberty et al.
`7.236,156 132
`712007 Liberty C! al.
`7.239.301 132
`7.262,760 B2
`Sl2007 Liberty
`Sl200!; Liberty
`7.414.611 132
`2009/0262074 AI· 1012009 Nasiri C( al. .................. 345/ 158
`
`• cited by examiner
`
`Primary Exallli"er - Lun-Vi Lao
`Assisfa" f li."xallli"er -
`lnsa Sadio
`(74) Attorney. Agent. or Firm - Ding Yu Tan
`
`(57)
`
`ABSTRACT
`
`A 3D ]X)inting device utilizing all orientation sensor, capable
`of accuwtely tronsfomling rotations and movements of the
`3D pointing device illlo a movement pallen! in the display
`plane of a display device is provided. The 3D pointing device
`includes the orien\<ltion sensor, a rotation sensor, and a com(cid:173)
`puting processor. The orientation sensor generotes an orien(cid:173)
`tation OUlpUl associated with the orientation orthe 3D point(cid:173)
`ing device associated with thrcc coordinate axes of a global
`refercncc frameassociakd with thc Earth. "nlc rotation sensor
`gcnerotes a rotation OUlput associated with the rotation of the
`3D pointing device associated with three coordilllltc axes ofa
`spatial rcference fmme associated with the 3D pointing
`device itself 111e computing processor II SCS the oril'ntation
`outplll and the rotation Olltplll to generate a transiOfillcd Ollt(cid:173)
`pili associated with a fixcd rdcrcncc fmme associaK"'d with
`the display dl"Vice aoovc. TIte transformed output represents
`a segment of the movement panem.
`
`18 Claims, 12 Drawing Sllel'ts
`
`r ----- - - - l
`,- :lO2
`,
`c'
`
`'"
`
`, , ,
`
`, ,
`,
`
`'"
`
`Dolo
`Tronsrritting
`
`""
`
`"" ,
`r---- ------- - ---- - .1-- 1 , , , , , ,
`p-{
`""""'"
`
`,
`L ______________________ ~
`
`p,~
`
`,
`
`'" ~,
`
`~ - 946
`~~"-~
`924
`'~
`; ----.. :.---!~-~~j,~,~,
`-----~~!~----t=~[===~~~-~9~
`'"
`91.
`
`d
`
`L ________ _
`
`ZTE Exhibit 1001
`Page 1 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 1 of 12
`
`US 8,552,978 B2
`
`/,Zo
`/ '
`120
`~"Xo ~
`
`- 122
`
`Yo
`
`Xp
`
`113
`
`112
`
`110
`
`\
`
`111
`
`Zp
`FIG. 1 (RELATED ART)
`Zo
`
`Xo
`
`Yo
`
`Xp
`
`Zp
`
`112
`
`110
`
`111
`
`FIG. 2 (RELATED ART)
`
`120
`
`122
`
`ZTE Exhibit 1001
`Page 2 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 2 of 12
`
`US 8,552,978 B2
`
`340
`
`322
`
`FIG. 3
`
`310
`
`320
`
`330
`
`ZTE Exhibit 1001
`Page 3 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 3 of 12
`
`US 8,552,978 B2
`
`342
`r----7 --l
`
`, - 302
`
`I
`I
`I
`I
`
`Rotation
`Sensor
`
`r
`
`344
`
`Accelerometer
`
`345
`
`r
`
`Magnetometer
`
`r ~
`I
`I
`
`:
`
`I
`I
`I
`I
`I
`I
`
`346
`
`346
`
`304
`
`i----~-----------~--f~-
`
`I
`I
`
`Computing
`Processor
`
`l
`I
`I
`Data
`I
`Transmitting
`I
`I
`I
`Unit
`I
`I
`I L _________________ _ ____
`I
`J
`
`I
`I
`I
`I
`I
`I
`I
`
`I
`I
`
`FIG. 4
`
`M02
`
`545
`
`542
`544
`
`560 ---,.
`
`522 520
`
`FI G. 5
`
`\
`
`570
`
`580
`
`554
`552
`
`ZTE Exhibit 1001
`Page 4 of 32
`
`

`

`U.S. Patent
`
`Del. 8, 2013
`
`Shcel4 of 12
`
`US 8,552,978 B2
`
`622
`
`64~ 644
`
`642
`
`646
`~ 648
`6~
`
`640
`
`682
`
`620
`
`610
`
`630
`
`FIG. 6
`
`ZTE Exhibit 1001
`Page 5 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 5 of 12
`
`US 8,552,978 B2
`
`705 '- Initialize on initial-value
`set
`!
`710 '- Obtain a previous state
`(1st quatemian) at T-l
`l
`715 '- Obtain measured angular
`velocities at T
`l
`'- Obtain a current state
`720
`(2nd quatemion) at T
`l
`Obtain "measured axial
`'- accelerations' of a
`725
`measured slate 01 T
`!
`730 '- Calculate "predicted axial
`accelerations" based on
`current slale at T
`1
`Obtain an updaled state
`'- (3rd qualemion) by
`735
`comparing current state
`with measured slale
`
`-
`
`,.... to 1st quatemian
`Output 3rd quatemian ~7
`1
`Oblain resultant
`deviation including yaw, V 7
`45
`pilch and roll angles
`
`40
`
`FIG. 7
`
`ZTE Exhibit 1001
`Page 6 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 6 of 12
`
`US 8,552,978 B2
`
`705 ',- Initialize on initial-value
`set
`1
`710 '- Obtain a previous state
`(1st quatemian) at T-l
`1
`715 '- Obtain measured angular
`velacilies 01 T
`1
`'- Oblain a currenl slate
`720
`(2nd quaterniDn) 01 T
`!
`Oblain • measured axial
`',- acce~atiDns' of a
`725
`measured state at T
`!
`Calculote 'predicted axial
`accelerations based on
`currenl stote ot T
`!
`Obloin on updaled stole
`'- (3rd quotemian) by
`735
`camporing currenl state
`with measured stote
`
`730
`'-
`
`f-
`
`40
`
`Output 3rd quotemion ~7
`,----; to 1 st quaterniDn
`l
`Obtain resu~anl
`deviation including yow, V 7
`45
`pitch and roll angles
`1
`Obtain display dota and
`lranslate the resullonl
`angles to movement
`pattern in the display
`reference frame
`
`50
`,.;- 7
`
`FIG. 8
`
`ZTE Exhibit 1001
`Page 7 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 7 of 12
`
`US 8,552,978 B2
`
`926
`
`910
`
`FIG. 9
`
`ZTE Exhibit 1001
`Page 8 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 8 of 12
`
`US 8,552,978 B2
`
`-----'>
`
`Output 3rd qualemion
`10 151 qualernion
`1
`Obtain resultant
`devialion including yaw,
`pilch and roll angles
`
`' - 1050
`
`-, 1060
`
`1005 Lj Initialize an inilial-value sel J
`~
`1010 1,- Obtain a previous slale
`(1st qualernion) 01 T-l
`
`~
`1015 ',- Oblain meosured angular
`velocilies 01 T
`
`~
`1020 1,- Oblain a currenl slale
`(2nd qualemion) 01 T
`~
`1025 ',- Obtain 'meosured axial
`accelerations of a
`measured state 01 T
`t
`1030 1,- Calculate 'predicted axial
`acceleralions' based on
`currenl stale 01 T
`
`~
`Oblain 'measured
`1035 ',- magnelism' of a
`measured slale 01 T
`
`1040 1,-
`
`~
`Calculate 'predicted
`magnetism based on
`current slate at T
`t
`Oblain an updaled slate
`1045 ',- (3rd quaternion) by
`comparing currenl state
`with measured slale
`
`-
`
`FIG. 10
`
`ZTE Exhibit 1001
`Page 9 of 32
`
`

`

`Initialize an initial-value set I
`~
`1110 ',- Obtain a previous state
`(1st quatemion) at T-l
`~
`1115 '- Obtain measured angular
`velocities at T
`!
`1120 ',- Obtain a current state
`(2nd quatemion) at T
`!
`Obtain "measured axial
`1125
`'- accelerations" of a
`measured state at T
`~
`Calculate "predicted axial
`accelerations" based on
`current state at T
`t
`Obtain a 1 st updated
`state (3rd quatemion) by
`comparing current state
`with measured state
`~
`Obtain and calculate
`"measured yaw angle" of
`a measured state at T
`~
`Calculate "predicted
`yaw angle" based on 15t
`updated state at T
`~
`Obtain a 2nd updated
`state (~th quatemion) by
`f-
`comparing current state
`with measured state
`
`1130 \..
`
`1135 \..
`
`1140 '\..
`
`1145 \..
`
`1150 \..
`
`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 9 of 12
`
`US 8,552,978 B2
`
`1105
`
`' - 1155
`
`--, 1160
`
`Output ~th quatemion
`~
`to 1st quatemion
`~
`Obtain resultant
`deviation including yaw,
`pitch and roll angles
`~
`Obtain display data and
`translate the resultant --, 1165
`ongles to movement
`pattem in the display
`reference frame
`
`FIG. 1 1
`
`ZTE Exhibit 1001
`Page 10 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 10 of 12
`
`US 8,552,978 B2
`
`1210
`
`1220
`
`1225
`
`1255
`
`Perform 2n
`data association
`and determine whether
`the result of comparison
`falls within a
`predetermined
`volue
`
`1260
`
`Yes
`
`No
`
`Obtain a 2nd updated
`state
`
`1265
`
`Output updated state
`to previous state
`
`1270
`r:c-:'-:-------":----,---------,
`Obtain resultant
`deviation including
`deviation angles in
`spatial reference frame
`
`Obtain a previous state at T-1
`
`Obtain a current state at T
`, ----- ------ ---1
`, obtain measured angular
`,
`, velocities at T
`,
`L _________ _____ J
`
`Obtain 1 st measured state at T
`,--------------1
`, obtain measured axial
`,
`'L accelerations at T
`J'
`- - - - - - - - - - - - - -
`
`1230
`, -______ ~ __ __ L_ ,
`Calculate 1st predicted
`meosurement
`(predicted axial accelerations)
`
`1235
`
`Perform
`st data ossociatio
`and determine whether the
`result of comparison falls withi
`a predetermined
`value
`
`N<>
`
`Yes
`
`1240
`
`Obtain a 1st updated state
`
`1245
`r:O""bt'"'oi:-n--:2:-n-:-d -m-e...lasLu-re""d-st-:-at-:-e...Lat:-T~
`, ------------- - 1
`,obtain measured magnetism ,
`~~~ r~_o~~e_ "!! ____ J
`1250
`
`Calculate 2nd predicted
`measurement (predicted yaw angle)
`FIG. 12
`
`ZTE Exhibit 1001
`Page 11 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 11 of 12
`
`US 8,552,978 82
`
`( Begin )
`l
`Generate on orientation output ....r
`1320
`associated with the 3D pointing
`device associated with Earth
`
`l
`Generate a rotation output ....r 1
`340
`associated with the 3D
`pointing device
`
`/-- 1 360
`- - - - - - - - - - ________ ...1_,
`I
`
`,(cid:173)
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`L
`
`the display device
`associated with Earth
`
`3D pointing device
`associated with the display
`device
`
`1362
`
`1364
`
`I
`I
`I
`I
`I
`
`I
`I
`I
`I
`
`Dbtain the orientation of ~
`l
`Obtain the orientation of the kJ-
`1
`:
`Generate a transformed ~
`1366
`rotation associated with the
`I
`display device
`I
`I
`I
`I
`
`Generate the transformed ~
`
`output bosed on the
`I
`transformed rotation
`I
`I
`---------- __________ --.J
`
`1368
`
`End
`
`FIG. 13
`
`ZTE Exhibit 1001
`Page 12 of 32
`
`

`

`u.s. Patent
`
`Oct. 8, 2013
`
`Sheet 12 of 12
`
`US 8,552,978 B2
`
`344
`
`3 42 -..r-
`
`Rotetion
`Sensor
`
`_0348
`r---------------~- - ,
`I
`I
`t- Accelerometer r------ Computing
`I
`I
`I
`Orientation Sensor ~ /
`FIG. 14
`
`I
`I
`:
`L ___________________ ~
`
`Processor
`
`Computing f
`Processor
`
`1420
`
`- 1410
`
`342 -..r-
`
`Rotetion
`Sensor
`
`r - - - - - - - - - - - - - - - _0348
`344
`l--- Accelerometer r------ Computing
`Processor
`
`I
`
`I
`I
`I
`I
`
`345
`l--- Megnetometer
`
`I
`I
`,
`I
`I
`~ /- 151O
`I
`I
`I
`I
`I
`I
`Orientation Sensor I
`I
`L _______ __ __________ ~
`
`Computing f
`Processor
`
`1420
`
`1420
`
`342 -..r-
`
`Rotetion
`Sensor
`
`FIG. 15
`
`_0348
`
`r - - - - - - - - - - - - - - - - - - ,
`I
`I
`I
`
`344
`
`345
`
`t- Accelerometer
`
`I
`I
`I
`I
`
`l--- Megnetometer
`
`---->
`
`Computing
`Processor
`
`Computing V-
`Processor
`
`I
`I
`~/- 161O
`I
`I
`I
`I
`I
`I
`Orientetion Sensor I
`I
`L ___________________ ~
`
`FIG. 16
`
`ZTE Exhibit 1001
`Page 13 of 32
`
`

`

`US 8,552,978 82
`
`:m POINTING IlI!:VICE AND METHOI) FOI~
`COMI'ENSATING ROTATIONS OF THE 31)
`POINTING DEVICE THEREOF
`
`CROSS-REFERENCE 'ID RELAI'ED
`APPLlCATION
`
`This application is a continuaTion in part application of and
`claims the priority bcncfitofU ,So application Ser. No. 13/072,
`794, filed on Mar. 28, 2011 , now pending. TIle prior applic3-
`tion Scr. No. 13/072.794 is a continuation in part application
`of and claims the priority benefit ofU.5. application Ser, No_
`12/943,934, filed on Nov_ 11,2010, now pending, which
`claims the priority benefit of U.s. provisional application Ser.
`No_ 61/292,558, filed on Jan. 6, 20 10. TIle entirety of each of
`the above-mentioned pMelll applications is hereby incorpo(cid:173)
`rated by reference herein and made a part of this specification _
`
`BACKGROUND OF nlE INVENTION
`
`I. Field of the Invention
`11le present invention genernlly rellltes to II 3D pointing
`d,:vice, more p<trlkularly to II 3D pointing dt-"Vil'e for nse in
`computers, motion detection or navigation utilizing II orien(cid:173)
`tmion sensor lind a method for compensating signals of the
`ori<:nlalion sensorsubjccllo movelllt-"111S and rolmions of said
`3D pointing device.
`2. Description of the [{elated Art
`FIG. I is a schematic diagram showing a uscr using a
`portable electronic device I I 0, such as a 3D pointing device
`or computer mouse, for detecting motions of the dt..",ice and
`lnmslaling the de\(."l:k""t.111101ions to a cursor display such as a
`cursor poiming on the scT"l-..... n 122 of a 2D display device 120.
`If the pointing device 110 emits a light beam, the correspond(cid:173)
`ing poim would be the location where the light beam hits the
`screen 122. For example, the pointing device 110 may be a
`mow;e of a computer or a pad of a video game eonsole. 'Ibe
`display d"'\Iice I 20 may be a part oftlwcomputer or the vid .. "O
`game console. 11lCre are two reference framcs , such as the
`spatilll pointer reference frame and the display frame. asso(cid:173)
`cialt--d with the poillling device no and the display device
`120, respectively. 111e first reference frnme or spatilll pointer
`reference fmme llssocimed with the pointing dcvice 110 is
`defint--d by the coordin1'tc aXt-'S Xp, Y p and Zp as shown in
`FIG. I . The second reference frnme or display frame assoei(cid:173)
`llted with the display device 120 is defined by the coordinate
`axt-'S X", Y n;md Zo as shown in FIG . I . Thescn:en 122 oftite
`display device 120 is a subset of the XvY D plane of the
`reference frame XD Y J.D associated with the display device
`120. 1berefore, the XD Y D plane is also known as the display
`plane associated with the display device 120.
`A uscr may perfortn control actions and movements lIliliz(cid:173)
`ing the pointing device for cerl;,in purposes including t-"1l1e[(cid:173)
`tainment such as playing a video game. on the display device
`120 through the aforementioned pointer on the screen 122 .
`For proper imeraction with the use of the pointing device,
`when the user moves the pointing device 110, the pointer on
`thescl\.>t.'Jl 122 is eXpt.>cted to move along with the orientation.
`direction and distance travelk-d by the poiming device JlO
`and the display 120 shall display such movement of the
`pointerto a new location on the screen I 22 of the display 120.
`lllCorientationofthepointingdevice I I 0 Illay bereprescmed
`by three deviation angles oflhe 3D poiming device 110 with
`respect to the reference frame Xp Y ;I.p. namely, the yaw
`angle Ill . the pitch angle I 12 and the roll angle 113 .TIle yaw,
`pitch and roll angles III , 112, 113 may be best understood in
`relation to the universal standard definition of spatial angles
`
`2
`relmed to commercial vehides or transportation such as ships
`and airplanes. Conventionally. the yaw angle 111 may repre(cid:173)
`scnt the rotation of the pointing device 110 about lhe Zp axis;
`lhe pitch angle 112 may represent the rotation of the pointing
`device 110 about theY paxis; the roll angle 113 may represcnt
`lhe rotation of the pointing devicc 110 about the Xp axis .
`In a known related art as shown in FIG . . 1, when the yaw
`angle 111 of the pointing device 110 changes. the aloremen(cid:173)
`tioned pointer on the screen I 22 mllSt1110ve horizontally or in
`to a horizontal direction with reference to the ground in
`l\.'"Sponse to the change of the yaw angle 111. FIG. 2 shows
`what happens when the mcr rotates the pointing device JlO
`counterclockwise by a degree such as a 90-degn.. ..... about the
`X p axis. In another known related an as shown in FIG. 2,
`15 when thc yaw angle I .1.1 changes, the aforementioned poiJ1Icr
`on the screen 122 is expected to move vertically in response.
`The change of the yaw angle .1.11 can be detected by a gyro(cid:173)
`sensor which detects the angular velocity w, of the pointing
`device 110 abou t the Xpaxis. FIG. I and FIG. 2 show that the
`20 sallie change of the yaw angle I 1.1 may be mapped to different
`movementsofthe point on theserccn 122 . 'Iberdorc, a proper
`compensation mechanism for the orientation of the pointing
`d"'\Ike 110 is rL'<.juirLxI sUl'h thai wTn:sponding mapping of
`lhe pointer on the screcn 122 of lhe display 120 may be
`25 obtained correctly and desirably. The 1enn compensation of
`the priorarls by Liberty (U.S . Pal. Nos. 7,158, 118, 7,262,760
`and 7,414.611) rders to the correction and compensation of
`signals subject to gravity ellects or extra rotations abom the
`axis rclatt--d to "roll··.lbekTIlI Of"eolllp..1rison·· of the prescnt
`30 invention may generally refer to the calculating andobtaining
`of the acmal deviation angles of the 3D pointing device 110
`Wilh rcspt.'l:t 10 the firsl rcft-'"J\."1lee frdllle or sp..1tial pointing
`frame Xp Y pZp utilizing sigmls generated by motion sensors
`while reducing or eliminating noises associated with said
`35 motion sensors; whereas the ternl mapping may refer to the
`calculating and translating of said deviation angles in the
`spatial pointing frame Xp Y;Zp onto the aloremelllioned
`pointer on the display plane assocl1lted with the 2D display
`device 120 of a second reference frame or display frame
`40 XDY J.D'
`It is known that a pointing dt-'Vice utilizing 5-axis motion
`sensors, namely,Ax,Ay,Az, w)"and wztTmy be compensated.
`Forcxample, U.S. Pal. No. 7,158,118 hy Liberty, U.S. Pm.
`No. 7,262.760 by Liberty and U.S. P1'1. No. 7,414,611 by
`45 Liberty provide such pointing device having a 5-axis motion
`sensor and discloses a compensation using two gyro-sensors
`w,. and Wz to deil-""I:t rotation about the Yp illid Zp axt-'S, ;Hld
`accelerometers Ax. Ay and Az to detect lhe acceleration of lhe
`pointing device along the three axes of the refert.'Ilce frame
`so XpY pZp- The pointing dt-"vice by Liberty utilizing a 5-axis
`motion sensor may not output deviation angles of the pointing
`device in, lorexample. a 3D relcrence IranIC; in other words,
`due to due to the limitation of the 5-axis motion sensor of
`accelerometers and gyro-sensors uti1iv..>U therein, the point-
`ing dt..'vice by Liberty cannot output d<..'Viation angles readily
`in 3D reference framc but rather a 2D reference frame only
`and the output of such device hadng 5-axis motion sensors is
`a planar pallem in 20 reference frame only. In addition. it has
`been found that the pointing device and compensation dis-
`60 closed therein cannot accurately or properly calculate or
`obtain movemelllS, angles and directions of the pointing
`device while being subject to undesirable interferences,
`external or internal, in the dyn. ... mic envirolUllent during the
`obtaining of the signals generated by the motion sensors, in
`65 particular, during unexpected drifting movclllents and/or
`aeeeleralions along with the direction of gravity. In other
`words, it has been found that dynamic actions or extra accc1-
`
`55
`
`ZTE Exhibit 1001
`Page 14 of 32
`
`

`

`US 8,552,978 82
`
`3
`crations induding additiotl.'l.l accclcmtiom, in particular the
`one acted upon the diJ't.'Ction substalltially parallel 10 or alollg
`with the gravity imposed on the pointing device w ith the
`compensation methods provided by Lib,my, said pointing
`device by Liberty CalUlot properly or accllwtcly ampUl the
`actual yaw, pitch and roll angles in the spatial reference fromc
`Xp Y plp<llld following which. consequcll11y, the tn.1pping of
`the spatial angles OlUO any 2D display reference frame slich as
`XD y c7.D may be greatly affoxtcd and erred. To be morc
`s{ll-'"Cific, as the 5-axis compensation by Liberty cannot detect 10
`or compensate rotation about the Xp axis directly or accu(cid:173)
`wldy, the rotation about the Xpuxisbas to bcdcrivcd from the
`grovit31ioI131 acceleration detected by the accelerometer. Fur(cid:173)
`thcnnorc, the reading of the accelerometer may be accurate
`only when the pointing device is static since duc to the limi- 15
`tmion on known accelerometers that these sensors may not
`distinguish the grovitational acceleration from the accelera(cid:173)
`tion oft he forccs including cent ri fugal forces or other types of
`additional accelerations imposed or exerted by the user,
`Funhemlore, it has been found that known prior ans may 20
`only be able to OUlpUl a "relative" movement pattern in a 2D
`rcierence lrnme based on the result calculated lium the sig(cid:173)
`n<tls of motion sensors. For example, thc <tbollCmentiOl)(:d
`prior arts by Libcrty may only output a 2D movemcnt pattcrn
`in a relative manner and a pointer on a display screen to show 25
`sll\;h I:orrespondinl; 2D rel<ttivc tuov(.·JlIent paUL TIl. To be
`more specific, the pointcr moves from a first location to a
`second new location relative to said tlrstlocation only, Such
`rel<ttive moV<."nK'Ilt from the previous location to the next
`location with respect to time cannot accurately dctcffiline 30
`andlor output the next location, particularly in situations
`whL"'Tt: the previous loc:dion nlliy h:we bL>cn an erred location
`or have been faultily dctcffilined as an incoTTL'"Ct reference
`point for the next location that is to be c.1lculated therefrom
`and obtaillL"(\ based Oil their rebtive rclmionship adaptL-rl. O ne 35
`illustrotion of such defect of known prior arts a(bpting a
`relative relationship in obtaining a movemcnt pattcrn may be
`cle1.rly illustrotl"(\ by all example showing the faultily output(cid:173)
`ted movements of a pointer intended to move out of a bound(cid:173)
`ary or an edge of display scrccn. It has bccn found that as the 4Q
`pointer ofknowll prior arts r<:<dches the L'(lge of a dispby <tnd
`continues to move out of the boundary or edge at a certain
`extra extent beyond said boundary, tbe pointer f.1ils to dcm(cid:173)
`onstmte a corrl'Ct or "absolute" paUL'f1l as it mov('s to a nL'W
`location either within the display or remaining outside of the 45
`bOlliubry; in other words. instead of returning to a ncw loca(cid:173)
`tion by wking into al:count s,aid certain extr.1 extend k)'und
`the bOWldary tTmde earlier in an "absolute" manner. the
`pointcr of known arts discards such virtual distancc of the
`extra extend bL)'ond the boundary .dready made and.m etTL"(\ so
`next position is lauitily outpUlled due to the relative relation(cid:173)
`ship adapted and utilized by thc poitUer,
`lllerefore, it is cle<tr that an improved dL'vice for usc in lur
`eX31nple motion detection. computers or navig3tion with
`enhanced calculating or comparison method capableof accu- 55
`rately obtaining and calculating actual deviation angles in the
`spatial poimer frome is needed. For applications of naviga(cid:173)
`tions or computers
`including portable conununication
`devices integrated with displays therein, the ele<:tronic device
`may too include the mapping of such actual angles onto a 60
`cursor. pointer or position infomwtion on the display frame in
`dytmmic CnvirotUllents and conditions including undesirable
`extemal interlcrences. In addition, as the trend of 3D tech(cid:173)
`nology advances and is applicable to various tlelds including
`displays, intcractive systems arKI navigation, there is a sig- 6S
`nitlcant need lor an electronic device, including for example
`a motion detector. a 3D pointing device, a navigation equip-
`
`4
`ment, or a communication device integrated with motion
`sensors therein. cap..1ble of accurately outpuning a dL"Viation
`of SUdl dL'Vkc readily uscful in .1 3D or spati<11 reference
`frame. Furthermore, there is a need to provide an enhanced
`comparison method and/or model applicable to the process(cid:173)
`ingof sign.ds of motion sensors SUdl thm errors and/or noises
`associated with such signals or fusion of signals from the
`motions sensors may be corrected or eliminatL"(\, In addition.
`aCl:ording to the field ofapplkation. such output ufdL'Vi<ttion
`in 3D rcference frame may too be furthcr mapped or trans(cid:173)
`lated to a pattern useful in a 2D reference frume.
`
`SUMlvlARY O F THE INVENTION
`
`According to one aspect of an exemplary cllllxxlilllent of
`the present invention, an electronic device utili7jng a nine(cid:173)
`axis motion sensor module for usc in for example complllers.
`motion detection or navigation is provided. The electronic
`device comprises an accelerometer to measure or detect axial
`accelerations Ax, Ay. Az. a magnetometer to measure or
`detect magnetism Mx, My, Mz and n rotation sensor to mea(cid:173)
`sureor dete<:t angular velocities !lJ" llly Ulz such that resulting
`d(''Vl:1tion induding resultant anl;lL"'S comprising y<tw. pitch
`and roll angles in a spatial pointer lhulle of the electronic
`device subject to movements and rotations in dyn.1mic envi(cid:173)
`ronments lIlay be obtained <tIKI sudl tlwt s<tid rL"'Suitinl; devia(cid:173)
`tion including said resultant angles may be obtained and
`outputted in an absolutc mannerrefiecting orassociating with
`the 1Ictu<t1 mO"'(.'1l1ents and rotations of the ek'l:tronic dL"'Vicc of
`thc present invention in said spatial pointer reference frame
`and preferably excluding undesirable external interferences
`in the dynanlil: environlllents,
`According to another aspL'"Ct of the present invcmion, the
`pre,cnt invemion provides an enlUlnCL>d comparison method
`1H)(Vor model to elimiJUltc the acculllulatL>d errors as well as
`noises over time associated with signals gencroted by a com(cid:173)
`bination of motion sensors, including the ones generated by
`llccelerometers A,.. A). A", the OIlL"'S genemted by maglletom(cid:173)
`eters Mo<' M). M:, and the ones gellerated by gyroscopes U)~.
`hI). (\I> indYllamicenvironments.ln otherwords, llccutllulated
`elTOrs assoeiaK"(\ with a fusion of signals from a motions
`sensor module comprising a plnmlity of motion sensors to
`detect 1ll0VClllents on aud rotations abom di/rerent axes of a
`reference fnnlle may be eliminated or corfL'Cted.
`According to still another aspect of the present invention,
`the present invention provides au enhanced comparison
`method to cOfl'L'"Ctly calcul<tting <tIKI outputting .1 resulting
`deviation comprising a set of resultant angles including yaw,
`pitch and roll angles in a spatial pointer frame, preferably
`about e.Kh ufthree orthogonill I:ourdillilte aXL"'S of the sp;di.d
`pointer reference frame, by comparing signals of rotation
`sensor related to angular velocities or rotes with the ones of
`aCl:derometer rel.dcd to axial acce1cmtions and the ones of
`magnetometcr related to magnetism such that these angles
`may be accurately outputted and obtained. which may too be
`furthcr mapping to another rcfcrcncc fmme different from
`said spatial pointer frame.
`In the event of interferenccs induding external interfer(cid:173)
`ences introduced by either the device user or the sUlTOunding
`envi roJUnent, such as extcrrt31 electromagnetic fidds. accord(cid:173)
`ing to still another aspect of the present invention. the present
`invcntion provides a unique uJXbte progrom comprising a
`data association model to intelligently process signals
`receival from a motion sensor module to omput a resultant
`deviation prefcmbly in 3D refe{Cnce frame such that the
`adverse elTects caused by the interferences may be advanta(cid:173)
`goously reduced or compensated.
`
`ZTE Exhibit 1001
`Page 15 of 32
`
`

`

`US 8,552,978 82
`
`6
`5
`said magnetism Mx, My, Mz in a 3D spatial refcr{'nee frame
`According to still anothcr aspect orthe pl"l'scm invl.'ntion,
`the present invention further provides a mapping of (hc
`of the electronic device. The rotation sensor detects the rota(cid:173)
`tion of the 3D poiming dCllice with respect to a reterence
`abovementioned resultant angles. preferably about each of
`frame associated with the 3D pointing device and provides a
`three orthoF,onal coordinate axes of Ihe spatial point{,T refer(cid:173)
`ence fmme. including yaw, pitch and roll angles in a spatial
`rotation rate or angnlar velocity output. The angular velocity
`outpUi includes three components corresponding to the rota(cid:173)
`pointer rcfcn.:ncc frame onto 11 display fromc either external to
`the dt..'Vicc orthe present invcllIion or integroted therein such
`tion rote or angular velocities wx ' iJJ
`lO, of the 3D pointing
`that a movelllent pa((crn in a display frame ditfcrcnt from (hc
`device aboltlthe first axis. thc sccondaxis and tlte third axisof
`spatial pointer reference frame may be obl<lincd according to
`the referen"e frame, namely, Xp, Yp and Zp of the 3D spatial
`the mapping or translation 0 fthe res\I ham angles 0 fthe result - 10
`frame. The accelerometer detects the axi31 accelerations of
`am deviation onto said movemetl1 pattern,
`the 3D pointing device with respect to the spatial reference
`According \0 another example embodiment orthe present
`fram" such as a 3D-pointer reference frame and provides an
`invention. an ciectronic device capable of generating 3D
`acceleration output, TIte acceleration output includes tluce
`deviation angles and for \lSI.' in for example computers,
`compon"nts corresponding to th" aecelerations, Ax, Az. Ay of
`the 3D pointing device along the first axis, theSL'Cond axis and
`motion dctection or navigation is provided. The electronic 15
`the third axis of the refen>nce frame, namely, Xp, Yp and Zp
`device may utilize a nine-axis motion sensor module with an
`enhanced comparison method or model for eliminating accu(cid:173)
`of the 3D spatial reference frome. The magnetometer deteets
`mulated errors of said nine-axis motion sensor module to
`the magnctism of the electronic dc'Vice with respect to the
`spatial rderence frame such as a 3D reterence tmme and
`ohtain deviation angles corresponding to movemenTS and
`20 provides au magnetism output. The magnetism output
`rotations of said electronic deviee in a sJXltial pointer refer(cid:173)
`ence frame , The comparison method or model may be advan(cid:173)
`ineludcs three components corresponding to thc magnetism,
`Mx. My, Mzofthe 3Dpointing device along the tirst axis, the
`tageously provided by comparing signals from the above(cid:173)
`sL"(;ond :lxis illld the third axis of the referenl:e frdme, namely,
`mentionL"ll nine-:1xis motion SL'JlSOr module I:apable of
`Xp, Yp and Zp of the 3D spatial framc , It can, however, bc
`detecting rotation rates or angular velocities of the electronic
`25 understood that the axes of Xp, Yp and Zp of the 3D sp..1tial
`device about all of the X p, Y p and Zp axes as well as axial
`ael:derdti(}ns and ambient nW);Iletistll including sUl:h as
`refeTL'Jll:e frdme may too be TLllresentL-d simply by tilL' deno(cid:173)
`tation of X, Y and Z,
`Eanh's magnetic field or that of other planets of the electronic
`device along all of the Xp. Y p and Zpaxes slleh tlwt deviation
`According to another example embodiment of the present
`invL'Jltion, a mdhod for I:ompensating accuJ1lulaK-d errors of
`angles of the resultant deviation () fthe ekclronk llL-vice of the
`30 signals of the abovementioned nine-axis motion sensor mod(cid:173)
`pl'L'Sent invention may be preferably obtained oroutpuned in
`an absolute manner. In other words, the pfL'Sent invention is
`ule in dynamic environments associated in a spatial reference
`I:apable of al:curately outpulling the abovetllL'JltionL-d devia(cid:173)
`frame is providL"ll. In one embodimL"'Ilt, the method may be
`performed or handl"d by a hardware processor. TIle processor
`tion angles including yaw, pitch and roll angles ina 3D spatial
`is capableof compensating theaccltntul1t"d errors associated
`pointer refCTL"1tce frame of the 3 [) point ing device to elimitwtc
`or 1'L>(luce accumulated errors und noises generuK-d over time 35 with the resulwtll dL'Vimion ill relmion to the signals of the
`abovementioned nille-axis motion sensor module of the 3D
`in a dynamic environment incli.lding conditions such as being
`pointing device subject to movements and rotations in a spa(cid:173)
`subject to a combin.1tion ofcontinllolts movcments, rotations,
`extemal gravity forces, m1'gJletic field and udditionul extru
`tiul r"ference frame und ill a dynamic envirolUllell1 by per(cid:173)
`accelemtions in multiple diTL><:tions or movcment and rota(cid:173)
`forming a data comparison to comp3re signals of rotation
`tions that are continuously nonlinear with resj'l'lXt totime; and 40
`senror related to angular velocities with the ones of acceler(cid:173)
`funhermore, based on the deviation angles bL~ng compen(cid:173)
`ometer relat<.-d to uxial occe1enllions and the ones of lllugJle-
`sated and accurately oUlpul1ed in 3D spatial reference fmme
`tometer related to magnetism such that the resultant deviation
`nmy be tilnher map[ICd onto or translated into another refer(c