(12) United States Patent
`Liberty
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,158,118 B2
`*Jan. 2, 2007
`
`(54) 31) I' OINTING ut:VICES WITH
`ORIENTATION COJ\1J>ENSATION AN))
`IMPROVEV USABILITY
`
`(75)
`
`Inventor: Mal1ill'w G . Libl' rly, PotOtll<lC, MD
`(US)
`
`(73) Assignee: !]IUcrest Lllborlliorlcs, Inc" Rockville,
`MD (US)
`
`( ' ) Notice:
`
`Subject 10 any disclaimer, the term of tltis
`patent is extended or adjusted under 35
`V .S.C. )54(b) by 0 days.
`
`This patent is subject \(l a tennillal dis(cid:173)
`claimer.
`
`(2 1) Appl. No.: 11 11 19,7 19
`
`(22) Filed:
`
`May 2, 2005
`
`(65)
`
`Prior I' ublicatioll Dahl
`
`US 2005/0243062 A I
`
`Nov. 3, 2005
`
`Related U.S. A pplication nata
`
`(60) Provisio nal application No. 601641 ,410, filed on Jan .
`5,2005, proviSional applil:ation No. 60/566,444, ilk'll
`onApr_ 30, 2004, provisional application No _ 601612,
`571, filed on Sep . 23, 2004.
`
`(SI)
`
`(S2)
`
`(58)
`
`(56)
`
`Inl. C I.
`G09G 5100
`G09G 5108
`U.S, C I. _.
`
`(2006.0 1)
`(2006.0 1)
`__ _ .. 345/ 158; 345/156; 3451157;
`3451163
`Field ofClassilkation Search .. ___ ._. 345/1S6- 169,
`345/173- 179; 178118_0 1- 18_07,19_01 - 19.06
`Sec application file for complete search histo1)'.
`
`I~ef ... renees Cit ... d
`
`U.S. PATENT DOCUMENTS
`
`4,839.838 A
`5,045.843 "
`5.138.1 S4 A
`5.181.181 "
`5,359.348 A
`5.440,326 A
`5,506.605 A •
`5,698,784 A
`5.703.623 A
`5.825,350 A
`5.835.156 A
`5,898.421 A
`5.912,612 A
`5,955,988 A
`6,002.394 A
`6.016.144 A
`6,049,823 A
`6.115,028 A
`6.164.808 A •
`
`611989 LaBkhe d at.
`911991 Hansen
`811992 HOlelling
`111993 Glynn -...
`1011994 Pi lcher c1 al.
`811995 Quinn
`411996 Paley __ •
`1211997 110lclling'" al.
`121 1997 Hall c1 al
`1011998 Case, Jr. ct al.
`1111998 BlonSlein c1 at.
`411999 Quinn
`&1999 Thlvolpi
`911999 Blonslcin '" al .
`1211999 Sdl.cin ct al.
`112000 Blonslcin '" al .
`412000 Ih,'ang
`912000 Balakrishnan "I al .
`1212000 ShibaTa el a1-
`
`._._._._._._.7021141
`
`..3451163
`
`.. .. .. 345/ 157
`__ 700185
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Navarrele. Po. ct aI., " Eigens.1po;e-bascd R<xogn ition of Faces:
`Compwisonsand a ""W 1\Il)fOa~h:'lmag" Analysis and Pr<X--.:ssing.
`2001.1l'- 1_6_
`
`(Colllinucd)
`
`Primary Examiner- Vijay Shankar
`(74) A/lOrney, Agerll, or Firm- Potomac Patent Group
`PLLC
`
`(57)
`
`AKSTI{;\ CT
`
`Systems and methoos according 10 the present invention
`describe 3D pointing devices which enhance llsability by
`transfo nnillg sensed motion dala from a first fro me of
`reference (e-s ., the body of Ihe 3D pointing device) into a
`second frome of reference (e.g., a user'. frame of reference).
`invention
`One exemplary embodiment of the present
`removes efleels associated wilh a lilt orientation in which Ihe
`3D pointing device is held by a lIser.
`
`4 .787,05 1 A
`
`1111988 Olson
`
`17 Claims. 9 I)rawing Sheels
`
`" I
`
`"
`
`ZTE Exhibit 1006
`Page 1 of 22
`
`

`

`US 7,158,118 82
`Page 2
`
`U ,So PATENT DOCUMENTS
`
`200610092U3 AI·
`
`512006 TOllma cl al. .. " " ... ,, ", . 345 / [58
`
`__ '_"' ___ H' _ 3821293
`
`6,466,831 BI· 1012002
`1212002
`6.492,981 BI
`612004
`6,753.849 81
`6,757,446 BI •
`612004
`812005
`6.933.923 Bl
`112006
`6.990,639 BI
`212006
`6,998.966 BI
`612(0)
`200310107551 l\1
`512004
`2004/0095317 Al
`12f2()()4
`200410239626 A I
`1212004
`2004/0268393 Al
`812005
`200s/0174324 Al
`912005
`200510212767 Al
`1112005
`2005/0243061 Al
`1112005
`2005/0253806 Al
`212006
`2006!OO2S446 Al
`
`Shibata ct aL
`Stork C( Ill.
`Curran ct aL
`Li ~1 al
`Feinslcin
`Wilson
`Pedersen cl al.
`Dunker
`Zhang cl al.
`Nog"c"'-
`Hunl<1h ~1 al.
`Liberty el al.
`Marvit ct al.
`Liberty et al.
`liberty cl al.
`Libcny ci al.
`
`__ 700185
`
`OTHER PUBLICATIONS
`
`Jakubowski. S .• C1 aL. "Higher Order Statistin and Neural Network
`for Tremor Rcwgnition." IEEE Transactions on Biomcdkal Engi(cid:173)
`neering. \01. 49, No 2, Feb 2002. pp_ 152-159_
`Liu. C. et aL. "Enhanced Fisher Linear DiocriminanT Models for
`Facc Rcwgnition." Proc_ 14th InTernational Conference on PatTern
`Recognition. Queensland. Austrnlia Aug. 17-20. 1998. pp. 1-5.
`International Search Report for PCT/US05/15096. mailed May 16.
`2006.
`Written Opinion for PCT!lJS05!15096. m.ailed May 15. 2006.
`International Search Report for PCT!lJS04/35369. mailed l\lay II.
`2006.
`Written Opinion for PCT!lJS04!35369. mailed May Il. 2006.
`GT\.'Cn, J.. et al.. MN"ew iMEMS Angular-Rate-Sensing Gyroscope:'
`Analog Dialogue. 3;-03 (2003). pp. 1-4.
`.. ciwd by examiner
`
`ZTE Exhibit 1006
`Page 2 of 22
`
`

`

`U.S. Patent
`
`Jan. 2, 2007
`
`Sheet 1 or 9
`
`US 7,158,118 B2
`
`FIG. 1
`(Prior Art)
`
`POIIffi -
`-
`CABLE TV DSS
`
`-E5 0 (0
`
`---IfUNCnON - -
`CABLE
`TV
`OSS
`000
`CODE SET
`A
`TE
`(=1
`TV/VIDEO TV/DSS JUMP
`000
`000
`000
`O~Y~ E~
`
`~ENU flDf
`EXIT
`00
`
`VOL
`
`CH
`
`§~
`
`( DSS
`
`J
`
`ZTE Exhibit 1006
`Page 3 of 22
`
`

`

`Cable
`Broadcast
`Input
`I,.r- 230
`
`Satellite
`Input
`__ 232
`
`~28
`
`System
`Controller
`
`1 4
`Video
`Cossette
`Recorder
`
`1
`Digital
`Video
`Disk
`
`VHF/UHF
`Input
`I,.r- 234
`
`6
`
`8
`
`1
`Audio/Video
`Turner
`
`I
`
`I/O BUS
`
`j 210
`
`I
`
`TV/
`Monitor
`
`Video
`Camera
`
`212 ~ 238 J
`
`224 )
`
`Compact
`Disk
`Player
`220 ~
`
`Microphone
`&
`Speakers
`,...J
`222
`
`236 J
`
`Wireless
`I/O
`Unit
`
`226
`
`Cable
`Modem
`Input
`
`FIG. 2
`
`Telephone
`Input
`
`•
`~ •
`
`c:
`en
`
`ZTE Exhibit 1006
`Page 4 of 22
`
`

`

`U.S. Patent
`
`Jan. 2,2007
`
`Sheet 3 of 9
`
`US 7,158,118 B2
`
`FIG. 3
`
`406
`
`402
`
`Y21
`
`X
`
`404
`
`./'
`.-/ 408
`/
`
`410
`
`•
`
`x2
`
`FIG. 4
`
`400
`
`406
`
`502
`
`ZTE Exhibit 1006
`Page 5 of 22
`
`

`

`FIG. 5
`
`Temp.
`Sensor
`
`, 619
`
`Rot.
`Calibration
`
`• ~18
`•
`~14
`r-- Conversion
`'- 616
`
`~
`
`-
`
`~26
`
`6;0
`
`Rotate -Wbst·l~
`
`~28
`
`L.. Scaling
`
`/ 608
`Stationary
`Detection
`
`~
`
`606 ,
`
`•
`- Acc.
`•
`
`Calibration
`
`Conversion
`
`Gain !r 6ID
`Matrix
`
`614
`
`LPF
`
`6f 2
`
`Tilt Calc.
`
`1-).
`604
`
`§J)Q
`
`~12
`Sampled
`lRot. OUtDU
`
`6f2
`
`Sampled
`kcc. Outpu
`
`•
`~ •
`
`c:
`en
`
`ZTE Exhibit 1006
`Page 6 of 22
`
`

`

`U.S. Patent
`
`Jan. 2, 2007
`
`Sheet 5 of 9
`
`US 7,158,118 B2
`
`FIG, 6A
`
`400 - /
`
`FIG. 68
`
`410
`
`408
`
`400\
`
`~--;.-~
`
`ZTE Exhibit 1006
`Page 7 of 22
`
`

`

`U.S. Patent
`
`Jan. 2, 2007
`
`Sheet 6 of 9
`
`US 7,158,118 B2
`
`FIG. 6e
`
`_____ I
`
`FIG. 6D
`
`.-
`
`410
`
`408
`
`-------,
`
`ZTE Exhibit 1006
`Page 8 of 22
`
`

`

`U.S. Patent
`
`Jan. 2,2007
`
`Sheet 7 of 9
`
`US 7,158,118 B2
`
`FIG. 7
`
`814 '\
`
`812
`
`Accelerometer
`
`r
`'\ Rotational r
`IR
`Pholodelector
`
`Sensors
`
`810
`'\
`
`'--
`
`800
`
`( 816
`Transceiver
`t
`
`-
`Processor -
`
`Scroll Wheel
`
`~
`
`802
`
`JTAG
`
`804
`.J
`
`r-
`
`L[Os
`
`..r-
`
`806
`
`t
`Switch Matrix
`
`1'--808
`
`ZTE Exhibit 1006
`Page 9 of 22
`
`

`

`U.S. Patent
`
`Jan. 2,2007
`
`Sheet 8 of 9
`
`US 7,158,118 B2
`
`Wakeup
`
`Sleep
`
`force sleep
`
`reset
`
`Active
`
`Inactive
`
`FIG. 8
`
`ZTE Exhibit 1006
`Page 10 of 22
`
`

`

`U.S. Patent
`
`Jan. 2, 2007
`
`Sheet 9 of 9
`
`US 7,158,118 B2
`
`FIG. 9
`9;1
`
`9)
`
`Interpret
`
`Sensors - Sensors - to User
`
`903
`(
`
`Convert
`
`Frame
`
`,5
`9;4
`I- Movement - Action
`
`Map
`
`Produce
`
`FIG. 10
`
`Body Frame
`
`User Frame
`
`ZTE Exhibit 1006
`Page 11 of 22
`
`

`

`US 7, 158, 11 8 B2
`
`3)) I'OINTING IU: VIC ES \\1TH
`ORIENTATION COMPENSf\TlON ANI)
`IMI'ROYED USAIllLlTY
`
`RELATED APPLICATIONS
`
`2
`being transmitted on those channels and (3) date and time.
`lbe television was tlUled to the desired ch:mnel by adjusting
`a tuner knob and the viewer watched the selected progrom.
`Later, remote control devices wcre inrroduced that p('rmitted
`viewers to tulle the television from a distance. This addition
`to the user-television interface created the phenomenon
`known as --dk1nne1 surfing" whereby a viC\\'er could rapidly
`view short segments being broadc.1st on a munber of dwn-
`ncls to quickly learn whnt progroms were available many
`given time.
`Despite the fnct that the number of channels :md amount
`of viewable content has dramatically increased, the gener(cid:173)
`ally avaibble user interface, control device options and
`frnmeworks for televisions has not chnnged much over the
`last 30 years. Printed guides arc still the most prevalent
`mechanism lor conveying programming inlonnation. 'I1m
`nmltiple button fCmote control with up and down arrows is
`still the most prevalent channel/content selection mecha(cid:173)
`nism. The reaction of those who design and implement the
`TV user inrerface to the increase in available media content
`has hccn a straightloN'a["d extension of the existing selec-
`tion procedures and interl~ee objects. llms, the number of
`rows in the printed guides has bt."(:n in\:re<lsl"d to :1\."\:OttuUO(cid:173)
`date more channels. The number of buttons on the remote
`control devices has been inermsed to suppon additional
`fUndioffitlity <lnu l"Ontent h<tJldling, e.g., as shown in FIG. I .
`However, this approach h.1S significantly increased both the
`time required for a viewer to review the available infol1na(cid:173)
`tion anu the l"Omplcxity of :u:tions required to implemcnt 11
`30 selection. Arguably, the cumbersome nature of the existing
`interface has hampered commercial implententation of some
`sl>J"vicl"S, e.g., video-on-Jemanu, sin\:c \:onsuJIlers are r"l"Sis(cid:173)
`tant to new services that will add complexity to an interface
`that they view as already too slow and complex .
`In addition to increases in bandwidth und content, the user
`interface bottleneck problem is being exacerbated by the
`aBYCg<ltion of technologies. Consumers arc reacting posi(cid:173)
`tively to having the option of buying integ.r<'ted systems
`rather than a number of segrt.>gable componems. An example
`40 of this trend is the combination televisiotVVOUDVD in
`which three previously independent components are fre(cid:173)
`quently sold tooay as an integrated unit . 'Ibis trend is likely
`to continue, porentially with an end result that most if not all
`of the cornmunicmion devices currently found in the house-
`45 hold will be packaged together as an integrated unir, e.g., a
`televisiotVVCKlDVDlintemet
`access/radio/stereo
`unit.
`Even thuse who continue to buy scpar;de components will
`likely desire seamless control of, alx:! inteN'orking between,
`the sep..1rate components. With this increased aggregation
`eoml'"S the potcmial for more complcxity in the uscr inter(cid:173)
`face. For example, when so-c.1lled "universal" remote units
`were introduced, e.g., to combine the lunctionality of TV
`remote units and VCR remote units, the number of buttons
`on these universal remote units was typically more than the
`55 number o f buttons on either the TV remote unit or VCR
`remote unit individually. This added number of buttons and
`functionality makes it very difJicult to control anything but
`the simplest aspects of a TV or VCR without hunting lor
`exactly the right button on the remote. Many timcs, these
`universal R'1ttotcs do not provide enough buttons to access
`many levels of control or features unique to certnin TVs. In
`these cases, the original device remote unit is still needed,
`and the original hassle of handling multiple remotes remains
`due to user interface issues arising from the complexity of
`nggrcgation. Some remote urlits have addressed tltis problem
`by adding "soft" buttons that can be programmed with the
`expert conunands. These soft buttons sometimes have
`
`This :lpplication is relmed to, and claims priority from,
`U.S . Provisional Patent Application Ser. No. 601566.444
`filed on Apr. 30, 2004. entitled "FR'Cspacc Pointing Device",
`the disclosure of which is incorpor:llcd here by reference. 10
`This application is also related to. and claims priority from.
`U.S. Provisional Patent Application Ser. No. 60/612,571.
`filed on Scp. 23, 2004, entitled "FI'l-'C Space Pointing
`Devices and Methods", the disclosure of which is inoorpo(cid:173)
`ruled here by reference. TItis application is also relmoo \0, 15
`and claims priority lium, U.S. Provisional Patent Applica(cid:173)
`tion Ser. No. 60/641.410, filed on Jan. 5, 2005. entitled
`"FR'CSpacc Pointing DC'vices and Methods for Using Same",
`the disclosure o f which is incorporated here by relerence.
`l1tis application is also rclated to U.S. patent applications 20
`Ser. Nos. 111119,987. 111119,688, and 111119,663, entitled
`"Methoos and Devices lor Removing Unintentional Move(cid:173)
`ment in 3D Pointing Dt:vkes", "Mcthods and Dcvkes lor
`Identifying Users Based on Tremor", and '"3D Pointing
`Devices and Methods", all of which were filed concurrently 25
`h<.'re with ,md all of whkh arc inl"Orporatlxi here by refl'"!"-
`ence.
`
`BACKGROUND
`
`The present invention relates generally to handheld,
`pointing devices and, more spe<:ifically to thrcc-dimensionnl
`(hereinafter '"3D") pointing devices and techniques for tilt
`compensation and improved usability nssociated thcn.·\vith.
`Te<:IUlologies nssociated with the communication ofinfor- 35
`mation have evolved rnpidly over the last several decades.
`Television, cellular telephony, the Intemct and optical com(cid:173)
`Illllnicmion tecluliques (to name just a few things) combine
`to inundate consumers with available infonnation and elller(cid:173)
`tainlllent options. Taking television as an example, the last
`three decades have sccnthe introduction of cable tclevision
`service, smellite television service, pay-pcr-vicw movies and
`video-on-demand. Whcreas television viewers of the I 960s
`could typically l"\.'·ceive perhaps four or fivc ovcr-the-air TV
`channels on their television sets, tod.:I.Y's lV wmehers have
`the opponunity to select from hundreds. thousnnds, and
`potentially millions of channels of shows and infonnmion.
`Vidco-on-demalx:! technology, currently used primarily in
`hotels :md the like, provides the potentinl lor in-home
`entertainment selection from among thousands of movie so
`titles.
`The technologicnJ nbility to provide so much inlonnation
`and content to Clx:! uscrs provides both opportunitics ;md
`challenges to system designers and service providers. O ne
`challenge is ttwt while end users typically preler hnving
`more choices rather tlwn fewer, this preference is COlllller(cid:173)
`weighted by their desire that the selection process be both
`fast and simple. Unfor1Unately. the development of the
`systems and interfaces by which end users access media
`items has resulted in selection processes wllich are neither 60
`fast nor simple. Consider again the example of telcvision
`progroms. When television was in its infancy. detcnnirling
`which program to watch was a rebtively simple process
`primarily due to the small number of choices. One would
`consult a printed guide wllich wns fonnaned, for ex:mlple, as 6S
`series of columns and rows which showed the correspon(cid:173)
`dence between (I) ncnrby television chatulels. (2) programs
`
`ZTE Exhibit 1006
`Page 12 of 22
`
`

`

`US 7, 158, 11 8 B2
`
`3
`accompanying LCD displays to indicate their action. 'Ibcsc
`too have (he flaw thm (hey are difllcull to use without
`looking away from the lV to the remOle control . Yet another
`!law in these remow units is the usc of modes in 1m attempt
`to fL>duce the number ofbultons. In thcse "maded" universal
`remote units, a sp<.'Cial button exists to select whether the
`remole should conmlllilicalc with the TV, DVD player, cable
`set-top box. VCR. CIC . lbis causes many usability issues
`including sending conunands \0 the wrong device, forcing
`the user to look at the remote to make sure 111.11 it is in the
`right mode. and it docs not provide any simplification to (he
`integration of multiple deviccs. TIle most advanced of these
`universal remote units provide :>orne imcgwlion by <ll1owing
`the user 10 progrom sequences of commands to multiple
`devices into the remote _ This is sllch a difficult task tlwt 15
`many users hire prolessiOlml installers
`to progrnm their
`universal remote units _
`Some attempts have also bl'Cn made to mooemize the
`screen interlace between end IIsers and media systems.
`However. these attempts typically suffer from. among other
`drawbacks, an inability to easily scale between largc col(cid:173)
`lections o t"media items and small collections of media items.
`For example, interl~ees whkh rely on lists of items m:l)"
`work wcll lor small collcctions of media itcms, but arc
`tedious to browse lor large collections of media items.
`Inl<:rfm;es whkh rely on hierdn:hkal navigation (e.g., tfl.'C
`structures) may be spcedicr to travcrsc than list intcrfaccs lor
`large collections of media items, but are not readily adapt(cid:173)
`able to small l:o][C\:tions of JIllxlia iK'JllS. Aduition'l][Y, uscrs
`tend to lose inteR'St in selection processes wherein the user 30
`has to move through three or more layers in a tree stnlcmre.
`For all of these cases, current remote unils make this
`selection processor ewn more tedious by forcing the user to
`repeatedly dcpfl.'Ss the up and down bunons to navigate the
`list or hiemrdlies_ When sell'"Ction skipping controls arc
`available such as page up and pagc down, the user usually
`has to look at the remote to find these special bUllons or be
`tmin,,'(! to know thUl they ewn exist. Accordingly, organiz(cid:173)
`ing frameworks, t~'Clutiques and systems which simplify lhe
`control and scrccn intcrface between users and media sys- 4Q
`tems as well as accelerate the selection proc<-'Ss, while Ullhe
`same time permitting service providers 10 take advantage of
`tile increases in available bandwidth to end user equipment
`by facilitating the supply of a brge number of 1Il<-'(lia iteills
`and nov.' services to the IIser have been proposed in U.S. 45
`patent application Ser. No. 101768,432, filed on Jan _ 30,
`2004, entitlL-d "A Control Fr;ull<:work with a Z.o<:Jm;lble
`Graphical User Interface for Organizing, Selecting and
`Launching Media Items", the disclosure of which is incor(cid:173)
`pomt~-d here by reference.
`Of particular interest lor this specification arc the remote
`devices usable to interact with such frameworks, as well as
`other applications amI systems. As mentiotl<.x1 in the above(cid:173)
`incorporated application, various difJerent types of remote
`devices can be used with such frameworks including, tor 55
`example. trackballs. "mouse"-type pointing devices, light
`pens, etc_ However, another category of remote devices
`which can be used with such frameworks (and other appli(cid:173)
`c<llions) is 3D pointing devices. 111e phrase "3D pointing" is
`used in this specification to refer to the ability of an input 60
`device to move in three (or more) dimensions in the air in
`front of, e_g., a display screen. and the corresponding ability
`of the user interface to translate those motions directly into
`user interface commands. e.g_. movement of a cursor on the
`display screen. llte transfer of data between the 3D pointing 6S
`device may be performed wirelessly or via a wife connecting
`the 3D pointing device to another device. Thus "3D lXlint-
`
`4
`ing" differs from, e.g., conventional computef mOllse point(cid:173)
`ing techniques which use a surface. e.g .. a desk sllrtace or
`mousqmd, as a proxy suriacc from which relativc move(cid:173)
`ment of the mouse is translated into cursor movement on the
`computer display screen. An example of a 3D pointing
`dcvice can be found in U.S . Pal. No. 5,440,326.
`The '326 (Xltent describes, among other things, a vertical
`gyroscope adapted for use as a pointing device for control(cid:173)
`ling the position of a cursor on the display of a computer_ A
`10 motor at the core of the gyroscope is sllspended by two pairs
`of orthogonal gimbals from a h.1ltd-held controller device
`and nominally oriented with its spin axis vertical by a
`pendulous device. Electro-optical shaft angle encoders sense
`the orientation of a hand-held controller device as it is
`manipulated by a uscr and the resulting electrical output is
`converted into a lonnat usable by a computer to control the
`movement ofa cursor on the screen of the compmer display_
`However, the freedom of usc associated with 3D pointers
`creates a(kiit ional challenges. For example, since there is
`20 genemlly no proxy surface on which a 3D lXlinting device
`rests, the orientation of the handheld control device may
`vary considerably lIum user to user or even IISC to use. If a
`3D pointing device is uS<.-d to, for example. control the
`movcmcnt of a cursor displayed on a screcn, then some
`25 mapping is perlonned between the detected movement of
`the handheld devke and the movell1<''Jlt of the cunor on the
`screen.
`One technique for perfonning this mapping is to nse the
`body frdme of the devil:e as the fr<Ul1e of reflTCnce lor
`mapping detected motion of the 3D poil11ing device into
`intended motion of the cursor. '[be temt "body frame" refers
`to a set o I' a xes associaKxI with the bod)" of the object being
`moved as described in more detail below. Using the booy
`frame of reference to perfomt tbe mapping, however, has
`35 certain drdwbacks _ For example, it rl'quires the user to hold
`the device in a certain orientation in order to obtain the
`cursor movement he or she desires . For example. if the user
`holds the device on its side lind moves the device left to
`right, the cursor will move vertically. not horizonUllly, on the
`screen.
`Accordingly. the present invention dl'Scribes 1Il<-'1hods lind
`devices for processing the data received from sensor\s) in a
`manner which addresses these and other problems associ(cid:173)
`med with conventional 3D ]Xlinting devic<-'S.
`
`SUMMA.RY
`
`Systems and methoos according to the present invention
`describe 3D pointing devices which enhance nsability by
`so transfonning sens<.-d motion data from a first frame of
`reference (e_g., the body of the 3D pointing device) into a
`second frame of reference (e.g .. a user's frame of refeR"Jlce).
`Onc excmplary cmbodimcnt of the present
`invention
`removes efiects associated with a tilt orientation in which the
`3D pointing device is held by a user.
`According to an excmplary embooimeltl of the present
`invention. a handheld, pointing, del'icc includes a first rota(cid:173)
`tional sensor for detemtining rotation of the pointing device
`about a first axis and geneMing a first rotational output
`associated therewith. a second rotational sensor for deter(cid:173)
`mining rotation of the pointing device about a second axis
`and generating a second rotational output associated there(cid:173)
`with, an accelerometer fo r detemlining an acceleration of the
`pointing device and outputting an acceleration omput asso(cid:173)
`ciated therov.·ith and a processing unit for receiving the first
`and second rotational outputs and the acceleration output
`and for: (a) converting the first and second rotational outputs
`
`ZTE Exhibit 1006
`Page 13 of 22
`
`

`

`US 7, 158, 11 8 B2
`
`5
`and the acccl{'ralion output from a body frame of rcll-r{'occ
`associated w ith the handheld poiJUing device into a uscr's
`frame of reference in order to remove the cfleets of tilt
`associated w ith the mrullcr in which a user is holding the
`handheld. poinling device: and (b) determining data associ(cid:173)
`a .... -d with x and y coordinates which arc in turn associ<ltoo
`with movement of a SCI'l->cn cursor, the data based on the
`converted first and second rotational OUlpllls and the COll(cid:173)
`verted acceleration output, wherein the step of converting
`renders the Dlovement of the SCR'"C1l cllrsor subst:lmi311y
`independent of an orientation in which a mer holds tile
`handheld device_
`According 10 :mothcr c);cmpl:JT)' embodiment of the
`present invention. a method for using a 3D pointing device
`includes the steps of detecting movement of the 3D pointing
`device and compel1S3Iing the detected movement by trans(cid:173)
`forming the detected movement from a body frame of
`reference associated with the 3D pointing de vice into an
`inertial fmme of reference.
`According to yet another exemplary embodiment of the 20
`present invention, a 3D, Iwndheld device includes m least
`one sensor for detecting movement of the 3D pointing
`device and a processing unit lor compensating the detected
`mov(:rnent by transfonning the detected movemem from a
`body fmme of reference associated with the 3D pointing 25
`device into an inertial frnme of reference.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`6
`A[so, the lallowing detail('d description does not limit the
`invention. Instead. the scope of the invemion is defined by
`the appended claims.
`In ord{'r to provide some eont('xt for this discussion, an
`exemplary aggreg..1ted media systenl 200 in which the
`present invention can be impkmented will fIrst be described
`with respect to FIG. 2. Those ski[led in the art will appre(cid:173)
`ciate. however. that tlte pl\.'Sent invention is not restrieud to
`imp[cmentation in this type of media system and that more
`10 or fewer components can be included therein. Therein, an
`inplllfoutput (1/0) bus 210 connects the system components
`in the media system 200 together. The!fO bus 210 represents
`any of a number of different of meclwnisms and tl'Clmiques
`for rollling signals betwC('n the media system components.
`15 For example, the 110 bus 210 Dlay include an appropriate
`number of independent audio "patch" cables thm route audio
`signals, coaxial cables that route video signals, two-wire
`serial li nes or infrared or radio frequency transceivers that
`route control signals, optical fiber or any other routing
`mechanisms that route other types of signals.
`In this exemplary embodiment, the media system 200
`includes a television/monitor 212, a video cassette recorder
`(VCR) 214, digit~1 video disk (DVD) rccorder/pl:lyb..1ck
`device 216, audio/video tuner 218 and compact dis k player
`220 coupled to the [lO bus 210 ·lbeVCR 214. DVD 2[6 and
`comp<1ct disk pl<1yer 220 J\l~y be singk disk or single
`cassette devices, or alternatively may be multiple disk or
`multiple cassette devices. They may be indcpcndentunits or
`int'-"gt<11t-"lI
`together. In addition. the J\lcdi~ system 200
`includes a microphone/speaker system 222, video camera
`224 and a wireless 110 control device 226. According to
`exempl<1ry embooiments of the present invention, the wire(cid:173)
`less [f0 control de vice 226 is a 3D pointing device according
`to one of the exemplary embodiments described below. 'llte
`35 wireless!fO cOlllro[ device 226 can eonullunicme with the
`entertainment system 200 using, e.g., an IR or RF trans mit(cid:173)
`ter or transceiver. Alternatively, the [/0 control device can be
`eOJUK'Cted to the ,-'tl1ertaiJUllent system 200 vb a wire.
`The enterlainment system 200 also includes a system
`controller 22g. According to one exemplary emoodiment of
`the present invention, the system cOlllroller 228 opeT1ltes to
`store and display entertainment system data available fro m
`a plurality of entertainment system data sources and to
`control a wide variety of features associat<--'d with each of the
`45 system components . As shown in FIG. 2, system controller
`22g is coupled either directly or indirectly, to each of the
`system components, as n,-'Cc:-;sary, through liD bus 210. In
`one exemplary embodiment, in addition to or in p[aee ofllO
`bus 210, system controller 228 is conflgurC([ with a wireless
`is
`lnlllseeiver), which
`so eommunie<1tion
`tnmsminer (or
`eapab[e of eOllUllunicating with the system components via
`IR sigmls or I{F signals. Reg..1rdless of the comrol medium.
`the syst'-'Iu controller 228 is eonfigurl-d to eontro[ the tlll.-di<1
`components of the media system 200 via a graphical user
`interlace described below.
`As fUrlher illustrated in FIG. 2, media system 200 llIay be
`configured to receive media items from various media
`sources and servicc providers. [n this exemplary embodi(cid:173)
`ment, media system 200 receives media input from and,
`optionally. sends information to, any or all of the following
`sources: eab[e broadcast 230. satdlite broadcast 232 (e.g.,
`via a wtellite dish), very high frequency (VHF) or ultra high
`frequency (UHF) mdio frequency communication of the
`broadcast television networks 234 (e·s ., via an a{'rial
`65 antenna), telephone network 236 and cable modem 238 (or
`another source of Internet content). Those skilled in the art
`will appreciate that the m{'dia components and media
`
`illustrate exemplary 30
`The accompanying drnwings
`embodiments of the present invention, wherein:
`FIG. 1 depicts a conventional remote control unit for an
`enterl3inntent system;
`FIG. 2 depiets an exemplary media system in which
`exemp[ury elllbodiments of the present invention can be
`implemented;
`FIG. 3 shows a 3D point ing device according to an
`exemplary elllbodiment of the present invention;
`FIG. 4 illustrates a cutaway view of the 3D pointing 4Q
`device in FIG. 4 including two rotational sellSOrs and one
`accelerometer;
`FIG. 5 is a block diagram illustmting processing of data
`associated with 3D pointing devices aewrding to an exem(cid:173)
`plary embodiment of the present invention;
`FIGS. 6(a)- 6(d) illustrate the effects of tilt:
`FIG. 7 depicts a hardware architecture of a 3D pointing
`device according to an exemplary embodiment of the
`present invention;
`FIG. 8 is a state diagram depicting a stationary detection
`mechanism according to an exemplary embodiment of the
`pl\.'Sent invention:
`FIG. 9 is ~ block diagrnm iIInstr.tting tmnsfonuation of
`sensed motion data from a fIrst frame of reference into a
`second frame of reference according to an exemplary
`embodiment of the pr,-'Sent invention; and
`FIG. ,10 graphically illustrates the transfonnation of
`sensed motion data from a flrst frame of reference into a
`to an exemplary 60
`second franle of reference according
`embodiment of the present invention.
`
`55
`
`DETA[ LED DESCRIPTION
`
`TIle following detailed description of the inve1l1ion refers
`to the accomp..1nying drawings. lbe same reference numbers
`in different drawings identify the wnl(' or similar clements.
`
`ZTE Exhibit 1006
`Page 14 of 22
`
`

`

`US 7, 158, 11 8 B2
`
`7
`sources illustrated and described ~,ith respect to FIG. 1 arc
`purely exemplary and that media system 200 may include
`more or fewL'!" of both. For example, other types of inputs to
`the syst ... m include AMlFM radio and 5.1tcllitc radio.
`More det.1ils regarding this exemplary enteJ1ailUucnt sys(cid:173)
`tem and frameworks associated therewith can be found in
`the above-incorpornled by reference U.S. p..1\Cnt Application
`"A Control Framework wilh a Zoolllabic Graphical Uscr
`Interface for Organizing, Selecting and Launching Media
`Items". Ahenwlivcly, remote devices in accordance with the
`pl\.'Scnt invention can be lISed in conjltnction with other
`systems, for example computer systems induding, e.g_, a
`display, a processor and a memory system or with various
`other systems and applications _
`As mentioned in the Background section, remote devices
`which operate as 3D pointers are ofpm1icubr interest lor the
`present specification_ Such devices enable me translation of
`movement, e.g_, geslllrcs, into commands to a user interfacc_
`An exemplary 3D pointing device 400 is depicted in FIG. 3.
`l11Creln, user movement of the 3D pointing can be defined,
`for example, in terms of a combination of x-axis nttitude
`(roll), y-nxis elevation (pitch) andlor z-nxis heading (yaw)
`motion of the 3D pointing dt:Vice 400. In addition, some
`exemplnry embodiments of the present invention c.1n nlso
`mensure linear movement 01" the 3D pointing device 400
`along the x, y, and z axes to genewte (;l[{sor mov<''I\lent or
`other IIscr interlace commnnds. In the exemplnry embodi(cid:173)
`ment of FIG. 3, the 3D pointing device 400 includes two
`bunoos 402 and 404 as we1l as a sl:rull wh<.'CI 406, althongh
`other exemplnry embodiments wi1l include other physical
`confignrations. According to exemplary embodiments of the
`pl't."Sent invcntion, it is antil:ipalL"ll that 3D pointing devices
`400 will be held by a user in front of a display 408 and tlwt
`motion of the 3D pointing device 400 will be translated by
`the 3D pointing device into output which is uS<lb1e to interact
`with the information displayed on display 408, e_g_, to move
`the cursor 41 0 on the display 408 . For example, rotation of
`the 3D poiming device 400 about the y-axis can be sens<.-xi
`by the 3D poillling device 400 and translated illlo an output
`usable by the systeltl to move CUTSor 410 along the Y2 axis
`of the display 408_ Lih~' ise, rotation of the 3D pointing
`device 408 about the z-axis enn be sensed by the 3D pointing
`device 400 and translated into an ontplll nsable by the
`system to move CUTSor 41 0 along the x2 axis of the display
`408. It wi1l be nppreciated that the outplll of 3D pointing
`device 400 can be used to itlteruct with the displny 408 in a
`number of ways other than (or in addition to) cursor movc(cid:173)
`ment, for example it can eOlllrol cursor fading, volume or
`media transport (play, pause, last-loIWard and l't.'Wind).
`Input I: