( 19) United States
`(12) Patent Application Publication
`Zhang ct at
`
`(10) Pub. No.: US 2004/0095317 Al
`May 20, 2004
`(43) Pub. Date:
`
`(54) METHOD ANVAI"'ARAT US 01-' UNIVERSAL
`REMOTE l'OINTING CONTROL FOR HOME
`ENTERTAINMENT SYSTEM AND
`COM I'UTER
`
`(76)
`
`Inventors: Jlngxl Zhang, Foster City, CA (US);
`Yim!.: ZhilOg, Foster City, CA (US);
`Iluifang NI, FOSier City, CA (U S)
`
`Corn;,spondtnce Mdrcss:
`,JINCXI ZHANG
`1403 MELBOURNE STREET
`FOSTER CITY, CA 94404 (US)
`
`(21) App\. No.;
`
`10/065,798
`
`(22) Filed:
`
`No\,. 20, 2002
`
`"ublication C lassification
`
`(51)
`
`Int. C I.7 . _______ ___ ___ ___ ___ _ ._ .. _ .. _ .. _ .. _ .. _ ..... _ .. _ .. _ .. _. G09G 5/08
`
`(52) U.S. CI. _ .. _ .. _ .. _ .. ______ ___ ___ ___ ___ . __ . __ . __ . __ . __ . __ .. _ .. _ .. _ ... 345/158
`
`(57)
`
`AllSTRACT
`
`A universaltc1evision and computer pointing oontrol system
`is disclosed. The system is comprised of a handheld pointing
`device, a display control unit, and a command delivery unit.
`The system allows the user to simply poim and click to
`control a computer or various home entertainment compo(cid:173)
`nt'nt de vil.'t's ~motely . lnsille tht' handhelll ilt'vi!:e, orit'nta(cid:173)
`tion SCI\SO/S detect [X'inting direction. The pointing direction
`signals are transmitted to the display control unit, and a
`cursor (pointer) is drawn onto the screen indicating the
`pointer's location. By interpreting the pointi ng direction
`signals and the button activities, the display control unit
`issucs a control signal to thc command dclivery unit. Thc
`command delivery unit then forwards the commands to the
`target device to execute the desired control function .
`
`, 502
`,
`.' ;- 501
`-' 500
`f
`
`"
`
`D
`
`'-"
`
`0
`
`o
`
`,~
`
`[ =:::::J
`::J
`
`/
`
`/
`/
`
`520
`
`410 \
`'-\. ,
`,
`<11 Q 1>,11>
`
`/
`/
`/
`
`.I
`
`\.
`200
`
`420,
`\
`"
`
`\
`\
`\
`
`\
`\
`
`•
`~I
`400
`
`\
`\
`\
`\
`\
`\
`\
`\
`\
`\
`\
`\
`
`351
`
`\
`430
`
`I
`"
`\/
`/, / 10
`I \.
`I
`\
`I
`"
`/ '
`/
`/
`/
`f
`/
`
`300 ' U
`
`~
`'-60
`
`50
`
`.... - 100
`
`ZTE Exhibit 1005
`Page 1 of 15
`
`

`

`Patent App lication l:tublication May 20, 2004 Sheet I of 8
`
`US 2004/0095317 A I
`
`\
`\
`\
`\
`
`\
`\
`
`• • I
`I ,
`400
`
`/ 510
`./
`
`\
`
`\
`
`200
`
`,
`'-
`
`60
`
`-- 50
`-'
`
`/ - 100
`
`~ 103
`,
`
`410 \ + I
`
`I
`
`<11 0 r> !I>
`
`•
`
`I
`
`/
`/
`
`/
`/
`/
`
`520
`
`,
`,
`/
`\/
`II / -10
`I ~/
`/
`/
`/
`/
`/
`/
`/
`/
`
`\
`\
`I
`\
`\
`
`I
`I
`
`\
`\
`430
`
`\
`\
`\
`\
`\
`\
`\
`\
`\
`\
`\
`351 -, \
`
`300 -,
`
`FIG. 1
`
`ZTE Exhibit 1005
`Page 2 of 15
`
`

`

`I'atent App lication l:.ublicalion May 20, 2004 Sheer 2 of 8
`
`US 2004/0095317 Al
`
`710 "lt-
`
`\
`\
`\
`
`720
`
`200
`600 \..D ~-d •
`i
`610
`-
`'~
`
`700'..
`
`\
`\
`\
`\
`\
`\
`. \y-10
`
`\
`\
`\
`
`50
`J
`
`103
`102
`101
`
`100
`
`o
`o
`'-~~~----_0-L __ ~
`
`FIG.2
`
`ZTE Exhibit 1005
`Page 3 of 15
`
`

`

`I)atent Application Publication May 20, 2004 Sheet 3 of 8
`
`US 2004{0095317 A I
`
`101
`
`102 103
`
`~ I
`
`x
`
`y
`
`17~ 110
`120
`~_O_\~ __ ~~~-r __ +-~ __ k 130
`
`140
`
`160
`
`FIG.3
`
`ZTE Exhibit 1005
`Page 4 of 15
`
`

`

`I'atent Application l:.ubl ica lion May 20, 2004 Sheer 4 of 8
`
`US 2004/00953 17 Al
`
`N
`
`300
`(
`
`320
`
`25
`}
`
`y
`
`s
`120
`
`cHo 21 _ _
`
`22
`1-" _ _ 1 ..... 1 _ _ -
`
`"
`
`X~;\~~-~~~~~~:~~~:== U(~:-~-~-~~Drn~)
`
`FIG.4a
`
`y
`
`(
`
`300
`\
`
`320
`I
`
`J
`
`1_ -
`- - - '
`1- 32
`
`/
`
`26
`
`FIG.4b
`
`ZTE Exhibit 1005
`Page 5 of 15
`
`

`

`I'atent Application l:.ublicalion May 20, 2004 Sheer 5 of 8
`
`US 2004/0095317 Al
`
`100
`\
`
`101 \ l 102\ l 103' 1
`--<>
`--<>
`
`-< l
`
`0 -
`
`0--
`
`/
`
`Transmitter I-
`
`I-
`Modulator
`"
`\ 113
`
`r
`
`MCU
`
`12{1
`
`AMP
`
`8::;
`rx::/
`Filter
`
`(112
`
`Filter
`
`132
`
`MUX
`
`110
`
`111
`
`ADC
`
`5~ 140
`121'&>
`~4 ~ J
`AMP ~
`123 -~ I--
`~
`1~Ol 1\&>-%
`131
`I 8>%
`
`"-
`
`Filler
`
`f-
`133
`• ../
`
`.
`Filter
`
`134
`
`FIG. 5
`
`171
`I
`170
`)
`\
`Battery
`-i~ Manage
`Unrt
`
`ZTE Exhibit 1005
`Page 6 of 15
`
`

`

`Palent Application l'ublication May 20, 2004 Sheel 6 of 8
`
`US 2004/009531 7 Al
`
`22 1)
`~ecelver
`
`231
`
`270
`I'
`
`DEMDD
`
`Memory
`
`221 I)
`,
`Receiver
`1
`DEMOD
`
`231 ,
`
`270
`
`r MemOry I
`251 ,
`
`v ideo
`Input
`s ignal
`
`\
`
`Video
`Decoder
`
`50
`
`21~
`\.
`
`CPU
`
`FIG.6a
`
`50
`
`21~
`\.
`CPU
`
`/ 200
`
`2fO
`)
`
`PC
`Interface
`
`To PC
`Peripheral
`Port
`
`222
`
`(
`
`~ 60
`, I T ransmitt~
`( 23 2
`
`/
`
`MOD
`
`~ 200
`
`270
`
`)
`
`2(2
`)
`
`To TV
`onitor
`M
`
`Graphlcs1
`Video MUX
`
`Video
`Encoder
`
`FIG.6b
`
`ZTE Exhibit 1005
`Page 7 of 15
`
`

`

`.'atent Application I)ublicalion May 20, 2004 Sheet 7 of 8
`
`US 2004/IlO')S317 Al
`
`320
`
`· 60
`
`Receiver
`
`DEMOD
`
`;
`
`300
`
`330
`
`351
`\
`
`350
`
`I
`IR
`Transmitter
`
`310
`
`360
`
`Meu
`
`IR
`Receiver
`
`r..(.,
`
`)
`361
`
`370
`
`Non·volatile
`Memory
`
`FIG.7a
`
`351
`
`300
`
`361
`
`800
`)
`
`s:
`
`FIG.7b
`
`ZTE Exhibit 1005
`Page 8 of 15
`
`

`

`I'atent Application Puhlication May 20, 2004 Sheet 8 of 8
`
`US 2004/0095317 A l
`
`221 ') ,
`
`Receiver
`
`231 '1
`
`'DEMOD
`
`270,
`
`\..
`
`memary
`
`~ 50
`
`21~
`\
`CPU
`
`J
`
`~60 222
`(
`, I T ransmitt;r
`".. 23 2
`
`".-
`
`MOD
`
`( 260
`
`/
`IR
`Receiver
`
`~r' 200
`
`v ideo
`Input
`s Ignal
`
`2~1
`\
`
`Video
`Decoder
`
`270
`
`)
`
`Graphicsl
`Video MUX
`
`2 61
`
`T oTV
`anitar
`M
`
`273
`
`J
`
`Video
`Encoder
`
`FIG.8a
`
`200
`
`/
`
`261
`s
`
`•
`
`800,
`O';! !lit iii \
`5i1a!QI ~
`SIOIt=l
`
`'"
`
`l/
`
`FIG.8b
`
`ZTE Exhibit 1005
`Page 9 of 15
`
`

`

`US 2004/00953 17 AI
`
`May 20, 2004
`
`1
`
`METHODANU A.'I'AKi\TUS OF UNIVERSAL
`REMon: )lOI N'n NG CONTROL .'OR HOME
`t;rrf'TI<: RTAIN 1.:N'1' SYSTI': M ANI} COM .. un : R
`
`BACKGRQUNI) OF INVENTION
`
`[000 11 Wi th advancing technology. more and moce fea(cid:173)
`tures are added 10 home video and ludio cDlenaiDmeOl
`s)'$lcms. For c;\:8 mple, inte ractive television sets allow users
`to purchasc a pay program by pressing bUlIonsoll the rem Ole
`control. However, the ri ch set of fnnctioIL" requires more
`bUlIons on the 1"I;11101C control UlliL Tne j am-packed bullon
`layout on the remote ,"'Oolrol unil makes the handheld device
`bulky and complicated. Moreover, an increasing number of
`audio and vidco oomponem devices. for example. VCRs,
`DVD players. digita l TV sct-Iop bo xes, arc added into home
`entertainment systems. Eac h device is usually controlled by
`a uniqu e remote control unit. '10 redu(."C the uscr'sconfusion
`of multiple reruote control units, uuiversal remote control
`devices were introduced to (."Ousumers. The universal remote
`control device can he either preprograrnrncd or trained with
`other remotc controls by the use r to provide multi-device
`control functionali ty. I lowcvcr, becausc more functions are
`being added to this type of hand he ld dev ice, and because of
`the lim ited numbe r of buttons available (which are already
`crowding the devk,,). each bnltOlI must serve mnlti ple
`functions. Un for1un~ t eJy, the: muhi-functiOfl bullous cannot
`provide clear visual feedback indicating their current func(cid:173)
`tion. This unfriendly user interface is obscure to the user
`operating the remote control unit ~nd leads 10 only a small
`subset o f the functiOn<;; being uti lized. Furthermore, the
`expandability of preliCnt universal remote control devices is
`ve ry poor. As new media modu les are introduced into home
`enterta in[llCnt systems, for instance, Internet browsers, it
`becomes cven more difticull to ada pt the exiSiing universal
`remote contro l to the new requirements, in the case of
`IntertlCt browscrs. that uscrs be able to move a pointer and
`select a visual objcct on the s<;:reen to operate a cen ain
`function. A handheld pointing control dcvice is desirable in
`such a case. W hile usin~ the pointing device, the on-sc reen
`graphica I user intcrfa(."C (GU I) provides friendly visual feed (cid:173)
`back. The dynamically displaycd selectable on-screen idcn(cid:173)
`tifiers (menu.s, ico t~<;. bUllOt~S. clc.) greatly reuuce the nu m(cid:173)
`ber of bullons on the pointing co ntrol device.
`
`In tIK: case of computer slitle presentations. a
`(0002)
`(.'Ou\"cnicnt handheld rcnlot..: pointing alKl control d..:vkc is
`ab>o cons ide rcd necessary. Conventional computer control
`depends on keyboard and mouse which are physically
`bou nded with computer hardware and a fixed surface such as
`a table. "10 control the Itow of the presen tation slides or to
`point out figures on lhe slide to Ihe aud ience, the presenter
`is forced to Slay with the computer keyboard alld IIlOllSC.
`111is constraint is "cry inconvenient fOf tlte prc.so;:nter trying
`to dcli"er hisjher tnlk to the audience. A remote pointing
`control device coulll help the pre...enter to freely walk about
`the stage and move a pointer on the screen to guide the
`aud icnce.
`[0003 J Beca use of the need for a remote pointing mecha(cid:173)
`nism for home entenainmem systems atld computer presen(cid:173)
`tations . many methods and devices have been invcnted . For
`examples, Fan (U.S. Pat. No. 5,926.168) has described
`several methods, iocludillg Il~ing light emission and elec(cid:173)
`tromagnetic fields. to develop femote pointing devices;
`Kahn (U.s. Pat. No. 6.40<1 ,416) described a pointing inter-
`
`face for (''Onll'uter s)'stems baso.:d on raster scanned ligbt
`emilled from display screen:;.. The me thods presented in
`those inventions ~re complicated, and some requ ire a new
`display apparatus to replace the existing one. Man;h el al.
`(U.S. I'll. No. 5.999. 167) introduced a pointer cootrol
`device based on an ult rasound source. Pilcher et al. (U.s.
`Pat. No. 5.359 • .l48). Uansen (U.S. Pat. No. 5,0<15.843),
`Odell (U.S. Pal . No. 5.574.479) and King. et al (U.S. Pat.
`No. 4,565,999) presented pointing dcvices based on detect(cid:173)
`ing fixed light sources. Auerbach (U .S. I'at. No. 4,796.( 19)
`explained a pointin~ devicc containing multiple
`light
`sources and thll lights are dlltectcd by n fi xcd light sensor.
`Wang et al. ( U.s. Pat. No . 5, 126,513) suggested a pointing
`measure melll method by detcctin!,;. the wave phases from a
`fixed transmillllT. 110wcvcr, in pract ice. all th e approachcs
`based on dctecting fi xed local so urces suffer from
`the
`limitatio [1S of the lixed source locatiortS and orientations. as
`wdl as tin; distance I>ctwccn the pointing device and fixed
`sources. Moreovcr. the control methods proposed in all the
`aforementioned invcnt ions are limited to oilly a single urget
`device. The control scope is narrow and canoot covcr all the
`related vidco/audio dcvices or equ ipment.
`[0004] Reu:ntly.low cost magnetic He kl .sensors based on
`magneto-resistivc. m~gne lo- i nd uct ivc and Ilall-c:fTect tech(cid:173)
`nologies wllre dcveloped . Those maWletic .sensors are sen(cid:173)
`siti\"C enough to measure earth's magnetic field and are
`widely used in such navigational dev ices as digital com(cid:173)
`pas.sc:s and the Global Positioning Syste m (G PS). Some
`magne tic sensors are packa ged 10 detect two-axis. even
`three-axis, magnetic field changi:S and provide a linear
`output to the direction of thll mngnetic field Dux, SIIch as
`IIMC I052 IWo-axis magnetic sensor fro m Uoneywell
`(www.sscc.honeywell.com). The two-axis magnetic Held
`sensor can Ix: casy and cost-perfect to implcmcnt a pointing
`device to detec11he yaw (azimuth) angle relative to eanh ' s
`North Polc. Howcver. using magnet ic field seoson; 10 detect
`a pitch (inc\in31ion) Hlljl,k: change would be a problem.
`particula rly when too p:!inting devicc's hcading direction is
`pcrpcndiculHr to ean h's North·South axis. 1·lall ct 31. (U.S.
`Pat. No . 5.703,623) prc.scnwd a poin ti ng dcvi(."C using three
`pairs of onhogonally mountcd onc-axis Ibll-cfl'cct sensors.
`10 ovcrcome thc problem in mea.surin~ pitch and ro ll angles,
`a set of piczoelectric sen'iOrs is used to detect the accelera(cid:173)
`tion changes. 'l1le authors suggesto:d using lOC dclecto:d
`accderation dat a to compensate the deficient of magnetic
`sensors. Howevcr, to mea.~lIre Ilevice angul ar movement an
`integration of the R(.x."e lc ra tion seeps is required. The piezo(cid:173)
`electric scn<;On> dctcct on ly the dyna mic changes of accel(cid:173)
`eration. "ille accelerat io n measurement errors are introduced
`because picl..oclectr ic scnsors are failed to meaSl.lre the
`constant acccicration. 'nle accumulated accele ration error in
`the integration proce:o;s would e\'entually rende r the dI:\'Lce
`unusable.
`[000.51
`'lb de te Ci a pointing device's pitch and roll angles,
`a static accclerometllr can be used. Rece ntly, low-cost,
`lightweight aC('"I.:\crull1e ter scn .. "WJs us in~ Micro-EIcc1ro- ~'h:­
`cha nica l Systems (MEMS) techno logy are available fro m
`many sou rces. MI!MS de ... ices in tegrate mecha nica l ele(cid:173)
`me nts. sensors, actu alors. and eleclronics on l common
`silicon substra tc usi ng micro-fabrication technology, which
`provides a cost ~ fTecti ... c and small-footprint component for
`consumer manufactories. 'lWo-axis linear MEM S acceler(cid:173)
`ometers. sud} as AJ)X L-202E fro m Analog Devices
`(www.analog.com). U S2 LOI
`from STMicroelectronics
`
`ZTE Exhibit 1005
`Page 10 of 15
`
`

`

`US 2004/00953 17 Al
`
`May 20, 2004
`
`2
`
`(www.st.com). and MXD201 0UI\V from MEMSIC (www(cid:173)
`. mcmsic.com), can measure both dynamic and sialic accel(cid:173)
`eration and are good candidates for lISC in poinTing devices
`to determine the pitch and roll angles. The carth's gravity
`exerts a constant acceleration on the MEMS accelerometer.
`By calculating the accelerometer's stalic acceleration out(cid:173)
`puts, a tilt angle (pilch or roll) can be obtained.
`
`Iksides magnetic field scnsors and accelerometer
`[0006]
`sensors, gyro sensors can also be used in pointing device
`design, Gyro scowrs, such as the A DXI{S 150 M EMS gyro(cid:173)
`swpe from Analog [)evictS (www.analog.com).can delCe\
`changes in the device's orientation angle and thus can be
`used in delecting the pointing device's heading.
`
`[0007] The object of the present invention is (0 provide a
`low-cost, practical, univefM I pointing devke to cuntrol
`home entertainment systems ami computer systems using
`spatial orientation sensor technologies.
`
`SUMM ARY OF INVENTION
`
`[0008] A univcrsal pointing control sySlcm for tclevisions
`and computer displays is disclosed . The sySiem is comprised
`of a remotc handheld devicc, a display control unit and a
`(.un][nand delivery unit. T he
`remote handheld device
`includes a set of orientation sensors that detect the device's
`current orientation , In the prclerred embodiment, a two-axis
`magneti\: sensor identifies the device's alimuth angle by
`detecting the earth's magnetic field , and a dna l-axis accel(cid:173)
`erometer sensor identifies the device's inclination angle by
`detecting the earth's gravity. The signals from the orienta(cid:173)
`tion sensors are translated and encoded into pointing direc(cid:173)
`tion information by a microprocessor or logic circuits on the
`pointing device and transmitted to the display control unit.
`Along with the directional information, data regarding the
`nser's selection activities collected by a selection unit in the
`handheld device is also encoded and sent to the display unit.
`The display control unit inclndes a data transceiver, a CPU,
`and a display control circnit for interfacing target device .
`T he pointing direction information received by the trans(cid:173)
`eeiver is decoded and manipulated by the on board C PU.
`Ba:;cd on the pointing information, the CPU instructs the
`controlled target device interface, either a television set or a
`computer, to display a pointer at the corresponding coordi(cid:173)
`nates on the target device screen. Uscr selection activities
`are abo interpreted by the CPU based on the current pointer
`location, and corresponding commands are sent to the com(cid:173)
`mand delivery unit. The command delivery uni t, which can
`be a stand-alone device or built imo the handheld poiming
`device, forwards the commands to any remote controllable
`target device usi ng an inhared beam to execnte a desired
`operation.
`
`[0009] The handheld remote control devke is simple and
`easy to usc. User directly points to any position of the screen
`and a cursor is displayed on the screen at the pointcd
`location . fl y selecting a menu or active control shape on the
`screen using a selection button on the device, the user can
`control the target device's operation intuitively. Because
`fewer buttons are required to operate the device (e .g. a
`selection bntton, a calibration bntton, and a button to show
`and hide the on·screen pointer), the device can be made
`smaller and lighter. The selectable items can vary and
`change their appearance dynamically ba:s<;:d on the status of
`the operations. With visnal feedback, the system provides a
`
`much beller and friendlier graphi\:al interface to users .
`Because the pointing signals are generated [rom the hand(cid:173)
`held remote control device without reference to any source
`from othcr devices or equipment, there is no significant
`change ne(X:ssary on the television or computer system . In
`the described em\xxliment, the remote pointing device can
`be directly used in cxisting televisions and compnters with(cid:173)
`out any modification. 11le control scope of this system is
`broad enough to cover all the audiolvideo devices which are
`originally controlled by their respective remote controls.
`111e extendibility of the system allows new types of uevices
`to be easily adapted and controlled.
`
`BRI EF DESCRII'TI ON O F DRAWINGS
`
`[0010] FIG. I is a perspective view of the universal
`point ing system in coJltrol1ing a variety of equipments in the
`home entertainment system.
`[0011 ] FIG. 2 is a perspcctive view of the pointing system
`in colltrol1ing computer pre:s<;:ntatiolls.
`[0012] FIG. 3 shows the components in the handheld
`pointing device.
`[0013] FIG. 4a and 4b demonstrate the principal mecha(cid:173)
`nism of the orientation sensor detecting the device's orien(cid:173)
`tation changes. and how the screen pointer to reflects these
`changes.
`[0014] FIG. 5 is lhe functional block diagram of the
`pointing devi(X: .
`
`[0015] FIG. 6a is the functional block diagram of the
`display (.'On trollinit for a computer.
`
`[0016] FIG. 6b is the functional block diagram of the
`display controllinit for a home entertainment system.
`
`[001 7] FIG. 7(1 is the functional block diagram of the
`command delivery uniL
`[001 8] FIG. 7b shows the command delivery unit being
`trained by an original remote controL
`[0019] FIG. 8a is the alternative functional block diagram
`of the display control nnit which inclndes the remote control
`training circuit.
`[0020] FIG. 8b shows the display control unit being
`trained by an original remote control.
`
`D ETAlLED DESCRIPTION
`
`[002 .1] 11le prcS<;;ll1 invention's universal pointing ,-umrol
`system consists of a handheld pointing device 100, a display
`control unit 21MI and a command deliver unit 300 as shown
`in t-IG. L In this example. the display control unit 200 is
`connected to a television 400 and a video component device
`500, wh ich can be a digital 1V set-top box, a VCR, or a
`DVD player, through video cables 520 and 510, respec(cid:173)
`tively. The display co[)\rol ullit 200 call also be embedded
`inside the 1V or other video component device in alternative
`emlxxliments. The handheld pointing device IIMI is aimed at
`the television screen 420 imlicated by a line of sight 10. On
`the other end of this line, a pointer 410 is displayed on the
`scrl'Cn . When the user points the uevice to an arbitrary
`pOSition of the screen, a set of orientation sensors inside the
`point ing device 100, which will be describeu in later, detects
`the device' s current orientation and generate the pcinting
`
`ZTE Exhibit 1005
`Page 11 of 15
`
`

`

`US 2004/00953 17 Al
`
`May 20, 2004
`
`3
`
`direct ion signal. T he pointing direction signal i1; cncOlkd
`and sent 10 the display control unit 200 through a transmis(cid:173)
`sion link 50. This transmission link can be any form of signal
`linkage_ For example, it could be implemented by using
`radio frequency (RF) wireless link, infrared (m ) link, or
`even a wired cable. Upon receiving the signal, a central
`process unit (CPU) inside the display unit 200 decodes and
`analyzes the pointing dire<:tion and determines the new
`coordinates of the pointer on the screen , A )Xlinter is drawn
`al the calculated coordinate and the poi nter image is then
`superimposed onto the input villoo signal, whkh is input
`from video component device 500 through cable 570. A sct
`of menus and control items 430 are also drawn and super(cid:173)
`imposed to the villeo signal. The composite video is then
`output to the television 400 through the output video cable
`520 and displayed on the television sereen 420. As a resuh,
`the pointer 470 is shown at a new location on the screen
`whe re the user poinlS to, l 11e user perceives that the pointer
`is moved following the aiming line of sight 70.
`
`[0022] Buttons arc located on the handheld pointing
`device to colloct the user's selection activities. lllree buttons
`are shown in this example, one for command selection
`( 1411), one to show and hide screen pointer ( 102), and
`another one for calibration purpose (703). When the user
`uses the device at first time, a calibration procedure is
`performed. The user aims the device at the center of the
`screen and presses button 703. The device's pointing direc(cid:173)
`tion information is recorded and stor~d into the display
`control unit as the sereen center reference . Any subsequent
`pointing information is then compared wi th this reference,
`and the diflcrence will be cakulated as the pointer displal."C(cid:173)
`ment distanl.'t,) away from the screen center.
`
`[0023] During normal usage, as lhe user points and clicks
`the selection button, the on-screen menu or selectable items
`under the pointer arc processed by the CPU in the display
`control unit. Selection information is generated and for(cid:173)
`warded to the command delivery unit 300 by means of
`transmission liuk 60. lllC link 60, again, can be any form of
`signal linkage. The command deliver unit 300 can be a
`stand-alone device facing the TV 4(MI and other egu ipments
`(500,570.520), or can be embedtltd inside the pointing
`device 100. All remote control command cooes for the
`devices in the home entertainment system are prerecorded in
`a memory module in lhe command delivery unit 300. Upon
`receiving selection information, the command delivery unit
`issues
`the corresponding command by searehing
`the
`memory module, and emits the command infrared (1R.)
`signal through the IR emitter 351 to the controlled equip(cid:173)
`menlS. l 11e target equipment performs a task as if it had
`received a command directly from its original remote con(cid:173)
`trol device.
`
`[0024] FlC.2 shows the pointing control system as used
`in a computer presentation scenario. In this case, the pre(cid:173)
`sentation is projected onto the screen 720 by a projector 700,
`which receives the villeo input from a computer 600 though
`a video cable 620. The display control unit 200 is connected
`to the peripheral port of a computer 64MI through the cable
`610. The presenter aims the pointing devke at the screen 720
`by a line of sight to. The aiming direction information
`generated by a set of orientation sensors in the pointing
`device 100 is lransmitted to display control un it 200 through
`transmission link SO. 111e CPU in the display control unit
`interprets the direction information. sends the pointer move
`
`commanll to the computer 's peripheral port, and instructs
`the computer to move the pointer 710 on screen to Ihe aimed
`place. This is analogoll'; to moving Ihe pointer by moving a
`regular computer mouse device. except that the moving
`information is in absolute coordinates instead of relative
`steps. The buttons 101 , 102, and 103 on the pointing device
`allow the presenter to select and exc<:ute a command
`remolely.
`
`[0025] FIG. 3 exposes the components inside of the
`handheld pointing device. On the top face of the device arc
`bul10ns 101, 102, and 103 for collecting ILSer selection
`activities. A set of orientation sensors 120 and 130 mounted
`on the print circuit board 160 detect device's orientation
`I:hanges. Note that the sensors are mounted orthogonally to
`each olher. The sensor 120 detects the device's yaw (azi(cid:173)
`mUll!) angle and sensor 130 detects device's pitch (inclina(cid:173)
`tion) angle. Allditional sensors (not show in the picture)
`could be used to detect device' s roll angle which may
`provide an additional dimension of controL A mierocontrol(cid:173)
`ler 110 provides computation power for calculating and
`encoding Ihe oril:ntation signal output from the orientation
`sensors. It also provides logic control for the transmitter 140
`and other electronic components. The device is powered by
`batteries 170.
`
`[0026] The orientation sensors' mechanisms are shown in
`FIGS. 4a and 4b. The orientation sensor demonst rated in
`FIG . 4<1 is a magnelic field sensor. whereas the one in FlC.
`4b is an accelerometer sensor. llowever, the orientation
`detection may not be limited to these types of sensors. Other
`sensors, for example, a gyro sensor, can also be used in the
`point ing control systl:m. In I' IG. 4(1, a two-axis magnetic
`field sensor 120 IS used to detect Ihe device 's orientation
`relative to the direction of the earth's magnetic field 25. The
`sensor contains two magnetic field detectors which arc
`arranged orlhogonal to each other. The sensor is mounted on
`the device's circuit board so that the two magnetic field
`detectors are laid on the x-z plane as shown in the picture.
`TIle azimuth angle <p betw~en the devicl:'s heading direction
`and the earth's North Pole direction can be calculated from
`the sensor's x and 7. output: 1>- arc tan(xh) . When the LL<;cr
`performs calibration, the llevil.\: reconis Ihe azimuth angle
`"'0 as thc rderence angle as the user points thc device to
`cenler of Ihe screen. When the device is rotated about the
`y-axis anll the pointing direction is moved away from
`screen 's center, the azimuth angle llifTcrence from the ref(cid:173)
`erence angle is q.-To. This difTerenee is interpreted by the
`llisplay control unit as Ihe degree of the pointer's horir.ontal
`dcpanure from the screen center. The amount by which the
`pointer moves horizontally (22) can be adjusted in the
`display control unit proportionally to the change in the
`Jlointer's azimuth angle 21.
`
`[0027] The orientation sensor 130 uses a similar method to
`detect the device's indination angle. The sensor could be an
`accelerometer or another orientation sensor that can sense
`the device's heading change in the y-"L plane. An acceler(cid:173)
`ometer sensor which can detect static acceleration is
`lleseribed in detail here. '11Ie accelerometer sensor 130
`contains two orlhogonally arranged acceleralion detectors.
`The sensor is mounted perpendicular to the circuit board 's
`plane so that one detector in the sensor detects y-axis
`the other detects z-axis al.'CCleration.
`acceleration anll
`Eanh's gravity 26 exerts a static acceleration on these
`detectors. When the device is placed on a horizontal level.
`
`ZTE Exhibit 1005
`Page 12 of 15
`
`

`

`US 2004/00953 17 Al
`
`May 20, 2004
`
`4
`
`the accdnometer 's z-axis lletector outputs zero accekra(cid:173)
`lion, while the y-axis outputs the maximum acceleration ( t
`g), If the device is rotated about the x-axis, the z and y
`channel outputs of the sensor are changed according \0 the
`inclination anglc. The inclination angle E thus can be
`calculated : ",_ arc
`tan( z/y), During
`the calibration,
`the
`device's indination angle to the screen center Eo is recorded
`and slOred as a reference ang\e. Any inclination angks
`sampled thereafter is compared Wilh this reference angle by
`determining Ihe onset, (,-Eo-H lisdifJercnce is interpreted by
`the display control unit as a ucgn;,c of the pointer's departure
`from the screen's center in the ver1ical direct ion, The
`amount by which the pointer moves vertically (32) can be
`adj usted in the lIisplay control unit proportionally to the
`change in the pointer's inclination angle 31.
`
`[0028] For a simplifiell version, a one-axis accelerometer
`sensor can be used . In such a case, the acceleration detector
`is mounted along the device's z-axis. The inclination angle
`(' thus can be calculated: ('-arc sin(z).
`
`[0029] nG, 5 is the functional block diagram of thc
`handheld pointing device. The signal conditioning cir<.'1lit for
`sensor 120 consists of two amplifiers 12 1, 123 and two low
`pass filtcrs 122, 124. Becanse we are intcrestcd in the static
`position and low frequenl:)' movement of ttJt, lIevice, the
`high frequency noises of the amplified x-axis and y-axis
`signals arc filtered in order to get a higher resolution of the
`azimuth angle changes. Two amplifiers 131, 133 anll two
`low pass filters 132, 134 are for conditioning the sensor 130
`output's x-axis and z-axis signals. We are interested in the
`sensor's static output relative to earth's conslanl gravity.
`11lerefore, the high frequency noises of the signals are also
`filtered in order to get a higher resolution of the inclination
`angle changes. llle conditioned signals from sensors 12()
`and ,13() arc thcn sem to an analog-to-digital convcrtcr
`(AOC) ill by an analog multiplexer 112. The digitizcd
`sensors dala are Ihen scntto a microconlroller (MCU) 110
`for further signal proccssing. Some variations of the orien(cid:173)
`tation sensor convert the analog signal internally to a digital
`or time period-based signal. In those cases, the signals can
`be directly sampled by Ihe microcomroller wi thoul an A])C
`I:hip. The M CU 110 wmpu ttS the azimuth and inclination
`angles. iJ ullons 101 , 102, and 103 produce aclivily signals
`that are sampled by M CU 100. The sensor orientation data
`and bUlIons activities are coded in such a way that the
`display conlrol unit can decodt them laler. T he encoded data
`is passed to a modulator 113 to modulate a carrier frequency
`for lTansmission. 'Ibe transmitter 140 emits Ihe m<xlulated
`signal 50 to thc display control nnit 200. Thc circuit is
`powcred by ballcrics 17(). A bal1cry manager unit 171
`condilions Ihe voltage for all components of the circuil . The
`M CU 110 constantly monitors the changes of sensor out(cid:173)
`pUIS. If the scllSOr outputs are not changed for a period, the
`M CU interprels the device as nol been used . The MCU
`inslmcts Ihe ballery manager unit 171 to sh ul down Ihe
`ballery power supply to the transmi11er and other I:ompo(cid:173)
`nenls in order to save power consumplion during Ihe idle
`stage.
`
`[0030] nG. 60 an ll FlG_ 6b show the functional block
`diagrams of display control unit 200 for a computer and a
`television set, respectively. A central process unit (CPU)
`210. a rece iver 221. a lIemodu lator 23 1, and a memory
`modu le 27() are common for both casts. TIle transrnilted
`signal SO from the pointing device. which includes band held
`
`lIevice orientation and user !;CIccI ion activities, is intercepted
`by the receiver module 221. After being demodulated by a
`dem<xlulalor 231 , the pointing device data is sent to CPU
`2.10 for funher processing. Thc CPU compares the device's
`azimuth and inclination angle data with tbe reference angles.
`which are sampled and slored in the memory module 270
`during the calibration proccdure. The difference angles
`calculated are translated into screen coordinates and the
`targc1 device is inslrucled 10 move Ihe poinler to the new
`local ion . 'Inc inlerface componcnts of thc disp lay conlrol
`unit are diflcrenl fm cadi I:ontrol targe t. In FIG. 6a, a
`computer peripheral inlerface module is used 10 connect 10
`a computer port. The pointer coordinates are sent to the
`compuler and, by the computer's processor anll video canl,
`the poinler on the screen is moved 10 Ihe corresponding
`location . l be bulton activities are also sent to the computer
`Ihough this interface and trigger certain actions for the
`computcr.
`
`[003.1] FIG. 6b demonstrates tbe display control interfaces
`to a television. The input video signal. which may come
`(rom othcr home entertainment devi~"Cs such as digital TV
`set-lop boxes, D VD players, ele ., is decoded by a video
`decoder 2S 1 frame by frame. A new pointcr imagc is drawn
`at the coordinate calculated by CPU 2 10. l be pointer image.
`along with menus and other control ilem images pre-slored
`in Ihe memory module 270, are sent to a graphic-video
`multiplexer 250 to superimpose onto a villeo frame . T he
`composite villeo frame is then encoded by a video tn(.wer
`252 and sent to the television for display. The process is
`about 30 frames persecond . As the result, a pointer is moved
`on lop of the vitleo following the handheh.l de\'i~"C·s pointing
`direction. If the CPU 2JO senses a bu