United States Patent (19)
`Numazaki
`
`USOO590O863A
`Patent Number:
`11
`(45) Date of Patent:
`
`5,900,863
`May 4, 1999
`
`54 METHOD AND APPARATUS FOR
`CONTROLLING COMPUTER WITHOUT
`TOUCHING INPUT DEVICE
`75 Inventor: Shunichi Numazaki, Yokohama, Japan
`73 Assignee: Kabushiki Kaisha Toshiba, Kawasaki,
`Japan
`
`21 Appl. No.: 08/614,502
`22 Filed:
`Mar 13, 1996
`30
`Foreign Application Priority Data
`Mar. 16, 1995
`JP
`Japan .................................... 7-083460
`Sep. 28, 1995
`JP
`Japan .................................... 7-2SO418
`(51) Int. Cl." ....................................................... G09G 5/08
`52 U.S. Cl. .......................... 345/158; 34.5/156; 34.5/157;
`345/159; 34.5/175
`58 Field of Search ..................................... 34.5/158, 157,
`345/156, 159, 160, 12, 7, 175; 348/140,
`141
`
`56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,782,328 11/1988 Denlinger ................................ 345/175
`4,988,981
`1/1991 Zimmerman et al.
`... 34.5/156
`5,168,531 12/1992 Sigel .
`... 34.5/157
`5,367,315 11/1994 Pan ........
`... 34.5/158
`5,459,488 10/1995 Geiser ..................................... 34.5/156
`
`5,686,940 11/1997 Kuga ....................................... 34.5/159
`5,686,942 11/1997 Ball ......................................... 34.5/158
`FOREIGN PATENT DOCUMENTS
`
`7-057103 3/1995 Japan.
`OTHER PUBLICATIONS
`30th General Conference (Zenkoku Taikai) of Information
`Processing, Society of Japan, pp. 1249-1250 (1985 Spring).
`Primary Examiner Jeffery A. Hofsass
`ASSistant Examiner John Tweel, Jr.
`Attorney, Agent, or Firm-Oblon, Spivak, McClelland,
`Maier & Neustadt, P.C.
`57
`ABSTRACT
`A distance image representing a distribution of distances,
`each measured between the apparatus and each conceivable
`point existing on an object is input. The minimum points in
`the distance image are detected. The minimum points are
`Some of the conceivable points on the object which have
`Smaller distance values than the other conceivable points.
`The time differences, each representing a change which each
`point in the distance image assumes as time passes, are
`detected. A designating point for designating a command to
`the System is determined. The designating point is one of the
`minimum points which has changed in the time difference
`detected. A System is controlled in accordance with motion
`of the designating point determined.
`
`33 Claims, 46 Drawing Sheets
`
`
`
`NFRARED
`LED
`
`O
`
`
`
`OBJECT
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 1
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 1 of 46
`
`5,900,863
`
`DSTANCE
`MAGE
`NPUTTNG
`SECTION
`
`D STANCE
`IMAGE
`ANALYZNG
`SECT ON
`
`PROCESSING
`SECTION
`
`
`
`
`
`D STANCE
`MAGE
`NPUTTNG
`SECTION
`
`
`
`
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PONT
`DETERMINING
`SECTION
`
`
`
`
`
`
`
`SYSTEM
`CONTROL
`SECTION
`
`F G. 2
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 2
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 2 of 46
`
`5,900,863
`
`
`
`2
`
`-
`-(
`- 1 N
`N 24
`-1
`N -
`Y&
`N
`-----
`
`1.
`N
`N
`N
`
`- - -
`
`- m -- - - - - - - - - - - - -
`
`23
`
`N
`
`-
`
`N
`
`REGON N. N.
`SCANNED
`
`F G. 4
`
`r
`
`LED ARRAY
`
`LENS
`
`N
`
`N
`
`SN N
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 3
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 3 of 46
`
`5,900,863
`
`ZZ
`
`LED ARRAY
`
`F G. 5
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LED ARRAY
`
`LED ARRAY
`CONTROL
`SECTION
`
`TMNG
`CONTROL
`SECTION
`
`
`
`F G. 6
`
`POST ON
`SENSING
`DETECTOR
`
`D STANCE
`CALCULATING
`SECTION
`
`42
`
`43
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`PONT DETERMINING
`SECT ON
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 4
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 4 of 46
`
`5,900,863
`
`50
`
`50)
`
`50
`
`50
`
`3-
`
`54
`
`--
`era
`SETERHSEESTERF-----
`REGISTER, 54
`54
`
`
`
`
`
`
`
`SHIFT
`REGISTER
`
`SHIFT
`REGISTER
`
`SHIFT
`REGISTER
`
`53
`
`
`
`CLOCK
`GENERATOR
`
`FREQUENCY
`DVDER
`
`PULSE
`EXTRACTOR
`
`GENERATING OF
`TIMING PULSES
`
`5
`
`F G.
`
`7
`
`DSTANCE
`CALCULATING
`SECTION
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 5
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 5 of 46
`
`5,900,863
`
`REC PROCAL
`GENERATOR
`
`
`
`REC PROCAL
`GENERATOR
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 6
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 6 of 46
`
`5,900,863
`
`€GNNOYDHOVE¢GNNOYDADVEtGNNOYSNIVEJUiiuu
`tONNOYDNIVE
`€ANNOYSNDVE&ONNOYDNIVE
`£Q341€031}a3)
`
`
`GSd40LAdiNd
`V6914
`
`OL
`
`Y599INLA/V
`TWNOIS
`86O14
`
`tL
`
`YA99IYLHOLV)
`
`“WNOIS
`Q6914
`
`el
`
`ol
`
`a/v
`
`4OANdLNO
`J6914
`
`HO1W1JOAdLNO
`469I4
`
`vl
`
`ONIGMNIIXSLAdLAO
`SMIVAGQNNOYOIVE
`
`46914
`
`Gl
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 7
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 7
`
`
`

`

`U.S. Patent
`
`May4, 1999
`
`Sheet 7 of 46
`
` QNNOYDNOVE
`QNNOYSNOVE
`
`Q/V
`
`28
`
`£8
`
`18
`
`ONIGMIX3=LAdLNO
`ANIWAQNNOYDNIVE
`
`QSd40LAdLNO
`YA99INLHOI)
`YANNIYLA/V
`JOANdLNO
`IWNOIS
`“WNOIS
`dotOld
`20+Ol4
`40+94
`dOtOld
`dOt9Is
`VOtOs
`
`5,900,863
`
`
`
`G8
`
`ANNOYSNIVEHOLV1401Nd1LNO
`
`v8
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 8
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 8
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 8 of 46
`
`5,900,863
`
`
`
`Do < D.
`
`Y-COORDINATE N
`D
`ND F G.
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 9
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 9 of 46
`
`5,900,863
`
`
`
`
`
`
`
`
`
`HOLVHWc][NOO
`
`| | Q | | £
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 10
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 10 of 46
`
`5,900,863
`
`
`
`V1V0 BONV1S10
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 11
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 11 of 46
`
`5,900,863
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LED ARRAY
`
`LED ARRAY
`CONTROL
`SECTION
`
`
`
`
`
`TMNG
`CONTROL
`SECTION
`
`
`
`
`
`
`
`42
`
`43
`
`POSITION
`SENSING
`DETECTOR
`
`D STANCE
`CAL CULATING
`SECTION
`
`
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`CAND DATE POINT
`STORNG SECTION
`
`
`
`PONT DETERMINING
`SECTION
`
`2
`
`F G.
`
`4
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 12
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 12 of 46
`
`5,900,863
`
`
`
`Ç HOLWT
`
`| HOLWT
`
`G |
`
`HETTOH1NOO
`H01||MS
`
`Z X10
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 13
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 13 of 46
`
`5,900,863
`
`STORAGE-NSTRUCTNG
`FLAG
`
`ESANCE
`COORDINATE -
`DATA
`
`-
`
`F G.
`
`6
`
`TO MEMORY
`- CRCUIT
`
`INPUT TRIGGER SGNAL
`TO FLP-FLOP
`CK
`
`X
`X
`
`X
`
`F.G. 7A X
`FIG 7B x2
`
`FG. 7C X3
`
`F.G. 7 D Q3
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 14
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 14 Of 46
`
`5,900,863
`
`
`
`TXIX-XI IX-X X X X X X
`
`08 | 9 | -!
`
`(18 | 9 | -!
`
`38 | 9 | -!
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 15
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 15 of 46
`
`5,900,863
`
`2
`
`LED ARRAY
`
`40
`
`LED ARRAY
`CONTROL
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TMING
`CONTROL
`SECTION
`
`
`
`
`
`
`
`N
`
`Position ?'
`SENSING
`DETECTOR
`
`D STANCE
`CAL CULATING
`SECTION
`
`43
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`
`
`
`
`
`
`44
`
`MNMUM
`PONT
`DETECTNG
`SECTION
`
`CAND DATE POINT
`STORING SECTION
`
`
`
`
`
`
`
`SECTION
`
`SECTION
`
`SECTION
`
`SWITCHING/52
`CONTROL
`SECTION
`
`F G.
`
`9
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 16
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 16 of 46
`
`5,900,863
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LATCH
`CRCUT
`
`
`
`
`
`LATCH
`CRCUIT
`
`
`
`LATCH
`CRCUIT
`
`LATCH
`CRCUT
`
`
`
`DSTANCE
`CALCULATING
`CRCUIT
`
`DSTANCE
`CALCULATING
`CRCUT
`
`DSTANCE
`CAL CULATING
`CIRCU
`
`
`
`COMPARATOR
`SECTION
`
`PONT RENEWING
`SECTION
`
`68
`
`F G. 2 O
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 17
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 17 of 46
`
`5,900,863
`
`40
`
`LED ARRAY
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LED ARRAY
`CONTROL
`SECTION
`
`
`
`TMNG
`CONTROL
`SECTION
`
`
`
`
`
`
`
`
`
`42
`
`43
`
`POST
`SERS
`DETECTOR
`
`DISTANCE
`CALCULATING
`SECTION
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`44
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`
`
`PONT
`DETECTING
`50\SEoN
`
`PONT
`STORING
`SECTION
`
`PONT
`TRACKING
`SECTION
`
`52
`
`
`
`SWITCHING
`CONTROL
`SECTION
`
`F G. 2
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 18
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 4, 1999
`May 4, 1999
`
`Sheet 18 of 46
`Sheet 18 of 46
`
`5,900,863
`5,900,863
`
`SAILOVYSSy4
`
`X3S0NI
`
`Uy7ee
`
`
`
`NOILVuYad0
`
`NOIL9SS
`
`YALYSANOD
`
`a/v
`
`v2ete
`
`cle
`
`“JOYULNOD
`
`NO11L93S
`
`
`
`
`
`
`
`
`
`
`
`
`
`{2,SY|\uff|_\_\
`OltOlt—p>—\
`[ft|77
`YalslNV
`
`ONIAIS9SY-LH917
`NOILOAS|-1—
`peO14
`ceOlS
`ONILLINS-LH9N1
`
`014
`Olt
`
`Ole
`
`Z Z 9 | -!
`
`NOIL93S
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 19
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 19
`
`
`
`
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 19 of 46
`
`5,900,863
`
`D STANCE
`MAGE
`NPUT
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PONT
`ENG
`
`
`
`TME
`DFFERENCE
`DETECTING
`SECTION
`
`PONT
`DETERMINING
`SECTION
`
`CURSOR
`CONTROL
`SECTION
`
`CONTROL
`EXECUTING
`
`SSMAN:
`SECTION
`
`F
`
`G. 23
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 20
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 20 of 46
`
`5,900,863
`
`CURRENT FOR
`FG.25A 5NN Eb
`
`FG.25B
`
`CURRENT OUTPUT
`SS's
`
`FG 25C TRGGER SGNAL FOR
`
`
`
`
`
`FIRST SAMPLE
`- AND - HOLD
`CIRCUIT
`
`SECOND SAMPLE
`-AND- HOLD
`CIRCUIT
`
`
`
`
`
`
`
`F. G. 27 A CURRENT FOR DRIVING LED
`
`
`
`F. G. 27B CURRENT OUTPUT FROM PSD
`
`SSESSES
`FG. 27C ESS
`E. E. Elio
`
`FG. 27D cott
`
`TRIGGER SIGNAL FOR
`F. G. 27E ADCONVERSION
`
`:
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 21
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 21 of 46
`
`5,900,863
`
`230
`
`
`
`23
`
`232
`
`LGHT
`-RECEWING
`SECTION
`
`AMPL FER
`
`LGHT
`-EMITTING
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`MAGNITUDE
`DETERMINING
`SECTION
`
`OPERATION
`SECTION
`
`OUTPUT
`SECTION
`
`A/D
`CONVERTER
`
`
`
`
`
`F G.
`
`28
`
`
`
`
`
`240
`
`24
`
`242
`
`243
`
`DISTANCE
`MAGE
`INPUT
`SECTION
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`MNMUM
`PONT
`SELECTING
`SECTION
`
`PROCESSING
`SECTION
`
`F G. 29
`
`240
`
`24
`
`242
`
`243
`
`DSTANCE
`MAGE
`INPUT
`SECTION
`
`
`
`
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`MNMUM
`PONT
`SELECTING
`SECTION
`
`PROCESSING
`SECTION
`
`F G. 3 O
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 22
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 22 of 46
`
`5,900,863
`
`25
`
`
`
`POINT
`MOTON
`DETECTING
`SECTION
`
`
`
`
`
`DSTANCE
`MAGE
`NPUT
`SECTION
`
`
`
`
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`
`
`
`
`MNIMUM
`PONT
`SELECTING
`SECTION
`
`PROCESSING
`SECTION
`
`OBJECT
`ANALYZNG
`253 - SECTION
`
`
`
`F G. 34
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 23
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 23 of 46
`
`5,900,863
`
`
`
`
`
`
`
`DSTANCE WALUE OF
`MNMUM POINT
`-- dm COUNTERFO
`
`COUNTER
`
`NO MORE SPREAD
`?o
`
`
`
`F G. 32
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 24
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 24 of 46
`
`5,900,863
`
`
`
`N0 1 10BS
`
`
`
`292
`
`292
`
`092
`
`2 2
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 25
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 25 of 46
`
`5,900,863
`
`DSTANCE
`MAGE
`INPUT
`SECTION
`
`27O
`
`MN MUM
`PONT
`DETECNG
`SECTION
`
`27
`
`PONT-ATNG
`TME
`DETECT NG
`SECTION
`
`273
`
`
`
`274
`
`PONT-MENU
`CORRESPONDENCE
`OETECT NG
`F G. 35 SECTION
`
`MNMUM
`PONT
`SELECTING ? 272
`SECTION
`
`275
`
`COMMAND
`EXECUTNG
`SECTION
`
`DSPLAY
`CONTROL
`SECTION
`
`D STANCE
`IMAGE
`INPUT
`SECTION
`
`
`
`
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`MN MUM
`POINT
`SELECNG
`SECTION
`
`272
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PONT- HALTING
`TME
`DETECTING
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`POINT-MENU
`CORRESPONDENCE
`DETECTING
`SECTION
`
`
`
`COMMAND
`EXECUTNG
`SECTION
`
`DISPLAY
`CONTROL
`SECTION
`
`
`
`F G. 36
`
`
`
`
`
`
`
`
`
`SPEECH
`OUTPUT
`SECTION
`
`
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 26
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 26 of 46
`
`5,900,863
`
`DSTANCE
`MAGE
`INPUT
`SECTION
`
`
`
`MNMUM
`PONT
`DETECTING
`SECTION
`
`
`
`
`
`282
`
`
`
`
`
`FRST
`PONT-MOTION
`DETECTING
`SECTION
`
`SECOND
`PONT-MOTON
`DETECTING
`SECTION
`
`
`
`COMMAND
`EXECUTING
`SECTION
`
`285
`
`
`
`MENU
`CONTROL
`SECTION
`
`
`
`F G. 37
`
`F G. 38
`
`290
`
`
`
`
`
`7 Saas BAR
`
`SNACK BAR CAFE
`
`7
`
`OOO~
`M 71 / /
`
`& 290
`
`
`
`
`
`FRENCE
`
`TALIAN
`
`JAPANESE OHERS
`
`7
`
`
`
`~ 3000
`
`- 6000
`
`- OOOO
`
`F G. 39
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 27
`
`

`

`U.S. Patent
`U.S. Patent
`
`May4, 1999
`May 4, 1999
`
`Sheet 27 of 46
`Sheet 27 Of 46
`
`5,900,863
`5,900,863
`
`On D
`
`Lo
`
`F G. 4
`FIG. 41
`
`Y I
`
`<
`
`PR2022-00090 - LGE
`
`Ex. 1005 - Page 28
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 28
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 28 of 46
`
`5,900,863
`
`|{
`
`ro--
`
`Blt
`
`|[ppee
`|YOLVYOSINI
`4aLY3ANO)|_|yo1jatad
`
`6LtOBI.
`Q/Vli|qaavis-aou
`YOLWYANSSWal4Luadid
`aintosay{|SSvd-aNveaud
`
`4yOLOVYLXS40194140
`
`dildouY3LNADALIAVYS
`18NO1L93S
`LINDH19|oniia
`
`NOILOAS98|NOILOSS|-~SBI
`
`LAdLAOLNdLNO
`
`JA8G-1HO17
`ONIALS9S94-1LH911O6t
`
`68t
`
`ONIATddV
`-LHOIT16}
`
`SN31
`
`v8t
`
`TWHYALN32
`
`40.194.140
`
`c6t
`
`Lt
`
`40193130
`
`-OLOHd
`
`MWY
`
`\SN31
`
`c6t
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 29
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 29
`
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 4, 1999
`May 4, 1999
`
`Sheet 29 of 46
`Sheet 29 of 46
`
`5,900,863
`5,900,863
`
`F G. 4 3
`FIG. 43
`
`
`
`
`
`
`
`
`
`
`
` CeCeCCCNSaCECeeeeeeCEPeeeer
`CEEEEEEE
`
`CECCre
`CECCETSeas
`CeCeCeeEe
`CECCORRE
`CECEGEST
`
`CEee
`SIEEECeCe
`COO
`Cee
`
`
`
`
`
`
`
`402
`02
`
`F G. 45
`FIG 45
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 30
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 30
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`j~—
`
`
`1O¢4OLNSYYNDLNdNIONTLLIWS-LHOI
`
`UMUU0NO1L93S)ON|LL1WS-LH917NO1193S
`
`
`
`May 4, 1999
`May 4, 1999
`
`
`
` -43LY3ANODA/1aZ20¢2)p~{HOLIMSSOTWNY
`
`vLE~
`
`FJOVLTIONoeTWNOISa
`
`Sheet 30 of 46
`Sheet 30 of 46
`
`GOs
`
`5,900,863
`5,900,863
`
`13S3u—13S38—
`
`boO14
`
`JOVITOA
`
`WNOISBlt
`
`JOVLIOA
`
`——
`
`
`WNOIS4Li—amvaaini0sav
`
`YOLVYANAS
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 31
`
`4OLNSYYNDLAdLNOYOLISL30OLOHd
`HOLIMSSOTWNYAVY
`
`
`
`
`
`
`HhSA»
`
`68t
`
`
`
`26h
`
`4314dWv
`
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 31
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 4, 1999
`May 4, 1999
`
`Sheet 31 of 46
`Sheet 31 of 46
`
`5,900,863
`5,900,863
`
`
`
`%
`×
`
`
`
`
`
`102
`
`
`
`F G. 48
`FIG 48
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 32
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 32
`
`

`

`U.S. Patent
`U.S. Patent
`
`May4, 1999
`May 4, 1999
`
`Sheet 32 of 46
`Sheet 32 of 46
`
`5,900,863
`5,900,863
`
`— a o
`
`O L
`
`CS
`CO
`
`s
`LAL
`
`
`
`i I
`
`PR2022-00090 - LGE
`
`1005 - Page 33
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 33
`
`

`

`U.S. Patent
`U.S. Patent
`
`May4, 1999
`May 4, 1999
`
`Sheet 33 of 46
`Sheet 33 of 46
`
`5,900,863
`5,900,863
`
`P(n.k)
`
`: N-N
`803J
`
`P(n.k)
`
`OS
`CN
`oO
`O
`@
`OO
`
`P(4.4)
`
`P(n.n)
`
`FIG.49
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 34
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 34
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 34 of 46
`
`5,900,863
`
`
`
`
`
`W
`
`D STANCE
`THE FINGER
`TP HAS MOVED
`
`- I
`
`-
`
`N N al THE
`
`
`
`GRAVITY CENTE
`HAS MOVED
`
`DSTANCE THE
`FNGER T P HAS
`MOVED
`
`N
`
`NN
`
`NYTNNNN
`
`NS I-NN
`NN.NN
`NEFERNNN
`
`F G. 5 A N -1 F G. 5 B
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 35
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 35 of 46
`
`5,900,863
`
`X-COORD NATE OF
`GRANTY ENER
`DOES NOT CHANGE
`
`DSTANCE THE
`ENERTIP'HAS
`MOVED
`
`(
`rise I
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 36
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 36 of 46
`
`5,900,863
`
`| L-BAGH---<
`
`L) – – – – – – – – – – – – – – – –~ ~— — —
`
`
`
`
`
`
`
`Y - AXS DRIVE CIRCUIT
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 37
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 37 of 46
`
`5,900,863
`
`TWINS) IS 9OTWNW
`
`9NISSE|00}}d
`
`NO 1 10BS
`
`NO 1 103S
`
`TOH1N00
`
`Y - AXS DRVE CRCUT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 38
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 38 of 46
`
`5,900,863
`
`
`
`ROD TP
`EXTRACTOR
`
`TP-EXTRACTING
`PARAMETER
`CALCULATOR
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 39
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 39 Of 46
`
`5,900,863
`
`
`
`J. G (9) | H
`
`80.103.130
`
`8 G (9) | -
`
`N0 1 103S
`
`80SNES
`
`N.01.108
`
`NO | 10BS
`
`HOSNES
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 40
`
`

`

`U.S. Patent
`U.S. Patent
`
`May4, 1999
`May 4, 1999
`
`Sheet 40 of 46
`Sheet 40 of 46
`
`5,900,863
`5,900,863
`
`802
`
`804.
`
`
`
`i802
`
`F G. 59
`FIG 59
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 41
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 41
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 41 of 46
`
`5,900,863
`
`costSMON16ev
`
`L061SNONNILNODSHLOLSNONNILNOD
`
`
`QNNOS4OSMOY9NVHLSS31
`
`
`
`[U]YO1NSO}U!
`
`
`
`S3LVGIONV)LI3EO
`
`
`
`tO6!”|G3dVHS-GO4ONIHOLSHO4
`
`
`
`
`
`SLINNJ9VYOLS3uvdaud
`
`
`
`ADVANISLVAIGNVD
`
`é03193130
`
`
`
`JOLYVLS40SLVNIGHOODX=—
`JHL40NOILVWYOSNISLVddN
`
`[Myoysx-Cu}yO1NSO}U!
`
`
`
`906SAONNTLNGS
`QAdVHS-d0dMANY31$193¥u
`
`
` NNe0coerNOILF1dWOD¢SNI1Wi-
`
`NOZ1YOHHOVSNiGNNOS
`AYOWHOd403NIVA
`ONIAVWHHOWS'ST3aXId
`
`ALVGIONVD
`
`S13Xid
`
`L93¢8O
`
`Q3SHOV3AYB06
`
`P<yjbua}[u]yotisojut
`
`éaNWAGaNIW
`
`G3HOVSYLON
`
`6061
`
`VO9914
`
`—431303udVCaHOVIY
`
`Q3-
`
`y914Soju!GSONALX3—SN}DJs[UW]4O14SOju!}+——4buajCu]
`
`
`
`
`40ON330SLVNIGHOO)X~—
`
`[0]pusx[uJxO1,So4u!
`
`MIILSON—SAYDISUYolSojul
`
`$1AXId
`
`4OONS40SLVNIGHOODX<—
`[!]puiax-[U]4914S04Ul
`
`4OLYVLS40SLVNIGHYOQO)X=—
`
`
`[Oo]1D}SX[u]yDI}SOjUI
`SIHLNIGalvddNLONJlvd
`
`
`IQNVDGadVHS-GOH314140
`
`$14X1d
`
`S13aXId
`
`
`
`3NI1JNO
`
`OaLVOdn_40HLONSTSVH
`
`193f80d3dVvHS-G0u
`
`GN31X3-——snjpjs(U]YOI4SOjUl
`
`
`1-—uyjbuaj(u]yosojut
`
`JNI1LXSNOLSAOW
`
`vO6t
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 42
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 42
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 42 of 46
`
`5,900,863
`
`(A) 2OO
`
`CALCULATE TH
`TH- (sum/mox)2X mox
`Sun AVERAGE OF TOTAL PXEL VALUES PER
`LNE WITH RESPECT TO FRST AND
`SECOND LNES (UPPERMOST S Oth
`LNE) FROM ROD-SHAPED OBJECT
`TOX LARGEST VALUE AMONG FRST AND
`SECOND LNES
`
`2OO2
`
`ASSUME PXEL VALUE SUM OF SOME OF
`PXELS IN iTH LINE (UPPERMOST S
`OTH LNE) OF ROD-SHAPED OBJECT S
`XSum (i), OBTAIN k and a ( 0< 0 < . )
`K
`SATSF ING Thisum (i)+O sum (k
`F
`
`2003
`
`MULTIPLY EACH LARGE-VALUE PXEL
`N K-TH ROW BY WARABLE O
`
`2OO4
`
`
`
`K
`
`OBTAIN GRAVITY CENTER (XX yy)
`K
`yy =
`sumti) (infostick (n) ystart + i) / TH
`XX =
`psum i/TH
`infoStick (U) Xend (i)
`x
`: P (j, infoStick (n) ystart + i)
`=infoStick (u). X starti)
`
`xpsum (i) =
`
`2005
`
`F G. 6 OB
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 43
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 43 of 46
`
`5,900,863
`
`
`
`2O2
`
`O
`
`N
`
`2O
`
`y
`
`l
`
`infoStick (nySt Ort
`infoStick (n). xstart (i)
`infoStick (n). Xend (i)
`
`infoStick (n), length
`
`infoStick (n). Stotus
`ST CK :
`NOSTICK.
`EXTENDED :
`
`F G. 6 B
`
`
`
`2O3
`
`AFTER LARGE-VALUE PIXELS FOUND IN LINE
`(y = 3) HAVE BEEN REGISTERED AS A NEW
`CAND DATE FOR A RODSHAPED OBJECT
`infoStick (Oystart =3
`infoStick (O.XStortO) =8
`infoStickLO). Xend (O) =8
`infoStick (O) length =
`infoStick O. St0tuS = EXTENDED
`
`F G. 6 C
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 44
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`Sheet 44 of 46
`
`5,900,863
`
`2O4
`
`AISR SCANN NG HAS PROCEEDED TO LINE
`y =
`infoStick (O). ystort = 3
`infoStick (O)xStort (O) = 8
`infoStiCKLO)xStort = 5
`infoStick (O). Xend (O) = 8
`infoStick (Oxend)= 7
`infoStick (O) length = 2
`infoStick O. status = EXTENDED
`F G. 6 D
`AFTER CONNECTING LARGE-VALUE PXELS
`AT THE MDDLE PART OF LINE (y = 5) TO
`A CAND DATE FOR A RODSHAPED OBJECT
`infoStickLO). ystort = 3
`infoStick (O)xStort (O)=8
`infoStick (O)xStort = 5
`infoStick (O)xStort 2) = 5
`infoStick (O).Xend (O) = 8
`infoStick (Oxend ( ) = 7
`infoStick(O).Xend (2) = 7
`info StickLO) length = 3
`infoStick O. St OtuS = EXTENDED
`
`2O5
`
`F G. 6 E
`
`
`
`X
`O 2 3 4 5 6 78 9 O
`y 0 || || || || ||
`|| || || || || - 230
`3 PHIN-1-1
`3 N1
`N N
`A
`5
`N N
`6
`N N
`I
`N
`||
`SN
`9
`N
`N
`ON N
`
`2302-N-
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 45
`
`

`

`U.S. Patent
`
`May 4, 1999
`
`
`
`Sheet 45 of 46
`
`5,900,863
`
`O
`2
`3
`
`7
`8
`9
`O
`
`22O2
`
`F G. 6 2 A
`
`
`
`
`
`
`
`
`
`
`
`22O3
`
`
`
`F G. 62 B
`
`
`
`2204
`
`
`
`F G. 62 C
`
`
`
`22O5
`
`
`
`F G. 62 D
`
`AFTER LINE ( y = 2) HAS BEEN
`SCANNED
`infoStick (O). ystart = 2
`infoStick (O)xStort (O) = 3
`infoStick (O). Xend O = 3
`infoStickIO), length = }
`infoStick (O) status = EXTENDED
`
`AFTER LINE (y = 3) HAS BEEN
`SCANNED
`infoStick (O). ystart = 2
`infoStick (O)xStort (O)=3
`infoStick (O). Xend (O) = 3
`infoStick (O), length =
`infoStick(O). Sto tuS = NOSTICK
`
`
`
`infoStick (Alystort = 3
`infoStick (). xStort (O) = 6
`infoStick (). Xend (O) = 6
`infoStick () length =
`infoStick () status = EXTENDED
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 46
`
`

`

`U.S. Patent
`U.S. Patent
`
`May4, 1999
`May 4, 1999
`
`Sheet 46 of 46
`Sheet 46 of 46
`
`5,900,863
`5,900,863
`
`
`
`
`
`F G. 64
`FIG. 64
`
`IPR2022-00090 - LGE
`
`Ex. 1005 - Page 47
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 47
`
`

`

`1
`METHOD AND APPARATUS FOR
`CONTROLLING COMPUTER WITHOUT
`TOUCHING INPUT DEVICE
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention relates to a device designed to input
`data for controlling or remote-controlling, for example, an
`electronic apparatus.
`2. Description of the Related Art
`Most of the data input devices (hereinafter called “control
`devices') hitherto developed to input control data are hand
`held ones. A user operates a control device of this type, while
`holding it in hand. Control devices most commonly used in
`households are remote controllers. A remote controller has
`Several buttons on it. The user holds it in a hand, aims it at
`an electronic apparatus and pushes one of the buttons. When
`the button is pushed, the remote controller emits an infrared
`beam to the apparatus, thus giving instructions to the appa
`ratuS.
`Among the control devices hitherto developed, other than
`the hand-held ones, is a So-called touch panel. The touch
`panel comprises a transparent or opaque panel. When a user
`pushes any part of the panel with a finger or an elongated
`object, the touch panel inputs a data item representing the
`pushing of that part or the position of that part. A transparent
`panel may be mounted on a display Screen displaying
`images of buttons. In this case, the user may push that part
`of the transparent panel located above one of the buttons
`displayed, thereby to input the data item indicating that the
`button has been Selected.
`In the field of image recognition, research has been done
`for a method of recognizing hand Signaling and gestures. No
`practically usable methods have been developed, however.
`This is because various requirements are imposed, which are
`hard to fulfill in practice. More specifically, it is required that
`the background on which to display hands or a perSon be
`monotonous So that image data may be processed more
`easily. It is also required that the hands or the perSon be
`displayed in a relatively large size at the central part of a
`display Screen.
`Also, research has been conducted to facilitate image
`recognition. More precisely, an attempt has been made to
`develop a method in which Stereoscopic analysis is per
`formed on an object, based on trigonometrical Survey, in
`order to recognize the three-dimensional shape of the object.
`Nonetheless, only a few cases are known in which image
`recognition techniques are applied to remote controllers.
`A user cannot operate a remote controller unless he or she
`holds it in a hand. It is undesirable that the remote controller
`is held in a hand which has gotten dirty.
`To operate a touch panel, the user need not hold anything
`at all. However, the panel may get dirty as the user repeat
`edly touches the panel with fingers. It may become So dirty
`that the images display on the display Screen can no longer
`be seen through the panel. If the panel is dirty after a long
`use by a user, other users are discouraged to touch the panel
`in order to input data.
`In the technique of recognizing hand Signaling, achieved
`by processing the image data, the results are hardly reliable,
`greatly influenced by noise. The technique should therefore
`be improved to eliminate the influence of noise.
`There is a great demand for control devices which can be
`operated by bare hands, without being held in hand or
`attached to a display Screen. However, no control devices of
`this type have been provided.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,900,863
`
`2
`Pointing devices, Such as mouses, have hitherto been
`commonly used to control personal computers and the like.
`A pointing device is a data input device which is operated to
`input position data to a computer. Various types of pointing
`devices are available. The most popular of the pointing
`device is the mouse. The mouse comprises a housing and a
`ball contained in the housing. The ball can rotate in any
`direction with respect to the housing.
`In use, the mouse is electrically connected to the computer
`and placed on a desk. Once the mouse is put on the desk, the
`ball contacts the desk top at one point. AS the mouse is
`moved back, forth, leftward or rightward on the desk, the
`ball rotates with respect to the mouse housing. The angles
`through which the ball rotate in two directions are detected,
`either mechanically or optically. The data representing the
`angles detected is Supplied to the computer In the computer,
`this data is processed into a direction and a distance, for
`which the cursor is moved on the display Screen.
`Pointing devices other than the mouse have been put to
`use for inputting data to computers. Among them are a touch
`pad, a track ball, a Small joystick, a touch panel and a pen
`device.
`The mouse is easy to operate to input data to the computer.
`However, the mouse can hardly be used unless there is a flat
`wide Surface Such as the top of a desk. To operate his
`computer outdoors, by using the mouse, the user needs to
`find Something having a flat wide Surface. In a train or a car,
`it would be impossible for him to use the mouse to input data
`to the computer placed on his laps. Even if a desk is
`provided, the mouse cannot be used if the desktop is covered
`with books, folders or documents. To use the mouse, the
`desktop has to be cleared.
`Portable computers, each having a pointing device other
`than the mouse, are now used in increasing numbers. The
`pointing device on the portable computer is a track ball, a
`touch pad, a Small joystick or the like. The track ball is
`rotated with a finger. The direction and distance the track
`ball is rotated are detected, and the data representing them
`is input to the computer. The touch pad, available in various
`types, can detect the position of any part touched. When
`Stroked with the user's finger, the touch pad generates data
`representing the direction and distance it is thus Stroked.
`A Small joystick is shaped like a lever. It is provided on
`a keyboard. In most cases, it protrudes upwards, from among
`“G”, “H” and “B” keys. When the joystick is rotated, the
`direction and angle by which it is rotated are detected. The
`direction and the angles are converted into two data items
`which represent the direction and distance for which the
`cursor is to move on the display Screen. The data items are
`input to the computer.
`Pointing accomplished by using a mouse, a track ball, a
`touch pad, or a Small joystick proceeds in the following
`Sequence of Steps:
`1. Detect the position the cursor assumes now.
`2. Determine a target position to which the cursor is to be
`moved.
`3. Determine the direction and distance for which the
`cursor is to be moved.
`4. Input data items representing the direction and the
`distance, thus determined.
`StepS 1 and 2 are interchangeable. Steps 3 and 4 may be
`repeated So that the cursor may keep moving in various
`directions in different directions.
`The movement of the operators hand for using the mouse
`differs from the movement of his hand for using other
`
`IPR2022-00090 - LGE
`Ex. 1005 - Page 48
`
`

`

`3
`pointing devices. When the pointing is made by using the
`mouse, the position of the mouse approximately corresponds
`to the position of the cursor. Of course, when the mouse is
`lifted up and then moved, the relation between the mouse
`position and the cursor position is changed. Further, if the
`mouse driver having a function of accelerating the cursor is
`used, the mouse position does not completely correspond to
`the cursor position. However, from a view point of short
`distance operation, it can be said that the mouse position
`approximately corresponds to the cursor position. In other
`words, when moving the cursor, the operator may move his
`hand holding the mouse Similar to the movement of the
`CUSO.
`By contrast, when a track ball, a touch pad or a Small
`joystick is used to move the cursor on the display Screen, the
`motion of the cursor is different from the motion of the
`user's hand.
`In the case of a track ball, the direction and angle by
`which the ball is rotated correspond to the direction and
`distance for which the cursor moves, as long as the ball is
`rotated at a Single touch to move the cursor for a relatively
`Short distance. To move the cursor for a longer distance, the
`user rotates the ball more than two times, repeatedly moving
`the handback and forth. Obviously, the motion of the cursor
`is no longer the same as the motion of the hand.
`In the case of a touch pad, the motion of the user's finger
`Stroking the pad is similar to the motion of the cursor on the
`display Screen, provided that the cursor is moved by a single
`Stroke on the pad to move the cursor for a relatively short
`distance. To move the cursor for a longer distance, the user
`Strokes the pad two or more times, moving his handback and
`forth. Hence, the motion of the cursor does not correspond
`to the motion of the hand.
`In the case of a Small joystick, the direction in which the
`Stick is rotated and the time for which the stick is held in the
`rotated position correspond to the direction and distance for
`which the cursor moves. While the user's hand is holding the
`Stick at a rotated position, the cursor keeps moving. The user
`must rotate the Stick minutely to adjust the position of the
`cursor as long as the Stick is rotated at a Single touch to move
`the cursor for a relatively short distance. To move the cursor
`for a longer distance, the user must rotate the Stick two times
`or more, by repeatedly moving the hand back and force.
`Thus, the motion of the cursor does not correspond to the
`motion of the hand. Therefore, the motion of the cursor does
`not correspond to the motion of the users hand.
`The difference between the mouse and any other pointing
`device with respect to the relationship between the motion of
`the cursor and that of the user's hand is conspicuous,
`particularly for Steps 3 and 4 of the pointing described
`above. In Steps 3 and 4, the cursor is first moved a long
`distance and then moved repeatedly, each time for a much
`Shorter distance. The user can move the cursor a long
`distance at a fairly high Speed by using a mouse if he has
`grasped the preset Speed ratio between the cursor and the
`OUSC.
`To move the cursor a long distance by using a track ball
`or a touch pad, however, the user must rotate the ball or
`Stroke the pad, Several times. To move the cursor a long
`distance by using a Small joystick or a touch pad, he must
`rotate the Stick, keep watching the cursor moving on the
`Screen and rotate the Stick back as the cursor gets near the
`target.
`To move the cursor a very short distance in a desired
`direction by using a mouse, the user only need to move the
`mouse in the Same direction. When the user uses a touchpad
`to move the cursor in the same way, the user may stroke the
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,900,863
`
`4
`pad in the same direction, but it is difficult for him to move
`his finger Smoothly on the pad due to the friction between
`the pad and the finger tip. If the user touches the pad rather
`lightly in order to move the cursor minutely, his finger will
`likely leave the pad, inevitably making a clicking error. To
`move the cursor a very short distance in a desired direction,
`the user may operate a Small joystick. In this cases the user
`must repeat slightly rotating the Stick and rotating it back,
`each time to move the cursor a little. It takes a long time to
`move the cursor, bit by bit.
`Generally, the mouse excels in operability but is disad
`Vantageous in terms of portability, whereas the touch pad,
`track ball and Small joystick are quite portable but inferior
`in operability.
`The user a touch panel in place of a mouse, to perform
`pointing. If a transparent touch panel is mounted on the
`display Screen, the user can designate any desired point on
`the Screen, merely by touching that part of the panel which
`is aligned with the desired point of the Screen.
`Pointing accomplished by using a touch panel proceeds in
`only the following two steps, far less than in the pointing
`achieved by means of a mouse, a track ball, a touch pad, or
`a Small joystick
`1. Determine a target position on the Screen, to which the
`cursor is to be moved.
`2. Touch that part of the panel which is aligned with the
`target position on the Screen.
`However, Some problems arise when a touch panel is used
`to accomplish pointing. First, the user's finger touching the
`panel conceals that part of the displayed data which is
`located at the target position. Second, it is impossible to
`point anything displayed that is Smaller than the finger tip.
`Third, the display Screen gets dirty as it is repeatedly
`touched with