United States Patent (19)
`Numazaki et al.
`
`US006144366A
`Patent Number:
`11
`(45) Date of Patent:
`
`6,144,366
`Nov. 7, 2000
`
`54 METHOD AND APPARATUS FOR
`GENERATING INFORMATION INPUT
`USING REFLECTED LIGHT IMAGE OF
`TARGET OBJECT
`75 Inventors: Shunichi Numazaki; Miwako Doi,
`both of Kanagawa; Akira Morishita,
`Tokyo; Naoko Umeki; Hiroki Miura,
`both of Kanagawa, all of Japan
`
`73 Assignee: Kabushiki Kaisha Toshiba, Kawasaki,
`Japan
`
`21 Appl. No.: 08/953,667
`22 Filed:
`Oct. 17, 1997
`30
`Foreign Application Priority Data
`Oct. 18, 1996
`JP
`Japan .................................... 8-275949
`Jan. 31, 1997 JP
`Japan ....
`... 9-019397
`Feb. 12, 1997
`JP
`Japan .................................... 9-O27752
`(51) Int. Cl. ................................................. G09G 5/08
`52 U.S. Cl. ........................... 345/156; 34.5/157; 34.5/158
`58 Field of Search ..................................... 34.5/158, 157,
`345/156,419, 425; 463/31, 32, 33, 36,
`37; 250/200, 206.1, 208.1, 214 R
`
`56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,320,292 3/1982 Oikawa et al. ......................... 250/227
`4,516,020 5/1985 Simpson et al. .
`... 250/214 L
`4,839,685 6/1989 Ishiguro et al. .
`... 354/403
`4,841,349 6/1989 Nakano ..................................... 357/30
`4,956,794 9/1990 Zeevi et al. ...
`... 364/559
`5,243,182 9/1993 Murata et al. ....................... 250/222.1
`
`5,297,061 3/1994 Dementhon et al. ................... 364/559
`5,323,174 6/1994 Klapman et al. ....................... 34.5/156
`5,353,042 10/1994 Klapman et al. ....................... 34.5/156
`5,423,554 6/1995 Davis ...................................... 273/437
`5,481,622
`1/1996 Gerhardt et al. ........................ 382/103
`5,627,565 5/1997 Morishita ................................ 34.5/158
`5,686,942 11/1997 Ball ................
`... 34.5/158
`5,703,356 12/1997 Bidiville et al.
`250/221
`5,729,475 3/1998 Romanik, Jr. ........................... 364/559
`5,801,704 9/1998 Oohara et al. .......................... 345/358
`5,819,206 10/1998 Horton et al. .......................... 702/150
`5,900,863 5/1999 Numazaki ............................... 34.5/158
`Primary Examiner William A. Cuchlinski, Jr.
`ASSistant Examiner Marthe Y. Marc-Coleman
`Attorney, Agent, or Firm-Oblon, Spivak, McClelland,
`Maier & Neustadt, P.C.
`57
`ABSTRACT
`A method and an apparatus for generating information input
`which are capable of realizing a direct command type
`information input Scheme by which the gesture or the
`motion can be inputted easily. The apparatus has a timing
`Signal generation unit for generating a timing Signal; a
`lighting unit for emitting a light whose intensity vary in
`time, according to the timing Signal generated by the timing
`Signal generation unit; and a reflected light extraction unit
`having a Sensor array for detecting a reflected light from a
`target object resulting from the light emitted by the lighting
`unit, in Synchronization with the timing Signal generated by
`the timing Signal generation unit, So as to obtain a Spatial
`intensity distribution of the reflected light in a form of a
`reflected light image indicative of an information input
`related to the target object, in Separation from an external
`light that is illuminating the target object.
`
`96 Claims, 101 Drawing Sheets
`
`
`
`101
`
`
`
`
`
`
`
`
`
`TIMING SEGNAL
`GENERATION
`UNIT
`
`LIGHTING UNIT
`
`
`
`REFLECTED LIGHT
`EXTRACTION UNIT
`
`103
`
`
`
`FEATURE DATA
`GENERATION UNIT
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 1
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 1 of 101
`
`6,144,366
`
`
`
`
`
`104
`
`TIMING SIGNAL
`GENERATION
`UNIT
`
`
`
`FIG.1
`
`101
`
`LIGHTING UNIT
`
`102
`
`REFLECTED LIGHT
`EXTRACTION UNIT
`
`103
`
`FEATURE DATA
`GENERATION UNIT
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 2
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 2 of 101
`
`6,144,366
`
`SIT
`
`AYOWYWN
`
`vit
`
`a/v
`
`¢Old
`
`CLT
`
`LINDTOULNOD
`
`DNIAIL
`
`NOILWINOTVO
`
`HONAAEAAIG
`
`LIND
`
`
`
`LINDNOLLOVaULXa
`
`LHOITGaLoeTa
`
`col
`
`Tol
`
`ONILHOIT
`
`LIND
`
`NOILOELAG
`
`-O.LOHd
`
`We
`
`SOILdO
`
`9
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 3
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 3
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`6,144,366
`
`N
`
`£OOAZOOAIOOASSA
`
`No.Ë
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`VO
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 4
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 4 of 101
`
`6,144,366
`
`
`
`
`
`
`
`
`
`Z PITCHWVS I PITCHWVS
`
`
`
`6 I I0ZI
`
`
`
`FINITI TIVNOIS
`
`9ZI
`
`
`
`
`
`LZI ANITI SSOETHGIGIV
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 5
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet S of 101
`
`6,144,366
`
`LIGHTING
`CONTROL PULSE
`LIGHTING
`POWER
`
`:
`
`|
`
`RESET
`
`STORING
`
`PERIOD-IN
`
`SAMPLE 1.
`
`SAMPLE 2
`
`128
`
`129
`
`130
`
`131
`
`-->
`STORING
`PERIOD-2
`
`132
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 6
`
`

`

`Nov. 7, 2000
`Nov.7, 2000
`
`Sheet 6 of 101
`Sheet 6 of 101
`
`
`
`U.S. Patent
`
`U.S. Patent
`
`6,144,366
`6,144,366
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 7
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 7
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 7 of 101
`
`6,144,366
`
`FIG.7
`
`&
`MXS.
`
`s
`
`s
`
`ress CY
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 8
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 8 of 101
`
`6,144,366
`
`FIG.8
`
`
`
`
`
`
`
`210
`
`211
`212
`
`M IJ
`NTV V
`Ng
`We \X2/
`|| |
`|
`|
`s
`
`|
`
`|
`
`|
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 9
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 9 of 101
`
`6,144,366
`
`FIG.9
`
`STICK SHAPED OBJECT
`DETECTION UNIT
`
`213
`
`
`
`
`
`STICK SHAPED OBJECT TIP
`END EXTRACTION UNIT
`
`214
`
`CENTER OF GRAVITY
`DETECTION UNIT
`
`215
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 10
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 10 of 101
`
`6,144,366
`
`
`
`0I’OIH
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 11
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 11 Of 101
`
`6,144,366
`
`FIG.11A
`
`GRAVITY
`MOVEMENT
`
`N-1
`
`
`
`N f
`N
`
`N
`
`CENTER OF
`GRAVITY
`MOVEMENT
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 12
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 12 of 101
`
`6,144,366
`
`FIG.12
`
`CALCULATED
`X-COORDINATE OF
`CENTER OF GRAVITY
`DOES NOT CHANGE
`
`: FINGER TIP
`;
`1 - MOVEMENT
`
`(
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 13
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 13 of 101
`
`6,144,366
`
`
`
`
`LISIOccTVUSANSANVdAYd
`
`
`
`
`
`
`
`SLNANATATANLONULSSHAGHdVHSMOILLSdOANVHIIYOLLSONIYOLSYOA¢L’DIJCaLOHNNOO
`
`
`
`
`
`
`
`
`
`97ZOLdNSALVGIGNVOLoardoAadVHSLOAfdO
`
`TaXId4OGNA
`
`AONANOAS
`
`CGACNALXA<smeys[U]yougoyur
`
`TaXIdJOANA
`
`
`[+<—Yysuay[U]yougosur
`dOALVNIGUYOOO-xX—
`
`[o]puex'[u}yongojur
`AONANO”S
`
`AGGNNOAdJAOdV
`
`TV.LNOZIYOH
`
`JOALVNIGYOOO-x<—
`TaXIdJOLUVLS
`
`SHNIVATAXIdHLA
`
`TaXIdJOLYVLS
`
`AONANOAS
`
`JOALVNIGUOOD-X<—
`[t]ueisx'[u]ongoyur
`MOILSMUNUALSIOUY
`
`
`
`LoaradoGadVHS
`
`
`
`LVHLSOVLIVdaLVddn
`
`ALVGIGNVD
`
`SNOTAdad
`
`é4dNII
`
`[w]yougoyut
`
`SALVGIGNVO
`
`[olweisx[u}yougoyur
`
`AYOWONDNIAVH
`
`ALVGIGNVO
`
`
`
`AONANOUSTAXId
`
`[t]puox'[u]yongojsut
`JOALVNIGYOOO-x—
`
`‘THXIdAAILNOASNOO°NVHL
`
`LCC
`
`GaLVddnJOHLONAT
`
`
`CAANALXA<snieis[u]yougoyur
`
`
`[<_Y)8u97[UW]yousosut
`
`
` GadVHSMOLLSALATA(CaHSINIS|ANIT
`
`AONANOS
`HNO/ON
`
`
`
`
`
`
`
`GSHOVAYLOSfd#OGAdVHSMOLLS
`
`
`
`ijHLONATGeeaTwOsdud
`
`
`
`PZmsuoy[u}yongoyur
`
`
`
`<—Snjejs'[U}OUSoyUT
`
`MOILSON
`
`
`
`WY)CCCANITLXHNOLOO
`
`STHLAdGHALVGdNLON
`
`
`
`ALVGIGNVOLOsArdO
`
`ANITHNO
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 14
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 14
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 14 of 101
`
`6,144,366
`
`CALCULATE TH
`TH=(sum/max)? Xmax
`Sum:MEAN OF PIXEL VALUE SUMS PER LINE FOR
`1ST & 2ND LINES FROM TOP OF STICK SHAPED
`OBJECT (TOPMOST IS 0-TH LINE)
`max:MAXIMUM PIXEL VALUE IN 1ST & 2ND LINES
`
`OBTAIN k & a (0<as 1) THAT SATISFY:
`k-1
`k
`a
`THE X
`i0 Xsumi)+axSumk)
`WHERE xsumi IS PIXEL VALUE SUM OF
`PARTIAL PIXEL SEQUENCE FOR i-TH LINE
`OF STICK SHAPED OBJECT
`
`230
`
`231
`
`MULTIPLY a WITH EACH PIXEL VALUE OF PARTIAL
`PIXEL SEQUENCE FOR k-TH LINE
`
`232
`
`
`
`
`
`OBTAIN CENTER OF GRAVITY (xx, yy)
`k
`
`sum (infoStickinly tatto /TH
`yy=
`xi. Xpsumi /TH
`infoStickn),xendi)
`Xpsumi=
`S. j. P(j.infoStickn).ystart+i)
`j=infoStickn),xstarti)
`
`k
`
`233
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 15
`
`

`

`Nov.7, 2000
`
`Sheet 15 of 101
`
`6,144,366
`
`CHANALXa=smets[OQ]yousoyur
`
`
`
`
`
`
`
`NITMANAG(GQHCNELXA)GALVddNSILE-GaaNaLXxXg
`
`¢=yisusz[O]yonsosut
`
`
`
`
`
`
`
`
`
`ALVGIGNVOLOaldOGadVHSMOLLSLONSILIHOLLSON
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 16
`
`U.S. Patent
`
`
`
`~79ANIT€=4NITAXIdIVILYVdONIGNIAYALAV
`
`
`
`
`
`
`
`SIDlH
`
`
`
`
`
`LOAfdOGHdVHSMOLLSMANSVLDONTYHLSTIONa
`
`ALVCGIGNVO
`
`
`
`ANITp=4OLdNGHHSINIASINVOSWHLHV
`
`
`
`CHANALXa=smeis[o}pougoyur
`
`
`
`9=[0]puex[o]yousosur
`
`
`
`[=w13uer[o}pougoyut
`
`
`
`9=[o]ueisx[o}pougoyut
`
`e=1e3sh-[Q]}yonsoyur
`“>9€Z
`
`
`
`
`
`g=[]ueisxyougosur[QO]
`
`
`
`
`
`c=[T]uesx[Oo]yonsgosur
`
`
`
`
`
`9=[0]puex'[9]yougosut
`
`
`
`
`
`L=[1]puex[og]yousojur
`
`
`
`
`
`Z=WNBu9][OQ]YouSosUt
`
`
`
`
`
`¢=yelsh[9]yousgosut
`
`HLVGIGNVOLOefdOGddVHSMOILLSOLGHLOENNOO
`
`
`
`
`
`SINITS=4dOAIGGINNITaXIdTWILYVdYALIVSc
`
`agwsusy[U}yougoyur
`BECALVNIGUOOD-XAOANda[t]puox'[U}yongoyur
`ANITHL!NI2LVNIGUOOO-XJOLUVLS
`
`
`LOardOGadWHSMOLLSJOdOLWOW
`
`c=yesh'[Q]yougoyur
`
`
`
`CHANALXa=snieis[O]yousgozur
`
`[T]ueisx:[u]yongoyur
`
`
`
`
`
`9=[0]eisx[9]yousosut
`
`¢=[T]veIsx'[(o]yousoyur
`
`
`
`
`
`¢=[Z]ye1sx[QO]yousoyur
`
`
`
`9=[0]puex[O]yousoyur
`
`
`
`L=[1]pusx[o]}yougoyur
`
`
`
`
`
`L=[Z]pucex[9]yongoyut
`
`
`
`ALVGIGNVOLOadOGadVHSMOLLSSILEAOLLS
`
`
`
`
`
`
`
`(NOLLOAMIGTVOLLYAANIH.LONAT)
`
`
`
`SANITJOWAGWONLNAWAND
`
`
`
`smqeys'[U]yongout
`
`
`
`
`
`
`
`LYVLSLOALdOGadVHSMOLLS
`
`
`
`HOIHMLV(ANIT)ALVNIGHYOOD-A
`
`weisf-[u]yougoyur
`
`LET
`
`||
`
`_|
`||
`Oo=H
`
`TIOI68L9SVETIO
`
`«—_—_—_———X
`
`DOMAIN NTPMHOMON
`=
`
`PET
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 16
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 16 of 101
`
`6,144,366
`
`9 I’OIH
`
`
`
`~ If,
`
`C -
`O - CN of r f \d Noo ON .
`D -se
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 17
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 17 of 101
`
`6,144,366
`
`FIG.17
`
`244
`
`2 3 4 5 6 789 1011
`/
`
`>> -->
`
`- | | | | | | | | | | | | | <> | | | | | | | | | | | | |
`
`Q> w-, CN C^ <† ur) \O T, OO ON ) _,
`
`245
`
`C •
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 18
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 18 of 101
`
`6,144,366
`
`18
`FIG
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 19
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 19 of 101
`Sheet 19 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
` tm
`
`e
`2
`
`=a
`N
`
`\O
`<=
`N
`
`oo
`CO
`r
`<t
`CN
`—N
`
`
`
`S246247
`
`g
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 20
`
`oO
`CO
`r
`CN
`
`=N
`
`nN
`
`— 2a
`
`e
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 20
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 20 0f 101
`
`6,144,366
`
`
`
`09Z 67Z
`
`(0)
`
`09Z 67Z
`
`(e)
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 21
`
`

`

`Nov.7, 2000
`
`U.S. Patent
`
`IcDi
`
`YOSAND ESCCSC[Sc
`
`LIND
`
`AVIdSIC
`
`NOILISOdYOSANO
`
`NOILVNINYALA
`
`dILYONI
`
`NOILISOd
`
`Sheet 21 of 101
`
`6,144,366
`
`TIVNOISMOTTO
`
`
`HLVLISAUVNOILV.LS
`
`
`
`LINQ)NOJLOYLAG
`
`wosano
`
`SSC
`
`SC
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 22
`
`TIND
`
`TINONOLMLOdLAC
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 22
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 22 of 101
`
`6,144,366
`
`LSZ
`
`
`
`
`
`TIVNOIS XIOITO RIO CINVIWIWIO O
`
`
`
`XVTdISIGH (HOS?O?C)
`
`JLIN[]
`
`ZZ'OIH
`
`
`
`
`
`$SZZSZISZ
`
`
`
`NOIJLV NIWTRI™IJLEICI
`
`
`
`
`
`NOIJLISOd? (IOS RIQO
`
`dII, IL RIHONIH
`
`
`
`NOI LISOd?
`
`JLIN[]
`
`
`
`JLINOI NOILOBIL™ICI
`
`
`
`R[{{DNIH RIO RIOS RIQO
`
`
`
`
`
`99 Z
`
`
`
`JLINQ NOIJLOEIJLEICI
`
`JLNHWNHAOWN
`
`
`
`NOIJLISOd dIJL
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 23
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 23 Of 101
`
`6,144,366
`
`
`
`JLTÍTSEIRI NOIJLV LEIRICIRIGH LNI
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`JLINQ XNIOWNSHWN
`
`
`
`ŒION VJLSICI
`
`XIRILVW
`
`JLINQ XQHOWNSHWN
`
`ROVVNI HONVRH || ALISNOEIINI
`
`
`
`JLHOIT
`
`{DNIWIJL
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 24
`
`

`

`6,144,366
`
`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 24 of 101
`
`
`
`LIIII,
`
`ILLILILLLLLLLL – ??l?j?ET)ã?????t?ji
`TTTTTTTTTTTTTTTTTTTT
`
`
`LILLE: IT-TITTI,II,II,II,II,II,III)
`DOELLITIITLIT
`LITTITTTT
`
`?
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 25
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 25 of 101
`
`6,144,366
`
`FIG.25
`
`START
`
`STORE RANGE IMAGE
`INTO RANGE IMAGE
`MEMORY UNIT
`
`401
`
`READ DISTANCE MATRIX
`
`402
`
`DISTANCE MATRIX
`PROCESSING
`
`403
`
`404
`
`
`
`
`
`MATCHING SHAPE
`INTERPRETATION
`RULE EXIST?
`
`NO
`
`406
`
`YES
`
`CHANGE THRESHOLD
`& GO TO 403
`
`OUTPUT COMMAND
`CORRESPONDING TO
`MATCHED SHAPE
`
`405
`
`END
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 26
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 26 of 101
`
`6,144,366
`
`FIG.26
`
`SHAPE INTERPRETATION RULES FOR POINTING
`
`
`
`RULE-1:ONE RECTANGLE--POINTING
`RULE -2:TWO RECTANGLES->TWO FINGER OPERATION
`RULE-3:OTHERS->MOTION INPUT
`
`PONTING RULE
`VERTICAL & HORIZONTAL CHECK
`VERTICAL/HORIZONTALast 1->OBJECT SELECT/MOVE FORWARD
`VERTICAL/HORIZONTAL(1->SLOPE CHECK-1
`VERTICAL/HORIZONTALe1->SLOPE CHECK-2
`SLOPE CHECK-1
`SLOPEast0 & CENTER OF GRAVITY RIGHT->ROTATION TO RIGHT
`SLOPEast0 & CENTER OF GRAVITY LEFT->ROTATION TO LEFT
`
`SLOPE CHECK-2.
`SLOPE NO & CENTER OF GRAVITY UPPER->TURNING UPWARD
`SLOPEast O & CENTER OF GRAVITY LOWER
`->TURNING DOWNWARD
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 27
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 27 of 101
`
`6,144,366
`
`ix)(m=)osewygit)
`
`
`LoOJA
`PTTTTTETTybitTTETTTT-SED
`
`
`PtTTTTPPet)ptttttTeeyt
`PTTETyyy
`PittTtTTr
`eaTTTPTel|TTTTP|TPPer
` ptitTTTTTPittTTPLitTTTTee
`
`PTTTTTTTeeeePEPTeTTrrPittTTeTpr
`
`
`PTTTTTeePePittTTTTeeeyerrpttitte
`
`pitTTTeePLTTTeTeTyetePe PTTTTTTPeeerVTTTeTTrPiTTPeerPr
`
`
` PitTTPeeerePTTETTeeeyPiiTTTTTTPrPittitTPePerey
`
`pttttTteTEPeePiTTTTTPPPr
`PEETTPeeeeerryttttte
`}tttTyTTLptTTTLtTTTTP
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 28
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 28
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 28 of 101
`
`6,144,366
`
`
`
`TU
`
`NY V
`SNY V
`Tu
`N
`
`IETFITTI IIIF | | | | | | | | | | | | | | | | | | DITTTTTT TTTTTTTTTT
`TIITETITT
`
`
`
`Ds.
`
`LLI,
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 29
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 29 of 101
`
`6,144,366
`
`FIG.29
`
`SHAPE INTERPRETATION RULES FOR POWER ON/OFF
`
`
`
`RULE - 1:ONE RECTANGLE->POINTING
`RULE-2:TWO RECTANGLES->TWO FINGERS
`RULE-3:FIVE RECTANGLES->FIVE FINGERS
`
`PONTING RULE
`AREA CHECK
`AREAdd y --> FIST
`AREAs a ->NULL
`
`COMMAND GENERATION
`FIVE FINGERS, FIST, TWO FINGERS->POWER OFF
`TWO FINGERS, FIST, FIVE FINGERS->POWER ON
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 30
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 30 0f 101
`
`6,144,366
`
`
`
`(^2)
`
`VIEWPOINT-3
`
`VIEWPOINT-2 VIEWPOINT-1
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 31
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 31 of 101
`Sheet 31 of 101
`
`6,144,366
`6,144,366
`
`b)VIEWPOINT-2
`
`
`c)VIEWPOINT-3
`
`a)VIEWPOINT-1
`
`FIG.31
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 32
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 32
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov. 7, 2000
`Nov.7, 2000
`
`Sheet 32 0f 101
`Sheet 32 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 33
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 33
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 33 0f 101
`
`6,144,366
`
`FIG.33
`
`SHAPE INTERPRETATION RULES FOR
`MOVING OBJECT PARALLAX
`
`
`
`RULE - 1:TWO RECTANGLES->FACE CHECK
`RULE -2:OTHERS->NULL
`
`FACE CHECK
`DISTANCE TO LARGER RECTANGLEDISTANCE TO SMALLER
`RECTANGLE->VIEWPOINT CHECK
`OTHERS->NULL
`VIEWPOINT CHECK
`L-FO-VIEWPOINT DIRECTION-arcsin((L-R)/L)
`L=0->VIEWPOINT DIRECTION--90
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 34
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 34 of 101
`
`6,144,366
`
`
`
`
`
`– – – – – – – – ~~~~ ~~ ~~~~ -- -- - ) ---- - - ---- ---- - - - - -
`
`I-JLNIOCHAAHIA\
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 35
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 35 0f 101
`
`6,144,366
`
`
`
`(RONVNI ONIONIAAS)
`
`
`
`RHOLOGHA NOILOWN
`
`(CIARCHS)
`
`(GITIONV)
`
`JLIN [] XQHOWNEIWN
`
`NOSIRIWCHWOO
`
`NOIJLISOd?
`
`99 "OIH
`
`
`
`
`
`
`
`
`
`
`
`NOIJLOVRIJLXEI
`
`JLINQ
`
`
`
`${ONVHO SHOVVANI
`
`${ONVJLSICI
`
`XINHLVVN
`
`JLINQ XQHOWNSHWN
`
`{{OVWI GHONVRH I Å LISNOEILNI
`
`
`
`{DNIWNIKL
`
`JLHOIT
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 36
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 36 of 101
`
`6,144,366
`
`FIG.36
`
`START
`
`INITIALIZE COMPARISON
`POSITION MEMORY UNIT
`
`STORE RANGE IMAGE
`INTO RANGE IMAGE
`MEMORY UNIT
`
`901
`
`902
`
`READ DISTANCE MATRIX
`
`903
`
`EXTRACT CENTER OF
`GRAVITY POSITION
`YES
`
`904
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`905
`
`CENTER
`OF GRAVITY
`POSITION STORED IN
`COMPARISON POSITION
`MEMORY UNIT?
`
`
`
`NO
`
`TO 907
`
`YES
`906
`CALCULATE MOTION VECTOR
`(CALCULATE SWINGING RANGE) (908)
`(CALCULATE SPEED)
`(909)
`(CALCULATE ANGLE)
`(910)
`
`
`
`STORE NEW CENTER OF GRAVITY
`POSITION INTO COMPARISON
`POSITION MEMORY UNIT
`
`907
`
`TO 902
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 37
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 37 0f 101
`
`6,144,366
`
`FIG.37
`- N
`NY \-
`
`FIG.38
`
`SWINGING RANGE(Xn-Xn-1)/M-30
`
`N.
`% Y
`AV
`
`SPEED(Yn-Yn-1)/M-30
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 38
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 38 0f 101
`
`6,144,366
`
`FIG. 39
`
`FINGER AT TIME t
`
`FINGER AT TIME t-1
`
`
`
`Xbt+1,Ybt+1
`
`Xst,Yst Xst-1,Yst-l
`
`|
`
`b)
`
`Xbt+1,Y"bt+1
`
`Xst,Yst X'st-1,Y'st-1
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 39
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 39 of 101
`
`6,144,366
`
`FIG.40
`
`
`
`YSWINGING
`ANGLE a
`
`SWINGING
`ANGLE cy
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 40
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 40 of 101
`
`6,144,366
`
`FIG.41
`
`
`
`.
`
`SWINGING
`ANGLE a
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 41
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 41 of 101
`Sheet 41 of 101
`
`FIG.42
`
`
`
`:
`
`6,144,366
`6,144,366
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 42
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 42
`
`

`

`
`
`(ATONV)SONVYONIONIMS
`
`
`YOLOAANOILOW‘daads
`ope~|LINDONIMV'G
`SveYALOVUVHO
`
`creNOLLOVULXA
`
`PONVHOAOVNI
`
`LINNTOMLNOO
`
`LINATOXLNOOD
`LINOAYONYAN
`NOSTAVdNOO
`daLANVaVd
`NOILVLAadaaINI
`tyDOI
`
`LINDAYOWAWNONITALL
`NOJLVLaaddaNi
`AONVY|ALISNALINI
`MALNAO‘VAUV
`
`
`LINOLINO
`ALIAVADAO
`
`AYOWAWAdVHS
`
`Cre
`
`NOILISOd
`
`LINN
`
`HAONVISIG
`
`XINLVW
`
`Tee
`
`ADVWI
`
`LHOIT
`
`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 42 of 101
`
`6,144,366
`
`bre
`
`ONIddVW
`
`adVHS
`
`LTNASAa
`
`ae
`
`Cee
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 43
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 43
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 43 of 101
`Sheet 43 of 101
`
`6,144,366
`6,144,366
`
`
`
`FIG.44
`
`i
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 44
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 44
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 44 of 101
`Sheet 44 of 101
`
`6,144,366
`6,144,366
`
`
`
`FIG.45
`
`:
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 45
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 45
`
`

`

`Nov.7, 2000
`
`Sheet 45 of 101
`
`6,144,366
`
`97DI
`
`NOLLOYTIAa
`
`CH.LOFTda
`
`LHOrT
`
`CaLOVULXA
`NOLLOVALXA
`
`qOVNNILIND
`
`ALISNALNI
`
`LHOIT
`
`
`
`LHOITATEISIA
`
`LINN
`
`
`
`NOLLVYANADVIVOAaYnLvVasa
`
`
`
`LINNNOLLOELAG
`
`AVUAV
`
`U.S. Patent COTLIND
`
`LINDAYOWAINATGISIA
`XIXLVINHOVWIAONVAqaDaa
`
`
`
`
`
`TINAAYOWAWNNOLLOVONINILaLXa
`
`
`
`
`
`
`AYOWAWNAOVAI-OLOHd
`
`ONIAIL
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 46
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 46
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 46 of 101
`
`6,144,366
`
`
`
`
`
`
`
`RIOJLO&HLEICI-OJLOHdH OIGHRIVNIHNI RIVEIN
`
`
`
`
`
`RIOJLO&HLEICH-OLOHA JLHOITI [[TºIISIA
`
`
`
`
`
`SNEHTI NOIJLOHJL HCI-OLOHdH
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 47
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 47 of 101
`
`6,144,366
`
`FIG.48
`- 1 -
`
`ORIGINAL
`IMAGE
`
`
`
`MASK
`
`EXTRACTED
`IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 48
`
`

`

`NOLLOYT4aa 67DIA
`
`
`
`LINOAYOWNYA
`
`LINDONIGXOOAa
`
`
`
`(NOISSHUAdNOD)
`
`CeLOVALXa
`
`HOVWI
`
`CALOVALXA
`AOVNUI
`NOILOVULXa
`
`
`
`
`
`XTALVANHOVWIFONVA
`
`LIND
`
`TINAYOWAN
`
`ALISNALNI
`
`ATEISIA
`
`LHOMIT
`
`GISREOCBEICR.|
`
`LHOIT
`
`LINN
`
`daLodTada
`
`LHOIT
`
`NOLLOVaLxXa
`ONIALL
`
`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 48 of 101
`
`6,144,366
`
`
`
`
`
`
`
`€OlLINDANOLLVYANASDVIVOqeaOLVast
`
`AYOWAWFOV
`
`LIND
`
`LTHODITATEISIA
`
`AVUAV
`
`-OLOHd
`
`NOJLOdLad
`ONIAIL
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 49
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 49
`
`

`

`NOILOATae OSDIA
`LINDAYOWHNATEHISIA
`
`
`LINNAYOWHNONIAIL
`
`AOVINI
`
`CaLOVULXTNOLLOVULXA
`
`LIN
`
`
`
`VLIVdFOVNI
`
`ALISNALNI
`
`LHOY]
`
`
`
`LHDITATEISIA
`
`
`
`
`
`XTaLVNHOVWIYONVA
`
`Ca.LOATada
`
`LHOYT
`
`NOILOVaLxa
`
`GHLOdTada
`
`LHOrT
`
`JIN
`
`(NOISSHYdNOD)
`
`NOISSINSNVaL
`
`LIN‘
`
`AYOWAWFOVWII-OLOHd
`NOLLOYLadONIWLL
`
`
`
`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 49 of 101
`
`6,144,366
`
`
`
`¢€OTLINDNOILVYANEYSVIVddaOLVds
`
`AVAAV
`
`LINN
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 50
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 50
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 50 of 101
`
`6,144,366
`
`FIG.51
`- -1 -
`
`BACKGROUND IMAGE
`
`EXTRACTED IMAGE
`
`
`
`COMPOSED IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 51
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 51 of 101
`
`6,144,366
`
`GaLOIVaLxa
`HDVWI
`NOILOVULXA
`
`LIND
`
`GaLOFWayCa.LOdTHY cSDla
`NOILISOdNODLIN?NOJLOALACG
`
`HOVNIAYOWAWFOVWE|ALISNALNI-OLOHd
`
`
`
`LINDAVUAV
`NOLLOATIFa
`LINDAYOWANNOHOVXPLLVWAVIFONVALHOTT
`
`aTdIsiA
`LHorl|LHOIT
`
`LINDAYOWAW
`
`LSet
`
`
`
`dHSOdNOODGQNNOYDAVE
`
`
`
`ADVIAOVUI
`
`LINNAYOWAN
`
`LINO
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 52
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 52
`
`

`

`NOLLOATI cSDJA
`NOLLOVULXALINOHOLIMS
`
`LINNAYOWAW
`
`GALOVaALXa
`
`SOVUI
`
`LIND
`
`LINDAYOWHN
`
`
`
`
`
`XIALVNAOVWIAONVY
`
`
`
`LHOITATEISIA
`
`LINN
`
`daLOATAA
`
`LHOYT
`
`NOJLOVULXA
`ONAL
`
`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 52 of 101
`
`6,144,366
`
`GISNFOCBEEe.
`
`LHOIT
`
`
`
`
`
`AYOWAWNAOVNWI-O.LOHd
`
`ALISNHINI
`
`ATHISIA
`
`LHOIT
`
`
`
`LIND.NOILLOFLAG
`
`AVAUAV
`
`ONDALL
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 53
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 53
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 53 of 101
`
`6,144,366
`
`COE
`
`LINNAYONYA
`
`ANIVA-Z
`
`SHDOVWI
`
`SHNTVA-Z
`
`+VLVG
`
`NOJLOVaLxad
`
`JLINA
`
`GaLOaTaeGaLOgTa pSOld
`LINDONIMVAGTINANOILOALad
`L9€CSElwivqGOVWIATAISIA
`
`
`HOVNITIND
`
`
`xT.LVNFOWWIFTONVULHOIT
`
`LINDAYOWSNNOLLOVULXA
`
`
`AYOWHAWAOVWI|ALISNALNI-O.LOHd
`
`NOLLOGae
`
`LHOTT
`
`
`
`LHorT|LHOrTaTdaisia
`
`AVUAV
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 54
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 54
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov. 7, 2000
`Nov.7, 2000
`
`Sheet 54 of 101
`Sheet 54 of 101
`
`6,144,366
`6,144,366
`
`
`
`LtPTTtTeTTTTTTT]91
`
`an
`
`maN NTO Or 00
`
`(9
`
`HNntinenon’d
`
`(e
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 55
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 55
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 55 of 101
`Sheet 55 0f 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 56
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 56
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 56 of 101
`
`6,144,366
`
`SHNTIVA-Z
`
`HOVWI
`
`+VLVO
`
`LINNAYOWAN
`
`
`
`
`
`XITYLVNAOVWIADNVU
`
`NOILLOATda
`
`dy.LOAaa
`
`LHOrl
`
`LSDIA
`
`LINDAYOWHIN
`
`ANIVA-Z
`
`HOVWI
`
`NOILOVULXa
`
`LIND
`
`NOILVNINAALAD
`
`NOLLOVaLxXd
`
`HONVA
`
`TINA
`
`ALISNALNI
`
`ATSISIA
`
`LHOMl]
`
`
`
`VIVdHOVNWI
`
`
`
`AYOWAWNADVANI
`
`LIND.
`
`-O.LOHd
`
`NOJLOALAGd
`ONITAIL
`
`
`
`LHOITATEISIA
`
`AVUAV
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 57
`
`LINA
`
`GyLOdTHe
`
`LHOIT
`
`NOILLOVaLxXd
`ONIALL
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 57
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 57 of 101
`
`6,144,366
`
`
`
`89’OIH
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 58
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 58 0f 101
`
`6,144,366
`
`FIG.59
`
`
`
`CONTROL
`SIGNAL
`
`720
`
`
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`721
`
`
`
`
`
`TIMING
`CONTROL
`UNIT
`
`SIGNALS
`CORRESPONDING
`TO REFLECTED
`LIGHT IMAGE
`
`
`
`NONLINEAR
`CONVERSION
`UNIT
`
`722
`
`SIGNALS CORRESPONDING
`TO RANGE IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 59
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 59 of 101
`
`6,144,366
`
`FIG.60
`
`
`
`
`
`CONTROL
`SIGNAL
`
`723
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`721
`
`TIMING
`CONTROL
`UNIT
`
`COORDINATE
`SIGNAL
`
`SIGNALS
`CORRESPONDING
`TO REFLECTED
`LIGHT IMAGE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A/D
`CONVERTER
`
`724
`
`DIGITAL
`REFLECTED
`LIGHT IMAGE
`
`CORRECTION
`UNIT
`
`725
`
`
`
`
`
`
`
`
`
`CORRECTED
`RANGE
`IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 60
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 60 0f 101
`
`6,144,366
`
`
`
`CONTROL
`SIGNAL
`
`723
`
`REFLECTED
`LIGHT IMAGE -721
`GENERATION
`UNIT
`
`
`
`
`
`
`
`
`
`
`
`
`
`TIMING
`CONTROL
`UNIT
`
`COORDINATE
`SIGNAL
`
`SIGNALS
`CORRESPONDING
`TO REFLECTED
`LIGHT IMAGE
`
`NONLINEAR
`CONVERSION
`UNIT
`
`726
`
`SIGNALS
`CORRESPONDING
`TO RANGE
`IMAGE
`
`
`
`A/D
`CONVERTER
`
`724
`
`DIGITAL
`RANGE
`IMAGE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`CORRECTION
`UNIT
`
`727
`
`CORRECTED
`RANGE
`IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 61
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 61 of 101
`Sheet 61 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 62
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 62
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 62 of 101
`
`6,144,366
`
`FIG.63
`
`
`
`CONTROL
`SIGNAL
`
`723
`
`REFLECTED
`LIGHT IMAGE -- 721
`GENERATION
`UNIT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TIMING
`CONTROL
`UNIT
`
`
`
`COORDINATE
`SIGNAL
`
`
`
`
`
`
`
`
`
`
`
`SIGNALS
`CORRESPONDING
`TO REFLECTED
`LIGHT IMAGE
`
`A/D
`CONVERTER
`
`724
`
`DIGITAL
`REFLECTED
`LIGHT IMAGE
`
`
`
`
`
`
`
`
`
`CORRECTION
`TABLE
`
`728
`
`CORRECTED
`RANGE IMAGE
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 63
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 63 of 101
`
`6,144,366
`
`|
`
`
`
`
`
`COORDI
`
`4094
`255
`
`FIG.64
`2
`255
`255
`255
`255
`255
`255
`255
`255
`255
`
`aoys
`GSP o
`255
`255
`0 ||
`255
`255
`255
`255
`255
`255
`255
`2 ||
`255
`255
`255
`255
`3H
`255
`255
`255
`4 ||
`255
`255
`255
`255
`5 ||
`255 T
`255
`255
`6 ||
`255 T 255
`255
`255
`255
`7 ||
`255
`255
`255
`8
`255
`255
`255
`255
`T-T
`-
`-
`- I -
`- I
`99
`39
`37
`38
`39
`100
`38
`36
`37
`38
`37
`38
`101 ||
`38
`36
`36
`37
`102 ||
`37
`35
`103 ||
`37
`35
`36
`37
`35
`36
`104 ||
`36
`34
`105 ||
`36
`34
`35
`36
`106
`35
`33
`34
`35
`34
`35
`107 H
`35
`33
`34
`34
`108 H
`34
`32
`109 ||
`34
`32
`-T-I-
`- -
`- ||
`|
`--
`247 ||
`1 ||
`0
`2
`248 ||
`1 ||
`0
`2
`249 ||
`1 ||
`0
`2
`250 ||
`1 ||
`0
`2
`251
`1
`0
`2
`252 ||
`0 ||
`0
`253 ||
`0 ||
`0
`254
`0 ||
`0
`255 ||
`0 ||
`0
`
`40
`39
`38
`37
`
`33 9
`
`1
`1
`
`
`
`
`
`|
`
`|
`
`1
`
`0 ||
`0 ||
`0 ||
`0 ||
`0 ||
`
`1
`0
`0
`0
`0
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 64
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 64 of 101
`
`6,144,366
`
`FIG.65
`
`
`
`CONTROL
`SIGNAL
`
`723
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`721
`
`TIMING
`CONTROL
`UNIT
`
`
`
`
`
`COORDINATE
`SIGNAL
`
`SIGNALS
`CORRESPONDING
`TO REFLECTED
`LIGHT IMAGE
`
`
`
`
`
`
`
`
`
`
`
`A/D
`CONVERTER
`
`724
`
`DIGITAL
`REFLECTED
`LIGHT IMAGE
`
`730
`
`CORRECTION
`TABLE
`
`CORRECTION
`CALCULATION
`CORRECTION | UNIT
`
`
`
`729
`
`CORRECTED
`RANGE IMAGE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 65
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 65 0f 101
`
`6,144,366
`
`COORDI
`NATE
`INPUT
`
`66
`FIG
`
`1
`
`1.
`
`141
`
`144
`116
`
`56
`50
`45
`40
`36
`32
`
`52
`47
`42
`
`2
`
`1
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 66
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 66 of 101
`
`6,144,366
`
`FIG.67
`
`
`
`CONTROL
`SIGNAL
`
`723
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`721
`
`TIMING
`CONTROL
`UNIT
`
`
`
`COORDINATE
`SIGNAL
`
`OUTPUT
`DATA
`
`
`
`
`
`
`
`
`
`IMAGE DATA
`STORAGE
`UNIT
`
`731
`
`IMAGE DATA
`TRANSFER
`UNIT
`
`732
`
`CORRECTION
`TABLE
`
`
`
`CORRECTION
`DATA
`
`CORRECTION
`CALCULATION
`UNIT
`
`729
`
`CORRECTED
`RANGE
`IMAGE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 67
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 67 of 101
`
`6,144,366
`
`FIG.68
`
`733
`
`
`
`
`
`REFERENCE
`OBJECT
`ACTIVATION
`UNIT
`
`
`
`
`
`
`
`
`
`
`
`OBJECT
`POSITION
`SPECIFYING
`DATA
`
`735
`
`
`
`CONTROL
`SIGNAL
`
`CONTROL
`UNIT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`734
`
`REFERENCE
`OBJECT
`
`721
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`COORDINATE
`DATA
`
`
`
`
`
`DISTANCE DATA
`
`OUTPUT
`DATA
`
`737
`
`
`
`
`
`
`
`
`
`
`
`
`
`CORRECTION
`DATA
`STORAGE
`UNIT
`
`
`
`CORRECTION
`DATA
`
`
`
`CORRECTION
`DATA
`CALCULATION
`UNIT
`
`
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 68
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 68 of 101
`Sheet 68 of 101
`
`6,144,366
`6,144,366
`
`FIG.69
`
`(a)
`
`(b)
`(b)
`
`
`
`256
`
`128
`
`160
`
`192
`
`224
`
`256
`
`
`
`128
`
`160
`
`192 224 256
`192
`224
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 69
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 69
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 69 of 101
`
`6,144,366
`
`FIG.70
`
`738
`
`USER
`COMMANDING
`UNIT
`
`CONTROL
`UNIT
`
`
`
`
`
`CONTROL
`SIGNAL
`
`REFLECTED
`LIGHT IMAGE
`GENERATION
`UNIT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`COORDINATE
`DATA
`
`
`
`
`
`DISTANCE DATA
`
`OUTPUT
`DATA
`
`741
`
`
`
`
`
`
`
`
`
`
`
`CORRECTION
`DATA
`STORAGE
`UNIT
`
`
`
`
`
`CORRECTION
`DATA
`
`
`
`CORRECTION
`DATA
`CALCULATION
`UNIT
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 70
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 70 of 101
`
`6,144,366
`
`FIG.71
`
`
`
`PLEASE PLACE THE REFERENCE
`PLATE AT DISTANCE OF 20cm.
`
`
`
`coRRECTION
`DATA
`PRODUCTION
`
`CANCEL
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 71
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 71 of 101
`
`6,144,366
`
`FIG.72
`
`BEFORE
`CORRECTION
`AFTER
`CORRECTION
`CORRECTION
`DATA
`
`BEFORE
`CORRECTION
`AFTER
`CORRECTION
`CORRECTION
`DATA
`
`BEFORE
`CORRECTION
`AFTER
`CORRECTION
`CORRECTION
`DATA
`
`PIXEL POSITION
`
`
`
`(b)
`
`
`
`PIXEL POSITION
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 72
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 72 of 101
`
`6,144,366
`
`FIG.73
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`COMMAND USER TO PLACE
`REFERENCE PLATE AT
`SPECIFIED DISTANCE
`
`ACQUIRE REFLECTED
`LIGHT IMAGE
`
`APPLY CORRECTION
`MODEL
`
`
`
`COMMAND USER TO MOVE
`REFERENCE PLATE
`PARALLEL
`
`CORRECTION
`DATA CORRECT?
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 73
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 73 of 101
`Sheet 73 0f 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`
`
`FIG.74
`
`
`
`/
`/
`/
`& 1
`
`/
`/
`
`/ / /
`/
`/
`/
`
`/ Y /
`/
`/ V /
`
`/
`/
`
`/
`/
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 74
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 74
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 74 of 101
`Sheet 74 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 75
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 75
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 75 of 101
`Sheet 75 0f 101
`
`6,144,366
`6,144,366
`
`FIG.77
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 76
`
`Do
`
`ON
`O
`N
`
`S=
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 76
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 76 of 101
`Sheet 76 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 77
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 77
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov.7, 2000
`Nov. 7, 2000
`
`Sheet 77 of 101
`Sheet 77 of 101
`
`6,144,366
`6,144,366
`
`
`
`
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 78
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 78
`
`

`

`U.S. Patent
`
`Nov.7, 2000
`
`Sheet 78 of 101
`
`ALVLS
`
`HOMLHOIT
`
`"IVNYALXA
`
`LIND
`
`ONILHOTT
` “IVNAALXA pOzl€071Z0ZIavNudixg 10Cl
`
`
`
`
`
`
`
`INANEodNfNOLLDALAG
`
`HOUNOSiLHOIT
`
`6,144,366
`
`Ca.LoaTa
`
`LHO!T
`
`FOV
`
`NOLLISINOOV
`
`LINN
`
`JLINA
`
`9OTLOXrdo
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 79
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 79
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 79 0f 101
`
`6,144,366
`
`
`
`[8’OIH
`
`
`
`{{WIVNIH OEHNO
`
`
`
`NOIJLVRHEIdO RHOSN'HS
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 80
`
`

`

`U.S. Patent
`
`FaNO
`
`
`
`03Ol
`
`Nov.7, 2000
`
`
`
`TWATLHOIl
`
`TVNUALXA(®)
`
`LHOM|ATHVadIOLNI|||“IVNAALXaA
`ZITDATAVaATOLNI()
`
`
`
`ATEdVayTOLNIALVLS
`
`Sheet 80 of 101
`
`6,144,366
`
`
`
`ONILHDITO)
`
`HSTOd
`
`NOILVdaadO
`
`ONIYOLS(P)
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 81
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 81
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 81 of 101
`
`6,144,366
`
`FIG.83
`
`START
`
`831
`
`RESET
`
`STORE CHARGES
`832 - INTO 1ST CHARGE
`STORAGE UNIT
`
`833
`
`RESET
`
`STORE CHARGES
`834 - INTO 2ND CHARGE
`STORAGE UNIT
`
`
`
`
`
`835
`
`EXTERNAL
`LIGHT STATE
`BECAME H DURING
`CHARGE STORING
`OPERATION?
`
`
`
`REPEATED
`FOR PRESCRIBED
`NUMBER OF TIMES
`
`NO
`
`NO
`
`YES
`
`836
`
`DATA OUTPUT
`
`END
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 82
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 82 of 101
`
`6,144,366
`
`ALVLS
`
`LINN
`
`ONLLHDIT
`
`LINN
`
`AOVWILHOIT|govwiLHOMLHon
`
`
`
`ONISSADOUdNOWISINOOV
`
`Gx.LOdTaeGHLOdTaCaLOaWaa
`
`
`IWNYELXALHOMT
`ntWIcorTWNYALXA|
`LNAWADdneNOLLOAULAG
`
`LVLSLHDIT
`
`IWNUALXAFIANOS
`
`LINO
`
`TINA
`
`901LOardo
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 83
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 83
`
`
`
`
`

`

`Sheet 83 of 101
`
`Nov.7, 2000
`
`LHOITIVNAEALXa
`
`“IVNAALX
`
`NOILVALONTA
`
`LHOITT
`
`
`
`LINONOLLOFLAGd
`
`LINN
`
`dolddd
`NOILOALAG
`
`U.S. Patent
`
`10¢]
`
`
`
`COIcOCI
`
`“IVNAALXa
`
`LHOIT
`
` qOUNOS
`
`-OLOHd%ONILHOTT
`
`NOILOFALAG
`
`NOILVdado
`
`ONILHOTT
`
`LINO,
`
`
`
`
`
`LINNTOULNOOD
`
`OOLOELdO
`
`6,144,366
`
`Ga.LOATa
`
`LHOITI
`
`ADVI
`
`NOLLISINOOV
`
`LINN
`
`IPR2022-00090 - LGE
`
`Ex. 1004 - Page 84
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 84
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 84 of 101
`
`6,144,366
`
`||||
`
`||||
`
`
`
`
`
`JLHOITI TIVNYI? LXR (e)
`TOEHAGHTI
`
`
`
`I, I,II,II,II,NOILVÆGIÃO ONIHOLS(p)
`
`
`
`||—|ŒSTIQq ?NIJLHOEDITI(0)
`
`
`
`SEISTI?ld OLNI
`
`
`
`
`
`||||JLHOEDITI TVN HEILXEI(q)
`
`
`
`CITHLNEHANOO STVNOIS
`
`IPR2022-00090 - LGE
`Ex. 1004 - Page 85
`
`

`

`U.S. Patent
`
`Nov. 7, 2000
`
`Sheet 85 0f 101
`
`6,144,366
`
`
`
`
`
`z I,II,IFNOILVRIRAO ?NIHOLS(p)
`
`
`
`EISTIQd ONIJLHOITI(O)
`
`
`
`SEISTIQd OLNI
`
`
`
`
`
`TLILTLILILILILILILILILILILILILILI CHI??A?99, ?TYNQIS
`
`
`
`JLHOITI TVNHEILXEI(q)
`
`
`
`JLHADITI TVNYHEILXEI(E)
`
`TOEH