US005880411A
`5,880,411
`[11] Patent Number:
`(15
`United States Patent
`Gillespie et al.
`[45] Date of Patent:
`Mar. 9, 1999
`
`
`[54] OBJECT POSITION DETECTOR WITH
`EDGE MOTION FEATURE AND GESTURE
`RECOGNITION
`
`[75]
`
`Inventors: David W. Gillespie, Palo Alto;
`Timothy P. Allen, Los Gatos; Ralph C.
`Wolf. Santa Clara. Shawn P. Day San
`Jose, all of Calif.
`
`....... GO6K 11/16
`8/1994 European Pat. Off.
`0609 021
`5/1990
`France eee . GO6K 11/16
`2 662528
`10/1985
`Japan ..
`«.. GO6F 3/03
`60-205625
`6/1987
`Japan ..
`.. GO6F 3/033
`62-126429
`4/1988
`Japan ..
`»- GO6F 3/033
`63-073415
`2/1990
`2 040014
`Japan...
`- GO2G 1/133
`4 OIS7T25 —/1992
`Japan woes GO6F 3/033
`(List continued on next page.)
`
`
`
`[73] Assignee: Synaptics, Incorporated, San Jose,
`Calif.
`
`[21] Appl. No.: 623,483
`[22]
`Filed:
`Mar. 28, 1996
`
`Related U.S. Application Data
`
`Wel
`
`.
`
`.
`
`:
`
`:
`
`Chun,et al., “A High-Performance Silicon Tactile Imager
`
`OTHER PUBLICATIONS
`“Pressure—Sensitive Icons’, IBM Technical Disclosure Bul-
`letin, Jun. 1990, vol. 33, No. 1B, pp. 277-278.
`“Scroll Control Box”, IBM ‘lechnical Disclosure Bulletin,
`Apr. 1993, vol. 36, No. 4, pp. 399-403.
`Wilton, Microsoft Windows 3 Developer’s Workshop, 1991,
`pp. 229-230.
`Tiburtius, “Transparente Folientastaturen”, Feinwerktechnik
`[63] Continuation-in-part of Ser. No. 320,158, Oct. 7, 1994, Pat.
`& Messtechnik 97, No. 7, Munchen, DE, Jul. 1989, pp.
`No. 5,543,591, which is a continuation-in-part of Ser. No.
`299-300.
`300,387,Sep. hetik apandoned,whichis PaNe “Double—Click Generation Method for Pen Operations”,
`78, which is a continuation-iapart of Ser. No. 895.934.
`IBM Technical Disclosure Bulletin, Nov. 1992, vol. 35, No.
`Jun. 8, 1992, abandoned.
`:
`6, p. 3.
`4G
`.
`“Three—Axis Touch—Sensitive Pad”, IBM Technical Disclo-
`[S51] Unt. Cheeee GO08C 21/00; aoteioG
`sure Bulletin, Jan. 1987, vol. 29, No. 8, pp. 3451-3453.
`a345‘sysaris,esa Based on a Capacitive Cell”, IEEE Transactions on Electron
`[58] Field of Search
`Mots 178/18 O1 19.01
`Devices, Jul. 1985, vol. ED-32, No. 7, pp. 1196-1201.
`178/20.01; 345/157, 159, 160, 173, 174;
`Primary Examiner—Vijay Shankar
`382/119, 186, 187, 316
`Attorney, Agent, or Firm—D’Alessandro & Ritchie
`References Cited
`[57]
`ABSTRACT
`U.S. PATENT DOCUMENTS
`Methods for recognizing gestures made by a conductive
`object on a touch-sensor pad and for cursor motion are
`disclosed. Tapping, drags, pushes, extended drags and vari-
`able drags gestures are recognized by analyzing the position,
`pressure, and movement of the conductive object on the
`sensor pad during the time of a suspected gesture, and
`signals are sent to a host indicating the occurrence of these
`gestures. Signals indicating the position of a conductive
`object and distinguishing between the peripheral portion and
`an inner portion of the touch-sensor pad are also sentto the
`host
`.
`
`[56]
`
`8/1948 Holt.
`Re. 23,030
`2,219,497 10/1940 Stevensetal. .
`3,128,458
`4/1964 Romero .
`
`(List continued on next page.)
`FOREIGN PATENT DOCUMENTS
`
`0 187 372 12/1985
`European Pat. Off.
`.......... GOIB 7/00
`
`0 394 614 10/1990 European Pat.
`sve
`GO6F 3/033
`0 490 001
`6/1992 European Pat. Off.
`........ GO6F 3/033
`0 574 213
`12/1993
`European Pal. Off.
`....... GOOK 11/16
`0 589 498
`3/1994 European Pat. Off.
`....... GO6K 11/16
`
`64 Claims, 28 Drawing Sheets
`
`s
`
`2
`
`‘
`xX INPUT
`PROCESSING
`
`|
`
`——\
`Y INPUT
`PROCESSING
`
`
`
`|
`
`16
`
`ARITH.
`UNIT
`
`x
`
`[Y
`
`r
`
`rain
`
`
`
`8 \
`
`
`VIRTUAL
`
`
`
`
`
`
`
`
`MOTION
`UNIT
`
`GESTURE
`UNIT
`
`BUTTONS
`
`Valve Exhibit 1014
`Valve Exhibit 1014
`Valve v. Immersion
`Valve v. Immersion
`
`

`

`5,880,411
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`6/1984 Schuyler wc eects reesee cere 178/18
`4,455,452
`4,475,235 10/1984 Graham 02...
`cece eeeceeeeceteeeeeees 382/3
`4,476,463 10/1984 Neget al.
`.
`340/712
`9/1965 Bray.
`3,207,905
`
`4,511,760
`4/1985 Garwinetal.
`. 178/18
`Por Sitoce Cissimbene -
`.. 340/365
`4,516,112
`5/1985 Chen ....
`van
`‘
`.
`
`73/862.04
`4,526,043
`7/1985 Boieetal
`3,437,795
`4/1969 Kuljian.
`costes
`.
`amen
`_
`.
`3,482,241
`12/1969 Johnson .
`4,550,221
`10/1985 Mabusth oeecteeteeeeeeenee 178/18
`3492440
`1/1970 Cerboneetal. .
`4,550,310 10/1985 Yamaguchiet al. 0... 340/365
`3,493,791
`2/1970 Adelsonetal. .
`4,554,409
`11/1985 Mitsui et al. oo.cecccececeeeeeee 178/19
`3.497.617
`2/1970 Ellis et al.
`.
`2/1986 Thornburget al.
`338/114
`4,570,149
`3,497,966
`3/1970 Gaven.
`
`4,582,955
`4/1986 Blesser....
`178/19
`.
`3,516,176
`6/1970 Cleary et al.
`
`.. 340/365
`4,595,913
`6/1986 Aubuchon
`3,522,664
`8/1970 Lambrightetal. .
`4,616,107 10/1986 Abe et al. oe eceeeesetcneereeeeee 178/18
`3,530,310
`9/1970 Adelsonet al.
`.
`4,639,720
`1/1987 Rympalski et al. oe 340/712
`3,543,056
`11/1970 Klein .
`
`...
`.. 178/19
`4,672,154
`6/1987 Rodgers et al.
`secsscssssssseessees 307/252
`3,549,909
`12/1970 Adelson et al.
`
`7/1987 Yoshikawaet al. oe 178/19
`4,680,430
`323/93
`3,593,115
`7/1971 Dym etal. ..
`
`
`
`3,598,903 8/1971 Johnson et al.wceeceeeeeeee 178/18 4,686,332 8/1987 Greanias et al. oe eee 178/19
`
`. 340/347 DD
`v 178/19
`3,662,378
`5/1972 MacArthur.....
`4,698,461
`10/1987 Meadowsetal.
`
`7/1972 Ackermanetal.
`. 340/365
`3/1988 Evans oo. cccceecseseetetteeteeee 340/365 C
`3,675,239
`4,733,222
`
`. 340/365
`3,683,371
`8/1972 Holz ......
`4,734,685
`3/1988 Watanabe 00.0000... 340/710
`
`
`« 340/365
`3,696,409
`10/1972 Braaten....
`4,736,191
`4/1988 Matzkeetal.
`. 340/365 C
`
`
`200/167 A
`5/1973 Kaelin et al.
`7/1988 Kimura oo. ee ecceteeteeeeeeenee 178/19
`3,732,389
`4,758,690
`
`6/1973 Larson ........
`w.. 307/116
`8/1988) Ono et al. eececceeseeeseees 340/709
`3,737,670
`4,766,423
`
`
`340/365 C
`.. 178/19
`3,757,322
`9/1973 Barkanetal.
`4,788,385
`11/1988 Kimura ...
`4,794,208 12/1988 Watson ....ccccceccceeeeneceeecees 178/19
`3,760,392
`9/1973 Stich .
`. 340/200
`3,773,989
`11/1973 Hacon ..
`200/52 R
`4,820,886
`4/1989) Watson oon. cceeescesecseseneeeceeee 178/19
`8/1989 Meadowsetal.
`. 178/19
`4/1975 Halsembalg w....ssscccsscsssssssecssesoee 178/19
`3,875,331
`4,853,498
`
`
`
`3,921,166 11/1975Volpeoo.eeeccreeeeeetenee 340/365 C 4,914,624 4/1990 Dunthorn .... 364/900
`
`
`
`
`4/1990 Flowersetal.
`3,931,610
`1/1976
`. 340/172.5
`4,918,262
`178/18
`
`
`
`1/1976 Graven ...........
`5/1990 Meadowsetal.
`3,932,862
`340/324 M
`4,922,061
`178/19
`8/1976 Abe et al. oe cteeeeeeeeeeeee 178/18
`3,974,332
`4,935,728
`6/1990
`340/709
`. 178/18
`.
`3,992,579
`11/1976 Dym etal.
`4,988,982
`1/1991
`345/173
`
`
`
`3,999,012 12/1976 Dym .........
`5/1991 Gruaz etal.
`...
`5,016,008
`341/33
`
`11/1977 Jordan w.ccccccecceseceseesereeees 200/5 A
`5/1992 Greaniasetal.
`4,056,699
`5,117,071
`178/19
`4,058,765
`11/1977 Richardsonetal.
`324/61 R
`5,120,907
`6/1992 Shinborietal.
`478/18
`
`
`eeesscceecssssseeeeeeeeeeeeee 178/19
`4,071,691
`1/1978 Pepper, Ite
`5,149,919
`9/1992 Greaniaset al.
`178/19
`5,153,572 10/1992 Caldwell et al.
`.
`. 340/712
`4,087,625
`5/1978 Dym et al.
`occ
`ees ceeeee cee 178/19
`
`3/1993 Edwards......
`« 341/20
`
`4,103,252
`7/1978 Bobick.....
` .. 331/48
`5,194,862
`4,129,747 12/1978 Pepper, Jt ccc
`cesses 178/19
`5,231,450
`7/1993 Daniels ...
`355/27
`
`
`8/1993 Kurodaetal.
`4/1979 Burson wo. ceeeeeeceeeteceeereees 340/709
`4,148,014
`5,239,140
`178/18
`
`12/1993 Knapp.........
`.. 341/34
`
`4,177,354 12/1979 Mathews..
` .. 178/18
`5,270,711
`
`
`12/1979 Thornburg 0...eee 324/61 R
`4,177,421
`5,327,161
`7/1994 Loganetal.
`345/157
`4,198,539
`4/1980 Pepper, Jt
`ieee
`eeceeeeseeeeee 178/18
`5,365,254
`11/1994 Kawamoto
`345/157
`
`9/1980 Lednicki etal.
`4,221,975
`. 307/116
`5,369,227
`11/1994 Stone
`9/1980 PeNZ on rcsccesseceeceeeeeceeereees 340/712
`4,224,615
`5,373,118 12/1994 Watson....
`
`1/1981) Chandler 0.0...cece 200/58 A
`4,246,452
`5,374,787 12/1994 Miller et al. wee eeeeereeeeee 178/18
`... 368/69
`.
`4,257,117
`3/1981 Besson .
`5,386,219
`1/1995 Greaniasetal.
`
`
`4/1981 Bigelow ov
`eceecssecseeccsenneenes 340/365 C
`4/1995 Kotaki et al. oeeee 395/122
`4,264,903
`5,408,593
`4,281,323
`7/1981 Burnett et al.
`.ccccscecssseesseee 340/712
`5,488,204
`1/1996 Mead eb ale eeeeececseeenece 345/179
`9/1981 Eichelbergeretal.
`340/365 C
`4,290,052
`
`sees 340/712
`4,290,061
`9/1981 Serrano ....
`
`9/1981 Cutler etal.
`4,291,303
`. 340/711
`
`
`4,293,734 10/1981 Pepper, Jr. 06 139022=5/1994 Japaneee GO6F 3/033.. .. 178/18
`
`
`
`
`
`
`
`4,302,011 07 072 976=.3/199511/1981 Pepper, Jr. .. w. 273/85 Japan..... tae . GO6F 3/033
`
`
`
`
`....
`11/1984 United Kingdom .
`4,310,839
`1/1982 Schwerdt
`. 340/712
`2 139 762
`. GO6F 3/033
`
`1/1982 Thornburg ..
`10/1993 United Kingdom .
`4,313,113
`. 345/159
`2 266 038
`. GO6F 3/033
`
`. 340/712
`4,334,219
`6/1982 Patilus et al.
`2 288 665
`4/1995 United Kingdom .
`GO6K 11/12
`2/1983 Pepper, Jr...
`4,371,746
`.. 178/18
`91/03039
`3/1991 WIPO ......0...
`. GO9G 3/02
`
`4,398,181
`8/1983 Yamamoto
`340/365 S
`91/05327
`4/1991 WIPO .
`.. GOIG 3/02
`
`
`
`
`4,423,286 96/07966=3/1996 WIPO .12/1983 Bergeron .. .. 178/19 . GO6F 3/033
`
`4,430,917
`2/1984 Pepper, Jr.
`.. 84/1.01
`96/11435
`4/1996 WIPO .
`. GO6F 3/033
`
`
`4,442,317
`4/1984 Jandrell vo. ee
`eesccenseeceeereneeee 178/18
`96/18179
`6/1996 WIPO wee eeeceeereteeeeees GO8C 21/00
`
`
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`
`
`

`

`AV
`
`SNOLLNG
`
`TWOLYIA
`
`LANIX
`
`ONISSADOYd
`
`LNdNIA
`
`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 1 of 28
`
`5,880,411
`
`XV IOld
`
`ONISSADOUd
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 2 of 28
`
`5,880,411
`
`30 30 30 30 3030 30 30 303030 30 303030
`
`
`
`34
`
`26
`
`
`
`
`
`24
`
`24
`
`FIG. 2A
`
`28
`
`26
`26
`26
`26
`
`26
`o-44@_¢ 4©¢ 0-4@
`26
`
`ale
`
`
`o-oo _¢_4_@_¢ 0 # © -@-©
`
`26
`
`26
`26
`26
`
`26
`
`26
`26
`
`FIG. 2B
`
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 3 of 28
`
`5,880,411
`
`
`
`
`22
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 4 of 28
`
`5,880,411
`
`40
`
`48-1
`
`50-1
`
`52
`
`16
`
`
`
`42-1 7
`
`FILTER
`S/H
`
`FILTER
`S/H
`
`44-n
`
`FILTER
`
`S/H
`
`Vmax VMIN
`
`
`
`
`
`
`
`
`
`
`
`Nun|
`
`
`7
`
`L
`42-n aa \
`
`ne”
`
`44-VMax
`
`FILTER
`S/H
`
`42-Vissax
`Ne
`_
`
`44-ViMiIN
`
`48-VMIN
`
`50-VMIN
`
`FILTER
`
`S/H
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 5 of 28
`
`5,880,411
`
`44
`
`Vpp
`
`OUT
`
`FIG. 4A
`
`Vppb
`
`RESETUP o—X}~ 90
`
`OUT
` VBIAS
`
`STEPDN
`
`ss
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`Sheet 6 of 28
`INO
`
`5,880,411
`
`|
`
`|
`
`EN
`
`|
`
`RESET UP
`
`RESET DN
`
`STEP UP
`
`STEP DN
`
`sup
`
`__/
`
`\
`
`/
`
`;
`
`\
`
`;
`
` \
`
`\
`
`\
`
`snOfLL
`
`CHUP
`
`/a
`
`SHARE
`
`/
`
`\
`
`OUT
`oS
`f—
`|_——_—— scan cycle ——————
`FIG. 5
`
`SAMPLE
`
`48
`
`;
`
`126
`
`120
`
`(
`422
`
`~
`T 124
`iE
`
`FIG. 6
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 7 of 28
`
`5,880,411
`
`rd
`
`50-1
`50-2
`50-3
`50-4
`
`50-5
`50-6
`
`50-8
`
`50-9
`50-10
`
`50-11
`50-12
`
`50-13
`50-14
`-
`50-15
`50-16
`
`50-17
`50-18
`50-19
`50-20
`
`50-21
`
`50-22
`
`50-23
`
`;
`
`I
`182-1 |
`!
`I
`
`|
`
`I
`132-2 |
`I
`;
`
`;
`|
`132-3 |
`!
`I
`|
`!
`i
`t
`
`I
`I
`
`;
`
`;
`I
`132-5 |
`!
`
`{
`|
`.
`i
`
`|
`1
`I
`
`|
`'
`.
`1
`|
`!
`
`|
`j
`
`I
`)
`!
`
`
`50-24
`
`A/D
`
`REG.
`
`138-1
`
`52-1
`
`D
`
`REG.
`
`138-2
`
`52-2
`
`A/D
`
`REG.
`
`138-3
`
`52-3
`
`AD
`
`REG.
`
`138-
`
`38-4
`
`52-4
`
`138-5
`
`138-6
`
`FIG. 7
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 8 of 28
`
`5,880,411
`
`174
`
`Xpos
`
`178
`
`Z
`
`Z
`
`+
`
`Ypos
`
`150
`
`x
`
`LiXi
`;
`
`166
`
`20
`
`Onx
`REG.
`
`
`
`
`x
`DENOMINATOR
`
`YXi
`i
`
`Opx
`REG.
`
`Y
`NUMERATOR
`
`viYi
`
`Ony
`
`156
`
`DENOMINATOR 4
`
`Y
`
`Di
`
`
`I
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 9 of 28
`
`5,880,411
`
`180
`
`J
`
`186
`
`CONTROL
`
`HISTORY
`BUFFER
`
`
`
`184
`
`
`REWIND
`
`
`
`
` ENABLE
`
`
`FINGER
`
`
`
`
`FILTER
`
` FINGER
`
`FIG. 9
`
`

`

`U.S. Patent
`
`5,880,411
`
`FIG.10
`
`Sheet 10 of 28
`
`Mar. 9, 1999
`
`
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 11 of 28
`
`5,880,411
`
`242
`
`-— ¥Y CENTER
`
`X CENTER
`
`
`
`
`240
`
`FIG. 11
`
`FIG. 12A
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 12 of 28
`
`5,880,411
`
`Y FingerOuter
`
`X FingerOuter
`
`FIG. 12B
`
`

`

`49Bul4
`
`5,880,411
`
`ElOla
`
`
`
`g|qeuguojow
`
`ble
`
`493NO
`
`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 13 of 28
`
`o9e
`
`dnoy
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 14 of 28
`
`5,880,411
`
`20
`
`ODP
`
`N<><
` Yer
`
`|[|
`
`ZIG-ZAG
`IT
`
`PUSH
`
`UNIT
`
`FIG. 14
`
`TAP GESTURE
`ZTAP™[—__\ CS
`41
`“tO
`
`OUT
`“te7
`
`zh
`
`FIG. 15A
`
`DRAG GESTURE
`
`tg|ts ts tr>«
`
`|
`zt
`
`
`OUT
`|
`
`FIG. 15B
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 15 of 28
`
`5,880,411
`
`TAP GESTURE WITH VARIABLE DRAG TIME
`
`EXPERT TAP
`
`NOVICE TAP
`
`zo
`“—y
`
`OUT
`
`|
`
`|
`
`toa
`
`tab
`
`FIG. 15C
`
`LOCKING DRAG GESTURE
`
`af\fVSVil
`
`tg|tg! tro ti tia! 13
`
`
`
`
`OUT
`
`FIG. 15D
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 16 of 28
`
`5,880,411
`
`DRAG EXTENSIONS GESTURE
`
`Z4
`
`OUT
`
`|
`
`ty4
`
`tis |
`
`tis
`
`ti7
`
`tia
`
`FIG. 15E
`
`zeUfLfLU
`ta tao
`tay
`
`DOUBLE TAP
`
`toa>]
`
`[«
`
`tes
`
`|
`
`FIG. 15F
`
`toa|tos
`
`HOP GESTURE
`
`zef
`
`OUT (L)
`
`OUT(R)
`
`|
`|
`
`|
`
`tog
`
`FIG. 15G
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 17 of 28
`
`5,880,411
`
`290
`
`288
`
`292
`
`FIG. 16A
`
`294
`
`CornerY
`
`296 onan eee ee CornerX
`
`FIG. 16B
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 18 of 28
`
`5,880,411
`
`300
`
`START
`
`302
`
`Y
`
`304
` PRE-
`VIOUSLY
`
`DOWN
`
`?
`
`N
`
`no,
`
`306
`
`TAP<a
`
`
`
`
`320
`
`TAPOKAY <— TRUE
`
`©
`
`
`
`
`
`308
`
`DIST >
`
`DRAGRADIUS
`?
`
`310
`
`NONE
`yiy
`
`DIST
`
`9
`
`314
`
`316
`
`
`DRAG-
`LOCK ENABLED
`?
`
`
`
`TAPSTATE — NONE
`
`312
`
`DOWNPOS ~— CURPOS
`DOWNTIME <— CURTIME
`
`318
`
`TAPOKAY <— FALSE
`SUPPRESS <— FALSE
`
`FIG. 17A
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 19 of 28
`
`5,880,411
`
`
`324 PRE-
`
`
`VIOUSLY
`UP
`?
`
`326
`
`
`TAP
`QUALIFIED
`
`
`
`?I
`
`REVERSE MOTION
`
`328
`
`330
`
`TAP
`
`332
`
`Y
`
`
`
`334
`
`LOCKED
`
`NONE
`
`TAPSTATE <— TAP
`
`336
`
`USE LONG DRAG TIME
`SUPPRESS <— TRUE
`
`N
`
`FIG. 17B
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 20 of 28
`
`5,880,411
`
`HOP
`QUALIFIED
`
`?
`
`340
`LEFT HOP
`
`DIRECTION
`
`RIGHT
`
`342
`
`344
`
`TAPBUTTON <— MIDDLE
`
`TAPBUTTON <— RIGHT
`
`
`
`346
`
`N
`
`CORNER
`TAP?
`
`Y
`
`3
`
`Ag
`
`350
`
`TAPBUTTON < LEFT
`
`TAPBUTTON < RIGHT
`
`UPPOS <— CURPOS
`
`352
`
`TAPSTATE < TAP
`
` 354
`
`FIG. 17C
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 21 of 28
`
`5,880,411
`
`
`
`TAPSTATE=
`TA
`
`356
`
`
`?
`
`DRAG-EXT
`ENABLED
`
`
`
`
`
`DRAGEXTSPEED
`?
`
`TAPSTATE <— LOCKED
`
`TAPSTATE <— NONE
`
`368
`
`UPPOS — CURPOS
`
`FIG. 17D
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 22 of 28
`
`5,880,411
`
`370
`
`UPTIME — CURTIME
`
`
`
`TIME >
`DRAG-LOCK
`DRAGTIME
`
`
`ENABLED
`
`
`
`
`?
`
`
`
`
` END
`
`SUPPRESSION
`?
`
`
`372
`
`LOCKED
`
`374
`
`380
`
`382
`
` TIME >
`DRAGEXTTIME
`?
`
`376
`
`TAPSTATE — NONE
`
`378
`
`384
`
`SUPPRESS < FALSE
`
`FIG. 17E
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 23 of 28
`
`5,880,411
`
`
`
`388
`
`OUT <— NONE
`
`
`
`
`
`= NONE
`?
`
` TAPSTATE
` OUT <— TAPBUTTON
`
`
`
`
`394
`
`OUT — NONE
`
`396
`
`FIG. 17F
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 24 of 28
`
`5,880,411
`
`START
`
`400
`
`WAIT ~3 SAMPLES
`
`402
`
`ZIGZ — “x”
`ZIGZ' — “x”
`
`404
`
`WAIT 1 SAMPLE
`
`406
`
`
`
`ZIGPOS < ZIGPOS'
`ZIGPOS' — CURPOS
`
`ZIGZ — ZIGZ'
`ZIGZ' <—Z
`
`FIG. 18A
`
`410
`
`412
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 25 of 28
`
`5,880,411
`
`|
`
`ZIGTIME — CURTIME
`
`ZIGLEFT < FALSE
`ZIGRIGHT <— FALSE
`
`WAIT 1-2 SAMPLES
`
`42
`
`42
`
`ey
`
`Zz
`
`Y
`
`414
`
`& — oO
`
`418
`
`424
`ZIGLEFT <— TRUE
`
`42y
`7ZIGRIGHT<——
`
`428
`
`WAIT 1 SAMPLE
`
`430
`
`43oi)
`
`FIG. 18B
`
`

`

`
`
`
`436
`
`ZAG
`QUALIFIED
`?
`
`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 26 of 28
`
`5,880,411
`
`
`
`WAIT 1-2 SAMPLES
`
`434
`
`
`
`
`
`
`442
`
`
`
`OUT <— LEFT
`
`
`
`OUT — NONE
`
`448
`
`FIG. 180
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 27 of 28
`
`5,880,411
`
`PUSH GESTURE
`
`ZPUSHDOWN
`
`ZTH
`
`zt
`
`OUT
`
`|
`
`FIG. 19
`
`ZPUSHUP
`
`456
`
`
`
`OUT <— LEFT
`
`OUT — NONE
`
`
`
`460
`
`
`
`
`

`

`U.S. Patent
`
`Mar. 9, 1999
`
`Sheet 28 of 28
`
`5,880,411
`
`SS3udddNs
`
`NOILOW
`
` dNNPLAIN
`
`
`
`
`

`

`5,880,411
`
`1
`OBJECT POSITION DETECTOR WITH
`EDGE MOTION FEATURE AND GESTURE
`RECOGNITION
`
`RELATED APPLICATIONS
`
`This application is a continuation-in-part of co-pending
`application Ser. No. 08/320,158,filed Oct. 7, 1994, which is
`a continuation-in-part of application Ser. No. 08/300,387,
`filed Sep. 2, 1994, now abandoned whichis a continuation-
`in-part of application Ser. No. 08/115,743, filed Aug. 31,
`1993, now US. Pat. No. 5,374,787, which is a continuation-
`in-part of application Ser. No. 07/895,934,filed Jun. 8, 1992,
`now abandoned.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to object position sensing
`transducers and systems. More particularly,
`the present
`invention relates to object position recognition useful in
`applications such as cursor movement
`for computing
`devices and other applications, and especially to cursor
`movement with enhanced edge-motion and gesture-
`recognition features.
`2. The Prior Art
`
`Numerous devices are available or have been proposed
`for use as object position detectors for use in computer
`systems and other applications. The most familiar of such
`devices is the computer “mouse”. While extremely popular
`as a position indicating device, a mouse has mechanical
`parts and requires a surface upon whichto roll its position
`ball. Furthermore, a mouse usually needs to be moved over
`long distances for reasonable resolution. Finally, a mouse
`requires the userto lift a hand fromthe keyboard to make the
`cursor movement,
`thereby upsetting the prime purpose,
`whichis usually typing on the computer.
`Trackball devices are similar to mouse devices. A major
`difference, howeveris that, unlike a mouse device, a track-
`ball device does not require a surface across which it must
`be rolled. Trackball devices are still expensive, have moving
`parts, and require a relatively heavy touch as do the mouse
`devices. They are also large in size and doe notfit well in a
`volume-sensitive application like a laptop computer.
`There are several available touch-sense technologies
`which may be employed for usc as a position indicator.
`Resistive-membraneposition sensors are knownandused in
`several applications. However, they generally suffer from
`poorresolution, the sensor surface is exposed to the user and
`is thus subject
`to wear. In addition, resistive-membrane
`touch sensors are relatively expensive. A one-surface
`approach requires a user to be grounded to the sensor for
`reliable operation. This cannot be guaranteed in portable
`computers. An example of a one-surface approach is the
`UnMouseproduct by MicroTouch, of Wilmington, Mass. A
`two-surface approach has poorer resolution and potentially
`will wear out very quickly in time.
`Resistive tablets are taught by U.S. Pat. No. 4,680,430 to
`Yoshikawa, U.S. Pat. No. 3,497,617 to Ellis and many
`others. The drawback of all such approaches is the high
`power consumption and the high cost of the resistive mem-
`brane employed.
`Surface Acoustic Wave (SAW) devices have potential use
`as position indicators. However, this sensor technology is
`expensive and is not sensitive to light touch. In addition,
`SAW devices are sensitive to residue buildup on the touch
`surfaces and generally have poor resolution.
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`Strain gauge or pressure plate approaches are an interest-
`ing position sensing technology, but suffer from several
`drawbacks. This approach may employpiezo-electric trans-
`ducers. One drawbackis that the piezo phenomena is an AC
`phenomena and may be sensitive to the user’s rate of
`movement.
`In addition, strain gauge or pressure plate
`approaches are somewhat expensive because special sensors
`are required.
`Optical approaches are also possible but are somewhat
`limited for several reasons. All would require light genera-
`tion which will require external components and increase
`cost and power drain. For example, a “finger-breaking”
`infra-red matrix position detector consumes high powerand
`suffers from relatively poor resolution.
`There have been numerous attempts to provide a device
`for sensing the position of a thumbor otherfinger for use as
`a pointing device to replace a mouseor trackball. Desirable
`attributes of such a device are low power, low profile, high
`resolution, low cost, fast response, and ability to operate
`reliably when the finger carries electrical noise, or when the
`touch surface is contaminated with dirt or moisture.
`
`Because of the drawbacks of resistive devices, many
`attempts have been made to provide pointing capability
`based on capacilively sensing the position of the finger. U.S.
`Pat. No. 3,921,166 to Volpe teaches a capacitive matrix in
`which the finger changes the transcapacitance between row
`and columnelectrodes. U.S. Pat. No. 4,103,252 to Bobick
`employs four oscillating signals to interpolate x and y
`positions between four capacitive electrodes. U.S. Pat. No.
`4,455,452 to Schuyler teaches a capacitive tablet wherein
`the finger attenuates the capacitive coupling between elec-
`trodes.
`
`U.S. Pat. No. 4,550,221 to Mabusth teaches a capacitive
`tablet wherein the effective capacitance to “virtual ground”
`is measured by an oscillating signal. Each row or column is
`polled sequentially, and a rudimentary formof interpolation
`is applied to resolve the position between two rows or
`columns. An attempt is made to address the problem of
`electrical interference by averaging over many cycles of the
`oscillating waveform. The problem of contamination is
`addressed by sensing when no finger was present, and
`applying a periodic calibration during such no-finger-present
`periods. U.S. Pat. No. 4,639,720 to Rympalski teaches a
`tablet for sensing the position of a stylus. The stylus alters
`the transcapacitance coupling between row and column
`electrodes, which are scanned sequentially. U.S. Pat. No.
`4,736,191 to Matzke teaches a radial electrode arrangement
`under the space bar of a keyboard,
`to be activated by
`touching with a thumb. This patent teaches the use of total
`touch capacitance, as an indication of the touch pressure, to
`control the velocity of cursor motion. Pulsed sequential
`polling is employed to address the effects of electrical
`interference.
`to Greanias,
`U.S. Pat. Nos. 4,686,332 and 5,149,919,
`teaches a stylus and finger detection system meant to be
`mounted on a CRT. As a finger detection system, its X/Y
`sensor matrix is used to locate the two matrix wires carrying
`the maximum signal. With a coding scheme these two wires
`uniquely determine the location of the finger position to the
`resolution of the wire stepping. For stylus detection, Gre-
`aniasfirst coarsely locates il, then develops a virtual dipole
`by drivingall lines on oneside of the object in one direction
`and all lines on the opposite side in the opposite direction.
`This is done three times with different dipole phases and
`signal polarities. Assuming a predetermined matrix response
`to the object,
`the three measurements present a set of
`simultaneous equations that can be solved for position.
`
`

`

`5,880,411
`
`3
`US. Pat. No. 4,733,222 to Evansis the first to teach a
`capacitance touch measurementsystem that interpolates to a
`high degree. Evans teaches a three terminal measurement
`system that uses a drive, sense and electrode signal set (3
`signals) in its matrix, and bases the measurement on the
`attenuation effect of a finger on the electrode node signal
`(uses a capacitive divider phenomena). Evans sequentially
`scans through each drive set to measure the capacitance.
`Fromthe three largest responses an interpolation routine is
`applied to determine finger position. Evans also teaches a
`zeroing technique that allows “no-finger” levels to be can-
`celed out as part of the measurement.
`US. Pat. No. 5,016,008 to Gruaz describes a touch
`sensitive pad that also uses interpolation. Gruaz uses a drive
`and sense signal set (2 signals) in the touch matrix and like
`Evansrelics on the attenuation effect of a finger to modulate
`the drive signal. The touch matrix is sequentially scanned to
`read the response of each matrix line. An interpolation
`program thenselects the two largest adjacent signals in both
`dimensions to determine the finger location, and ratiometri-
`cally determines the effective position from those 4 num-
`bers.
`
`Gerpheide, PCT application US90/04584,publication No.
`W091/03039, U.S. Pat. No. 5,305,017 applies to a touch pad
`system a variation of the virtual dipole approach of Grean-
`ias. Gerpheide teaches the application of an oscillating
`potential of a given frequency and phaseto all electrodes on
`oneside of the virtual dipole, and an oscillating potential of
`the same frequency and opposite phase to those on the other
`side. Electronic circuits develop a “balance signal” which is
`zero when no finger is present, and which has one polarity
`if a finger is on one side of the center of the virtual dipole,
`and the opposite polarity if the finger is on the opposite side.
`To acquire the position of the finger initially, the virtual
`dipole is scanned sequentially across the tablet. Once the
`finger is located,it is “tracked” by movingthe virtual dipole
`toward the finger once the finger has moved more than one
`row or column.
`
`Because the virtual dipole method operates by generating
`a balance signal that is zero when the capacitance does not
`vary with distance, it only senses the perimeter of the finger
`contact area, rather than the entire contact area. Because the
`method relies on synchronous detection of the exciting
`signal, it must average for long periods to reject electrical
`interference, and hence it
`is slow. The averaging time
`required by this method,
`together with the necessity to
`search sequentially for a new finger contact once a previous
`contact is lost, makes this method, like those before it, fall
`shortof the requirementsfor a fast pointing devicethat is not
`affected by electrical interference.
`It should also be noted that all previous touch pad
`inventions that used interpolation placed rigorous design
`requirements on their sensing pad. Greanias and Evansuse
`a complicated and expensive drive, sense and electrode line
`scheme to develop their signal. Gruaz and Gerpheide use a
`two signal drive and sense set. In the present invention the
`driving and sensing is done on the sameline. This allows the
`row and column sections to be symmetric and equivalent.
`This in turn allows independent calibration of all signal
`paths, which makes board layout simpler and less
`constraining, and allows for more unique sensor topologies.
`The shortcomings of the inventions and techniques
`describedin the prior art can also betracedto the use of only
`one set of driving and sensing electronics, which was
`multiplexed sequentially over the electrodes in the tablet.
`This arrangement wascost effective in the days of discrete
`components, and avoided offset and scale differences among
`circuits.
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`The sequential scanning approach of previous systems
`also made them more susceptible to noise. Noise levels
`could change between successive measurements,
`thus
`changing the measured signal and the assumptions used in
`interpolation routines.
`Finally, all previous approaches assumed a particular
`signal response for finger position versus matrix position.
`Because the transfer curve is very sensitive to many param-
`eters and is not a smooth linear curve as Greanias and
`
`Gerpheide assume, such approaches are limited in the
`amount of interpolation they can perform.
`In prior co-pending application Ser. No. 08/115,743, filed
`Aug. 31,1993, now U.S. Pat. No. 5,734,787, a two-
`dimensional capacitive sensing system equipped with a
`separate set of drive/sense electronics for each row and for
`each column of a capacitive tablet is disclosed. All row
`electrodes are sensed simultaneously, and all column elec-
`trodes are sensed simultaneously. The sensed signals are
`processed by analogcircuitry.
`Of the touchpad devices currently available, only the
`Alps/Cirque GlidcPoint includes gesture recognition. The
`GlidePoint supports basic tap, double-tap, and drag gestures
`to simulate actions on a primary mouse button. It does not
`support multiple-finger gestures, nor are there gestures for
`simulating secondary button clicks. No information is
`known about the implementation methods employed in the
`GlidePoint. However,
`the GlidePoint
`is known to have
`difficulty with double-taps, one of the problems addressed
`by the present invention. The GlidePoint exhibits a hesita-
`tion on each finger-motion stroke which may be an attempt
`to stabilize the cursor during tap gestures. Also, the Glide-
`Point must rely on physical switches or extremely high gain
`or acceleration in order to allow drags over long distances.
`One touchpad product, the UnMouse, mounts a switch
`underneath its resistive sensor so that the user simply presses
`down onthe padto activate the button. Aside from requiring
`fragile and complex mechanical mounting, this device also
`is reported to be very tiring to the user.
`Graphics tablets operated by a pressure sensitive stylus
`instead of a finger are well knownin the art. These devices
`typically use a mechanism like the “push” gesture of the
`present invention to simulate actuator switches. No other
`gestures of the sort described herein have been seen in stylus
`operated tablets.
`It is thus an object of the present invention to provide a
`two-dimensional capacitive sensing system equipped with a
`separate set of drive/sense electronics for each row and for
`each column of a capacitive tablet, wherein all row elec-
`trodes are sensed simultaneously, and all columnelectrodes
`are sensed simultaneously.
`It is a further object of the present invention to provide an
`electronic system that
`is sensitive to the entire area of
`contact of a finger or other conductive object with a capaci-
`tive tablet, and to provide as output the coordinates of some
`measure of the center of this contact area while remaining
`insensitive to the characteristic profile of the object being
`detected.
`
`It is a further object of the present invention to provide an
`electronic system that provides as output some measure of
`area of contact of a finger or other conductive object with a
`capacitive tablet.
`Yet another object of the present invention is to provide a
`two-dimensional capacitive sensing system equipped with a
`separate set of drive/sense electronics for each row and for
`each column of a capacitive tablet, wherein all row elec-
`trodes are sensed simultaneously, and all column electrodes
`
`

`

`5,880,411
`
`5
`are sensed simultaneously and wherein the information
`defining the location of a finger or other conductive object
`1s processed in digital form.
`It is a further object of the present invention to provide a
`two-dimensional capacitive sensing system wherein all row
`electrodes are sensed simultaneously, and all columnelec-
`trodes are sensed simultaneously and wherein the location of
`a finger or other conductive object within a peripheral region
`of a sensing plane can optionally cause cursor “edge
`motion” on a display screen allowing controlof large cursor
`excursions from a small sensing plane with a single gesture.
`A further object of the invention is to provide for the
`recognition of a drag extension gesture made bya finger or
`other object on a touch-sensor pad in a manner which
`permits control of large cursor excursions from a small
`sensing plane with a single gesture.
`A further object of the invention is to provide for the
`recognition of gestures made by a finger or other object on
`a touch-sensor pad in a manner which compensates for
`unintended motion of the finger or other object during ,,
`expression of the gesture.
`Yet another object of the present invention is to provide
`for the recognition of multiple-finger gestures and for simu-
`lating secondary button clicks.
`It is a further object of the present inventionis to provide
`for the recognition of the difference between gestures made
`by novice and expert users.
`BRIEF DESCRIPTION OF THE INVENTION
`
`10
`
`15
`
`With the advent of very high levels of integration, it has
`become possible to integrate many channels of driving/
`sensing electronics into one integrated circuit, along with the
`control logic for operating them, and the interface electron-
`ics to allow the pointing device to communicate directly
`with a host microprocessor. The present
`invention uses
`adaptive analog techniques to overcome offset and scale
`differences between channels, and can thus sense either
`transcapacitance or self-capacitance of all tablet rows or
`columns in parallel. This parallel-sensing capability, made
`possible by providing one set of electronics per row or
`column,allows the sensing cycle to be extremely short, thus
`allowing fast response while still maintaining immunity to
`very high levels of electrical interference.
`The present invention comprises a position-sensing tech-
`nology particularly useful for applications where finger
`position information is needed, such as in computer
`“mouse” or trackball environments. However the position-
`sensing technology of the present invention has much more
`general application than a computer mouse, because its
`sensor can detect and report if one or more points are being
`touched. In addition, the detector can sense the pressure of
`the touch.
`
`According to a preferred embodiment of the present
`invention,
`referred to herein as a “finger pointer”
`embodiment, a position sensing, system includes a position
`sensing transducer comprising a touch-sensitive surface
`disposed on a substrate, such as a printed circuit board,
`including a matrix of conductive lines. A first set of con-
`ductive lines runs in a first direction and is insulated from a
`
`second set of conductive lines running in a second direction
`generally perpendicular to the first direction. An insulating
`layer is disposed overthe first and secondsets of conductive
`lines. The insulating layer is thin enough to promote sig-
`nificant capacitive coupling between a finger placed on its
`surface and the first and second sets of conductive lines.
`
`Sensing electronics respond to the proximity of a finger,
`conductive object, or an object of high dielectric constant
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`to transl