USOO7663607B2
`
`(12) United States Patent
`Hotelling et al.
`
`(10) Patent No.:
`45) Date of Patent:
`
`US 7.663,607 B2
`Feb. 16, 2010
`
`9
`
`(54) MULTIPOINT TOUCHSCREEN
`
`(75) Inventors: Steve Hotelling, San Jose, CA (US);
`Joshua A. Strickon, San Jose, CA (US);
`rt:
`r
`s
`Rita Q. Huppi, San Francisco, CA
`
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`c
`-
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 754 days.
`
`(21) Appl. No.: 10/840,862
`
`(22) Filed:
`(65)
`
`May 6, 2004
`Prior Publication Data
`
`3/1988 Watanabe ................... 340,710
`4,734,685 A
`5/1988 McAvinney
`... 178/18
`4,746,770 A
`9/1988 Parks ......................... 340,712
`4,771,276 A
`4,788,384 A 11/1988 Bruere-Dawson et al. ..... 178/18
`4,806,846 A
`2, 1989 Kerber .................. 324f60 CD
`4,898,555 A
`2, 1990 Sampson ..................... 445/22
`4.914,624 A
`4, 1990 Dunthornet al.
`4,968,877 A 11/1990 McAvinney et al. ........ 250,221
`5,003,519 A
`3/1991 Noirjean ....
`... 368,73
`5,017,030 A
`5/1991 Crews .......
`... 400,485
`5,178.477 A
`1/1993 Gambaro ...
`... 400,489
`5,189,403 A
`2f1993 Franz et al. .
`... 340,711
`5,194,862. A
`3/1993 Edwards ...................... 341?20
`5,224,861 A
`7/1993 Glass et al. ................... 434/35
`5,241.308 A
`8/1993 Young ......................... 341 (34
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`US 2006/OO97991 A1
`
`May 11, 2006
`
`CA
`
`1243.096
`
`10/1988
`
`................. 340,180
`
`(51) Int. Cl
`(2006.01)
`G06F 3/04
`(52) U.S. Cl. ........ r 345/173; 34.5/174; 178/18.01
`(58) Field of Classification Search ......... 345/173–179;
`178/18.01 18.04
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`3,333,160
`3,541,541
`3,662,105
`3,798,370
`4,246,452
`4.550,221
`4,672,364
`4,672,558
`4,692.809
`4,695,827
`4,733,222
`
`7, 1967
`11, 1970
`5, 1972
`3, 1974
`1, 1981
`10, 1985
`6, 1987
`6, 1987
`9, 1987
`9, 1987
`3, 1988
`
`Gorski
`Englebart
`Hurst et al. ................... 178/18
`Hurst ......
`... 178/18
`Chandler ....................... 2005
`Mabusth ...................... 178/18
`Lucas ..........
`... 340,365 P
`Beckes et al. ............... 364,518
`Beining et al. .............. 358,247
`Beining et al. ..
`... 340,365 P
`Evans ..................... 340,365 C
`
`(Continued)
`OTHER PUBLICATIONS
`U.S. Appl. No. 10/654,108, filed Sep. 2, 2003.
`
`(Continued)
`Primary Examiner Richard Hjerpe
`Assistant Examiner—Kimnhung Nguyen
`(74) Attorney, Agent, or Firm Morrison & Foerster LLP
`(57)
`ABSTRACT
`
`A touch panel having a transparent capacitive sensing
`medium configured to detect multiple touches or near touches
`that occur at the same time and at distinct locations in the
`plane of the touch panel and to produce distinct signals rep
`resentative of the location of the touches on the plane of the
`touch panel for each of the multiple touches is disclosed.
`
`11 Claims, 14 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`102
`
`"
`
`.
`
`4ZZZZZZZZZ7.
`assa ZZZZZZZZZZZ
`
`seat
`
`ZZZZZZZZZZ 7
`
`
`
`
`
`106
`
`Page 1 of 33
`
`Tianma Exhibit 1017
`
`

`

`US 7,663,607 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,252,951 A 10/1993 Tannenbaum et al. ....... 345,156
`5,281,966 A
`1/1994 Walsh ..............
`... 341/22
`5,305,017 A
`4/1994 Gerpheide ..
`345,174
`5,345,543 A
`9/1994 Capps et al.
`395,137
`5,376,948 A 12/1994 Roberts ......
`345,173
`5,398.310 A
`3, 1995 Tchao et al. ...
`... 395,144
`5,442,742 A
`8/1995 Greyson et al.
`... 395.146
`5,463,388 A 10, 1995 Boie et al. .....
`... 34.1/33
`5,463,696 A 10, 1995 Beernink et al.
`382, 186
`5,483.261 A
`1/1996 Yasutake .......
`345,173
`5.488.204 A
`1/1996 Mead et al. .
`178/18
`5,495,077 A
`2f1996 Miller et al. ...
`... 178/18
`5,513,309 A
`4, 1996 Meier et al. .
`395/155
`5,523,775 A
`6/1996 Capps ........
`345,179
`5,530.455 A
`6, 1996 Gillicket al. ..
`345,163
`5,543,590 A
`8/1996 Gillespie et al. ..
`178/18
`5,543,591 A
`8/1996 Gillespie et al. .....
`... 178/18
`5,563,632 A 10, 1996 Roberts .........
`345,173
`5,563,996 A 10, 1996 Tchao ...........
`395,144
`5,565,658 A 10/1996 Gerpheide et al. ...
`... 178.19
`5,579,036 A 1 1/1996 Yates, IV ..........
`... 345,173
`5,581,681 A 12/1996 Tchao et al.
`... 395/804
`5,583,946 A 12/1996 Gourdol ..................... 382,187
`5,589,856 A 12/1996 Stein et al.
`5,590,219 A 12/1996 Gourdol ..................... 382,202
`5,592,566 A
`1/1997 Pagallo et al...
`... 382,187
`5,594,810 A
`1/1997 Gourdol .....
`382,187
`5,596,694 A
`1/1997 Capps ...........
`395,152
`5,612,719 A
`3, 1997 Beernink et al. ...
`... 345,173
`5,631,805 A
`5/1997 Bonsall ............
`... 361/681
`5,633,955 A
`5, 1997 BOZinovic et all
`381,187
`5,634,102 A
`5/1997 Capps .......,
`395,334
`5,636,101 A
`6, 1997 Bonsall et al. .
`361,681
`5,642,108 A
`6/1997 Gopher et al. ................ 341 (22
`5,644,657 A
`7/1997 Capps et al. ...
`382,229
`5,666,113 A
`9/1997 Logan ........
`... 34.1/34
`5,666,502 A
`9/1997 Capps ........................ 345,352
`5,666,552 A
`9/1997 Grayson et al. ............. 395/802
`5,675,361 A 10, 1997 Santilli ................
`... 345,168
`5,677.710 A 10/1997 Thompson-Rohrlich .... 345/173
`5,689.253 A 11/1997 Hargreaves et al. ........... 341 (22
`5,710,844 A
`1/1998 Capps et al. .........
`... 382.317
`5,729,250 A
`3/1998 Bishop et al. ............... 34.5/175
`5,730,165 A
`3/1998 Philipp .......................... 137/1
`5,736,976 A
`4, 1998 Cheung ...
`345,168
`5,741,990 A
`4, 1998 Davies ..................... 84/423 R
`5,745,116 A
`4, 1998 Pisutha-Arnond ........... 345,358
`5,745,716 A
`4, 1998 Tchao et al. ...
`... 395,350
`5,746,818 A
`5/1998 Yatake .................... 106.31.86
`5,748,269 A
`5/1998 Harris et al. .................. 349/58
`5,764.222 A
`6, 1998 Shieh ...............
`345,173
`5,767.457 A
`6/1998 Gerpheide et al. ............ 178/18
`
`5,767,842 A
`
`6, 1998 Korth - - - - - - - - - - - - - - - - - - - - - - - - - 345,168
`
`345,173
`8, 1998 Shieh .........
`5,790,104 A
`8/1998 Kasser et al. ............... 345,174
`5,790,107 A
`9, 1998 Shwarts et al. ................. 707/6
`5,802,516 A
`9, 1998 McCloud .......
`... 341 (20
`5,808,567 A
`9, 1998 Moran et al. ................ 395,358
`5,809,267 A
`5,821,690 A 10, 1998 Martens et al. ............. 313,506
`5,821,930 A 10/1998 Hansen ..........
`... 345.340
`5,823,782. A 10, 1998 Marcus et al. .............. 434,156
`5,825.351 A 10/1998 Tam
`5,825,352 A 10/1998 Bisset et al. ................ 345,173
`5,854,625. A 12/1998 Frisch et al. ................ 345,173
`5,880,411 A
`3/1999 Gillespie et al. ......... 178/1801
`5,898,434 A
`4/1999 Small et al. .......
`... 345.348
`5,920,309 A
`7, 1999 Bisset et al. ................ 345,173
`5,923,319 A
`7, 1999 Bishop et al. ............... 34.5/175
`5,933,134 A
`8/1999 Shieh ......................... 345,173
`5,942,733. A
`8, 1999 Allen et al.
`5,943,044 A
`8, 1999 Martinelli et al. ........... 345,174
`
`6,002,389 A 12/1999 Kasser ....................... 345,173
`6,002,808. A 12/1999 Freeman .....
`382.288
`6,020,881 A
`2/2000 Naughton et al. ........... 345,327
`6,031,524 A
`2/2000 Kunert ....................... 345,173
`6,037,882 A
`3/2000 Levy .......
`... 341 (20
`6,050,825 A
`4/2000 Nichol et al. ............... 434,227
`6,052,339 A
`4/2000 Frenkel et al. .............. 368,230
`6,061,177 A
`5/2000 Fujimoto .
`359,443
`6,072,494 A
`6, 2000
`345,358
`6,084,576 A
`T/2000
`345,168
`6,107,997 A
`8, 2000
`345,173
`6,128,003 A 10/2000 Smith et al. ................. 345,157
`6, 131,299 A 10/2000 Raab et al. .................... 33,503
`6,135,958. A 10/2000 Mikula-Curtis et al.
`600/443
`6,144,380 A 1 1/2000 Shwarts et al. .............. 345,350
`6,188,391 B1
`2/2001 Seely et al.
`348,836
`6,198.515 B1
`3/2001 Cole
`345,173
`6,208,329 B1
`3/2001 Ballare .
`... 341 (20
`6,222.465 B1
`4/2001 Kumar et al. ...
`345,174
`6,239,790 B1
`5, 2001 Martinelli et al.
`345/146
`6,243,071 B1
`6/2001 Shwarts et al. ..
`... 455,90
`6,246,862 B1
`6/2001 Grivas et al. .
`382,195
`6,249,606 B1
`6/2001 Kiraly etal
`345,168
`6,288,707 B1
`9/2001 Philipp ...
`705/7O2
`6,289,326 B1
`9, 2001 LaFleur ...
`6,292,178 B1
`9/2001 Bernstein et al. ............ 345,173
`6,323,846 B1
`1 1/2001 Westerman et al.
`6,337,678 B1
`1, 2002 Fish
`6,347.290 B1
`2/2002 Bartlett ....................... 7O2/15O
`6,377,009 B1
`4/2002 Philipp ...
`318,468
`6,380,931 B1
`4/2002 Gillespie et al.
`345,173
`6,411,287 B1
`6/2002 Scharff et al. ...
`345,177
`6,414,671 B1
`7/2002 Gillespie et al.
`345,157
`6.421,234 B1
`7/2002 Ricks et al. ...
`361,683
`6,452,514 B1
`9/2002 Philipp ...
`... 341.33
`6.457,355 B1
`10/2002 Philipp .
`... 73,304
`6,466,036 B1
`10/2002 Philipp .
`324,678
`6,515,669 B1
`2/2003 Mohri .....
`345/474
`6,525,749 B1
`2/2003 Moran et al. .
`345,863
`6,535,200 B2
`3/2003 Philipp .......
`345,168
`6,543,684 B1
`4/2003 White et al. .
`234,379
`6,543,947 B2
`4/2003 Lee ............................ 400,489
`6,570,557 B1
`5/2003 Westerman et al.
`6,593.916 B1
`7/2003 Aroyan
`178.18.01
`6,610,936 B2
`8/2003 Gillespie et al.
`... 345,863
`6,624,833 B1
`9/2003 Kumar et al. ...
`6,639,577 B2 10/2003 Eberhard .................... 345,102
`6,650,319 B1
`1 1/2003 Hurst et al.
`6,658,994 B1
`12/2003 McMillan .................... 99/468
`6,670.894 B2 12/2003 Mehring ...................... 341 (22
`6,677,932 B1
`1/2004 Westerman
`6,677,934 B1
`1/2004 Blanchard ................... 345,173
`6,723,929 B2
`4/2004 Kent
`6,724,366 B2
`4/2004 Crawford .................... 345,157
`
`6,757,002 B1
`
`6/2004 Oross et al. ..
`
`345,864
`
`10/2004 Morrison et al. ............ 345,173
`6,803,906 B1
`1/2005 Straub et al. ................... TO1/3
`6,842,672 B1
`6,856.259 B1* 2/2005 Sharp ............
`... 341/5
`6,888,536 B2 * 5/2005 Westerman et al. ......... 345,173
`6,900,795 B1
`5/2005 Knight, III et al. .......... 345,173
`6,927,761 B2
`8/2005 Badaye et al. .....
`345,173
`6,942,571 B1
`9/2005 McAllister et al. ............ 463,20
`6,965,375 B1
`1 1/2005 Gettemyet al. ............. 345,173
`6,972,401 B2 12/2005 Akitt et al. ...
`250,221
`6,977.666 B1
`12/2005 Hedrick ...................... 345,690
`6,985,801 B1
`1/2006 Straub et al. ................... TO1/3
`6,992,659 B2
`1/2006 Gettemy ..
`345,173
`7,030,860 B1
`4/2006 Hsu et al.
`345,173
`7,031,228 B2
`4/2006 Born et al. .................... 368/69
`2002fOO15024 A1
`2/2002 Westerman et al.
`345,173
`2002/01 18848 A1
`8/2002 Karpenstein ................ 381,119
`2003/0006974 A1
`1/2003 Clough et al. ............... 345,179
`2003/0076301 A1
`4/2003 TSuk et al. .................. 345,159
`
`
`
`Page 2 of 33
`
`

`

`US 7,663,607 B2
`Page 3
`
`4/2003 Huppi ........................ 345,163
`2003/0076303 A1
`4/2003 Zadesky et al.
`345,173
`2003/0076306 A1
`5/2003 Pihlaja ....................... 345,156
`2003/OO95095 A1
`5/2003 Robbin et al. ............... 345,156
`2003/OO95096 A1
`5, 2003 Perski et al. ...
`... 345,173
`2003/0098858 A1
`1 1/2003 Moriya ....................... 345,846
`2003/0206202 A1
`2003/0234768 A1
`12/2003 Rekimoto et al.
`345/169
`2003/0234770 A1* 12/2003 MacKey ...........
`... 345,173
`2004/0227736 A1* 11/2004 Kamrath et al. ............. 345,173
`2004/0263484 A1 12/2004 Mantysalo et al. .......... 345,173
`2005, OOO7349 A1
`1/2005 Vakil et al. ........
`... 345,173
`2005, OO12723 A1
`1/2005 Pallakoff .................... 345,173
`2005/005.2425 A1
`3/2005 Zadesky et al. ............. 345,173
`2005, 0083307 A1* 4/2005 Aufderheide et al. ....... 345,173
`2005, 0104867 A1
`5/2005 Westerman et al. ......... 345,173
`2005, 0110768 A1
`5/2005 Marriott et al. ............. 345,173
`2005, 0146511 A1*
`7/2005 Hill et al. .......
`... 345,173
`2/2006 Kennedy ....
`2006/0022955 A1
`345,173
`2/2006 Lengeling et al. .
`2006/0022956 A1
`... 345,173
`2/2006 Hotelling et al. .....
`2006, 0026521 A1
`... 157702
`2/2006 Hotelling et al. .....
`2006, OO26535 A1
`... 715,863
`2/2006 Hotelling et al. ..
`2006, OO26536 A1
`... 715,863
`2006, OO32680 A1
`178.18.06
`2, 2006 Elias et al. ........
`2006, OO33724 A1
`... 345,173
`2/2006 Chaudhri et al. ...
`3/2006 Ording ..........
`2006.0053387 A1
`... 715,773
`3/2006 Lyon et al. ..
`2006/0066582 A1
`... 345,173
`2006/0O85757 A1
`... 715,771
`4/2006 Andre et al. ...
`... 345,173
`5/2006 Hotelling et al.
`2006/0097991 A1
`9/2006 Hotelling .................... 345,173
`2006/O197753 A1
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`5, 2004
`10251 296
`DE
`10, 1985
`O 156593
`EP
`1, 1988
`O 25.0931
`EP
`7, 1993
`0 288 692
`EP
`1, 1995
`O 664 504
`EP
`9, 1996
`O 464908
`EP
`1, 2002
`1 O14 295
`EP
`12, 1984
`59-214941
`JP
`10, 1985
`60-211529
`JP
`1, 1996
`08-016307
`JP
`4/2000
`2000-112642
`JP
`11, 2002
`2002-342033
`JP
`1, 2003
`2003-029899
`JP
`5, 1997
`97, 18547
`WO
`7/1997
`97.23738
`WO
`4f1998
`98.14863
`WO
`4/2001
`WO-01 (27868 A1
`WO
`10, 2003
`O3,O88176
`WO
`2, 2004
`WO WO-2004/O13833 A2
`2, 2004
`WO WO-2004/O13833 A3
`WO WO-2005,114369 A2 12/2005
`WO WO-2005,114369 A3 12/2005
`WO
`2006/023569
`3, 2006
`
`OTHER PUBLICATIONS
`U.S. Appl. No. 10/789,676, filed Feb. 27, 2004.
`U.S. Appl. No. 10/903,964, filed Jul. 30, 2004.
`U.S. Appl. No. 1 1/015,978, filed Dec. 17, 2004.
`U.S. Appl. No. 11/038,590, filed Jan. 18, 2005.
`U.S. Appl. No. 1 1/048.264, filed Jan. 31, 2005.
`“Touch Technologies Overview.” 2001, 3M Touch Systems, Massa
`chusetts.
`“Touchscreen Technology Choices.” http://www.elotouch.com/
`products/detech2.asp, downloaded Aug. 5, 2005.
`Jun Rekimoto, "SmartSkin: An Infastructure for Freehand Manipu
`lation on Interactive Surfaces.” CHI 2002, Apr. 20-25, 2002, Minne
`apolis, Minnesota.
`http://www.
`“Wacom
`Components Technology.”
`Wacomeomponents.com/english tech.asp., downloaded Oct. 10,
`2004.
`
`“Comparing Touch Technologies.” http://www.touchscreens.com/
`intro-touchtypes.html, downloaded Oct. 10, 2004.
`“GlidePoint R.” http://www.cirque.com/technology/technology gp.
`htm, downloaded Aug. 5, 2005.
`"Captive Position Sensing.” http://www.synaptics.com/technology/
`cps.cfm, downloaded Aug. 5, 2005.
`“How do touchscreen monitors know where you're touching?”
`http://electronics.howstuffworks.com/question716.htm,
`down
`loaded Aug. 5, 2005.
`“How Does a Touchscreen Work?.” http://www.touchscreens.com/
`intro-anatomy.html, downloaded Aug. 5, 2005.
`“4-Wire Resistive Touchscreens.” http://www.touchscreens.com/
`intro-touchtypes-4 resistive.html, downloaded Aug. 5, 2005.
`“5-Wire Resistive Touchscreens.” http://www.touchscreens.com/
`intro-touchtypes-resistive.html, downloaded Aug. 5, 2005.
`"Capacitive Touchscreens.' http://www.touchscreens.com/intro
`touchtypes-capacitive.html, downloaded Aug. 5, 2005.
`“PenTouch Capacitive Touchscreens.” http://www.touchscreens.
`com/intro-touchtypes-pentouch.html, downloaded Aug. 5, 2005.
`"Surface Acoustic Wave Touchscreens.” http://www.touchscreens.
`com/intro-touchtypes-saw.html, downloaded Aug. 5, 2005.
`"Near Field Imaging Touchscreens.” http://www.touchscreens.com/
`intro-touchtypes-nfi.html, downloaded Aug. 5, 2005.
`“Infrared
`Touchscreens.”
`http://www.touchscreens.com/intro
`touchtypes-infrared.html, downloaded Aug. 5, 2005.
`“Watershed Algorithm.” http://rsb.info.nih.gov/ij/plugins/watershed.
`html, downloaded Aug. 5, 2005.
`U.S. Appl. No. 10/654,108, filed Sep. 2, 2003 entitled “Ambidextrous
`Mouse'.
`U.S. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled “Shape
`Detecting Input Device'.
`“4-Wire Resistive Touchscreens' obtained from http://www.
`touchscreens.com/intro-touchtypes-4 resistive.html generated Aug.
`5, 2005.
`“5-Wire Resistive Touchscreens' obtained from http://www.
`touchscreens.com/intro-touchtypes-resistive.html generated Aug. 5,
`2005.
`“A Brief Overview of Gesture Recognition” obtained from http://
`www.dailed.ac.uk/Cvonline/LOCA COPIES, COHEN
`gesture overview.html, generated Apr. 20, 2004.
`"Capacitive Touchscreens' obtained from http://www.touchscreens.
`com/intro-touchtypes-capacitive.html generated Aug. 5, 2005.
`"Capacitive Position Sensing” obtained from http://www.synaptics.
`com/technology/cps.cfm generated Aug. 5, 2005.
`“Comparing Touch Technologies' obtained from http://www.
`touchscreens.com/intro-touchtypes.html generated Oct. 10, 2004.
`“Gesture
`Recognition”
`http://www.fingerworks.com/
`gesture recognition.html.
`“GlidePoinar R” obtained from http://www.cirque.com/technology/
`technology gp.html generated Aug. 5, 2005.
`“How do touchscreen monitors know where you're touching?”
`obtained from http://www.electronics.howstuffworks.com/ques
`tion716.html generated Aug. 5, 2005.
`“How does a touchscreen work?” obtained from http://www.
`touchscreens.com/intro-anatomy.html generated Aug. 5, 2005.
`“iGesture Products for Everyone (learn in minutes) Product Over
`view” FingerWorks.com.
`“Infrared Touchscreens' obtained from http://www.touchscreens.
`com/intro-touchtypes-infrared.html generated Aug. 5, 2005.
`“Mouse Emulation' FingerWorks obtained from http://www.
`fingerWorks.com/gesture guide mouse.html generated Aug. 30.
`2005.
`“Mouse Gestures in Opera” obtained from http://www.opera.com/
`products/desktop/mouse/index.dml generated Aug. 30, 2005.
`“Mouse Gestures.” Optimoz, May 21, 2004.
`“MultiTouch Overview” FingerWorks obtained from http://vvvvw.
`fingerworks.com/multoverview.html generated Aug. 30, 2005.
`"Near Field Imaging Touchscreens' obtained from http://www.
`touchscreens.com/intro-touchtypes-nfi.html generated Aug. 5, 2005.
`“PenTouch Capacitive Touchscreens' obtained from http://www.
`touchscreens.com/intro-touchtypes-pentouch.html generated Aug.
`5, 2005.
`
`Page 3 of 33
`
`

`

`US 7,663,607 B2
`Page 4
`
`"Surface Acoustic Wave Touchscreens' obtained from http://www.
`touchscreens.com/intro-touchtypes-saw.html generated Aug. 5,
`2005.
`“Symbol Commander' obtained from http://www.sensiva.com/
`symbolcomander?, generated Aug. 30, 2005.
`"Tips for Typing” FingerWorks http://www.fingerworks.com/
`mini typing.html generated Aug. 30, 2005.
`“Touch Technologies Overview” 2001, 3M Touch Systems, Massa
`chusetts.
`“Wacom Components—Technology” obtained from http://www.
`Wacom-components.com/english/tech.asp generated on Oct. 10,
`2004.
`“Watershed Algorithm” http://rsb.info.nih.gov/ij/plugins/watershed.
`html generated Aug. 5, 2005.
`“FingerWorks—Gesture Guide—Application Switching,” obtained
`from http://www.fingerworks.com/gesture guide apps.html, gen
`erated on Aug. 27, 2004, 1-pg.
`“FingerWorks—Gesture Guide—Editing,” obtained from http://
`www.fingerWorks.com/geSure guide editing.html, generated on
`Aug. 27, 2004, 1-pg.
`“FingerWorks—Gesture Guide—File Operations,” obtained from
`http://www.fingerworks.com/gesture guide files.html, generated
`on Aug. 27, 2004, 1-pg.
`“FingerWorks—Gesture Guide TextManipulation” obtained from
`http://www.fingerworks.com/gesture guide text manip.html,
`generated on Aug. 27, 2004, 2-pg.
`“FingerWorks—Gesture Guide Tips and Tricks,” obtained from
`http://www.fingerworks.com/gesture guide tips.html, generated
`Aug. 27, 2004, 2-pgs.
`“FingerWorks—Gesture Guide Web,” obtained from http://www.
`fingerWorks.com/gesture guide web.html, generated on Aug. 27.
`2004, 1-pg.
`“FingerWorks—Guide to Hand Gestures for USB Touchpads.”
`obtained from http://www.fingerworks.com/igesture userguide.
`html, generated Aug. 27, 2004, 1-pg.
`“FingerWorks—iGesture Technical Details,” obtained from http://
`www.fingerWorks.com/igesture tech.html, generated Aug. 27.
`2004, 1-pg.
`“FingerWorks—The Only Touchpads with Ergonomic Full-Hand
`Resting and Relaxation” obtained from http://www.fingerworks.
`com/resting.html. Copyright 2001, 1-pg.
`“FingerWorks Tips for Typing on the Mini,” obtained from http://
`www.fingerWorks.com/mini typing.html, generated on Aug. 27.
`2004, 2-pgs.
`“iGesture Pad the MultiFinger USB TouchPad with Whole-Hand
`Gestures,” obtained from http://www.fingerworks.com/igesture.
`html, generated Aug. 27, 2004, 2-pgs.
`Bier, et al., “Toolglass and Magic Lenses: The see-through interface'
`In James Kijiya, editor, Computer Graphics (SIGGRAPH '93 Pro
`ceedings), vol. 27, pp. 73-80. Aug. 1993.
`Douglas et al., The Ergonomics of Computer Pointing Devices
`(1997).
`European Search Report received in EP 1621989 (a) Beyer Weaver
`& Thomas, LLP) dated Mar. 27, 2006.
`EVB Elektronik 'TSOP6238 IR Receiver Modules for Infrared
`Remote Control Systems” dated Jan. 2004 1-pg.
`Fisher et al., “Repetitive Motion Disorders: The Design of Optimal
`Rate- Rest Profiles.” Human Factors, 35(2):283-304 (Jun. 1993).
`Fukumoto, et al., “ActiveClick: Tactile Feedback for Touch Panels.”
`In CHI 2001 Summary, pp. 121-122, 2001.
`Fukumoto and Yoshinobu Tonomura, “Body Coupled Fingering:
`Wireless Wearable Keyboard.” CHI 97, pp. 147-154 (Mar. 1997).
`Hardy, “Fingerworks' Mar. 7, 2002; BBC World On Line.
`Hillier and Gerald J. Lieberman, Introduction to Operations
`Research (1986).
`International Search Report dated Mar. 3, 2006 (PCT/US 05/03325:
`119-0052WO).
`Jacob et al., “Integrality and Separability of Input Devices.” ACM
`Transactions on Computer-Human Interaction, 1:3-26 (Mar. 1994).
`Kinkley et al., “Touch-Sensing Input Devices,” in CHI '99 Proceed
`ings, pp. 223-230, 1999.
`KionX “KXP84 Series Summary Data Sheet” copyright 2005.dated
`Oct. 21, 2005, 4-pgs.
`
`Lee et al., “A Multi-Touch Three Dimensional Touch-Sensitive Tab
`let,” in CHI '85 Proceedings, pp. 121-128, 2000.
`Lee, “A Fast Multiple-Touch-Sensitive Input Device.” Master's The
`sis, University of Toronto (1984).
`Matsushita et al., “HoloWall: Designing a Finger, Hand, Body and
`Object Sensitive Wall.” In Proceedings of UIST '97, Oct. 1997.
`Quantum Research Group “QT510 / QWheelTM Touch Slider IC”
`copyright 2004-2005, 14-pgs.
`Quek, “Unencumbered Gestural Interaction.” IEEE Multimedia,
`3:36-47 (Winter 1996).
`Radwin, "Activation Force and Travel Effects on Overexertion in
`Repetitive Key Tapping.” Human Factors, 39(1): 130-140 (Mar.
`1997).
`Rekimoto “SmartSkin: An Infrastructure for Freehand Manipulation
`on Interactive Surfaces” CHI 2002, Apr. 20-25, 2002.
`Rekimoto et al., “ToolStone: Effective Use of the Physical Manipu
`lation Vocabularies of Input Devices.” In Proc. Of UIST 2000, 2000.
`Rubine et al., “Programmable Finger-Tracking Instrument Control
`lers.” Computer Music Journal, vol. 14, No. 1 (Spring 1990).
`Rutledge et al., “Force-To-Motion Functions for Pointing.” Human
`Computer Interaction INTERACT (1990).
`Subatai Ahmad, "A Usable Real-Time 3D Hand Tracker.” Proceed
`ings of the 28' Asilomar Conference on Signals, Systems and
`Computers—Part 2 (of2), vol. 2 (Oct. 1994).
`Texas Instruments “TSC2003 I2C Touch Screen Controller Data
`Sheet SBAS 162, dated Oct. 2001, 20-pgs.
`Wellner, “The Digital Desk Calculators: Tangible Manipulation on a
`Desk Top Display” in ACMUIST '91 Proceedings, pp. 27-34, Nov.
`1991.
`Williams, "Applications for a Switched-Capacitor Instrumentation
`Building Block” Linear Technology Application Note 3, Jul. 1985,
`pp. 1-16.
`Yamada et al., “A Switched-Capacitor Interface for Capacitive Pres
`Sure Sensors' IEEE Transactions on Instrumentation and Measure
`ment, vol. 41, No. 1, Feb. 1992, pp. 81-86.
`Yeh et al., “Switched Capacitor Interface Circuit for Capacitive
`Transducers' 1985 IEEE.
`Zhai et al., “Dual Stream Input for Pointing and Scrolling.” Proceed
`ings of CHI '97 Extended Abstracts (1997).
`Zimmerman et al., “Applying Electric Field Sensing to Human
`Computer Interfaces.” In CHI '85 Proceedings, pp. 280-287, 1995.
`U.S. Appl. No. 10/774,053, filed Feb. 5, 2004.
`.S. Appl. No. 1 1/140,529, filed May 27, 2005 which is a Reissue of
`570,557 listed above.
`.S. Appl. No. 1 1/381.313, filed May 2, 2006 entitled “Multipoint
`ouch Surface Controller'.
`.S. Appl. No. 1 1/332,861, filed Jan. 13, 2006 which is a Reissue of
`,677.932 listed above.
`.S. Appl. No. 1 1/380,109, filed Apr. 25, 2006 entitled “Keystroke
`actility Arrangement on Smooth Touch Surface.”
`.S. Appl. No. 1 1/428,501, filed Jul. 3, 2006 entitled “Capacitive
`ensing Arrangement,” which is a Continuation of US 2005/0104867
`listed above.
`U.S. Appl. No. 1 1/428,503, filed Jul. 3, 2006 entitled “Touch Sur
`face’ which is a Continuation of US 2005, 0104867 listed above.
`U.S. Appl. No. 1 1/428,506, filed Jul. 3, 2006 entitled “User Interface
`Gestures which is a Continuation of US 2005, 0104867 listed above.
`U.S. Appl. No. 1 1/428.515, filed Jul. 3, 2006 entitled “User Interface
`Gestures which is a Continuation of US 2005, 0104867 listed above.
`U.S. Appl. No. 1 1/428,522, filed Jul. 3, 2006 entitled “Identifying
`Contacts on a Touch Surface' which is a Continuation of US 2005/
`0.104867 listed above.
`U.S. Appl. No. 1 1/428,521, filed Jul. 3, 2006 entitled “Identifying
`Contacts on a Touch Surface' which is a Continuation of US 2005/
`0.104867 listed above.
`U.S. Appl. No. 1 1/426,078, filed Jun. 23, 2006 entitled “Electronic
`Device Having Display and Surrounding Touch Sensitive Bezel for
`User Interface and Control' which is a Continuation-In-Part of 2006/
`O197753 listed above.
`
`Page 4 of 33
`
`

`

`US 7,663,607 B2
`Page 5
`
`U.S. Appl. No. 1 1/278,080, filed Mar. 30, 2006 entitled “Force Imag
`ing Input Device and System”.
`U.S. Appl. No. 1 1/382,402, filed May 9, 2006 entitled “Force and
`Location Sensitive Display” which is a Continuation of U.S. Appl.
`No. 1 1/278,080 listed above.
`International Search Report received in corresponding PCT applica
`tion No. PCT/US2006/008349 Oct. 6, 2006.
`
`International search report for International Application No. PCT/
`US2005/014364 mailed Jan. 12, 2005.
`U.S. Appl. No. 12/267,522, filed Nov. 7, 2008, by Hotelling et al.
`U.S. Appl. No. 12/267,532, filed Nov. 7, 2008, by Hotelling et al.
`U.S. Appl. No. 12/267,540, filed Nov. 7, 2008, by Hotelling et al.
`* cited by examiner
`
`Page 5 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 1 of 14
`
`US 7.663,607 B2
`
`FG. 1A
`
`
`
`FIG. 1B
`
`Row projection
`Y-1 signal
`
`S Column projection
`Signal
`
`Page 6 of 33
`
`

`

`U.S. Patent
`US. Patent
`
`Feb. 16, 2010
`Feb. 16, 2010
`
`Sheet 2 of 14
`Sheet 2 0f 14
`
`US 7.663,607 B2
`US 7,663,607 B2
`
`30
`30
`
`S.
`S4
`Y
`S.
`s3 \
`S.
`S2
`Ya
`
`
`
`S,
`S1
`
`\
`
`38
`38
`
`Page 7 of 33
`
`Page 7 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 3 of 14
`
`US 7.663,607 B2
`
`38
`
`
`
`
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`|
`
`||||||||||1||
`|
`|
`|
`|
`|
`|
`|
`|
`|
`|
`|
`I
`SY
`|| ||
`T4 || ||
`-7 IV
`1 N
`44
`| RF
`FSSI)
`NY
`YaY
`
`44
`
`2N
`
`44
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`44
`
`
`
`42A
`
`Page 8 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 4 of 14
`
`US 7.663,607 B2
`
`45-N
`
`RECEIVE MULTIPLE
`TOUCHES ON
`SURFACE OF TOUCH
`SCREEN
`
`RECOGNIZE EACH OF
`THE MULTIPLE
`TOUCHES
`
`REPORT TOUCH DATA
`BASED ON MULTIPLE
`TOUCHES
`
`FIG. 4
`
`
`
`FIG. 5
`
`Page 9 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 5 of 14
`
`US 7.663,607 B2
`
`102
`
`108
`
`?
`
`w
`
`y 3.
`Eas 44%-444-44444.
`E4 as 22
`
`
`
`
`
`
`
`
`
`
`
`SERADATA
`BUS
`
`106
`
`
`
`Page 10 of 33
`
`

`

`U.S. Patent
`U.S. Patent
`
`Feb. 16, 2010
`Feb. 16, 2010
`
`Sheet 6 of 14
`Sheet6of14
`
`US 7.663,607 B2
`US 7,663,607 B2
`
`
`
`
`
`mm?
`
`#3
`
`Page 11 0f33
`
`Page 11 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 7 of 14
`
`US 7.663,607 B2
`
`150 -
`
`156
`
`
`
`190
`
`-17
`
`174
`
`18O
`182
`176
`178
`
`172
`
`FIG. 10
`
`Page 12 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 8 of 14
`
`US 7.663,607 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`/ 200
`
`(3)
`
`(3)
`
`(6)
`
`NSNNNNNNNNNNNNNNNNNNNNNESNNSNSNNNN
`NNSNSNNNNNNNSNNNNNNNNNNNNNN
`
`2O6
`
`&N NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
`NNNNNNNSNSNNNNNNNNNNNNNNNNNNNNNNNNNN &N NNNNNNNNNNNNNNSNSNNNNNNNNNNNNNNNNN
`
`NSNNSNNSNSNNNNNNNSNSNNNNNNSNNNNNN
`NNNNNNNNNNNNNNNNNNNNNNNSN
`NNNSNNNNNNNNNSNNNNNNNNNNNNNS
`S
`O SSN
`NSNNNNNNNNNNNNNSNNNNNNNNNNNN
`NNNNNNNNNNNSNNNNNNNNNNNNNNNNNNNNNSNS
`NSNSNNNNNNNNNNNNNNNNNNNNNNNNNN
`
`204
`
`NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN &N NNNNSNNNNNNNSNSNNNNNNNNNNSNSN
`
`NNNNNNN IN INNNNNNNNNNNNNNNNNNNNSNNNNN
`
`NX
`
`FIG. 11A
`
`
`
`
`
`2
`
`
`
`
`
`
`
`i 1
`
`2
`
`
`
`
`
`
`
`2
`
`Z.
`
`204
`
`Page 13 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 9 of 14
`
`US 7.663,607 B2
`
`22O
`
`
`
`232
`
`IN
`
`OUT
`
`FIG. 13
`
`Page 14 of 33
`
`

`

`U.S. Patent
`US. Patent
`
`Feb. 16, 2010
`Feb. 16, 2010
`
`Sheet 10 of 14
`Sheet 10 0f 14
`
`US 7.663,607 B2
`US 7,663,607 B2
`
`FIG.14
`262
`
`264
`
`e
`
`
`
`s
`260
`
`266
`
`Page 15 0f33
`
`
`
`CHANNEL
`
`(N+1)
`
`Page 15 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 11 of 14
`
`US 7.663,607 B2
`
`280 Y
`
`DRIVE SENSING POINTS
`
`READ OUTPUT FROM SENSING
`POINTS
`
`PRODUCE AND ANALYZE
`TOUCHSCREEN DATA
`
`COMPARE CURRENT DATA TO PAST
`DATA
`
`
`
`
`
`
`
`
`
`PERFORMACTION BASED ON
`COMPARISON
`
`FIG. 15
`
`282
`
`284
`
`286
`
`288
`
`290
`
`Page 16 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 12 of 14
`
`US 7.663,607 B2
`
`-
`
`RECEIVE RAW DATA
`
`FILTERRAW DATA
`
`
`
`
`
`
`
`
`
`GENERATEGRADENT DATA
`
`CALCULATE BOUNDARES FOR TOUCH
`REGIONS
`
`CALCULATE COORDINATES FOREACH
`TOUCH REGION
`
`PERFORM MULTIPOINT TRACKING
`
`FIG. 16
`
`302
`
`304
`
`306
`
`308
`
`310
`
`312
`
`Page 17 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 13 of 14
`
`US 7.663,607 B2
`
`
`
`
`
`FIG. 17B
`GRADIENT DATA
`
`FIG. 17C
`
`TOUCH REGIONS
`
`FIG. 17D
`
`COORDINATES OF TOUCH REGIONS
`
`a=15.00 p=121.93
`x=172.04, y=234.237288
`
`a.
`a=33.00 p=133,97
`x=707.07.04, y=331.323230
`
`a=9.00 p=113.33
`x=417.29, y=333.666667
`
`s
`a=35.00 p=133.74
`x=290.16, y=570.155950
`
`FIG. 17E
`
`Page 18 of 33
`
`

`

`U.S. Patent
`
`Feb. 16, 2010
`
`Sheet 14 of 14
`
`US 7.663,607 B2
`
`350
`1.
`
`370
`
`
`
`362
`
`FIG. 18
`
`FIG. 19
`
`Page 19 of 33
`
`

`

`US 7,663,607 B2
`
`1.
`MULTIPOINT TOUCHSCREEN
`
`BACKGROUND OF THE INVENTION
`
`10
`
`15
`
`1. Field of the Invention
`The present invention relates generally to an electronic
`device having a touch screen. More particularly, the present
`invention relates to a touchscreen capable of sensing multiple
`points at the same time.
`2. Description of the Related Art
`There exist today many styles of input devices for perform
`ing operations in a computer system. The operations gener
`ally correspond to moving a cursor and/or making selections
`on a display Screen. By way of example, the input devices
`may include buttons or keys, mice, trackballs, touchpads, joy
`Sticks, touchscreens and the like. Touchscreens, in particular,
`are becoming increasingly popular because of their ease and
`Versatility of operation as well as to their declining price.
`Touch screens allow a user to make selections and move a
`cursor by simply touching the display screen via a finger or
`stylus. In general, the touch screen recognizes the touch and
`position of the touch on the display Screen and the computer
`system interprets the touch and thereafter performs an action
`based on the touch event.
`Touch screens typically include a touch panel, a controller
`and a software driver. The touch panel is a clear panel with a
`touch sensitive surface. The touch panel is positioned in front
`of a display screen so that the touch sensitive Surface covers
`the viewable area of the display screen. The touch panel
`registers touch events and sends these signals to the control
`ler. The controller processes these signals and sends the data
`to the computer system. The software driver translates the
`touch events into computer events.
`There are several types of touch screen technologies
`including resistive, capacitive, infrared, Surface acoustic
`wave, electromagnetic, near field imaging, etc. Each of these
`devices has advantages and disadvantages that are taken into
`account when designing or configuring a touch screen. In
`resistive technologies, the touch panel is coated with a thin
`metallic electrically conductive and resistive layer. When the
`panel is touched, the layers come into contact thereby closing
`a switch that registers the position of the touch event. This
`information is sent to the controller for further processing. In
`capacitive technologies, the touch panel is coated with a
`material that stores electrical charge. When the panel is
`45
`touched, a small amount of charge is drawn to the point of
`contact. Circuits located at each corner of the panel measure
`the charge and send the information to the controller for
`processing.
`In Surface acoustic wave technologies, ultrasonic waves
`are sent horizontally and vertically over the touch screen
`panel as for example by transducers. When the panel is
`touched, the acoustic energy of the waves are absorbed. Sen
`sors located across from the transducers detect this change
`and send the information to the controller for processing. In
`infrared technologies, light beams are sent horizontally and
`vertically over the touch panel as for example by light emit
`ting diodes. When the panel is touched, some of the light
`beams emanating from the light emitting diodes are inter
`rupted. Light detectors located across fr