111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US007844914 B2
`
`c12) United States Patent
`Andre et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 7,844,914 B2
`Nov.30,2010
`
`(54) ACTIVATING VIRTUAL KEYS OF A
`TOUCH-SCREEN VIRTUAL KEYBOARD
`
`(75)
`
`Inventors: Bartley K. Andre, Menlo Park, CA
`(US); Bas Ording, San Francisco, CA
`(US); Greg Christie, San Jose, CA (US)
`
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 333 days.
`
`(21) Appl. No.: 11/228,737
`
`(22) Filed:
`
`Sep.16, 2005
`
`(65)
`
`Prior Publication Data
`
`US 2006/0085757 AI
`
`Apr. 20, 2006
`
`Related U.S. Application Data
`(63) Continuation-in-part of application No. 10/903,964,
`filed on Jul. 30, 2004.
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 31048
`(52) U.S. Cl. oooooooooooooooooooooooooooooooooooooo. 7151773; 715/863
`(58) Field of Classification Search 00000000000000000 715/863,
`7151771, 773; 345/173
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3,333,160 A
`3,541,541 A
`3,662,105 A
`3,798,370 A
`4,246,452 A
`
`7/1967 Gorski
`11/1970 Englehart
`5/1972 Hurst eta!. 0 00 00 00 00 00 0000 000 0 178/18
`3/1974 Hurst 00000000000000000000 0 0 0 0 0 0 178/18
`1/1981 Chandler 0000 00 000000 000000 00 0 0 . 200/5
`
`4,550,221 A
`4,672,364 A
`4,672,558 A
`
`10/1985 Mabusth 0000000000000000000000 178/18
`6/1987 Lucas 0000000000000000000000 340/365 P
`6/1987 Beckes et al. 0 00 0 00 00 0 00 0 00 0 364/518
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CA
`
`1243096
`
`10/1988
`
`00 00 00 00 00 00 00 00 0 340/180
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`U.S. Appl. No. 10/840,862, filed May 6, 2004.
`
`(Cont,inued)
`
`Primary Examiner-Boris Pesin
`Assistant Examiner--Omar Abdul-Aii
`(74) Attorney, Agent, or Firm-Morgan, Lewis & Bockius
`LLP
`
`(57)
`
`ABSTRACT
`
`A method of operating a touch screen to activate one of a
`plurality of virtual keys is provided. A touch location is deter(cid:173)
`mined based on location data pertaining to touch input on the
`touch screen, wherein the touch input is intended to activate
`one of the plurality of virtual keys. Each of the plurality of
`virtual keys has a set of at least one key location correspond(cid:173)
`ing to it. For each of the virtual keys, a parameter (such as
`physical distance) is determined for that virtual key that
`relates the touch location and the set of at least one key
`location corresponding to that virtual key. The determined
`parameters are processed to determine one of the virtual keys.
`For example, the determined one virtual key may be the
`virtual key with a key location (or more than one key location,
`on average) being closest to the touch location. A signal is
`generated indicating activation of the determined one of the
`virtual keys.
`
`49 Claims, 6 Drawing Sheets
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`5021:!
`("m1
`
`EXHIBIT 1012
`IPR Petition for U.S. Patent No. 8,059,015
`
`

`

`US 7,844,914 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`4,692,809 A
`4,695,827 A
`4,733,222 A
`4,734,685 A
`4,746,770 A
`4,771,276 A
`4,788,384 A
`4,806,846 A
`4,898,555 A
`4,968,877 A
`5,003,519 A
`5,017,030 A
`5,178,477 A
`5,189,403 A
`5,194,862 A
`5,224,861 A
`5,241,308 A
`5,252,951 A
`5,281,966 A
`5,305,017 A
`5,345,543 A
`5,376,948 A
`5,398,310 A
`5,442,742 A
`5,463,388 A
`5,463,696 A
`5,483,261 A
`5,488,204 A
`5,495,077 A
`5,513,309 A
`5,523,775 A
`5,530,455 A
`5,543,590 A
`5,543,591 A
`5,563,632 A
`5,563,996 A
`5,565,658 A
`5,579,036 A
`5,581,681 A
`5,583,946 A
`5,590,219 A
`5,592,566 A
`5,594,810 A
`5,596,694 A
`5,612,719 A
`5,627,567 A *
`5,631,805 A
`5,633,955 A
`5,634,102 A
`5,636,101 A
`5,642,108 A
`5,644,657 A
`5,666,113 A
`5,666,502 A
`5,666,552 A
`5,675,361 A
`5,677,710 A
`5,689,253 A
`5,710,844 A
`5,729,250 A
`5,730,165 A
`5,736,976 A
`5,740,390 A
`5,741,990 A
`5,745,116 A
`5,745,716 A
`5,746,818 A
`5,748,269 A
`5,748,512 A
`5,757,358 A *
`5,764,222 A
`
`911987 Beining eta!. .............. 3581247
`911987 Beining eta!. ........... 3401365 P
`311988 Evans ... ... ... .. ... ... ... . 3401365 C
`311988 Watanabe ................... 340/710
`511988 McAvinney .................. 178118
`911988 Parks ......................... 340/712
`1111988 Bruere-Dawsonetal. ..... 178118
`211989 Kerber .................. 324160 CD
`211990 Sampson ..................... 445122
`1111990 McAvinney et al ......... 2501221
`311991 Noirjean ...................... 368/73
`511991 Crews ........................ 4001485
`111993 Gambaro .................... 4001489
`211993 Franzetal. ................. 340/711
`311993 Edwards .. .. ... ... ... ... ... .. . 341120
`711993 Glass et al .................... 434135
`811993 Young ......................... 341134
`1011993 Tannenbaum
`111994 Walsh ......................... 341122
`411994 Gerpheide .................. 3451174
`911994 Capps et a!.
`1211994 Roberts ...................... 3451173
`311995 Tchao eta!.
`811995 Greyson eta!.
`1011995 Boie et al ..................... 341133
`1011995 Beernink et al.
`111996 Yasutake .................... 3451173
`111996 Mead eta!. ................... 178118
`211996 Miller eta!. .................. 178118
`411996 Beier et a!.
`611996 Capps
`611996 Gillick eta!. ............... 3451163
`811996 Gillespie eta!. .............. 178118
`811996 Gillespie et a!.
`1011996 Roberts ...................... 3451173
`1011996 Tchao
`1011996 Gerpheide eta!. ............ 178119
`1111996 Yates, IV .................... 3451173
`1211996 Tchao eta!.
`1211996 Gourdol
`1211996 Gourdol
`111997 Pagallo et a!.
`111997 Gourdol
`111997 Capps
`311997 Beernink et al.
`511997 Davidson .................... 3451173
`511997 Bonsall ...................... 3611681
`511997 Bozinovic et a!.
`511997 Capps
`611997 Bonsall eta!. .............. 3611681
`611997 Gopher eta!. ................ 341122
`711997 Capps eta!.
`911997 Logan .. ... ... .. ... ... ... ... .. . 341134
`911997 Capps
`911997 Greyson et a!.
`1011997 Santilli ....................... 3451168
`1011997 Thompson-Rohrlich
`1111997 Hargreaves eta!. ........... 341122
`111998 Capps et a!.
`311998 Bishop eta!. ............... 3451175
`311998 Philipp .......................... 13711
`411998 Cheung ...................... 3451168
`411998 Pickover et al .............. 3951348
`411998 Davies . ... .. ... ... ... ... ... 841423 R
`411998 Pisutha-Arnond
`411998 Tchao eta!.
`511998 Yatake
`.................... 106131.86
`511998 Harris eta!. .................. 349158
`511998 Vargas
`511998 Osga .......................... 7151862
`611998 Shieh ......................... 3451173
`
`5,764,818 A
`5,767,457 A
`5,767,842 A
`5,790,104 A
`5,790,107 A
`5,798,760 A
`5,802,516 A
`5,805,165 A *
`5,808,567 A
`5,809,267 A
`5,821,690 A
`5,821,930 A
`5,823,782 A
`5,825,308 A
`5,825,351 A
`5,825,352 A
`5,854,625 A
`5,874,948 A
`5,877,751 A
`5,880,411 A
`5,898,434 A
`5,920,309 A
`5,923,319 A
`5,933,134 A
`5,943,044 A
`5,963,671 A
`6,002,389 A
`6,002,808 A
`6,020,881 A
`6,028,271 A
`6,031,524 A
`6,037,882 A
`6,049,326 A
`6,050,825 A
`6,052,339 A
`6,072,494 A
`6,084,576 A
`6,104,384 A *
`6,107,997 A
`6,128,003 A
`6,131,299 A
`6,135,958 A
`6,144,380 A
`6,188,391 B1
`6,198,515 B1
`6,208,329 B1
`6,222,465 B1
`6,239,790 B1
`6,243,071 B1
`6,246,862 B1
`6,249,606 B1
`6,259,436 B1 *
`6,278,443 B1
`6,288,707 B1
`6,289,326 B1
`6,292,178 B1
`6,323,846 B1
`6,347,290 B1
`6,377,009 B1
`6,380,931 B1
`6,411,283 B1
`6,411,287 B1
`6,414,671 B1
`6,421,234 B1
`6,452,514 B1
`6,457,355 B1
`6,466,036 B1
`6,515,669 B1
`6,525,749 B1
`6,535,200 B2
`6,543,684 B1
`6,543,947 B2
`6,567,109 B1 *
`
`611998 Capps eta!.
`611998 Gerpheide eta!. ............ 178118
`611998 Korth ......................... 3451168
`811998 Shieh ......................... 3451173
`811998 Kasser eta!.
`............... 3451174
`811998 Vayda et al.
`911998 Shwarts eta!.
`911998 Thorne et al ................ 7151823
`911998 McCloud ..................... 341120
`911998 Moran eta!.
`1011998 Martens et al .............. 3131506
`1011998 Hansen ....................... 3451340
`1011998 Marcus eta!.
`.............. 4341156
`1011998 Rosenberg
`1011998 Tam ........................... 3451173
`1011998 Bisset eta!.
`................ 3451173
`1211998 Frisch eta!. ................ 3451173
`211999 Shieh
`311999 Kanemitsu et al.
`311999 Gillespie et al .......... 178118.01
`411999 Small eta!.
`................ 3451173
`711999 Bisset eta!.
`711999 Bishop eta!. ............... 3451175
`811999 Shieh ......................... 3451173
`811999 Martinelli et al ............ 3451174
`1011999 Comerford et al. . . . . . . . . . . 3 821230
`1211999 Kasser ....................... 3451173
`1211999 Freeman
`212000 Naughton eta!. ........... 3451327
`212000 Gillespie et al.
`212000 Kunert ....................... 3451173
`312000 Levy . ... ... ... ... ... .. ... ... ... 341120
`412000 Beyda eta!. ................ 3451157
`412000 Nichol eta!.
`............... 4341227
`412000 Frenkel et a!.
`.............. 3681230
`612000 Nguyen
`712000 Leu eta!. .................... 3451168
`812000 Moon eta!. ................. 3451168
`812000 Ure ............................ 3451173
`1012000 Smith eta!.
`1012000 Raab eta!. .................... 331503
`1012000 Mikula-Curtis eta!. ..... 6001443
`1112000 Shwarts eta!.
`212001 Seely eta!. ................. 3451173
`312001 Cole .......................... 3481836
`312001 Ballare ....................... 3451173
`412001 Kumar eta!.
`512001 Martinelli et al ............ 3451174
`612001 Shwarts eta!.
`612001 Grivas et al ................... 455190
`612001 Kiraly et al.
`712001 Moon eta!. ................. 3451173
`812001 Arnro eta!. ................. 3451173
`912001 Philipp ....................... 3451168
`912001 LaFleur ...................... 705/702
`912001 Bernstein eta!. ............ 3451173
`1112001 Westerman eta!.
`212002 Bartlett
`412002 Philipp ....................... 3181468
`412002 Gillespie et al.
`612002 Murphy ...................... 3451173
`612002 Scharff eta!. ............... 3451177
`712002 Gillespie et al.
`712002 Ricks eta!. ................. 3611683
`912002 Philipp . ... ... ... ... .. ... ... ... 341133
`1012002 Philipp . ... ... ... ... .. ... ... ... 731304
`1012002 Philipp ....................... 3241678
`212003 Mohri ........................ 3451474
`212003 Moran eta!. ................ 3451863
`312003 Philipp ....................... 3451168
`412003 White eta!.
`................ 2341379
`412003 Lee ............................ 4001489
`512003 Todd .......................... 7151862
`
`

`

`US 7,844,914 B2
`Page 3
`
`6,570,557 Bl
`6,593,916 Bl
`6,610,936 B2
`6,624,833 Bl
`6,639,577 B2
`6,643,824 Bl
`6,650,319 Bl
`6,658,994 Bl
`6,670,894 B2
`6,677,932 Bl *
`6,677,934 Bl
`6,724,366 B2
`6,757,002 Bl
`6,803,906 Bl
`6,842,672 Bl
`6,856,259 Bl
`6,888,536 B2
`6,900,795 Bl
`6,927,761 B2
`6,938,220 Bl
`6,942,571 Bl
`6,965,375 Bl
`6,972,401 B2
`6,977,666 Bl
`6,985,801 Bl
`6,992,659 B2
`7,031,228 B2
`7,277,088 B2
`7,352,365 B2
`7,489,306 B2
`7,605,804 B2
`2002/0051018 AI
`2002/0118848 AI
`2003/0006974 Al
`2003/0063073 Al
`2003/0076301 Al
`2003/0076303 Al
`2003/0076306 Al
`2003/0095095 Al
`2003/0095096 Al
`2003/0098858 Al
`2003/0193481 Al
`2003/0206202 Al
`2003/0234768 Al
`2004/0021644 Al
`2004/0053661 Al
`2004/0134238 Al
`2004/0136564 Al
`2004/0155869 Al *
`2004/0183833 AI *
`2004/0263484 Al
`2005/0012723 AI
`2005/0027622 Al *
`2005/0052425 Al
`2005/0104867 Al
`2005/0110768 Al
`2005/0116927 Al *
`2005/0134578 Al
`2005/0190970 Al
`2006/0007174 Al
`2006/0022955 Al
`2006/0022956 Al
`2006/0026521 Al
`2006/0026535 Al
`2006/0026536 Al
`2006/0032680 Al
`2006/0033724 Al
`2006/0053387 Al
`2006/0066582 Al
`2006/00857 57 AI
`2006/0097991 Al
`2006/0197753 Al
`
`5/2003 Westerman et al.
`7/2003 Aroyan ....................... 345/173
`8/2003 Gillespie et a!.
`9/2003 Kumar et al.
`10/2003 Eberhard .................... 345/102
`1112003 Bates et al ............... 715/501.1
`1112003 Hurst et al .................. 345/173
`12/2003 McMillan .................... 99/468
`12/2003 Mehring ...................... 341122
`112004 Westerman ................. 345/173
`112004 Blanchard ................... 345/173
`4/2004 Crawford .................... 345/157
`6/2004 Oross eta!. ................. 345/864
`10/2004 Morrison et al ............. 345/173
`112005 Straub et al .................... 70113
`2/2005 Sharp ............................ 34115
`5/2005 Westerman et al.
`5/2005 Knight, III eta!. .......... 345/173
`8/2005 Badaye eta!. .............. 345/173
`8/2005 Shigematsu eta!. ......... 715/863
`9/2005 McAllister eta!. ............ 463/20
`1112005 Gettemy eta!. ............. 345/173
`12/2005 Akitt eta!. .................. 250/221
`12/2005 Hedrick ...................... 345/690
`112006 Straub et al .................... 70113
`112006 Gettemy ..................... 345/173
`4/2006 Born eta!. .................... 368/69
`10/2007 Robinson eta!. ............ 345/173
`4/2008 Trachte ...................... 345/173
`212009 Kolmykov-Zotov eta!. . 345/173
`10/2009 Wilson ....................... 345/173
`5/2002 Yeh ............................ 345/784
`8/2002 Karpenstein ................ 3811119
`112003 Clough et al ................ 345/179
`4/2003 Geaghan eta!. ............. 345/173
`4/2003 Tsuk eta!. .................. 345/159
`4/2003 Huppi ........................ 345/163
`4/2003 Zadesky eta!. ............. 345/173
`5/2003 Pihlaja ....................... 345/156
`5/2003 Robbin et al ................ 345/156
`5/2003 Perski eta!. ................ 345/173
`10/2003 Sokolsky
`1112003 Moriya ....................... 345/846
`12/2003 Rekimoto eta!. ........... 345/169
`2/2004 Enomoto
`3/2004 Jones et al.
`7/2004 Buckroyd et al.
`7/2004 Roeber et a!.
`8/2004 Robinson eta!. ............ 345/168
`9/2004 Chua .......................... 345/773
`12/2004 Montysalo eta!. .......... 345/173
`112005 Pallakoff .................... 345/173
`2/2005 Walker et al .................. 705/28
`3/2005 Zadesky eta!. ............. 345/173
`5/2005 Westerman et al .......... 345/173
`5/2005 Marriott et al.
`.. ........... 345/173
`6/2005 Voelckers ................... 345/157
`6/2005 Chambers et al.
`9/2005 Griffin ........................ 382/209
`112006 Shen .......................... 345/173
`2/2006 Kennedy .................... 345/173
`2/2006 Lengeling et al ............ 345/173
`2/2006 Hotelling eta!. ............ 715/702
`2/2006 Hotelling eta!. ............ 715/863
`2/2006 Hotelling eta!. ............ 715/863
`2/2006 Elias eta!. ............... 178/18.06
`2/2006 Chaudhri et al ............. 345/173
`3/2006 Ording ....................... 715/773
`3/2006 Lyon eta!. .................. 345/173
`4/2006 Andre eta!. ................ 715/771
`5/2006 Hotelling eta!. ............ 345/173
`9/2006 Hotelling .................... 345/173
`
`2009/0284471 Al * 1112009 Longe et al ................. 345/168
`
`FOREIGN PATENT DOCUMENTS
`
`DE
`DE
`DE
`DE
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`GB
`GB
`JP
`JP
`KR
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`wo
`
`10042300 Al
`100 59 906 Al
`101 40 874 Al
`102 51 296
`0 288 692
`0 664 504
`0 464 908
`1 014 295
`1 271 295 A2
`1 569 079 Al
`1505484 Al
`1 674 976 A2
`2344894 A
`2 351 639 A
`10039748
`2000163193
`4057131 A
`97/18547
`97/23738
`98/14863
`wo 98/14863
`99/28813 Al
`99/38149 Al
`99/54807 Al
`00/38042 Al
`wo 00/38042
`01102949 Al
`02/01482 Al
`03/088176
`03/098421 Al
`2005/114369 A2
`2006/023569
`2006/026012 A2
`
`3/2002
`6/2002
`3/2003
`5/2004
`7/1993
`111995
`9/1996
`1/2002
`112003
`8/2005
`9/2005
`6/2006
`6/2000
`1/2001
`2/1998
`6/2006
`7/2004
`5/1997
`7/1997
`4/1998
`4/1998
`6/1999
`7/1999
`10/1999
`6/2000
`6/2000
`1/2001
`1/2002
`10/2003
`1112003
`12/2005
`3/2006
`3/2006
`
`OTHER PUBLICATIONS
`
`U.S. Appl. No. 10/903,964, filed Jul. 30, 2004.
`U.S. Appl. No. 10/927,925, filed Aug. 26, 2004.
`U.S. Appl. No. 111048,264, filed Jan. 31, 2005.
`"Touch Technologies Overview." 3M Touch Systems, Massachusetts
`2001.
`Hardy, Ian. "Fingerworks." Mar. 7, 2002.
`"Symbol Commander" (printed Aug. 30, 2005) http://www.sensiva.
`corn/symbolcommander.
`"Mouse Gestures in Opera" (printed Apr. 20, 2004) http://www.
`opera.com.features/mouse/.
`"A Brief Overview of Gesture Recognition" (printed Apr. 20, 2004)
`http://www.dai.ed.ac.uk/Cvonline/LOCAL_COPIES/COHEN/
`gesture_overview.htrnl.
`Rekimoto, Jun. "SmartSkin: An Infrastructure for Freehand Manipu(cid:173)
`lation on Interactive Surfaces." CHI 2002. Apr. 20-25, 2002.
`"Mouse Gestures." Optim oz. May 21, 2004.
`"iGesture Products for Everyone (Learn in Minutes)." FingerWorks.
`Aug. 30, 2005.
`"MultiTouch Overview." FingerWorks. (printed Aug. 30, 2005)
`http://www.fingerworks.com/multoveriew.htrnl.
`"Gesture Recognition." (printed Aug. 30, 2005) http://www.
`fingerworks.corn/gesture_recognition.htrnl.
`"Tips for Typing." FingerWorks. (printed Aug. 30, 2005)http:/ /www.
`fingerworks.corn/mini_typing.htrnl.
`"Mouse Emulation." FingerWorks. (printed Aug. 30, 2005).
`"Touch Technologies: Touch is Everywhere." (printed Aug. 30, 2005)
`http://www.3m.com/3MTouchSystems/downloads/PDFs/
`TouchTechOV.pdf.
`U.S. Appl. No. 10/654,108, filed Sep. 2, 2003 entitled "Ambidextrous
`Mouse".
`U.S. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled "Shape
`Detecting Input Device".
`
`

`

`US 7,844,914 B2
`Page 4
`
`"4-Wire Resistive Touchscreens" obtained from http://www.
`touchscreens.corn/intro-touchtypes-4resistive.htrnl generated Aug.
`5, 2005.
`"5-Wire Resistive Touchscreens" obtained from http://www.
`touchscreens.corn/intro-touchtypes-resistive.html generated Aug. 5,
`2005.
`"A Brief Overview of Gesture Recognition" obtained from http://
`www.dai.ed.ac.uk/Cvonline/LOCA_COPIES/COHEN/
`gesture_overview.htrnl, generated Apr. 20, 2004.
`"Capacitive Touchscreens" obtained from http:/ /www.touchscreens.
`corn/intro-touchtypes-capacitive.htrnl generated Aug. 5, 2005.
`"Capacitive Position Sensing" obtained from http:/ /www.synaptics.
`corn/technology/cps.cfm generated Aug. 5, 2005.
`"Comparing Touch Technologies" obtained from http://www.
`touchscreens.corn/intro-touchtypes.htrnl generated Oct. 10, 2004.
`http:/ /www.fingerworks.com/
`"Gesture
`Recognition"
`gesture_recognition.htrnl.
`"GlidePoint®" obtained from http:/ /www.cirque.com/technology/
`technology_gp.htrnl generated Aug. 5, 2005.
`"How do touchscreen monitors know where you're touching?"
`obtained
`from http:/ /www.electronics.howstuffWorks.com/ques(cid:173)
`tion716.htrnl generated Aug. 5, 2005.
`"How does a touchscreen work?" obtained from http://www.
`touchscreens.corn/intro-anatomy.htrnl generated Aug. 5, 2005.
`"iGesture Products for Everyone (learn in minutes) Product Over(cid:173)
`view" FingerWorks.com.
`"Infrared Touchscreens" obtained from http:/ /www.touchscreens.
`corn/intro-touchtypes-infrared.htrnl generated Aug. 5, 2005.
`"Mouse Emulation" FingerWorks obtained from http://www.
`fingerworks.corn/gesture_guide_mouse.htrnl generated Aug. 30,
`2005.
`"Mouse Gestures in Opera" obtained from http://www.opera.com/
`products/desktop/mouse/index.dml generated Aug. 30, 2005.
`"Mouse Gestures," Optim oz, May 21, 2004.
`"MultiTouch Overview" FingerWorks obtained from http://www.
`fingerworks.corn/multoverview.htrnl generated Aug. 30, 2005.
`"Near Field Imaging Touchscreens" obtained from http://www.
`touchscreens.corn/intro-touchtypes-nfi.htrnl generated Aug. 5, 2005.
`"PenTouch Capacitive Touchscreens" obtained from http://www.
`touchscreens.corn/intro-touchtypes-pentouch.htrnl generated Aug.
`5, 2005.
`"Surface Acoustic Wave Touchscreens" obtained from http://www.
`touchscreens.corn/intro-touchtypes-saw.htrnl generated Aug. 5,
`2005.
`"Symbol Commander" obtained from http:/ /www.sensiva.com/
`symbolcomander/, generated Aug. 30, 2005.
`for Typing" FingerWorks http://www.fingerworks.com/
`"Tips
`mini_typing.htrnl generated Aug. 30,2005.
`"Touch Technologies Overview" 2001, 3M Touch Systems, Massa(cid:173)
`chusetts.
`"Wacom Components-Technology" obtained from http://www.
`wacom-components.corn/english/tech.asp generated on Oct. 10,
`2004.
`"Watershed Algorithm" http:/ /rsb.info .nih. gov/ij/plugins/watershed.
`htrnl generated Aug. 5, 2005.
`"FingerWorks---Gesture Guide-Application Switching," obtained
`from http:/ /www.fingerworks.com/ gesture_guide_apps.htrnl, gen(cid:173)
`erated on Aug. 27, 2004, 1-pg.
`"FingerWorks---Gesture Guide-Editing," obtained from http://
`www.fingerworks.com/gesure_guide_editing.htrnl, generated on
`Aug. 27, 2004, 1-pg.
`"FingerWorks---Gesture Guide-File Operations," obtained from
`http://www.fingerworks.com/gesture_guide_files.html, generated
`on Aug. 27,2004, 1-pg.
`"FingerWorks---Gesture Guide-Text Manipulation," obtained from
`http:/ /www.fingerworks.com/ gesture_guide_text_manip.html,
`generated on Aug. 27, 2004, 2-pg.
`"FingerWorks---Gesture Guide-Tips and Tricks," obtained from
`http://www. fingerworks. corn/ gesture _guide _tips .htrnl, generated
`Aug. 27, 2004, 2-pgs.
`"FingerWorks---Gesture Guide-Web," obtained from http://www.
`fingerworks.corn/gesture_guide_web.html, generated on Aug. 27,
`2004, 1-pg.
`
`"FingerWorks---Guide to Hand Gestures for USB Touchpads,"
`obtained
`from http:/ /www.fingerworks.com/igesture_userguide.
`htrnl, generated Aug. 27,2004, 1-pg.
`"FingerWorks-iGesture-Technical Details," obtained from http:/ I
`www.fingerworks.com/igesture_tech.htrnl, generated Aug. 27,
`2004, 1-pg.
`"FingerWorks-The Only Touchpads with Ergonomic Full-Hand
`Resting and Relaxation!" obtained from http://www.fingerworks.
`corn/resting.htrnl, Copyright 2001, 1-pg.
`"Finger Works-Tips for Typing on the Mini," obtained from http:/ I
`www.fingerworks.com/mini_typing.html, generated on Aug. 27,
`2004, 2-pgs.
`"iGesture Pad-The MultiFinger USB TouchPad with Whole-Hand
`Gestures," obtained from http:/ /www.fingerworks.com/igesture.
`htrnl, generated Aug. 27,2004, 2-pgs.
`Bier, eta!., "Toolglass and Magic Lenses: The see-through interface"
`In James Kijiya, editor, Computer Graphics (SIGGRAPH '93 Pro(cid:173)
`ceedings), vol. 27, pp. 73-80, Aug. 1993.
`Douglas et a!., The Ergonomics of Computer Pointing Devices
`(1997).
`European Search Report in EP 1 621 989 (@Beyer Weaver & Tho(cid:173)
`mas, LLP) dated Mar. 27, 2006.
`EVB Elektronik "TSOP6238 IR Receiver Modules for Infrared
`Remote Control Systems" dated Jan. 2004 1-pg.
`Fisher eta!., "Repetitive Motion Disorders: The Design of Optimal
`Rate- Rest Profiles," Human Factors, 35(2):283-304 (Jun. 1993).
`Fukumoto, eta!., "ActiveClick: Tactile Feedback for Touch Panels,"
`In CHI 2001 Summary, pp. 121-122, 2001.
`Fukumoto and Yoshinobu Tonomura, "Body Coupled Fingering:
`Wireless Wearable Keyboard," CHI 97, pp. 147-154 (Mar. 1997).
`Hardy, "Fingerworks" Mar. 7, 2002; BBC World On Line.
`Hillier and Gerald J. Lieberman, Introduction to Operations
`Research (1986).
`International Search Report dated Mar. 3, 2006 (PCT/US 05/03325;
`119-0052WO).
`Jacob et a!., "Integrality and Separability of Input Devices," ACM
`Transactions on Computer-Human Interaction, 1:3-26 (Mar. 1994).
`Kinkley eta!., "Touch-Sensing Input Devices," in CHI '99 Proceed(cid:173)
`ings, pp. 223-230, 1999.
`Kionx "KXP84 Series Summary Data Sheet" copyright 2005,dated
`Oct. 21, 2005, 4-pgs.
`Lee eta!., "A Multi-Touch Three Dimensional Touch-Sensitive Tab(cid:173)
`let," in CHI '85 Proceedings, pp. 121-128, 2000.
`Lee, "A Fast Multiple-Touch-Sensitive Input Device," Master's The(cid:173)
`sis, University of Toronto (1984).
`Matsushita et al., "HoloWall: Designing a Finger, Hand, Body and
`Object Sensitive Wall," In Proceedings ofUIST '97, Oct. 1997.
`Quantum Research Group "QT510 I QWheel™ Touch Slider IC"
`copyright 2004-2005, 14-pgs.
`Quek, "Unencumbered Gestural Interaction," IEEE Multimedia,
`3:36-47 (Winter 1996).
`Radwin, "Activation Force and Travel Effects on Overexertion in
`Repetitive Key Tapping," Human Factors, 39(1):130-140 (Mar.
`1997).
`Rekimoto "SmartSkin: An Infrastructure for Freehand Manipulation
`on Interactive Surfaces" CHI 2002, Apr. 20-25, 2002.
`Rekimoto eta!., "ToolStone: Effective Use of the Physical Manipu(cid:173)
`lation Vocabularies oflnput Devices," In Proc. ofUIST 2000, 2000.
`Rubine eta!., "Programmable Finger-Tracking Instrument Control(cid:173)
`lers," Computer Music Journal, vol. 14, No. 1 (Spring 1990).
`Rutledge eta!., "Force-To-Motion Functions for Pointing," Human(cid:173)
`Computer Interaction-INTERACT (1990).
`Subatai Ahmad, "A Usable Real-Time 3D Hand Tracker," Proceed(cid:173)
`
`ings of the 281h Asilomar Conference on Signals, Systems and
`Computers-Part 2 (of2), vol. 2 (Oct. 1994).
`Texas Instruments "TSC2003 I I2C Touch Screen Controller" Data
`Sheet SBAS 162, dated Oct. 2001, 20-pgs.
`Wellner, "The Digital Desk Calculators: Tangible Manipulation on a
`Desk Top Display" in ACM UIST '91 Proceedings, pp. 27-34, Nov.
`1991.
`Williams, "Applications for a Switched-Capacitor Instrumentation
`Building Block" Linear Technology Application Note 3, Jul. 1985,
`pp. 1-16.
`
`

`

`US 7,844,914 B2
`Page 5
`
`Yamada et al., "A Switched-Capacitor Interface for Capacitive Pres(cid:173)
`sure Sensors" IEEE Transactions on Instrumentation and Measure(cid:173)
`ment, vol. 41, No. 1, Feb. 1992, pp. 81-86.
`Yeh et a!., "Switched Capacitor Interface Circuit for Capacitive
`Transducers" 1985 IEEE.
`Zhai eta!., "Dual Stream Input for Pointing and Scrolling," Proceed(cid:173)
`ings of CHI '97 Extended Abstracts (1997).
`Zimmerman et a!., "Applying Electric Field Sensing to Human(cid:173)
`Computer Interfaces," In CHI '85 Proceedings, pp. 280-287, 1995.
`U.S. Appl. No. 10/774,053, filed Feb. 5, 2004.
`U.S. Appl. No. 111140,529, filed May 27,2005 which is a Reissue of
`6,570,557.
`U.S. Appl. No. 111381,313, filed May 2, 2006 entitled "Multipoint
`Touch Surface Controller".
`U.S. Appl. No. 111332,861, filed Jan. 13,2006 which is a Reissue of
`6,677,932.
`U.S. Appl. No. 111380,109, filed Apr. 25, 2006 entitled "Keystroke
`Tactility Arrangement on Smooth Touch Surface."
`U.S. Appl. No. 111428,501, filed Jul. 3, 2006 entitled "Capacitive
`Sensing Arrangement," which is a Continuation of US 2005/
`0104867.
`U.S. Appl. No. 111428,503, filed Jul. 3, 2006 entitled "Touch Sur(cid:173)
`face" which is a Continuation of US 2005/0104867.
`U.S. Appl. No. 111428,506, filed Jul. 3, 2006 entitled "User Interface
`Gestures" which is a Continuation of US 2005/0104867.
`U.S. Appl. No. 111428,515, filed Jul. 3, 2006 entitled "User Interface
`Gestures" which is a Continuation of US 2005/0104867.
`U.S. Appl. No. 111428,522, filed Jul. 3, 2006 entitled "IdentifYing
`Contacts on a Touch Surface" which is a Continuation of US 2005/
`0104867.
`U.S. Appl. No. 111428,521, filed Jul. 3, 2006 entitled "IdentifYing
`Contacts on a Touch Surface" which is a Continuation of US 2005/
`0104867.
`U.S. Appl. No. 111426,078, filed Jun. 23, 2006 entitled "Electronic
`Device Having Display and Surrounding Touch Sensitive Bezel for
`User Interface and Control" which is a Continuation-In-Part of 2006/
`0197753.
`U.S. Appl. No. 111278,080, filed Mar. 30, 2006 entitled "Force Imag(cid:173)
`ing Input Device and System".
`U.S. Appl. No. 111382,402, filed May 9, 2006 entitled "Force and
`Location Sensitive Display" which is a Continuation of 111278,080.
`
`International Search Report in corresponding PCT application No.
`PCT/US2006/008349.
`Annex to Form PCT/ISA/206 Communication Relating to the
`Results of the Partial International Search in corresponding PCT
`Application No. PCT/US2005/025641 dated Feb. 19, 2007.
`PCT International Search Report in corresponding PCT Application
`No. PCT/US2006/031523 dated Feb. 27,2007.
`PCT International Search Report in corresponding PCT Application
`No. PCT/US2005/025657 dated Feb. 26, 2007.
`PCT International Search Report in corresponding PCT Application
`No. PCT/US2006/031527 dated Feb. 27,2007.
`PCT International Search Report in corresponding PCT Application
`No. PCT/US2006/031526 dated Feb. 14,2007.
`Schiphorst, et a!.; "Using a Gestural Interface Toolkit for Tactile
`Input to a Dynamic Virtual Space;" Conference on Human Factors in
`Computing Systems, Proceedings, Apr. 25, 2002, pp. 754-755.
`Chen, et al.; "Flowfield and Beyond: Applying Pressure-Sensitive
`Multi-Point Touchpad Interaction;" Multimedia and Expo, 2003,
`ICME '03, Proceedings, Jul. 9, 2003, pp. I-49-I52.
`Jones; "MTC Express Multi-touch Controller," Computer Music
`Journal25.1, 2001, pp. 97-99.
`Centroid, http:/ /faculty,evansville.edu/ck6/tcenters/class/centroid.
`htrnl, printed Apr. 28, 2006, 4 pages.
`International Search Report and Written Opinion for International
`Application No. PCT/US2008/050426, mailed Jun. 13, 2008.
`International Search Report and Written Opinion for International
`Application No. PCT/US2007/077645, mailed Jan. 10, 2008.
`Office Action dated Oct. 14, 2008, in the European Patent Application
`which corresponds to U.S. Appl. No. 111228,737.
`Office Action dated Sep. 8, 2009 in Australian Patent Application No.
`2006291338, which corresponds to U.S. Appl. No. 111228,737.
`Office Action dated Dec. 7, 2009, in European Patent Application No.
`08 727 399.1, which corresponds to U.S. Appl. No. 111969,796.
`Office Action dated Dec. 11, 2009, in U.S. Appl. No. 111696,703.
`Office Action dated Jun. 7, 2010, in U.S. Appl. No. 111850,015.
`Office Action dated Feb. 18, 2010, in Australian Patent Application
`No. 2007292384, which corresponds to U.S. Appl. No. 111850,015.
`Office Action dated Feb. 16, 2010, in European Patent Application
`No. 07 814 691.7, which corresponds to U.S. Appl. No. 111850,015.
`* cited by examiner
`
`

`

`U.S. Patent
`
`Nov. 30,2010
`
`Sheet 1 of 6
`
`US 7,844,914 B2
`
`108 \
`1o2q ;
`I
`o
`106q*
`
`102w
`0
`106w
`
`102e
`0
`106e
`
`102r
`0
`106r
`
`102t
`0
`106t
`
`102a
`0
`106a
`
`102s
`0
`106s
`
`102d
`0
`106d
`
`102f
`0
`106f
`
`102g
`0
`106g
`
`102z
`0
`106z
`
`102x
`0
`106x
`
`102c
`0
`106c
`
`102v
`0
`106v
`
`102b
`0
`106b
`
`For Each
`Virtual Key
`
`FIG. 1
`
`f 202
`Determine Touch
`Location
`.....
`~I
`Determine Distance
`Between Touch Location
`and Key Location
`Corresponding to
`Virtual Key
`
`{204
`
`206
`I
`---------------~---------~---
`Optional-Weight the
`_______________ J: ______________ ,
`Determined Distances
`8
`{20
`
`Compute Shortest
`Determined Distance
`~
`r2t
`0
`Generate Signal
`Indicating Virtual Key
`Based on Computed
`Shortest Determined
`Distance
`
`FIG. 2
`
`

`

`U.S. Patent
`
`Nov. 30, 2010
`
`Sheet 2 of 6
`
`US 7,844,914 B2
`
`Touch Sensitive
`Layer
`
`614
`
`16
`Screen
`
`10w:y
`
`Target
`
`~ 18 Touch Area
`20
`Mid Point of
`Touch Area
`
`FIG. 1-18
`
`12
`
`' I
`
`FIG. 1-1C
`
`

`

`U.S. Patent
`
`Nov. 30,2010
`
`Sheet 3 of 6
`
`US 7,844,914 B2
`
`708
`
`302w
`0
`
`302e
`0
`
`302r
`0
`
`302t
`0
`
`FIG. 3
`
`FIG. 3-1
`
`

`

`U.S. Patent
`
`Nov. 30, 2010
`
`Sheet 4 of 6
`
`US 7,844,914 B2
`
`dq is Shortest
`
`FIG. 4
`
`0
`
`0
`
`0
`
`0
`
`0
`
`502a
`("y")
`
`0
`
`0
`
`502b
`("h"} 0 o
`504z
`
`0
`
`0
`
`504j
`
`("i")
`
`0
`
`0
`
`0
`
`0
`
`50~~
`("m"
`
`502f
`("n")
`
`FIG. 5
`
`

`

`U.S. Patent
`
`Nov. 30, 2010
`
`Sheet 5 of 6
`
`US 7,844,914 B2
`
`From 202
`
`L...._
`
`For Each
`Key Location
`
`~
`
`, ..
`I Distance Determination I
`
`(602
`
`For Each
`Virtual Key
`
`L604
`~
`--••••~----••~-----•-•• ----w
`Weight Distances 1
`-------------~-------------!
`(606
`
`Average
`
`lr
`To Shortest
`206 or 208
`
`FIG. 6
`
`FIG. 7
`
`

`

`U.S. Patent
`
`Nov. 30, 2010
`
`Sheet 6 of 6
`
`US 7,844,914 B2
`
`804 ~ Hello, this is an arc shaped
`keyboard~
`
`802
`
`FIG. 8
`
`Keep
`
`Delete
`' . ..,.__ __ _..,
`---
`
`FIG. 9
`
`

`

`US 7,844,914 B2
`
`1
`ACTIVATING VIRTUAL KEYS OF A
`TOUCH-SCREEN VIRTUAL KEYBOARD
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a Continuation-In-Part of prior applica(cid:173)
`tion Ser. No.: 10/903,964 filed Jul. 30, 2004, from which
`priority under 35 U.S.C. §120 is claimed, which is hereby
`incorporated by reference in its entirety. This application is
`also related to the following co-pending applications: U.S.
`Ser. No.: 10/840,862, filed May 6, 2004; U.S. Ser. No.:
`11/048,264, filed Jul. 30, 2004; U.S. Ser. No.: 11/038,590,
`filed Jul. 30, 2004; U.S. Ser. No.: 11/228,758, filed Sep. 16,
`2005; and U.S. Ser. No.: 11/228,700, filed Aug. 16, 2005; all
`of which are hereby incorporated herein by reference in their
`entirety for all purposes.
`
`TECHNICAL FIELD
`
`The present invention is in the field of touch screens and, in
`particular, relates to operating a touch screen to activate one
`of a plurality of virtual keys.
`
`BACKGROUND
`
`A touch screen is a type of display screen that has a touch(cid:173)
`sensitive transparent panel covering the screen. When using a
`touch screen, a user makes a selection on the display screen by
`pointing directly to GUI objects on the screen (usually with a
`stylus or finger). The GUI objects may be considered as
`virtual keys (for example, of a keyboard). It is desirable to
`increase the usability