111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US008749507B2
`
`c12) United States Patent
`DaCosta et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,749,507 B2
`*Jun. 10,2014
`
`(54) SYSTEMS AND METHODS FOR ADAPTIVE
`INTERPRETATION OF INPUT FROM A
`TOUCH-SENSITIVE INPUT DEVICE
`
`(75)
`
`Inventors: Henry DaCosta, Montreal (CA);
`Christophe Ramstein, San Francisco,
`CA (US); Danny Grant, Laval (CA)
`
`(73) Assignee: Immersion Corporation, San Jose, CA
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis(cid:173)
`claimer.
`
`(21) Appl. No.: 13/441,108
`
`(22) Filed:
`
`Apr. 6, 2012
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2,972,140 A
`3,157,853 A
`3,220,121 A
`3,497,668 A
`3,517,446 A
`3,623,064 A
`3,902,687 A
`3,903,614 A
`3,911,416 A
`4,127,752 A
`
`2/1961 Hirsch
`1111964 Hirsch
`1111965 Cutler
`211970 Hirsch
`6/1970 Carlyon et a!.
`1111971 Kagan
`9/1975 Hightower
`9/1975 Diamond eta!.
`10/1975 Feder
`1111978 Lowthorp
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`
`111990
`0349086
`0536715 A2
`4/1993
`(Continued)
`OTHER PUBLICATIONS
`
`(65)
`
`Prior Publication Data
`
`US 2012/0194472Al
`
`Aug. 2, 2012
`
`Adelstein, "A Virtual Environment System For The Study of Human
`Arm Tremor," Ph.D. Dissertation, Dept. of Mechanical Engineering,
`MIT, Jun. 1989.
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/723,778, filed on
`Nov. 26, 2003, now Pat. No. 8,164,573.
`
`(51)
`
`(2006.01)
`
`Int. Cl.
`G09G5/00
`(52) U.S. Cl.
`USPC ........................................ 345/173; 178/18.01
`(58) Field of Classification Search
`CPC ....... G06F 3/016; G06F 3/017; G06F 3/0487;
`G06F 3/0488; G06F 2203/014
`USPC ......... 345/173, 174; 178/18.01, 18.03, 18.06;
`715/701, 702
`See application file for complete search history.
`
`102 \__
`
`104
`
`\
`
`X,Y,Z
`
`(Continued)
`
`Primary Examiner- Regina Liang
`(74) Attorney, Agent, or Firm- Kilpatrick Townsend &
`Stockton LLP
`
`ABSTRACT
`(57)
`Systems and methods for adaptively interpreting a user's
`intent based on parameters supplied by a touch-sensitive
`input device are described. In one of the methods described, a
`processor is programmed for receiving contact data from an
`input device; determining whether to output a haptic effect
`based on the contact data; and outputting the haptic effect
`based on the contact data.
`
`18 Claims, 4 Drawing Sheets
`
`106
`
`Processor
`
`108
`
`Various
`Interfaces
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 1
`
`

`
`US 8,749,507 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,160,508 A
`4,236,325 A
`4,262,549 A
`4,333,070 A
`4,464,117 A
`4,484,191 A
`4,513,235 A
`4,581,491 A
`4,599,070 A
`4,708,656 A
`4,713,007 A
`4,758,165 A
`4,772,205 A
`4,794,392 A
`4,885,565 A
`4,891,764 A
`4,926,879 A
`4,930,770 A
`4,934,694 A
`5,019,761 A
`5,022,384 A
`5,022,407 A
`5,035,242 A
`5,038,089 A
`5,078,152 A
`5,121,091 A
`5,159,159 A
`5,165,897 A
`5,175,459 A
`5,186,695 A
`5,212,473 A
`5,240,417 A
`5,271,290 A
`5,275,174 A
`5,283,970 A
`5,299,810 A
`5,302,132 A
`5,309,140 A
`5,334,027 A
`5,389,849 A
`5,436,622 A
`5,437,607 A
`5,461,711 A
`5,466,213 A
`5,488,204 A
`5,547,382 A
`5,575,761 A
`5,600,777 A
`5,638,060 A
`5,719,561 A
`5,736,978 A
`5,766,016 A
`5,785,630 A
`5,880,411 A *
`5,887,995 A
`5,889,511 A
`5,917,906 A
`5,977,867 A
`6,008,800 A
`6,067,081 A
`6,081,536 A
`6,111,577 A
`6,118,435 A *
`6,128,007 A
`6,140,987 A
`6,151,332 A
`6,154,210 A *
`6,160,489 A
`6,198,206 B1
`6,215,778 B1
`6,218,966 B1
`6,219,034 B1
`6,236,647 B1
`6,243,080 B1
`6,262,717 B1
`
`7 I 1979 Salsbury
`1211980 Hallet a!.
`411981 Schwellenbach
`611982 Barnes
`811984 Forest
`1111984 Vavra
`411985 Acklarn et a!.
`411986 Boothroyd
`711986 Hladky eta!.
`1111987 De Vries eta!.
`1211987 Alban
`7 I 1988 Tieman eta!.
`911988 Chlumsky eta!.
`1211988 Selinko
`1211989 Embach
`111990 Mcintosh
`511990 Sevrain et al.
`611990 Baker
`611990 Mcintosh
`511991 Kraft
`611991 Freels
`611991 Horch et a!.
`711991 Franklin
`811991 Szakaly
`111992 Bond
`611992 Fujiyarna
`1011992 Asher
`1111992 Johnson
`1211992 Dania! eta!.
`211993 Mangseth et a!.
`511993 Louis
`811993 Smithson eta!.
`1211993 Fischer
`111994 Cook
`211994 Aigner
`411994 Pierce
`411994 Corder
`511994 Everett, Jr. eta!.
`811994 Wherlock
`211995 Asano et al.
`711995 Gutman eta!.
`811995 Taylor
`1011995 Wang et al.
`1111995 Hogan
`111996 Mead
`811996 Yamasaki
`1111996 Hajianpour
`211997 Wang et al.
`611997 Kataoka et al.
`211998 Gonzales
`411998 Hasser eta!.
`611998 Sinclair
`7 I 1998 Bobick et a!.
`311999 Gillespie eta!. .............. 3451173
`311999 Holehan
`311999 Ong
`611999 Thronton
`1111999 Blouin
`1211999 Pryor
`512000 Hahlganss eta!.
`612000 Gorsuch eta!.
`812000 Zilles et al.
`912000 Fujita eta!. ................... 3451173
`1012000 Seybold
`1012000 Stein eta!.
`1112000 Gorsuch eta!.
`1112000 Anderson ..................... 3451173
`1212000 Perry eta!.
`312001 Saarmaa eta!.
`412001 Lomp eta!.
`412001 Goodwin et al.
`412001 Elbing eta!.
`512001 Amalfitano
`612001 Moine
`712001 Donohue et al.
`
`6,307,465 B1
`6,326,901 B1
`6,337,678 B1
`6,373,463 B1
`6,388,655 B1
`6,388,999 B1
`6,414,674 B1
`6,422,941 B1
`6,429,846 B2
`6,445,284 B1
`6,469,695 B1
`6,487,421 B2
`6,492,979 B1
`6,509,847 B1
`6,509,892 B1
`6,518,958 B1
`6,535,201 B1
`6,590,568 B1
`6,610,917 B2
`6,610,936 B2
`6,628,195 B1
`6,636,202 B2
`6,639,582 B1
`6,647,145 B1
`6,730,863 B1 *
`6,801,191 B2
`7,030,860 B1 *
`7,046,235 B2
`200110035854 A1
`200210033795 A1
`200210149561 A1
`200210149570 A1
`200210156807 A1
`200310006892 A1
`200310016211 A1
`200310022701 A1
`200310025679 A1
`200310030628 A1
`200310038776 A1
`200310048260 A1
`200310058265 A1
`200310063073 A1
`200310067449 A1
`200310071795 A1
`200310095105 A1
`200310128191 A1
`200310128192 A1
`200310151597 A1
`200310174121 A1
`200310179190 A1
`
`1012001 Kayma et al.
`1212001 Gonzales
`1/2002 Fish
`412002 Beeks
`512002 Leung
`512002 Gorsuch eta!.
`712002 Kamper et al.
`712002 Thorner et al.
`812002 Rosenberg eta!.
`912002 Cruz-Hernandez eta!.
`1012002 White
`1112002 Hess eta!.
`1212002 Kent eta!.
`112003 Anderson
`112003 Cooper et al.
`212003 Miyajima et a!.
`312003 Cooper et al.
`712003 Astala et al.
`812003 Ludwig
`812003 Gillespie et al.
`912003 Coudon
`1012003 Ishmael, Jr. et a!.
`1012003 Shrader
`1112003 Gay
`512004 Gerpheide eta!. ............ 3451174
`1012004 Mukai eta!.
`412006 Hsu eta!. ...................... 3451173
`512006 Katoh
`1112001 Rosenberg eta!.
`312002 Shahoia et a!.
`1012002 Fukumoto eta!.
`1012002 Knowles eta!.
`1012002 Die berger
`112003 Church
`112003 Woolley
`112003 Gupta
`212003 Taylor et al.
`212003 Sato et al.
`212003 Rosenberg eta!.
`312003 Matusis
`312003 Robinson eta!.
`412003 Geaghan et a!.
`412003 Yoshikawa et a!.
`412003 Baldauf et a!.
`512003 Vaananen
`712003 Strasser et a!.
`712003 van Os
`812003 Roberts eta!.
`912003 Poupyrev et a!.
`912003 Franzen
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`wo
`wo
`wo
`wo
`
`0556999 B1
`1182851 A1
`01-003664
`02-109714
`04-007371
`06-282369
`05-193862
`08-044493
`09-016318
`11-506559
`2001-350592 A
`2002-259059 A
`2003-337659
`WO 97118546 A1
`WO 02112991 A1
`wo 02127645
`WO 02131807 A1
`
`511998
`212002
`711990
`111992
`811993
`1011994
`111995
`211996
`111997
`611999
`1212001
`912002
`1112003
`511997
`212002
`412002
`412002
`
`OTHER PUBLICATIONS
`
`Adelstein, "Design and Implementation of a Force Reflecting
`Manipulandum for Manual Control research," DSC- vol. 42,
`Advances in Robotics, Edited by H. Kazerooni, pp. 1-12, 1992.
`Aukstakalnis eta!., "Silicon Mirage: The Art and Science of Virtual
`Reality," ISBN 0-938151-82-7, pp. 129-180, 1992.
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 2
`
`

`
`US 8,749,507 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Baigrie, "Electric Control Loading:A Low Cost. High Performance
`Alternative," Proceedings, pp. 247-254, Nov. 6-8, 1990.
`Bejczy et a!. "A Laboratory Breadboard System For Dual-Arm
`Teleoperation," SOAR '89 Workshop, JSC, Houston, TX, Jul. 25-27,
`1989.
`Bejczy et al., "Kinesthetic Coupling Between Operator and Remote
`Manipulator," International Computer Technology Conference, The
`American Society of Mechanical Engineers. San Francisco, CA,
`Aug. 12-15, 1980.
`Bejczy, "Sensors, Controls, and Man-Machine Interface for
`Advanced Teleoperation," Science, vol. 208, No. 4450, pp. 1327-
`1335, 1980.
`Bejczy, et a!., "Universal Computer Control System (UCCS) For
`Space Telerobots." CH 2413-3/87/0000/0316501.00 1987 IEEE,
`1987.
`Bejczy, "Generalization of Bilateral Force-Reflecting Control of
`Manipulators," Proceedings Of Fourth CISM• IFToMM, Sep. 8-12,
`1981.
`Bliss, "Optical-to-Tactile Image Conversion for the Blind," IEEE
`Transactions on Man- Machine Systems. vol. MMS-11, No. 1, Mar.
`1970.
`Brooks eta!., "Hand Controllers for Teleoperation- A State-of-the(cid:173)
`Art Technology Survey and Evaluation," JPL Publication 85-11;
`NASA-CR-175890; N85-28559, pp. 1-84, Mar. 1, 1985.
`Burdea eta!., "Distributed Virtual Force Feedback, Lecture Notes for
`Workshop on Force Display in Virtual Environments and its Appli(cid:173)
`cation to Robotic Teleoperation," 1993 IEEE International Confer(cid:173)
`ence on Robotics and Automation, pp. 25-44, May 2, 1993.
`Cadler, "Design of A Force-Feedback Touch-Introducing Actuator
`For Teleoperator Robot Control," Bachelor of Science Thesis, MIT,
`Jun. 23, 1983.
`Caldwell eta!., "Enhanced Tactile Feedback (Tele-Taction) Using a
`Multi-Functional Sensory System," 1050-4729/93, pp. 955-960,
`1993.
`"Cyberman Technical Specification," Logitech Cyberman SWIFT
`Supplement, Apr. 5, 1994.
`Eberhardt et a!., "Including Dynamic Haptic Perception by The
`Hand: System Description and Some Results," DSC-vol. 55-1,
`Dynamic Systems and Control: vol. 1, ASME 1994.
`Eberhardt et al., "OMAR-A Haptic display for speech perception by
`deaf and deaf-blind individuals," IEEE Virtual Reality Annual Inter(cid:173)
`national Symposium, Seattle, WA, Sep. 18-22, 1993.
`Fokumoto, "Active Click: Tactile Feedback For Touch Panels," ACM
`CHI200 1, Extended Abstracts, pp. 121-122, Apr. 2001.
`Force Feedback Touch Panel, Represented by CSC Division, Sales
`Department., SIXIK Corporation, Tokyo, Japan, www.srnk.co.jp.
`Gobel eta!., "Tactile Feedback Applied to Computer Mice," Interna(cid:173)
`tional Journal of Human-Computer Interaction, vol. 7, No. 1, pp.
`1-24, 1995.
`Gotow et al.,"Controlled Impedance Test Apparatus for Studying
`Human Interpretation of Kinesthetic Feedback," WA11-11:00, pp.
`332-337.
`Howe. "A Force-Reflecting Teleoperated Hand System for the Study
`of Tactile Sensing in Precision Manipulation," Proceedings of the
`1992 IEEE International Conference on Robotics and Automation,
`Nice, France, May 1992.
`IBM Technical Disclosure Bulletin. "Mouse Ball-Actuating Device
`With Force and Tactile Feedback," vol. 32, No. 98, Feb. 1990.
`Iwata, "Pen-based Haptic Virtual Environment," 0-7803-1363-1/93
`IEEE, pp. 287-292, 1993.
`Jacobsen et at., "High Performance, Dextrous Telerobotic Manipu(cid:173)
`lator With Force Reflection," Intervention/ROV'91 Conference &
`Exposition, Hollywood, Florida. May 21-23, 1991.
`Johnson, "Shape-Memory Alloy Tactile Feedback Actuator,"
`Armstrong Aerospace Medical Research Laboratory, AAMRL-TR-
`90-039, Aug. 1990.
`Jones eta!., "A perceptual analysis of stiffness," ISSN 0014-4819
`Springer International
`(Springer-Verlag); Experimental Brain
`Research, vol. 79, No. 1, pp. 150-156, 1990.
`
`Kaczmarek eta!., "Tactile Displays," Virtual Environment Technolo(cid:173)
`gies.
`Kontarinis et a!., Tactile Display of Vibratory Information in
`Teleoperation and Virtual Environments, Presence, 4(4):387-402,
`1995.
`Lake, "Cyberman from Logitech," GarneBytes, 1994.
`Kontarinis eta!., "Display of High-Frequency Tactile Information to
`Teleoperators," Telemanipulator Technology and Space Telerobot(cid:173)
`ics, Won S. Kim, Editor, Proc. SPIE vol. 2057, pp. 40-50, Sep. 7-9,
`1993.
`Marcus, "Touch Feedback in Surgery." Proceedings ofVirtual Real(cid:173)
`ity and Medicine The Cutting Edge, Sep. 8-11, 1994.
`McAffee, "Teleoperator Subsystern!Telerobot Demonsdtrator: Force
`Reflecting Hand Controller Equipment Manual," JPL D-5172, pp.
`1-50, A1-A36, 81-B5, C1-C36, Jan. 1988.
`Minsky. "Computational Haptics: The Sandpaper System for Syn(cid:173)
`thesizing Texture for a Force-Feedback Display," Ph.D. Dissertation.
`MIT, Jun. 1995.
`Ouh-Young eta!., "The Development of A Low-Cost Force Feedback
`Joystick and Its Use in the Virtual Reality Environment," Proceedings
`of the Third Pacific Conference on Computer Graphics and Applica(cid:173)
`tions. Pacific Graphics '95, Seoul, Korea, Aug. 21-24, 1995.
`Ouh-Young, "A Low-Cost Force FeedbackJoystickandlts Use in PC
`Video Garnes," IEEE Transactions on Consumer Electronics, vol. 41,
`No.3, Aug. 1995.
`Ouh-Young,"Force Display in Molecular Docking," Order No.
`9034744, p. 1-369, 1990.
`Patrick et a!., "Design and Testing of A Non-reactive. Fingertip,
`Tactile Display for Interaction with Remote Environments," Coop(cid:173)
`erative Intelligent Robotics in Space, Rui J. deFigueiredo et a!.,
`Editor, Proc. SPIE vol. 1387, pp. 215-222, 1990.
`Patrick, "Design, Construction, and Testing of a Fingertip Tactile
`Display for Interaction with Virtual and Remote Environments,"
`Master of Science Thesis, MIT, Nov. 8, 1990.
`Pimentel eta!., 'Virtual Reality: through the new looking glass, 2'•
`Edition; McGraw-Hill, ISBN 0-07-050167-X, pp. 41-202, 1994.
`Rabinowitz et a!., "Multidimensional tactile displays: Identification
`of vibratory intensity, frequency, and contactor area," Journal of The
`Acoustical Society of America, vol. 82, No.4, Oct. 1987.
`Russo, "Controlling Dissipative Magnetic Particle Brakes•in Force
`Reflective Devices," DSC-vol. 42, Advances in Robotics, pp. 63-70,
`ASME 1992.
`Russo, "The Design and Implementation of a Three Degree of Free(cid:173)
`dom Force Output Joystick," MIT libraries Archives Aug. 14, 1990,
`pp. 1-131, May 1990.
`Safe Flight Instruments Corporation, "Coaxial Control Shaker," Part
`No. C-25502, Jul. 1, 1967.
`Scannell. 'Taking a Joystick Ride,' Computer Currents. Boston Edi(cid:173)
`tion. vol. 9, No. 11. Nov. 1994.
`Shimoga, "Finger Force and Touch Feedback Issues in Dexterous
`Telemanipulation," Proceedings of Fourth Annual Conference on
`Intelligent Robotic Systems for Space Exploration, Rensselaer Poly(cid:173)
`technic Institute, Sep. 30-0ct. 1, 1992.
`Snow et al.,"Model-X Force-Reflecting-Hand-Controller," NT Con(cid:173)
`trol No. MP0-17851; JPL Case No. 5348, pp. 1-4, Jun. 15, 1989.
`SMK Corporation, "Multi-Functional Touch Panel, Force-Feedback
`Type, Developed: A Touch Panel Providing a Clicking Feeling,"
`http://www.srnk.co.jp/whatsnewe/628csc.corn!htrnl, Sep. 30, 2002.
`SMK Corporation, "Force Feedback Type Optical Touch Panel
`Developed," SMK Corporation Website, Oct. 30, 2002.
`Stanley et a!., "Computer Simulation of Interacting Dynamic
`Mechanical Systems Using Distributed Memory Parallel Proces(cid:173)
`sors," DSC-vol. 42, Advances in Robotics, pp. 55-61. ASME 1992.
`Synaptics, Inc., "Synaptics Touchpad Interfacing Guide," Second
`Edition, Downloaded Aug. 26, 2003.
`Tadros, "Control System Design for a Three Degree of Freedom
`Virtual Environment Simulator Using Motor/Brake Pair Actuators",
`MIT Archive© Massachusetts Institute ofTechnology, pp. 1-88, Feb.
`1990.
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 3
`
`

`
`US 8,749,507 B2
`Page 4
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Terry eta!., "Tactile Feedback In A Computer Mouse," Proceedings
`ofFourteenth Annual Northeast Bioengineering Conference. Univer(cid:173)
`sity of New Hampshire, Mar. 10-11, 1988.
`Wiker, "Teletouch Display Development: Phase 1 Report," Technical
`Report 1230, Naval Ocean Systems Center, SanDiego,Apr. 17, 1989.
`European Patent Office, European Search Report, European Appli(cid:173)
`cation No. 10010754, dated Nov. 8, 2010.
`Japanese Patent Office, Notice of Reasons for Rejection, Application
`No. 2006-541126, dated Oct. 13, 2009.
`Japanese Patent Office, Notice of Reasons for Rejection, Application
`No. 2006-541126, dated Oct. 7, 2008.
`Japanese Patent Office, Notice of Reasons for Rejection, Application
`No. 2010-091566, dated Mar. 13, 2012.
`
`European Patent Office, Supplemental European Search Report,
`European Application No. 04779422, dated Jan. 22, 2007.
`European Patent Office, Communication Pursuant to Article 96(2)
`EPC Application No. 04779422, dated Aug. 22, 2007.
`European Patent Office, Communication Pursuant to Article 96(2)
`EPC, Application No. 04779422, dated Nov. 9, 2010.
`Chinese State Intellectual Property Office, Office Action, Application
`No. 200480040718, dated Dec. 25, 2007.
`Chinese State Intellectual Property Office, Office Action, Application
`No. 200480040718, dated Sep. 11,2008.
`Chinese State Intellectual Property Office, Office Action, Application
`No. 200480040718, dated Mar. 25, 2009.
`German Patent and Trademark Office, Office Action, Application No.
`11 2004 000918, dated Apr. 20, 2012.
`* cited by examiner
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 4
`
`

`
`U.S. Patent
`
`Jun.10,2014
`
`Sheet 1 of 4
`
`US 8,749,507 B2
`
`00
`0 .......
`
`H
`0
`U)
`U)
`<!)
`(.)
`0
`H p...
`
`\0
`0
`
`N
`:>-<
`~
`><
`
`-q_)
`
`0
`
`N
`0
`
`.........
`d
`~
`~
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 5
`
`

`
`y
`
`........___
`
`1-l...f'""'\1 UL._LJ
`
`/
`
`Yt:<:>
`
`I
`
`e •
`
`7J).
`•
`~
`~
`~
`
`~ = ~
`
`Start
`
`)
`
`204
`
`--....
`
`...
`
`...
`
`216 ----
`
`/
`
`........_
`
`TOUCHPAD
`
`FINGER ON ~~ ~I
`·--t
`I
`
`21
`
`206 -----
`
` ~WD~:f£ ~ "'D~T
`
`18
`
`i
`
`T'-1.01=~\.-4{)\ !'"'\, t..M""\\11=
`
`I
`
`I
`
`r
`
`I
`
`no
`
`~ = := ....
`0 ....
`
`~
`
`0
`N
`
`.j;o.
`
`YES
`
`/
`
`FINGER
`
`........___
`
`2/0 --.....
`
`I
`
`~ ----.....__/
`
`...
`~ NO I I START RELE~SE TICK COUNT ~
`
`I
`
`··-- T
`
`/
`
`212 ----
`
`214
`
`226
`
`228
`
`('D
`
`rFJ =-('D
`.....
`N
`0 .....
`
`.j;o.
`
`FIG. 2
`
`d
`rJl
`00
`~
`~
`
`\C u. = -....l = N
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 6
`
`

`
`START
`
`3/4
`
`318
`
`306
`
`--....._
`
`YES..Ir
`
`_j
`
`YES
`
`SPEED <SPEED
`THRESHOLD
`
`NO
`,,
`
`-
`
`··-
`
`324
`
`NO
`
`326
`
`308
`
`--....._
`
`y
`
`~ ~
`
`NO~ -:::.
`312 ~
`
`TAPPING
`
`310
`
`FIG. 3
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`~ = := ....
`0 ....
`
`~0
`N
`
`.j;o.
`
`rFJ =-('D
`.....
`
`('D
`
`(.H
`
`0 .....
`
`.j;o.
`
`d
`rJl
`00
`~
`~
`
`\C u. = -....l = N
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 7
`
`

`
`Step
`
`402
`
`Pulse
`
`404
`
`Ramp
`
`406
`
`Triangle
`
`408
`
`Quarter Cosine
`
`Quarter Sine
`
`412
`
`Half Cosine
`
`414
`
`Half Sine
`
`416
`
`Full Cosine
`
`418
`
`FIG. 4
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`2'
`:= ....
`0 ....
`
`.j;o.
`
`~0
`N
`
`rFJ =(cid:173)
`.....
`
`('D
`('D
`
`.j;o.
`
`0 .....
`
`.j;o.
`
`d
`rJl
`00
`~
`~
`
`\C u. = -....l = N
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 8
`
`

`
`US 8,749,507 B2
`
`1
`SYSTEMS AND METHODS FOR ADAPTIVE
`INTERPRETATION OF INPUT FROM A
`TOUCH-SENSITIVE INPUT DEVICE
`
`2
`Thus, a method and system are needed for accurately deter(cid:173)
`mining a user's intent based on data supplied by a touch(cid:173)
`sensitive input device.
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica(cid:173)
`tion Ser. No. 10/723,778, filed Nov. 26, 2003, entitled "Sys(cid:173)
`tems and Methods for Adaptive Interpretation oflnput from a
`Touch-Sensitive Device," now U.S. Pat. No. 8,164,573, the
`entirety of which is hereby incorporated by reference.
`
`NOTICE OF COPYRIGHT PROTECTION
`
`A section of the disclosure of this patent document and its
`figures contain material subject to copyright protection. The
`copyright owner has no objection to the facsimile reproduc(cid:173)
`tion by anyone of the patent document, but otherwise reserves
`all copyright rights whatsoever.
`
`FIELD OF THE INVENTION
`
`SUMMARY
`
`An embodiment of the present invention provides systems
`and methods for adaptive interpretation of input received
`from a touch-sensitive input device by receiving a pressure
`10 signal indicating a pressure from the input device, comparing
`the pseudo pressure signal to an adaptive pressure threshold
`value, and outputting a signal if the pseudo pressure signal is
`greater than the adaptive pressure threshold value. Further
`details and advantages of embodiments of the present inven-
`15 tion are set forth below.
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`These and other features, aspects, and advantages of the
`20 present invention are better understood when the following
`Detailed Description is read with reference to the accompa(cid:173)
`nying drawings, wherein:
`FIG.1 illustrates an exemplary environment for implemen(cid:173)
`tation of one embodiment of the present invention;
`FIG. 2 is a flow chart illustrating a process or algorithm for
`detecting finger presses on a touchpad in one embodiment of
`the present invention;
`FIG. 3 is a flow chart illustrating a process for detecting a
`finger press on a touchpad in another embodiment of the
`30 present invention; and
`FIG. 4 is a group of charts illustrating various filters that
`may be utilized in embodiments of the present invention.
`
`The present invention generally relates to receiving input
`from a touch-sensitive input device. This invention more par- 25
`ticularly relates to adaptive interpretation of input received
`from a touch-sensitive input device.
`
`BACKGROUND
`
`A variety of input devices may be used to provide position
`and control data to programs executing on computers, cell
`phones, and other processor-equipped devices. These input
`devices include mice, trackballs, touchpads, touch screens,
`touch panels, and various other devices. While the mouse and 35
`trackball provide distinct control elements for performing
`positioning and other control actions, the touchpad combines
`positioning and control.
`For example, a conventional mouse includes a ball or opti-
`cal sensor for determining changes in position of the mouse. 40
`The mouse also includes one or more buttons for performing
`a control function, such as selecting a graphical representa(cid:173)
`tion on a screen. In these systems, a user's intent to make a
`positional change or provide control input is apparent to the
`system.
`In contrast, conventional touchpads combine the position
`and control functionality in a way that often masks the user's
`intent to make a positional change to provide control input. A
`user moves a finger along a touchpad to reposition a cursor. A
`user may also perform gestures to simulate functions of the 50
`buttons of a mouse, such as drag, click, and double-click. In
`either case, the user's finger is in contact with the surface of
`the touchpad. Changes in position on the touchpad and in the
`pressure exerted on the surface of the touchpad must be used
`to determine the user's intent. Because of the variety of users 55
`that may interact with a touchpad and the variety of functions
`that may be performed, determining the user's intent based on
`a gesture on a touchpad is difficult. Variables affecting the
`ability of a program to determine what a user is attempting to
`do include the following: the physical difference between 60
`users; the different angles at which a user may place their
`finger while using a touchpad; the variance in pressure
`between different users and between the same user; the move(cid:173)
`ment of the finger across the touchpad while simultaneously
`attempting to perform actions on the touchpad. U.S. Pat. No. 65
`6,414,671 to Gillespie, et a!. describes one conventional
`method for recognizing a user's gesture as a drag gesture.
`
`DETAILED DESCRIPTION
`
`Referring now to the drawings in which like numerals
`indicate like elements throughout the several figures, FIG. 1
`illustrates an exemplary environment for implementation of
`an embodiment of the present invention. The embodiment
`shown includes a touch-sensitive device commonly called a
`touchpad 102. Touchpad 102 senses the position of a conduc-
`tor, such as a finger, on the surface of the touchpad (102). The
`touchpad (1 02) is further able to provide a position, compris(cid:173)
`ing X andY parameters, as well as a pressure, Z parameter, as
`45 an output signal. Conventional touchpads are very accurate in
`determining and providing the position of the conductor. For
`example, some conventional touchpads have resolutions
`greater than 1000 dpi. However, conventional touchpads are
`less accurate in determining and providing the pressure
`exerted on the touchpad. Other embodiments of the present
`invention may use other touch-sensitive input devices, such as
`a touch panel or touch screen.
`The touchpad 102 shown does not sense an actual pressure.
`Instead, the pressure reading from the touchpad 102 is a
`pseudo pressure. Touchpads work by utilizing resistance,
`capacitance, or membrane switches. Thetouchpad 102 shown
`in FIG. 1 utilizes capacitance, however, an embodiment of the
`present invention may be implemented in conjunction with
`any touch-sensitive input device, including resistive and
`membrane-switch touchpads. In other embodiments, actual
`pressure may be sensed. For example, in one embodiment, a
`touch screen with an attached explicit pressure sensor is uti(cid:173)
`lized.
`Capacitance-based touchpads are well known to those
`skilled in the art, and therefore, only a basic description of
`their function is provided herein. A capacitance touchpad,
`such as touchpad 102 shown in FIG. 1, includes two sets of
`
`APPLE INC.
`EXHIBIT 1001 - PAGE 9
`
`

`
`US 8,749,507 B2
`
`3
`wires, which are perpendicular to one another and configured
`so that a gap is formed between them. When a user places a
`conductor, such as a finger, on the touchpad 102, wires of the
`two perpendicular sets are brought together and form a
`capacitance. The touchpad 102 measures which of the wires
`in each of the two sets has the most capacitance to determine
`where the conductor is touching the touchpad 102 and, based
`on this information, provides the X andY coordinates of the
`position of the conductor on the touchpad 102.
`The touchpad 102 also provides a pseudo pressure, Z. The
`pseudo pressure is based on the amount of capacitance result(cid:173)
`ing from the conductor touching the touchpad 102. Accord(cid:173)
`ingly, the amount of capacitance is not a direct measure of
`pressure but rather a pseudo pressure.
`In other words, the pseudo pressure or Z parameter pro(cid:173)
`vided by the touchpad 102 is not a measure of the actual
`vertical displacement by a conductor at a single point on the
`touchpad 102, but rather an estimation of the vertical dis(cid:173)
`placement based on the size of the capacitance change. The
`pseudo pressure may not accurately represent the amount of
`pressure actually exerted on the touchpad 102. For example,
`the larger the surface of the conductor used on the touchpad
`102, e. g., a user's finger, the larger the change in capacitance
`per amount of pressure exerted. As would be expected, if a
`user presses heavily against the touchpad 102 with a fleshy
`part of the finger, the amount of touchpad 102 area covered by
`the finger is greater than then when the same part of the finger
`is touching lightly. However, what is less obvious is that the
`area covered, and the corresponding pseudo pressure, is also
`greater than when the user presses heavily with a bony part of
`a finger.
`Additionally, the difference in the features of different
`conductors, for instance the size or makeup of different users'
`fingers, affects the capacitance change for any given change
`in pressure. For example, if a first user with a large finger
`applies the same pressure as a second user with a small finger,
`the pseudo pressure signal output by the touchpad 102 is
`greater for the first person than for the second person for the
`same amount of applied pressure.
`The difficulty in determining a user's intent by evaluating
`the data provided by the touchpad 102 is compounded by the
`different ways in which a conductor may be utilized. For
`example, the pressure exerted across the surface of the touch(cid:173)
`pad may vary as the user's finger moves in relation to the
`hand. The user's finger covers a larger area of the touchpad
`when the finger is extended horizontally away from the hand
`on the touchpad 102 than when the finger is close to the hand.
`Similarly, a pointing device held vertical in relation to the
`touchpad 102 may cover a smaller surface area than one held
`at an angle to the touchpad 102.
`Referring again to FIG. 1, the touchpad 102 transmits the
`X, Y, and Z parameters 104 to a processor 106. The touchpad
`102 in various embodiments of the present invention may be
`capable of sending several types of coordinate information.
`For example, a Synaptics TouchPad is able to send either
`relative or absolute coordinates. Relative coordinates provide
`the movement of the conductor on the touchpad 102 since the
`last coordinates were transferred. Absolute coordinates pro(cid:173)
`vide the position of the conductor on the touchpad 102 at that
`moment. An embodiment of the present invention may utilize
`additional parameters as well. For example, the Synaptics
`TouchPad provides a "W" parameter, which reports the char(cid:173)
`acter of a contact with the touchpad, such as "accidental." An
`embodiment of the present invention may utilize such a
`parameter to accurately determine a user's intent.
`Referring again to FIG. 1, the processor 106 and touchpad
`102 may be connected directly or indirectly and may be
`
`4
`connected via wires or a wireless connection. For example,
`the touchpad 102 may utilize the PS/2, Serial, Apple Desktop
`Bus (ADB), or other communication protocol in communi(cid:173)
`cating with the processor. The processor 106 is capable of
`executing program code stored on a computer-readable
`medium. Although the processor shown is separate from the
`touchpad 102, some conventional touchpads include a pro(cid:173)
`cessor, such as an Application Specific Integrated Circuit
`(ASIC). An ASIC may provide some processing of the move-
`10 ments on the touchpad 102 to determine whether or not the
`user is making gestures. This integrated processor may be
`utilized alone or in combination with the processor 106
`according to the present invention.
`Processor 106 may include, for example, digital logic pro-
`15 cessors capable of processing input, executing algorithms,
`and generating output as necessary in response to the inputs
`received from the touch-sensitive input device. Such proces(cid:173)
`sors may include a microprocessor, the aforementioned
`ASIC, and state machines. Such processors include, or may
`20 be in communication with, media, for example computer(cid:173)
`readable media, which stores
`instructions
`that, when
`executed by the processor 106, cause the processor 106 to
`perform the steps described herein.
`Embodiments of computer-readable media include, but are
`25 not limited to, an electronic, optical, magnetic, or other stor(cid:173)
`age or transmission device capable of providing a processor,
`such as the processor 106 in communication with a touch(cid:173)
`sensitive input device, with computer-readable instructions.
`Other examples of suitable media include, but are not limited
`30 to, a floppy disk, CD-ROM, magnetic disk, memory chip,
`ROM, RAM, an ASIC, a configured processor, all optical
`media, all magnetic tape or other magnetic media, or any
`other medium from which a computer processor can read
`instructions. Also, various other forms of computer-readable
`35 media may transmit or carry instructions to a computer,
`including a router, private or public network, or other trans(cid:173)
`mission device or channel, both wired and wireless. The
`instructions may comprise code from any computer-pro(cid:173)
`gramming language, including, for example, C, C#, Visual
`40 Basic, Java, and JavaScript.
`The embodiment shown in FIG. 1 may be implemented in
`a variety of devices. Such devices include personal comput(cid:173)
`ers, many of which include an integrated touchpad. Such
`devices may also include handheld devices, such as handheld
`45 organizers, cellular telephones, handheld communicators,
`MP3 players, GPS receivers, and the like.
`Embodiments of the present invention may also be utilized
`to implement haptic effects in devices such as those men(cid:173)
`tioned above. In such an embodiment, the haptic effects result
`50 from various actions by a user interfacing with a touch-sen(cid:173)
`sitive input device, and the effects may be based on the user's
`intent as determined by the processor 106. Haptic effects may
`also result from interaction with software executing on a
`device in communication with the touch-sensitive input
`55 device.
`Embodiments of the present invention address the difficul(cid:173)
`ties faced in attempting to d