a2) United States Patent
`US 8,164,573 B2
`(10) Patent No.:
`(45) Date of Patent:
`Apr. 24, 2012
`DaCosta et al.
`
`
`US008164573B2
`
`(54) SYSTEMS AND METHODSFOR ADAPTIVE
`INTERPRETATION OF INPUT FROM A
`TOUCH-SENSITIVE INPUT DEVICE
`
`(75)
`
`Inventors: Henry DaCosta, Montreal (CA);
`Christophe Ramstein, San Francisco,
`CA (US); Danny Grant, Montreal (CA)
`
`(73) Assignee:
`
`Immersion Corporation, San Jose, CA
`(US)
`
`EP
`
`4,127,752 A
`4,160,508 A
`4,236,325 A
`4,262,549 A
`4,333,070 A
`4,464,117 A
`A
`4,484,191
`
`11/1978 Lowthorp
`7/1979 Froschetal.
`12/1980 Hallet al.
`4/1981 Schwellenbach
`6/1982 Barnes
`8/1984 Forest
`11/1984 Vavra
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`0349086
`1/1990
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C. 154(b) by 1560 days.
`
`(21) Appl. No.: 10/723,778
`
`(22)
`
`Filed:
`
`Nov. 26, 2003
`
`(65)
`
`Prior Publication Data
`
`US 2005/0110769 Al
`
`May 26, 2005
`
`(51)
`
`Int. Cl.
`(2006.01)
`G09G 5/00
`(52) US. CL cece ceeesteceeeneeees 345/173; 178/18.01
`(58) Field of Classification Search.......... 345/156-163,
`345/168, 169, 173-179; 178/18.1, 18.01,
`178/18.03-18.09; 715/701, 702
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2,972,140 A
`3,157,853 A
`3,220,121 A
`3,497,668 A
`3,517,446 A
`3,623,064 A
`3,902,687 A
`3,903,614 A
`3,911,416 A
`
`2/1961 Hirsch
`11/1964 Hirsch
`11/1965 Cutler
`2/1970 Hirsch
`6/1970 Corlyonet al.
`11/1971 Kagan
`9/1975 Hightower
`9/1975 Diamondet al.
`10/1975 Feder
`
`PCT Search Report, corresponding to PCT/US04/24358, mailed on
`Feb. 11, 2005.
`
`(Continued)
`
`Primary Examiner — Regina Liang
`(74) Attorney, Agent, or Firm — Kilpatrick Townsend &
`Stockton LLP
`
`(57)
`
`ABSTRACT
`
`Systems and methods for adaptively interpreting a user’s
`intent based on parameters supplied by a touch-sensitive
`input device are described. In one ofthe methodsdescribed, a
`processor receives a pressure signal indicating a pressure
`from an input device, such as a touchpad, compares the
`pseudo pressure signal to a pressure threshold value, and
`outputs a signal if the pseudo pressure signal is greater than
`the pressure threshold value. In another embodiment, the
`processoralso calculates the speed ofmovementof a conduc-
`tor, for instance a user’s finger, across the input device, and
`comparesthe speed to a threshold. If the speed is greater than
`the threshold, the processor determines that although the
`pressure may be great enoughto signal a press, no press is
`intended. The various parameters supplied by the input device
`maybedigitally filtered to increase the accuracy ofthe deter-
`mination ofuser intent.
`
`26 Claims, 4 Drawing Sheets
`
`FIRST TICK
`COUNT
`ELAPSED
`
`6
`
`248
`
`
`
`FINGER ON
`‘TOUCHPAD
`S,
`
`ve
`FINGER
`PREVIOUSLY ON
`‘TOUCHPAD
`NG
`STARTFIRSTTICK COUNT
`
`206
`Yes
`
`208
`
`
`204
`
`
`
`
`
`
`210
`
`
`NY DETERMINE KEY ON WHICHFINGER IS POSITIONED
`22
`
`214
`
`RELEASETICK
`ELAPSED
`
`
`
`
`
`220
`
`Valve v. Immersion
`
`Valve Exhibit 1020
`Valve Exhibit 1020
`Valve v. Immersion
`
`

`

`US 8,164,573 B2
`
`Page 2
`
`etal.
`
`5/2002 Leung
`6,388,655 Bl
`§/2002 Gorsuchetal.
`6,388,999 Bl
`7/2002. Gillespie etal.
`6,414,671 BL
`aoe wampet et a
`6,414,674 Bl
`ab) Cromeret
`Oak BI
`5
`50°
`8/2002 Rosenberg et al.
`6,429,846 B2
`9/2002 Cruz-Hernandezetal.
`6,445,284 Bl
`11/2002 Fleasetal
`6487421 B2
`>
`,
`07
`6,492,979 BL* 12/2002 Kentet al. ccc 345/173
`6,509,847 B1*
`1/2003 Anderson cece 341/34
`6,509,892 Bl
`1/2003 Kamper etal.
`6,518,958 BL
`2/2003 Miyajimaetal.
`6.535.201 BL
`3/2003 Cooperet al.
`6,590,568 BL*
`7/2003 Astala et al.
`...cccccccc0.. 345/173
`CeleoxG BS
`Sooo qutwig
`Oe
`5
`mesple
`6,628,195 Bl
`9/2003 Coudon
`6,636,202 B2
`10/2003 Ishmael,Jr. et al.
`ee BI
`hhooos Sirader
`647,
`2
`ay
`6,801,191 B2* 10/2004 Mukai et al. cece 345/179
`7,046,235 B2
`5/2006 Katoh
`2001/0035854 Al
`11/2001 Rosenberget al.
`2002/0033795 Al
`3/2002 Shahoian etal.
`2002/0149561 Al
`10/2002 Fukumotoet al.
`2002/0149570 Al
`10/2002 Knowlesetal.
`2002/0156807 Al
`10/2002 Dieberger
`2003/0006892 Al
`‘1/2003 Church
`2003/0016211 Al
`1/2003 Woolley
`2003/0022701 Al
`‘1/2003 Gupta
`2003/0025679 Al
`2/2003 Tayloret al.
`2003/0030628 Al
`2/2003.
`Satoet al.
`2003/0038776 Al
`2/2003 Rosenberget al.
`2003/0048260 Al
`3/2003 Matusis
`2003/0058265 Al
`3/2003. Robinsonetal.
`2003/0063073 Al*
`4/2003 Geaghanet al. wc... 345/173
`2003/0067449 Al
` 4/2003-- Yoshikawaetal.
`2003/0071795 Al
`4/2003 Baldaufetal.
`2003/0095105 Al
`5/2003. _Vaananen
`2003/0128191 Al
`7/2003 Strasseret al.
`2003/0128192 Al
`7/2003 van Os
`2003/0151597 Al
`8/2003 Robertset al.
`2003/0174121 Al
`9/2003 Poupyrevet al.
`2003/0179190 Al
`9/2003. Franzen
`FOREIGN PATENT DOCUMENTS
`
`U.S. PATENT DOCUMENTS
`ae, A
`ie AcKfam etal
`4509070 A
`7/1986. Hladky et al.
`4,708,656 A
`11/1987 De Vrieset al.
`/1987 Alban
`asevee ‘ ae “
`;
`4777005 A
`9/1988 Chiumakyetal
`4,794,392 A
`12/1988 Selinko
`/1989 Embach
`qeorve a
`tong panache
`071)
`;
`4,926,879 A
`5/1990 Sevrain etal.
`4,930,770 A
`6/1990 Baker
`4,934,694 A
`Ooo, welqtosh
`a7984 ‘
`CooL na
`5,022,407 A
`6/1991 Horchetal.
`5,035,242 A
`7/1991 Franklin
`359,
`ee ‘
`F000 palaly
`5,121,091 A
`6/1992 Fujiyama
`121,
`jeeny a
`Ih1003 tener
`1103,
`l
`;
`aed A 003 Rania’ “al
`5,186,695
`993
`Mangseth
`etal.
`oodLs ‘
`ito03 sous
`;
`acer haeyarees Pron etal
`ells
`oon ‘
`stood Cook
`5°900'810 A
`4/1994 Sra
`10995
`11904 Cond
`oelao ‘
`Vie9fe an etal
`2309,
`Jt.
`5,334,027 A
`8/1994 Wherlock
`5,389,849 A
`2/1995 Asanoet al.
`ooeee ‘
`gies, cuiman et al.
`prriert iy GETAree Won. tal
`wo
`/
`ang et
`al.
`oteoa ‘
`4Nove Reean
`354738) A
`8/1996 Yamasaki
`,
`,
`S
`5,575,761 A
`11/1996 Hajianpour
`oe ‘ ae ane etal. ;
`oeorsey A
`81907 Denetal
`5638-060 A
`6/1997 Katnoke ot al
`5,719,561 A
`2/1998 Gonzales
`5,736,978 A
`4/1998 Hasseretal.
`
`5,766,016 A 0 536715 A2—4/19936/1998 Sinclair EP
`
`
`5,785,630 A
`7/1998 Bobicket al.
`EP
`0556999 BL
`5/1998
`
`5,880,411 A 1182851 Al—2/20023/1999 Gillespie et al. EP
`
`
`5,887,995 A
`3/1999 Holehan
`JP
`01-003664
`7/1990
`5,889,511 A
`3/1999 Ong
`JP
`02-109714
`1/1992
`5,917,906 A
`6/1999 Thronton
`JP
`04-007371
`8/1993
`5,977,867 A
`11/1999 Blouin
`JP
`05-193862
`1/1995
`5,995,084 A
`L1/1999 Chan et al.
`JP
`2001-350592 A
`12/2001
`6,008,800 A
`12/1999 Pryor
`JP
`2002-259059 A
`9/2002
`6,057,830 A
`5/2000 Chan etal.
`WO
`WO 97/18546 Al
`5/1997
`6,067,081 A
`5/2000 Hahlganssetal.
`WO
`WO 02/12991 Al
`2/2002
`6,081,536 A
`6/2000 Gorsuchetal.
`WO
`WO 02/27645
`4/2002
`6,111,577 A
`8/2000 Zillesetal.
`WO
`WO 02/31807 Al
`4/2002
`6,118,435 A
`9/2000 Fujita etal.
`
`6.128.007 A*10/2000 Seybold ..ccccsssesesesee 345/179 OTHER PUBLICATIONS
`eet ‘ 1ooo qeineta al
`Adelstein, “A Virtual Environment System for the Study of Human
`6. 160,489 A
`12/2000 Perryetal.
`Arm Tremor,” Ph.D. Dissertation, Dept. of Mechanical Engineering,
`6,198,206 Bl
`3/2001 Saarmaaetal.
`MIT,Jun. 1989.
`6,215,478 Bl
`4/2001 Yeh etal.
`Adelstein, “Design and Implementation of a Force Reflecting
`6,215,778 Bl
`4/2001 Lompetal.
`Manipulandum for Manual Control
`research,” DSC—vol. 42,
`6,218,966 BI
`4/2001 Goodwinet al.
`Advancesin Robotics, Edited by H. Kazerooni, pp. 1-12, 1992.
`eteoay BI 40 Apinget al.
`Aukstakalnis et al., “Silicon Mirage: The Art and Science ofVirtual
`6243080 Bl
`6/2001 Molne ano
`Reality,” ISBN 0-938151-82-7, pp. 129-180, 1992.
`6,262.7 17 BL
`7/2001 Donohueetal.
`Baigrie, “Electric Control Loading—A Low Cost, High Performance
`6,307,465 Bl
`10/2001 Kaymaetal.
`Alternative,” Proceedings, pp. 247-254, Nov. 6-8, 1990.
`6,326,901 Bl
`12/2001 Gonzales
`Bejezyet al., “Kinesthetic Coupling Between Operator and Remote
`6,337,678 Bl
`1/2002 Fish
`Manipulator,” International Computer Technology Conference, The
`6,373,463 Bl
`4/2002 Beeks
`American Society of Mechanical Engineers, San Francisco, CA,
`6,380,931 Bl
`4/2002. Gillespie et al.
`Aug. 12-15, 1980.
`
`

`

`US 8,164,573 B2
`Page 3
`
`and Man-Machine Interface for
`“Sensors, Controls,
`Bejezy,
`Advanced Teleoperation,” Science, vol. 208, No. 4450, pp. 1327-
`1335, 1980.
`Bejezy, “Generalization of Bilateral Force-Reflecting Control of
`Manipulators,” Proceedings of Fourth CISM-IFToMM,Sep. 8-12,
`1981.
`Bejezy, et al., “Universal Computer Control System (UCCS) for
`Space Telerobots,” CH2413-3/87/0000/0318501.00 1987 IEEE,
`1987.
`Bejezy et al., “A Laboratory Breadboard System for Dual-Arm
`Teleoperation,” SOAR ’89 Workshop, JSC, Houston, TX, Jul. 25-27,
`1989.
`Bliss, ““Optical-to-Tactile Image Conversion for the Blind,’ IEEE
`Transactions on Man-Machine Systems, vol. MMS-11, No. 1, Mar.
`1970.
`Brookset al., “Hand Controllers for Teleoperation—A State-of-the-
`Art Technology Survey and Evaluation,’ JPL Publication 85-11,
`NASA-CR-175890; N85-28559, pp. 1-84, Mar. 1, 1985.
`Burdeaetal., “Distributed Virtual Force Feedback, Lecture Notes for
`Workshop on Force Display in Virtual Environments and its Appli-
`cation to Robotic Teleoperation,” 1993 TEEE International Confer-
`ence on Robotics and Automation, pp. 25-44, May 2, 1993.
`Calder, “Design ofa Force-Feedback Touch-Introducing Actuator for
`Teleoperator Robot Control,” Bachelor of Science Thesis, MIT, Jun.
`23, 1983.
`Caldwellet al., “Enhanced Tactile Feedback (Tele-Taction) Using a
`Multi-Functional Sensory System,” 1050-4729/93, pp. 955-960,
`1993.
`“Cyberman Technical Specification,’ Logitech Cyberman SWIFT
`Supplement, Apr. 5, 1994.
`Eberhardt et al., “OMAR—A Haptic display for speech perception
`by deaf and deaf-blind individuals,’ IEEE Virtual Reality Annual
`International Symposium, Seattle, WA, Sep. 18-22, 1993.
`Eberhardt et al., “Including Dynamic Haptic Perception by The
`Hand: System Description and Some Results,’ DSC—vol. 55-1,
`Dynamic Systems and Control: vol. 1, ASME 1994.
`Fukumoto, “Active Click: Tactile Feedback for Touch Panels,” ACM
`CHI2001 Extended Abstracts, pp. 121-122, Apr. 2001.
`Force Feedback Touch Panel, Represented by CSC Division, Sales
`Department., S[IXIK Corporation, Tokyo, Japan, www.smk.co.jp.
`Gobeletal., “Tactile Feedback Applied to Computer Mice,” Interna-
`tional Journal of Human-Computer Interaction, vol. 7, No.
`|, pp.
`1-24, 1995.
`Gotowet al., “ Controlled Impedance Test Apparatus for Studying
`Human Interpretation of Kinesthetic Feedback,” WA11-11:00, pp.
`332-337.
`Howe,“A Force-Reflecting Teleoperated Hand System for the Study
`of Tactile Sensing in Precision Manipulation,” Proceedings of the
`1992 TEEFE International Conference on Robotics and Automation,
`Nice, France, May 1992.
`IBM Technical Disclosure Bulletin, “Mouse Ball-Actuating Device
`With Force and Tactile Feedback,” vol. 32, No. 9B, Feb. 1990.
`Iwata, “Pen-based Haptic Virtual Environment,” 0-7803-1363-1/93
`IEEE,pp. 287-292, 1993.
`Jacobsen et al., “High Performance, Dextrous Telerobotic Manipu-
`lator With Force Reflection,” Intervention/ROV ’91 Conference &
`Exposition, Hollywood, Florida, May 21-23, 1991.
`Johnson,
`“Shape-Memory Alloy Tactile Feedback Actuator,”
`Armstrong Aerospace Medical Research Laboratory, AAMRL-TR-
`90-039, Aug. 1990.
`Jones et al., “A perceptual analysis of stiffness,’ ISSN 0014-4819
`Springer
`International
`(Springer-Verlag); Experimental Brain
`Research, vol. 79, No. 1, pp. 150-156, 1990.
`Kaczmareket al., “Tactile Displays,” Virtual Environment Technolo-
`gies.
`Kontarinis et al., “Display of High-Frequency Tactile Information to
`Teleoperators,” Telemanipulator Technology and Space Telerobot-
`ics, Won S. Kim, Editor, Proc. SPIE vol. 2057, pp. 40-50, Sep. 7-9,
`1993.
`Kontarinis et al., “Tactile Display of Vibratory Information in
`Teleoperation and Virtual Environments,’ PRESENCE, 4(4):387-
`402, 1995.
`Lake, “Cyberman from Logitech,” GameBytes, 1994.
`
`Marcus,“Touch Feedback in Surgery,” Proceedings ofVirtual Real-
`ity and Medicine The Cutting Edge, Sep. 8-11, 1994.
`McAffee, “Teleoperator Subsystem/Telerobot Demonstrator: Force
`Reflecting Hand Controller Equipment Manual,” JPL D-5172, pp.
`1-50, Al-A36, B1-B5, C1-C36, Jan. 1988.
`Minsky, “Computational Haptics: The Sandpaper System for Syn-
`thesizing Texture for a Force-Feedback Display,’ Ph.D. Dissertation,
`MIT, Jun. 1995.
`Ouh-Young, “Force Display in Molecular Docking,” Order No.
`9034744, p. 1-369, 1990.
`Ouh-Young,“A Low-Cost Force Feedback Joystick and Its Use in PC
`Video Games,” IEEE Transactions on ConsumerElectronics,vol. 41,
`No.3, Aug. 1995.
`Ouhyounget al., “The Development of a Low-Cost Force Feedback
`Joystick andIts Use in the Virtual Reality Environment,” Proceedings
`of the Third Pacific Conference on Computer Graphics and Applica-
`tions, Pacific Graphics ’95, Seoul, Korea, Aug. 21-24, 1995.
`Patrick et al., “Design and Testing of a Non-reactive, Fingertip,
`Tactile Display for Interaction with Remote Environments,’ Coop-
`erative Intelligent Robotics in Space, Rui J. deFigueiredo et al.,
`Editor, Proc. SPIE vol. 1387, pp. 215-222, 1990.
`Patrick, “Design, Construction, and Testing of a Fingertip Tactile
`Display for Interaction with Virtual and Remote Environments,”
`Master of Science Thesis, MIT, Nov. 8, 1990.
`Pimentelet al., “Virtual Reality: through the new looking glass,” 2”?
`Edition; McGraw-Hill, ISBN 0-07-050167-X, pp. 41-202, 1994.
`Rabinowitz et al., “Multidimensional tactile displays: Identification
`of vibratory intensity, frequency, and contactor area,” Journal of The
`Acoustical Society of America, vol. 82, No. 4, Oct. 1987.
`Russo, “The Design and Implementation of a Three Degree of Free-
`dom Force Output Joystick,” MIT Libraries Archives Aug. 14, 1990,
`pp. 1-131, May 1990.
`Russo, “Controlling Dissipative Magnetic Particle Brakes in Force
`Reflective Devices,’ DSC—vol. 42, Advances in Robotics, pp.
`63-70, ASME 1992.
`Safe Flight Instruments Corporation, “Coaxial Control Shaker,” Part
`No. C-25502, Jul. 1, 1967.
`Scannell, “Taking a Joystick Ride,’ Computer Currents, Boston Edi-
`tion, vol. 9, No. 11, Nov. 1994.
`Shimoga, “Finger Force and Touch Feedback Issues in Dexterous
`Telemanipulation,” Proceedings of Fourth Annual Conference on
`Intelligent Robotic Systems for Space Exploration, Renssclacr Poly-
`technic Institute, Sep. 30-Oct. 1, 1992.
`SMKCorporation, “Multi-Functional Touch Panel, Force-Feedback
`Type, Developed: A Touch Panel Providing a Clicking Feeling,”
`http://www.smk.co.jp/whatsnew__e/628csc_e.html, Sep. 30, 2002.
`SMK Corporation, “Force Feedback Type Optical Touch Panel
`Developed,” SMK Corporation Website, Oct. 30, 2002.
`Snowet al., “Model-X Force-Reflecting-Hand-Controller,’ NT Con-
`trol No. MPO-17851; 17851; JPL Case No. 5348, pp. 1-4, Jun. 15,
`1989.
`Stanley et al., “Computer Simulation of Interacting Dynamic
`Mechanical Systems Using Distributed Memory Parallel Proces-
`sors,” DSC—vol. 42, Advancesin Robotics, pp. 55-61, ASME 1992.
`Tadros, “Control System Design for a Three Degree of Freedom
`Virtual Environment Simulator Using Motor/Brake Pair Actuators”,
`MITArchive © Massachusetts Institute ofTechnology, pp. 1-88, Feb.
`1990.
`Terry etal., “Tactile Feedback in a Computer Mouse,” Proceedings of
`Fourteenth Annual Northeast Bioengineering Conference, Univer-
`sity of New Hampshire, Mar. 10-11, 1988.
`Vwiker, “Teletouch Display Development: Phase 1 Report,” Techni-
`cal Report 1230, Naval Ocean Systems Center, San Diego, Apr. 17,
`1989.
`European Patent Office, Communication,
`10010754, dated Nov. 8, 2010.
`European Patent Office, Communication Pursuant to Article 94(3)
`EPC,Application No. 04779422, dated Nov. 9, 2010.
`
`Application No.
`
`* cited by examiner
`
`

`

`U.S. Patent
`
`Apr. 24, 2012
`
`Sheet 1 of 4
`
`US 8,164,573 B2
`
`Various
`
`Interfaces
`
`w9
`
`°a
`na
`vo
`
`FIG.1
`
`o°~
`
`~
`au
`
`106
`
`Touchpad
`
`102ae
`
`

`

`U.S. Patent
`
`ce
`
`917
`
`SSA
`
`
`
`MOLLLSUld
`
`LNNOD
`
`gasdv13
`
`817
`
`ON
`
`NO4¥39NI4
`
`QVdHONOL
`
`407
`
`
`
`SAA
`
`
` LYVLaMOHSASHL°8=<ZVSnN+vzaGVdHONOL=NOATISNOIASudaYAONISSSAA<Aa»YOGIOHSSYHLAayYOQIOHSSYHL
`nceBAOWo0z =CIOHS3eHLLSuld=!CIOHSSSHL
`
`
`
`
`
`
`
`
`
`wtONISSBad
`
`So°zIza9c
`
`SNINYSLSGaoiz
`GANOILISOdSIUJONIJgalmosay<ZHOIHMNOAZ»
`—2MOHSAHHL
`
`#07
`
`827
`
`ON
`
`LNNODMOMLSYld
`
`US 8,164,573 B2
`
`¢OFF
`
`ONgasdvia
`
`
`
`MOLL3SV314y
`
`LNNOD
`
`17
`
`
`
`AINNOOHOLLASV37SYLHWLSS34
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Apr. 24, 2012
`
`Sheet 3 of 4
`
`US 8,164,573 B2
`
`PIE
`
`AISNOI3ad
`
`ONIHONOL
`
`ait
`
`OzE
`
`ONIHONOL
`
`
`
`ANNODMILLSHldLYVLS
`
`yaddn<Z
`
`QUOHSSYHL
`
`ATSNOIARYd
`
`ONIHONOL
`
`92E
`
`OoQ3Sdv13
`
`
`
`TWAM3LNIELSYIS
`
`ONIddVL
`
`MOLLSU
`
`LNNOD
`
`g3sdv14
`
`QIOHS3YHL
`
`<2
`
`gaSd$>G33dS
`
`COHS3YHL
`
`QIOHSSYHL
`
`$34
`
`YyaMO1<Z
`
`¢Old
`
`
`
`
`
`LNNODMOLLSuyldYV3a19
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Apr. 24, 2012
`
`Sheet 4 of 4
`
`US 8,164,573 B2
`
`vyOIA
`
`
`
`{L
`
`CIPaugsauuenn
`
`
`
`
`i1
`
`
`
`CORdaig
`
`
`
`aUulsodJEM
`
`
`
`(heeree
`;|yo—‘|90%dwey|vOrasing
`
`
` 51Paulsog|Ingaugs12H
`
`
`i| i
`
`Reneen bndeededdoen
`
`Footmanberm
`
`i
`perennehneleheePoseddoayadicandicacbecs hepebsabernBosna
`
`
`
`!
`tL
`
`
`
`
`
`

`

`US 8,164,573 B2
`
`1
`SYSTEMS AND METHODS FOR ADAPTIVE
`INTERPRETATION OF INPUT FROM A
`TOUCH-SENSITIVE INPUT DEVICE
`
`NOTICE OF COPYRIGHT PROTECTION
`
`A section of the disclosure of this patent documentandits
`figures contain material subject to copyright protection. The
`copyright ownerhas no objection to the facsimile reproduc-
`tionby anyoneofthe patent document, but otherwise reserves
`all copyright rights whatsoever.
`
`FIELD OF THE INVENTION
`
`The present invention generally relates to receiving input
`from a touch-sensitive input device. This invention morepar-
`ticularly relates to adaptive interpretation of input received
`from a touch-sensitive input device.
`
`BACKGROUND
`
`A variety of input devices may be usedto provide position
`and control data to programs executing on computers, cell
`phones, and other processor-equipped devices. These input
`devices include mice, trackballs, touchpads, touch screens,
`touch panels, and various other devices. While the mouse and
`trackball provide distinct control elements for performing
`positioning and other control actions, the touchpad combines
`positioning and control.
`For example, a conventional mouse includesa ballor opti-
`cal sensor for determining changesin position of the mouse.
`The mousealso includes one or more buttons for performing
`a control function, such as selecting a graphical representa-
`tion on a screen. In these systems, a user’s intent to make a
`positional change or provide control input is apparent to the
`system.
`In contrast, conventional touchpads combinethe position
`and control functionality in a way that often masksthe user’s
`intent to make a positional change to provide control input. A
`user movesa finger along a touchpadto reposition a cursor. A
`user may also perform gestures to simulate functions of the
`buttons of a mouse, such as drag, click, and double-click. In
`either case, the user’s finger is in contact with the surface of
`the touchpad. Changes in position on the touchpadandin the
`pressure exerted on the surface of the touchpad must be used
`to determine the user’s intent. Because of the variety of users
`that may interact with a touchpad andthe variety of functions
`that may be performed, determining the user’s intent based on
`a gesture on a touchpadis difficult. Variables affecting the
`ability of a program to determine whata useris attempting to
`do include the following: the physical difference between
`users; the different angles at which a user may place their
`finger while using a touchpad;
`the variance in pressure
`betweendifferent users and between the sameuser; the move-
`mentof the finger across the touchpad while simultaneously
`attempting to perform actions on the touchpad. U.S. Pat. No.
`6,414,671 to Gillespie, et al. describes one conventional
`method for recognizing a user’s gesture as a drag gesture.
`Thus, amethod and system are neededfor accurately deter-
`mining a user’s intent based on data supplied by a touch-
`sensitive input device.
`
`SUMMARY
`
`An embodimentofthe present invention provides systems
`and methods for adaptive interpretation of input received
`from a touch-sensitive input device by receiving a pressure
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`signal indicating a pressure from the input device, comparing
`the pseudopressure signal to an adaptive pressure threshold
`value, and outputting a signalif the pseudo pressure signalis
`greater than the adaptive pressure threshold value.
`Further details and advantages of embodiments of the
`present invention are set forth below.
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`These and other features, aspects, and advantages of the
`present invention are better understood when the following
`Detailed Description is read with reference to the accompa-
`nying drawings, wherein:
`FIG.1 illustrates an exemplary environmentfor implemen-
`tation of one embodimentof the present invention;
`FIG.2 is a flow chart illustrating a process or algorithm for
`detecting finger presses on a touchpad in one embodimentof
`the present invention;
`FIG.3 is a flow chart illustrating a process for detecting a
`finger press on a touchpad in another embodiment of the
`present invention; and
`FIG.4 is a group of charts illustrating variousfilters that
`maybe utilized in embodiments of the present invention.
`
`DETAILED DESCRIPTION
`
`Referring now to the drawings in which like numerals
`indicate like elements throughout the several figures, FIG. 1
`illustrates an exemplary environmentfor implementation of
`an embodiment of the present invention. The embodiment
`shown includes a touch-sensitive device commonlycalled a
`touchpad 102. Touchpad 102 sensesthe position of a conduc-
`tor, such asa finger, on the surface ofthe touchpad (102). The
`touchpad (102) is further able to provide a position, compris-
`ing X andY parameters, as well as a pressure, Z parameter, as
`an output signal. Conventional touchpadsare very accurate in
`determining and providing the position of the conductor. For
`example, some conventional
`touchpads have resolutions
`greater than 1000 dpi. However, conventional touchpads are
`less accurate in determining and providing the pressure
`exerted on the touchpad. Other embodiments of the present
`invention mayuse other touch-sensitive input devices, suchas
`a touch panelor touch screen.
`The touchpad 102 showndoesnot sense an actualpressure.
`Instead, the pressure reading from the touchpad 102 is a
`pseudo pressure. Touchpads work by utilizing resistance,
`capacitance, or membrane switches. The touchpad 102 shown
`in FIG.1 utilizes capacitance, however, an embodimentofthe
`present invention may be implemented in conjunction with
`any touch-sensitive input device,
`including resistive and
`membrane-switch touchpads. In other embodiments, actual
`pressure may be sensed. For example, in one embodiment, a
`touch screen with an attached explicit pressure sensor is uti-
`lized.
`
`Capacitance-based touchpads are well known to those
`skilled in the art, and therefore, only a basic description of
`their function is provided herein. A capacitance touchpad,
`such as touchpad 102 shown in FIG.1, includes twosets of
`wires, which are perpendicular to one another and configured
`so that a gap is formed between them. Whena userplaces a
`conductor, such as a finger, on the touchpad 102, wires of the
`two perpendicular sets are brought together and form a
`capacitance. The touchpad 102 measures which of the wires
`in each ofthe twosets has the most capacitance to determine
`where the conductoris touching the touchpad 102 and, based
`on this information, provides the X and Y coordinates of the
`position of the conductor on the touchpad 102.
`
`

`

`US 8,164,573 B2
`
`3
`The touchpad 102 also provides a pseudo pressure, Z. The
`pseudopressure is based on the amountof capacitanceresult-
`ing from the conductor touching the touchpad 102. Accord-
`ingly, the amount of capacitance is not a direct measure of
`pressure but rather a pseudopressure.
`In other words, the pseudo pressure or Z parameter pro-
`vided by the touchpad 102 is not a measure of the actual
`vertical displacement by a conductor at a single point on the
`touchpad 102, but rather an estimation of the vertical dis-
`placement based on the size of the capacitance change. The
`pseudo pressure may not accurately represent the amount of
`pressure actually exerted on the touchpad 102. For example,
`the larger the surface of the conductor used on the touchpad
`102, e.g., a user’s finger, the larger the change in capacitance
`per amountof pressure exerted. As would be expected, if a
`user presses heavily against the touchpad 102 with a fleshy
`part ofthe finger, the amountoftouchpad 102 area covered by
`the finger is greater than then whenthe samepart ofthe finger
`is touching lightly. However, what is less obviousis that the
`area covered, and the corresponding pseudopressure, is also
`greater than whentheuserpresses heavily with a bonypart of
`a finger.
`Additionally, the difference in the features of different
`conductors, for instance the size or makeupofdifferent users’
`fingers, affects the capacitance change for any given change
`in pressure. For example, if a first user with a large finger
`applies the same pressure as a second user with a small finger,
`the pseudo pressure signal output by the touchpad 102 is
`greater for the first person than for the second person for the
`same amountof applied pressure.
`The difficulty in determining a user’s intent by evaluating
`the data provided by the touchpad 102 is compounded by the
`different ways in which a conductor may be utilized. For
`example, the pressure exerted across the surface ofthe touch-
`pad may vary as the user’s finger moves in relation to the
`hand. The user’s finger covers a larger area of the touchpad
`whenthe finger is extended horizontally away from the hand
`on the touchpad 102 than whenthe finger is close to the hand.
`Similarly, a pointing device held vertical in relation to the
`touchpad 102 maycover a smaller surface area than one held
`at an angle to the touchpad 102.
`Referring again to FIG. 1, the touchpad 102 transmits the
`X, Y, and Z parameters 104 to a processor 106. The touchpad
`102 in various embodiments ofthe present invention may be
`capable of sending several types of coordinate information.
`For example, a Synaptics TouchPad is able to send either
`relative or absolute coordinates. Relative coordinates provide
`the movementof the conductor on the touchpad 102 since the
`last coordinates were transferred. Absolute coordinates pro-
`vide the position of the conductor on the touchpad 102at that
`moment. An embodimentofthe present invention mayutilize
`additional parameters as well. For example, the Synaptics
`TouchPad provides a “W”parameter, which reports the char-
`acter of a contact with the touchpad, such as “accidental.” An
`embodiment of the present invention may utilize such a
`parameter to accurately determine a user’s intent.
`Referring again to FIG.1, the processor 106 and touchpad
`102 may be connected directly or indirectly and may be
`connected via wires or a wireless connection. For example,
`the touchpad 102 mayutilize the PS/2, Serial, Apple Desktop
`Bus (ADB), or other communication protocol in communi-
`cating with the processor. The processor 106 is capable of
`executing program code stored on a computer-readable
`medium. Although the processor shownis separate from the
`touchpad 102, some conventional touchpads include a pro-
`cessor, such as an Application Specific Integrated Circuit
`(ASIC). An ASIC mayprovide someprocessing ofthe move-
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`4
`ments on the touchpad 102 to determine whether or not the
`user is making gestures. This integrated processor may be
`utilized alone or in combination with the processor 106
`according to the present invention.
`Processor 106 mayinclude, for example, digital logic pro-
`cessors capable of processing input, executing algorithms,
`and generating output as necessary in responseto the inputs
`received from the touch-sensitive input device. Such proces-
`sors may include a microprocessor,
`the aforementioned
`ASIC,and state machines. Such processors include, or may
`be in communication with, media, for example computer-
`readable media, which stores
`instructions
`that, when
`executed by the processor 106, cause the processor 106 to
`perform the steps described herein.
`Embodiments of computer-readable media include, but are
`notlimitedto, an electronic, optical, magnetic, or otherstor-
`age device capable of providing a processor, such as the
`processor 106 in communication with a touch-sensitive input
`device, with computer-readable instructions. Other examples
`of suitable media include, butare not limited to, a floppy disk,
`CD-ROM, magnetic disk, memory chip, ROM, RAM, an
`ASIC, a configured processor,all optical media, all magnetic
`tape or other magnetic media, or any other medium from
`which a computer processor can read instructions. The
`instructions may comprise code from any computer-pro-
`gramming language, including, for example, C, C#, Visual
`Basic, Java, and JavaScript.
`The embodiment shownin FIG. 1 may be implemented in
`a variety of devices. Such devices include personal comput-
`ers, many of which include an integrated touchpad. Such
`devices mayalso include handheld devices, such as handheld
`organizers, cellular telephones, handheld communicators,
`MP3players, GPSreceivers, and the like.
`Embodimentsof the present invention may also be utilized
`to implement haptic effects in devices such as those men-
`tioned above. In such an embodiment,the haptic effects result
`from various actions by a user interfacing with a touch-sen-
`sitive input device, and the effects may be based onthe user’s
`intent as determinedbythe processor 106. Haptic effects may
`also result from interaction with software executing on a
`device in communication with the touch-sensitive input
`device.
`
`Embodimentsof the present invention address the difficul-
`ties faced in attempting to determinethe intent of a user based
`on the X, Y, and Z parameters supplied bythe touchpad 102.
`Examples of determining a user’s intent include determining
`when a useris tapping or pressing on a specific portion of a
`touch-sensitive input device that corresponds to a control
`displayed on the input device or displayed on a separate,
`synchronized display.
`Embodimentsofthe present invention provide systems and
`methodsfor adaptive interpretationof the intent ofa user ofa
`touch-sensitive input device. In one embodiment of the
`present invention, a processorreceives a pressure signal indi-
`cating a pressure from the input device, compares the pres-
`sure signal to an adaptive pressure threshold value, and out-
`puts a signal if the pressure signal is greater than the adaptive
`pressure threshold value. The pressure may be a pseudopres-
`sure or an explicit pressure. Also, the pressure maybefiltered.
`Embodimentsof the present invention mayalso utilize the
`velocity of the conductoracross the touchpad in determining
`a user’s intent. Additionally, an embodiment may utilize
`adaptive thresholds alone or in combination with digitalfil-
`tering to more accurately determine a user’s intent.
`Thresholds for pressure, pseudo pressure, pseudo-pressure
`change, velocity, and other measures may be stored in a
`computer-readable medium when the device is manufac-
`
`

`

`US 8,164,573 B2
`
`5
`tured. Alternatively, software executed by a processor may
`provide settings for the thresholds. Thresholds set by soft-
`ware maybestatic or adaptive. Adaptive thresholds mayrely
`on various parameters, including, for example, the length of
`time the input device has been active, the placement of the
`conductor on the surface of the input device, and the current
`userof the device.
`
`FIG. 2 is a flow chart illustrating a process or algorithm for
`detecting and interpreting finger presses on the touchpad
`(102) according to the present invention. In the embodiment
`shown, a keypadis displayed on the touchpad (102) or on a
`corresponding display. In various embodiments, the keypad
`may bevirtual or physical, and may be displayed or not
`displayed. A processor executing the process shown com-
`pares the pseudopressure against a minimum threshold value
`and compares changesin pseudopressure against additional
`minimum thresholds.
`
`6
`(102) orafter the first tick count is started, the processor (106)
`determines wherethe finger is positioned 210. The processor
`(106) makesthis determination based on the X and Y coordi-
`nates provided by the touchpad (102).
`In the embodiment shown,the processor(106) then utilizes
`the coordinates to determine whether the finger is on a key
`212. Each key displayed on the touchpad (102) or corre-
`sponding display is associated with numerous attributes.
`Theseattributes include characteristics of the key, such as the
`size, posilion and behavior of the key. The processor (106)
`determinesif the finger is on the key by comparing the X and
`Y position data reported by the touchpad (102) to the charac-
`teristics of the key.If the finger is not on a key, the processor
`(106) repeats the process beginning at step 204. If the finger
`is on a key, the processor (106) determines whether the
`release tick counthas elapsed 214. Ifthe release tick count has
`not elapsed, then the processor (106) repeats the process
`The processor (106) may use adaptive thresholds. For
`beginningat step 204. Ifthe release tick count has elapsed, the
`example, the processor (106) mayutilize different threshold
`processor (106) determines whetheror notthefirst tick count
`values based onthe