(12)
`
`United States Patent
`Martin et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,336,260 B2
`Feb. 26, 2008
`
`US007336260B2
`
`(54) METHOD AND APPARATUS FOR
`PROVIDING TACTILE SENSATIONS
`
`3,220,121 A 11/1965 Cutler
`3,497,688 A
`2/1970 Hirsch
`
`(75) Inventors: Kenneth M. Martin, Los Gatos, CA
`
`(Continued)
`
`Steven P- Vassallo, Redwood City, CA (US); Alex S. Goldenberg,
`
`Portola Valley, CA (US); Alexander
`Jasso, Los Altos, CA (US); Kollin
`Tierling, Milpitas, CA (US)
`
`EP
`
`0349086
`
`I/ 1990
`
`_
`(Con?rmed)
`
`(73) Assignee: Immersion Corporation, San Jose, CA
`(US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 730 days.
`
`(21) APP1~ NOJ 10/285,450
`
`(22) Filed:
`
`Nov. 1, 2002
`
`(65)
`
`Prior Publication Data
`Us 2003/0122779 A1
`]u1_ 3, 2003
`
`Related US. Application Data
`(60) Provisional application No. 60/335,493, ?led on Nov.
`1 2001 provisional application No 60/399 883 ?led
`’ J 1 151 2002
`'
`’
`’
`On u '
`’
`'
`
`(51) Int Cl
`GobG /00
`52 U 5 Cl
`(58) F: l'd
`(
`)
`1e
`
`0
`
`(2006 01)
`345/169 3 4 5 /1 5 6
`'
`’345/156
`"" """
`""
`’
`assl ca Ion earc """"""" "
`_
`345/173’ 157’ 161’ 163} 167T169
`_
`See apphcanon ?le for Complete Search hlstory'
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`2/1961 Hirsch
`2,972,140 A
`3,157,853 A 11/1964 Hirsch
`
`OTHER PUBLICATIONS
`Adelstein, “A Virtual Environments System For The Study of
`Human Arm Tremor,” Ph.D. Dissertation, Dept. of Mechanical
`Engineering, MIT, JllI1~ 1989
`(Continued)
`
`Primar ExamineriRicardo Osorio
`y
`(74) Attorney, Agent, or F irmiKilpatrick Stockton LLP
`
`(57)
`
`ABSTRACT
`
`Products and processes for providing tactile sensations to
`input devices or electronic devices are provided. Input
`devices include mechanical input devices (such as, for
`example, mechanical switches) and non-mechanical input
`fievlces-ésgcél as’ -for example’ touchp?dsy Tacnl-e feedback
`15- pro-vl e - y uslpg an acu-lator or 0t er means n Commu
`n1cat1on W1th the 1nput device or electromc device. A con
`troller may be employed to receive signals from the input
`devices and control the actuator. Tactile feedback to an in ut
`P
`device or electronic device may be provided in response to
`one or more events or situations. Such an event or situation
`may be any one designated. Examples of such events and
`situations include the level of pressure placed on an input
`device; the availability or lack of availability of a function
`associated With an input device; and the function, menu, or
`mode of operation associated With an input device’s activa
`tion. A variety of feedback types and combinations may be
`selected.
`
`9 Claims, 11 Drawing Sheets
`
`18
`\
`W Z)
`
`j 4
`34
`'
`
`33\
`
`24
`
`s \_
`
`\
`
`\‘Q
`
`35
`/
`
`é \
`
`21
`
`20
`
`/ §
`64
`
`/
`22
`
`13
`
`V
`
`l8
`
`W A
`\
`30
`
`\_
`34
`
`21
`
`A
`
`SCEA Ex. 1006 Page 1
`
`

`
`US 7,336,260 B2
`Page 2
`
`US. PATENT DOCUMENTS
`
`6/1970 Corlyon et al.
`3,517,446 A
`3,623,064 A 11/1971 Kagen
`3902687 A
`9/1975 Hightower
`3,903,614 A
`9/1975 Diamond et a1.
`3,911,416 A 10/1975 Peder
`4,127,752 A 11/1978 Lowthorp
`4,160,508 A
`7/1979 Salsbury
`4,236,325 A 12/1980 Hall et 31.
`4,262,549 A
`4/1981 Schwellenbach
`4,311,980 A
`1/1982 Prudenziati
`4,333,070 A
`6/1982 Barnes
`4,362,408 A 12/1982 Cordes et a1. ............ .. 400/481
`4,464,117 A
`8/1984 Foerst
`4,484,191 A 11/1984 Vavra
`4,513,235 A
`4/1985 Acklam et 31.
`4,581,491 A
`4/1986 Boothroyd
`4,581,972 A
`4/1986 Hoshino
`4,599,070 A
`7/1986 Hladky er 91
`4692756 A
`9/1987 Clark
`4,708,656 A 11/1987 De Vries et 31.
`4,713,007 A 12/1987 Alba“
`4,725,817 A
`2/1988 Wihlborg
`4,791,416 A 12/1988 Adler
`4,794,392 A 12/1988 Selinko
`4,795,296 A
`V1989 Jau
`4,798,919 A
`V1989 Suita
`4,821,030 A
`4/1989 Batson
`4,823,106 A
`4/1989 Pope
`4,840,634 A
`6/1989 Muller
`4,885,565 A 12/1989 Embach
`4,891,764 A
`1/1990 McIntosh
`4,930,770 A
`6/1990 Baker
`4,934,694 A
`6/1990 McIntosh
`4982916 A
`l/199l DuPOIlt
`4,983,786 A
`1/1991 Stevens
`5,019,761 A
`5/1991 Kraft
`5,022,384 A
`6/1991 Freels
`5,022,407 A
`6/ 1991 Horch er 91-
`5,035,242 A
`7/1991 Franklin et a1.
`5,038,089 A
`8/1991 Szakaly
`5,053,585 A 10/1991 Yaniger
`5,078,152 A
`1/1992 Bond
`5,116,051 A
`5/1992 Moncrief
`5,165,897 A 11/1992 Johnson
`5,175,459 A 12/1992 Danial @1111
`5,182,557 A
`V1993 Lang
`5,186,685 A
`2/1993 Grossmann @1111
`5212473 A
`5/1993 Louis
`5223658 A
`6/1993 Suzuki
`5,237,327 A
`8/1993 Saitoh
`5,240,417 A
`8/1993 Smithson er 91-
`5,241,308 A
`8/1993 Young
`5,246,316 A
`9/1993 Smith
`5,271,290 A 12/1993 Fischer
`5,275,174 A
`V1994 (399k
`5,283,970 A
`2/1994 Aigner
`5,289,273 A
`2/ 1994 Lang
`5,299,810 A
`4/1994 Pierce et a1.
`5,309,140 A
`5/1994 Everett
`5,334,027 A
`8/1994 Wherlock
`5,355,148 A 10/1994 Anderson
`5,390,128 A
`2/1995 Ryan
`5,390,296 A
`2/1995 Crandall
`5,402,499 A
`3/1995 Robinson
`5,402,680 A
`4/1995 Korenaga
`5,436,622 A
`7/1995 Gntrnan eta1.
`5,437,607 A
`8/1995 Taylor
`5,451,924 A
`9/1995 Massimino
`5,461,711 A 10/1995 Wang
`5,466,213 A 11/1995 Hogan eta1.
`5,489,812 A
`2/1996 Furuhata
`
`3/1996 Garner
`5,496,174 A
`5/1996 Rostoker
`5,514,150 A
`5/1996 Buchanan
`5521336 A
`8/1996 Yamasaki et a1.
`5,547,382 A
`5,575,761 A 11/1996 Hajianponr
`5,631,861 A
`571997 Kramer
`5,684,722 A 11/1997 Thorner
`5,691,747 A l1/1997 Anlan0
`5,709,219 A
`1/1998 Chen
`5,729,249 A
`3/1998 Yasutake
`5,766,016 A
`6/1998 Sinclair et 31‘
`5,767,457 A
`6/199g Gerpheide
`5,785,630 A
`7/1998 Bobick eta1.
`5,791,992 A
`8/1998 clump
`5,844,392 A 12/1998 Peurach
`5,857,986 A
`1/1999 Moriyasn
`5,887,995 A
`3/1999 Holehan
`5,889,670 A
`3/1999 Schuler
`5,889,672 A
`3/1999 Schuler
`5,917,906 A
`6/1999 Thornton
`5,943,044 A
`8/1999 Martine11i
`5,945,772 A
`g/1999 Macnak
`5,977,867 A 11/1999 Bouin
`5,988,902 A 11/1999 Holehan
`6,059,506 A
`5/2000 Kramer
`6,078,126 A
`6/2000 Rollins
`6,097,964 A
`8/2000 Nuovo
`6,111,577 A
`8/2000 Zilles eta1.
`6,118,435 A
`9/2000 Fujita et a1.
`6,131,097 A 10/2000 Peurach
`6,160,489 A 12/2000 Perry eta1.
`6,195,592 B1
`2/2001 Schuler
`6,198,206 B1
`3/2001 Saarmaa
`6,218,966 B1
`4/2001 Goodwin
`6,219,034 B1
`4/2001 Elbing et a1.
`6,225,976 B1
`5/2001 Yates
`6,292,173 B1* 9/2001 Rarnhaldi eta1. ......... .. 345/157
`6,307,465 B1
`10/2001 Kayarna eta1.
`. 338/114
`6,344,791 B1* 2/2002 Arnistrong ..
`6,369,803 B2* 4/2002 Brisebois et a1. ......... .. 245/173
`6,373,463 B1
`4/2002 Beeks
`6,374,255 B1
`4/2002 Peurach
`6,388,655 B1
`5/2002 Leung
`6,422,941 B1
`7/2002 Thornereta1.
`6,429,846 B2
`8/2002 Rosenberg
`6,543,487 B2
`5/2003 Bazinet
`6,597,347 B1* 7/2003 Yasutake .................. .. 345/173
`6,657,617 B2* 12/2003 Pao1inieta1. ..
`345/173
`6,781,569 B1* 8/2004 Gregorio eta1.
`345/157
`6,801,191 B2* 10/2004 Mukai et a1.
`345/179
`6,976,562 B1* 12/2005 Perreteta1. ........... .. 188/24.11
`2002/0177471 A1
`1/2002 Kaaresoja
`2002/0033795 A1
`3/2002 shahoian
`2002/0128048 A1
`9/2002 Aa1tonen
`2002/0149561 A1 10/2002 Fukumoto eta1.
`2002/0171621 A1* 11/2002 Johnson .................... .. 345/156
`2005/0099393 A1* 5/2005 Johnson .................... .. 345/163
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`WO
`W0
`W0
`
`7/1990
`01003664
`7/1990
`H2-185278
`1/1992
`02109714
`1/1992
`H4-8381
`8/1993
`04007371
`8/1993
`H5-192449
`1/1995
`05193862
`1/1995
`H7-24147
`2001350592 A 12/2001
`2002259059 A
`9/2002
`W09520787
`8/1995
`W0 09718546 A1
`5/1997
`W0 02/27645
`4/2002
`
`SCEA Ex. 1006 Page 2
`
`

`
`US 7,336,260 B2
`Page 3
`
`W0
`W0
`
`W0 0231807 A1
`W0 019110A1 A1
`
`4/2002
`11/2002
`
`OTHER PUBLICATIONS
`
`Adelstein, “Design and Implementation of a Force Re?ecting
`Manipulandum for Manual Control research,” DSC-vol. 42,
`Advances in Robotics, Edited by H. KaZerooni, pp. 1-12, 1992.
`Aukstakalnis et al., “Silicon Mirage: The Art and Science of Virtual
`Reality,” ISBN 0-938151-82-7, pp. 129-180, 1992.
`Baigrie, “Electric Control LoadingiA Low Cost, High Perfor
`mance Alternative,” Proceedings, pp. 247-254, Nov. 6-8, 1990.
`BejcZy, “Sensors, Controls, and Man-Machine Interface for
`Advanced Teleoperation,” Science, vol. 208, No. 4450, pp. 1327
`1335, 1980.
`BejcZy, “Generalization of Bilateral Force-Re?ecting Control of
`Manipulators,” Proceedings Of Fourth CISM-IFToMM, Sep. 8-12,
`1981.
`BejcZy, et al., “Universal Computer Control System (UCCS) For
`Space Telerobots,” CH2413-3/87/0000/0318501.00 1987 IEEE,
`1987.
`BejcZy et al., “A Laboratory Breadboard System For Dual-Arm
`Teleoperation,” SOAR ’89 Workshop, J SC, Houston, TX Jul. 25-27,
`1989.
`Bliss, “Optical-to-Tactile Image Conversion for the Blind,” IEEE
`Transactions on Man-Machine Systems, vol. MMS-11, No. 1, Mar.
`1970.
`Brooks et al., “Hand Controllers for TeleoperationiA State-of-the
`Art Technology Survey and Evaluation,” JPL Publication 85-11;
`NASA-CR-175890; N85-28559, pp. 1-84, Mar. 1, 1985.
`Burdea et al., “Distributed Virtual Force Feedback, Lecture Notes
`for Workshop on Force display in Virtual Environments and its
`Application to Robotic Teleoperation,” 1993 IEEE International
`Conference on Robotics and Automation, pp. 25-44, May 2, 1993.
`Cadler, “Design of A Force-Feedback Touch-Introducing Actuator
`For Teleoperator Robot Control,” Bachelor of Science Thesis, MIT,
`Jun. 23, 1983.
`Caldwell et al., “Enhanced Tactile Feedback (Tele-Traction) Using
`a Multi-Functional Sensory System,” 1050-4729/93. pp. 955-960,
`1993.
`Eberhardt et al., “OMARiA Haptic display for speech perception
`by deaf and def-blind individuals,” IEEE Virtual Reality Annual
`International Symposium, Seattle, WA, Sep. 18-22, 1983.
`Eberhardt et al., “Including Dynamic Haptic Perception by The
`Hand: System Description and Some Results,” DSC-vol. 55-1,
`Dynamic Systems and Control: vol. 1, ASME 1994.
`Fokumoto, “Active Click: Tactile Feedback for Touch Panels,”
`ACM CHI2001 Extended Abstracts, pp. 121-122, Apr. 2001.
`Gobel et al., “Tactile Feedback Applied to Computer Mice,” Inter
`national Journal of Human-Computar Interaction, vol. 7, No. 1, pp.
`1-24, 1995.
`Gotow et a1, “Controlled Impedance Test Apparatus for Studying
`Human Interpretation of Kinesthetic Feedback, ” WA11-11:00. pp.
`332-337.
`Howe, “A Force-Re?ecting Teleoperated Hand System for the
`Study of Tactile Sensing in Precision Manipulation,” Proceedings of
`the 1992 IEEE International Conference on Robotics and Automa
`tion, Nice, France, May 1992.
`IBM Technical Disclosure Bulletin, “Mouse Ball-Actuating Device
`With Force and Tactile Feedback,” vol. 32, No. 98, Feb. 1990.
`International Search Report ?led Nov. 1, 2002 corresponding to
`PCT/US02/35016.
`IWATA, “Pen-based Haptic Virtual Environment,” 0-7803-1363-1/
`93 IEEE, pp. 287-292, 1993.
`Jacobsen et al., “High Performance, Dextrous Telerobotic Manipu
`lator With Force Re?ection,” Intervention/ROV ’91 Conference &
`Exposition, Hollywood, Florida, May 21-23, 1991.
`Johnson, “Shape-Memory Alloy Tactile Feedback Actuator,”
`Armstrong Aerospace Medical Research Laboratory, AAMRL-TR
`90-039, Aug. 1990.
`Jones et al., “A perceptual analysis of stiffness,” ISSN 0014-4819
`Springer International (Springer-VeItag); Experimental Brain
`Research, vol. 79, No. 1, pp. 150-156, 1990.
`KacZmarek et al., “Tactile Displays,” Virtual Environment Tech
`nologies.
`
`Kontarinis et al., “Display of High-Frequency Tactile Information to
`Teleoperators,” Telemanipulator Technology and Space Telerobot
`ics, Won S. Kim, Editor, Proc. SPIE vol. 2057, pp. 40-50, Sep. 7-9,
`1993.
`Kontarinis et al., “Tactile Display of Vibratory Information in
`Teleoperation and Virtual Environments,” PRESENCE, 4(4):387
`402, 1995.
`Lake, “Cyberman from Logitech,” GameBytes, 1994.
`Marcus, “Touch Feedback in Surgery,” Proceedings of Virtual
`Reality and Medicine The Cutting Edge, Sep. 8-11, 1994.
`McAffee, “Teleoperator Subsystem/Telerobot Demonsdtrator:
`Force Re?ecting Hand Controller Equipment Manual,” JPL
`D-5172, pp. 1-50, A1-A36, B1-B5, C1-C36, Jan. 1988.
`Minsky, “Computational Haptics: The Sandpaper System for Syn
`thesiZing Texture for a Force-Feedback Display,” Ph.D. Disserta
`tion, MIT, Jun. 1995.
`Ouh-Young, “Force Display in Molecular Docking,” Order No.
`9034744, p. 1-369, 1990.
`Ouh-Young, “A Low-Cost Force Feedback Joystick and Its Use in
`PC Video Games,” IEEE Transactions on Consumer Electronics,
`vol. 41, No. 3, Aug. 1995.
`OuhYoung et al., “The Development of a Low-Cost Force Feedback
`Joystick and Its Use in the Virtual Reality Environment,” Proceed
`ings of the Third Paci?c Conference on Computer Graphics and
`Applications, Paci?c Graphics ’95, Seoul, Korea, Aug. 21-24, 1995.
`Patrick et al., “Design and Testing of A Non-reactive, Fingertip,
`Tactile Display for Interaction with Remote Environments,” Coop
`erative Intelligent Robotics in Space, Rui J. deFigueiredo et al.,
`Editor, Proc. SPIE vol. 1387, pp. 215-222, 1990.
`Pimentel et al., “Virtual Reality: through the new looking glass,” 2nd
`Edition; McGraw-Hill, ISBN 0-07-050167-X, pp. 41-202, 1994.
`RabinowitZ et al., “Multidimensional tactile displays: identi?cation
`of vibratory intensity, frequency, and contractor area,” Journal of
`The Accoustical Society of America, vol. 82, No. 4, Oct. 1987.
`Russo, “The Design and Implementation of a Three Degree of
`Freedom Force Output Joystick,” MIT Libraries Archives Aug. 14,
`1990, pp. 1-131, May 1990.
`Russo, “Controlling Dissipative Magnetic Particle Brakes in Force
`Re?ective Devices,” DSC-vol. 42, Advances in Robotics, pp. 63-70,
`ASME 1992.
`Scannell, “Taking a Joystick Ride”, Computer Currents, Nov. 1994,
`Boston Edition, vol. 9, No. 11.
`Shimoga, “Finger Force and Touch Feedback Issues in Dexterous
`Telemanipulation,” Proceedings of Fourth Annual Conference on
`Intelligent Robotic Systems for Space Exploration, Rensselaer
`Polytechnic Institute, Sep. 30-Oct. 1, 1992.
`SMK Corporation, “Multi-Functional Touch Panel, Force-Feedback
`Type, Developed: A Touch Panel Providing a Clicking Feeling,”
`http://www.smk.cojp/whatsnewie/628csciehtml, Sep. 30, 2002.
`SMK Corporation, “Force Feedback Type Optical Touch Panel
`Developed,” SMK Corporation Website, Oct. 30, 2002.
`Snow et al., “Model-X Force-Re?ecting-Hand-Controller,” NT
`Control No. MPO-17851; JPL Case No. 5348, pp. 1-4, Jun. 15,
`1989.
`Stanley et al., “Computer Simulation of Interacting Dynamic
`Mechanical Systems Using Distributed Memory Parallel Proces
`sors,” DSV-vol. 42, Advances in Robotics, pp. 55-61, ASME 1992.
`Tadros, Control System Design for a Three Degree of Freedom
`Virtual Environment Simulator Using Motor/Brake Pair Actuators,
`MIT Archive© Massachusetts Institute of Technology, pp. 1-88,
`Feb. 1990.
`Terry et al., “Tactile Feedback In A Computer Mouse,” Proceedings
`of Fourteenth Annual Northeast Bioengineering Conference, Uni
`versity of New Hampshire, Mar. 10-11, 1988.
`Wiker, “Teletouch Display Development: Phase 1 Report,” Tech
`nical Report 1230, Naval Ocean Systems Center, San Diego, Apr.
`17, 1989.
`Wakiwaka, et al., “In?uence of Mover Support Structure on Linear
`Oscillatory Actuator for Cellular Phones,” The Third International
`Symposium on Linear Drives for Industry Applications, 2001, p.
`260-263, Nagano, Japan.
`
`* cited by examiner
`
`SCEA Ex. 1006 Page 3
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 1 6f 11
`
`US 7,336,260 B2
`
`SCEA Ex. 1006 Page 4
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 2 6f 11
`
`US 7,336,260 B2
`
`SCEA Ex. 1006 Page 5
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 3 6f 11
`
`US 7,336,260 B2
`
`/
`
`SCEA Ex. 1006 Page 6
`
`

`
`U.S. Patent
`
`Feb. 26,2008
`
`Sheet 4 of 11
`
`US 7,336,260 B2
`
`
`
`3mp§s\\\..@.\W§ Kpwmw.\-\.w
`
`as2we.5.“.2
`
`¢.O_n_
`
`
`
` & xx5L\\§\\..\..:\;.mmWs.
`
`SCEA EX. 1006 Page 7
`
`SCEA Ex. 1006 Page 7
`
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 5 6f 11
`
`US 7,336,260 B2
`
`FIG.5
`
`SCEA Ex. 1006 Page 8
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 6 6f 11
`
`US 7,336,260 B2
`
`§\ \ E
`
`owl/r
`
`31/!
`
`i v /
`
`\ \g
`
`a w.
`
`\
`7/l\| a
`a
`
`\\ \6.
`x
`
`SCEA Ex. 1006 Page 9
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 7 6f 11
`
`US 7,336,260 B2
`
`Input
`40/ Delnoes
`
`.
`
`Electronic
`(3:283? \43
`
`D '
`
`f3?!
`
`41
`\ Controller
`
`_
`44
`Drsploy /
`
`Control
`45 j Circultry
`
`Memory f42
`
`Actuator
`
`45f
`
`FIG.7
`
`SCEA Ex. 1006 Page 10
`
`

`
`U.S. Patent
`
`Feb. 26,2008
`
`Sheet 8 0f 11
`
`US 7,336,260 B2
`
`Access
`Memory
`
`72
`
`Obtain
`Ambiguous
`Feedback
`Info.
`
`Produce
`Tactile
`Sensation
`
`Pressure
`Data?
`
`Obtain
`Pressure
`Data
`
`FIG.8
`
`SCEA Ex. 1006 Page 11
`
`

`
`U.S. Patent
`
`Feb. 26,2008
`
`Sheet 9 of 11
`
`US 7,336,260 B2
`
`zorcxmzmm5:95
`
`.\.cocamcumwcocamcommcocmmcumvcocmmcummcozamcomNcosmmcum#cocdmcum
`
`ficocmwcomSconmmcumZco_:mmcumEcozmmcommcocmmcomwcosmmcum
`
`¢._cozficmm.
`
`
`
` M:cocmmcumEcocmmcomEcocmmcum,2cocmwcum
`
`omcocmmcum2cocmwcom
`
`mmcozmmcomScocmmcum
`
`mmcozmmcom
`
`mGE
`
`Ssznmcoconsm_
`
`SCEA EX. ‘I006 Page ‘I2
`
`%
`
`ao:2m2<H_cozocnmflPfimmflm n3.5w .mBnwmoimcofiwomo5.5
`
`I m
`
`:o.su_.§..HNBsmmuiNcocaomm3.5o_cocopnm~0580.5Hcozaomv3.5H,.233;-.m _.-m4.NBsmmoi
`-.m~838$..m252 ,NBsaflm..m:35wW:52_EsmmflmHW2325--Qi
`:0-._copsom--
`N,V233;IB5.5Xmofimmuion3.5m3NEEE--%imA233iI«N5.5N8225.-.N
`
`
`
`
`
`
`..Hnv._BsmmfimH,cocmooq #Pfimmoiv~conmooa
`
`
`
`
`
`
`
`zofiuzbm<H<Dmmsmmmma<._.<DzorzmomizoaSane.mu:/mo5&5
`
`
`
`
`
`
`
`SCEA Ex. 1006 Page 12
`
`
`

`
`U.S. Patent
`
`Feb. 26,2008
`
`Sheet 10 of 11
`
`US 7,336,260 B2
`
`Mcocmmcom
`
`N—COCMMCDW:cozmmcumScozmmcum0cozficummcocnmcommcosmmcum©CO.5mmCD%mcosamcomVcocmmcummcocmmcom
`
`wéCO.SNmEDm.3cozficom2cocmwcom
`
`CgCOCmmCU®
`
`zoE<mzmmmSC.U<.H.
`
`ZO:.UZbm<53mmammfla
`
`
`
`/2.40zoEmom442055.57:
`
`.
`
`mmcocmmcum_|Zcocmmcom
`o 32$
`2823%22.22E225.5%I LEammuiUcEnmmuimm228$<N23395NHo.=._m$£% m:52m233$.:52_233$NmPfimmflm>VBsmmflmXmEsmmoiH:23H glH..
`uv#ofimmui_V
`Eg
`..mm5.5I«N5.5
`HqoéomIflcocmuoq_3::_cocaooq
` .-m:o:9m2<Ncosmooqw3.5_cocmooqn:15mcocfiomW«San:Ncofmomm8.5H8.3.53v5.5
`..m39:
`..mm§.£
`--om5.5
`.-om3::
`
`.2cocmmcum
`
`.2cocmmcum
`
`.©~COCmmCU®
`
`omcocmmcom
`
`
`
`mmcosmmcom
`
`..PEEKcosoasm
`
`3.
`DE
`
`M
`
`mB>mE:37:
`
`SCEA EX. ‘I006 Page 13
`
`SCEA Ex. 1006 Page 13
`
`
`
`
`

`
`U.S. Patent
`
`Feb. 26, 2008
`
`Sheet 11 6f 11
`
`US 7,336,260 B2
`
`Ctrl+N
`Ctr|+0
`
`dit ?ew
`—[ ||e
`D New
`E Qpen
`112*‘
`Qlose
`\ LEW
`*5 Erint
`Ctrl+P
`//
`Send To
`/}
`Properties
`63 Hgii Recipient (as Attachment)‘
`Exit
`l? Microsoft EowerPoini
`\—-‘| 14
`
`110-_\/ y
`
`102——/
`
`SCEA Ex. 1006 Page 14
`
`

`
`US 7,336,260 B2
`
`1
`METHOD AND APPARATUS FOR
`PROVIDING TACTILE SENSATIONS
`
`RELATED APPLICATIONS
`
`This application claims priority to US. Provisional Appli
`cation No. 60/335,493, ?led Nov. 1, 2001, and US. Provi
`sional Application No. 60/399,883 ?led Jul. 31, 2002. The
`entire disclosure of both applications are incorporated herein
`by reference.
`
`FIELD OF THE INVENTION
`
`The present invention relates to methods and apparatus
`for providing tactile sensations.
`
`BACKGROUND
`
`Conventional electronic devices, such as mobile tele
`phones and Personal Digital Assistants (PDAs), include
`visual displays. A user of such devices interacts With the
`visual display using any one of a number of input devices.
`Examples of such input devices include computer mice,
`joysticks, trackballs, steering Wheels, stylus, tablets, pres
`sure-sensitive spheres, scroll Wheels, keyboards, and key
`pads. The user provides instructions, responses, and other
`input to the device using such input devices.
`In conventional mobile telephones and PDAs, con?rma
`tion of the input provided by the user is primarily limited to
`visual or audible con?rmation. In some such devices, physi
`cal feedback is provided by conventional mechanical
`sWitches in the form of the conventional mechanical feed
`back of sWitches, for example the sWitch closure force
`displacement pro?le. Typically, in such devices, the
`mechanical feedback provided by each button is identical. In
`addition, in such conventional devices, for those buttons that
`serve multiple functions, the mechanical feedback generally
`remains the same regardless of the current function of the
`button.
`In addition to providing extremely limited and rudimen
`tary mechanical con?rmation of button selection, conven
`tional buttons as used, for example, in keypads for mobile
`telephones and PDAs, provide simple passive touch cues
`regarding the alignment of keys. Such cues include raised
`bumps on the center key of a telephone keypad or on the “F”
`and “G” keys of a keyboard that assist a user in orienting to
`the pattern of keys in the keypad and keyboard. Again, these
`physical queues are very limited, and users typically need to
`vieW a keypad or keypad for visual con?rmation that the
`correct instructions or information is being entered.
`When a ?at surface interface device is used, such as a
`touchpad for a computer or PDA, these simple mechanical
`cues are unavailable to the user. Often, touchpads are
`combined With ?at-panel display screens that display one or
`more graphically generated buttons or softkeys. Normally,
`the softkeys are visible through the touchpad. A user’s
`contact With the touchpad in an area de?ned by a softkey
`provides the electronic device having the touchpad With the
`input associated With that softkey.
`The use of electronic devices using such conventional
`mechanical buttons and touchpad arrangements are particu
`larly difficult to use in distracting environments or When the
`user is attempting to perform another task simultaneously
`With using the electronic device. For example, if the other
`task involves operating a motor vehicle or heavy machinery,
`it may be dif?cult or impossible for a user to simultaneously
`use such an electronic device because such devices typically
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`require the user to look at the device, at least brie?y, When
`interacting With the device. In addition, electronic devices
`relying on softkeys can be dif?cult to read in bright light
`environments such as in bright sunlight and can contain very
`small fonts and graphics that are dif?cult to read and select.
`Some conventional touchpads include vibratory feedback
`to the user of the touchpad. US. Pat. No. 5,977,867 is one
`example. Such conventional systems and methods are lim
`ited, though. They lack a full range of functionality assis
`tance to a user interacting With an electronic device. More
`over, such systems and methods still require considerable
`visual attention from the user.
`
`SUMMARY OF THE INVENTION
`
`The present invention comprises products and processes
`for providing tactile sensations to input devices or electronic
`devices. Input devices include mechanical input devices
`(such as, for example, mechanical sWitches) and non-me
`chanical input devices (such as, for example, touchpads).
`Tactile feedback is provided by using an actuator or other
`means in communication With the input device or electronic
`device. A controller may be employed to receive signals
`from the input devices and to control the actuator. Tactile
`feedback to an input device or electronic device may be
`provided in response to one or more events or situations.
`Such an event or situation may be any one designated.
`Examples of such events and situations include the level of
`pressure placed on an input device; the availability or lack
`of availability of a function associated With an input device;
`and the function, menu, or mode of operation associated
`With an input device’s activation. A variety of feedback
`types and combinations may be selected. Mobile telephones
`and PDAs bene?t from employing such products and pro
`cesses, but other devices bene?t as Well. The advantages
`offered by the various embodiments of the present invention
`may be understood by examining this speci?cation.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a perspective vieW of an embodiment of an
`apparatus according to the present invention;
`FIG. 2 is a plan vieW of another embodiment of an
`apparatus according to the present invention;
`FIG. 3 is a plan vieW of an electronic device including an
`embodiment of the present invention;
`FIG. 4 is a schematic representations through line 4-4 of
`FIG. 3;
`FIG. 5 is a plan vieW of another electronic device includ
`ing another embodiment of the present invention;
`FIG. 6 is a schematic representations through line 6-6 of
`FIG. 5;
`FIG. 7 is a block diagram illustrating an embodiment of
`the apparatus in an electronic device;
`FIG. 8 is a How chart illustrating a method according to
`the present invention;
`FIG. 9 is a table illustrating a ?rst set of data to be used
`in one embodiment of the present invention; and
`FIG. 10 is a table illustrating a second set of data to be
`used in another embodiment of the present invention.
`FIG. 11 is a plan vieW of another embodiment of the
`present invention.
`
`DETAILED DESCRIPTION
`
`The present invention includes methods and systems for
`providing tactile sensations. One embodiment includes
`
`SCEA Ex. 1006 Page 15
`
`

`
`US 7,336,260 B2
`
`3
`methods and systems for providing tactile sensations to
`input devices, both mechanical and non-mechanical (for
`example soft-keys that are computer generated and dis
`played on a screen). Embodiments of the present invention
`can be utilized in Wide variety of electronic devices includ
`ing telephones, mobile telephones, remote controls, game
`pads, joystick handles, automotive controls (radios, Com
`pact Disc (CD) players, automobile functions, etc.),
`consumer electronics devices, Personal Digital Assistants
`(PDAs), personal computers, laptop computers, portable
`gaming devices, pagers, I-pagers, audio equipment, televi
`sions, security or alarm systems, Automated Teller Machines
`(ATM), calculators, home appliances, and White goods.
`FIG. 1 shoWs one embodiment of the present invention.
`The apparatus 1 shoWn in FIG. 1 includes an input device 2
`having multiple positions for communicating a plurality of
`input signals. The input device 2 can be any device capable
`of transmitting an input signal. In the embodiment shoWn,
`the input device 2 is a rocker-type sWitch. The rocker sWitch
`2 shoWn can pivot or rock betWeen tWo positions in Which
`the rocker sWitch contacts and activates one of tWo rubber
`sWitches 3 containing conductive pads. The use of rubber
`sWitches 3 provides the advantage of alloWing the user to
`still feel a substantial vibration or force through the input
`device 2 When the user had fully depressed the sWitch.
`Suitable rubber sWitches are available and knoWn in the art.
`In other embodiments, the input device may include an
`analog sWitch, a force sending resistor, a strain gauge based
`sensor, a capacitative touch sWitch, a scroll Wheel, a mini
`joystick, a touchpad, a touch screen, a 3-Way sWitch, a 4-Way
`sWitch, a 5-Way sWitch, or other input device. Each position
`of the input device 2 corresponds to one of the input signals.
`The input device 2 and rubber sWitches 3 are mounted on
`a Printed Circuit Board (PCB) 4 in the embodiment shoWn
`to facilitate electrical communication betWeen the input
`device 2 and an electronic device (not shoWn). The PCB 4
`can be custom shaped according to the device into Which the
`apparatus 1 is placed. The PCB 4 also provides for secure
`mounting Within the device by including, for example, a
`plurality of holes 5 to accept fasteners for securing to the
`electronic device. In another embodiment, the input device
`2 can be directly connected or mounted in the electronic
`device.
`The apparatus 1 shoWn in FIG. 1 also includes a vibro
`tactile actuator 6 in communication With the input device 2.
`Preferably, the actuator 6 is con?gured to output a plurality
`of distinct tactile feedback sensations to the input device 2.
`Suitable tactile sensations include vibrations, for example,
`jolts and textures, and a plurality of distinct tactile sensations
`can be created by varying the frequency, amplitude and
`Waveform output by the actuator 6. The actuator 6 is selected
`to deliver the desired tactile sensations to the input device 2
`The actuator 6 shoWn in FIG. 1 is a voice coil actuator. Other
`suitable actuators include, for example, pieZo-electric actua
`tors, eccentric mass actuators, moving magnet actuators, and
`friction brakes in contact With metal shafts. In addition, the
`actuator can include a ?exure, for example an arrangement
`of ?exible material, couple to the rotating shaft of a DC
`motor or step motor to transform the rotation of the motor
`shaft into vibrations or other haptic sensations. Various
`arrangements of a ?exure coupled to a motor may be used
`as an actuator. For example, US. patent application Ser. No.
`09/585,741, ?led Jun. 2, 2000, illustrates suitable arrange
`ments of ?exures and motors for use as an actuator in
`embodiments of the present invention. The entire disclosure
`of the application Ser. No. 09/ 585,741 is incorporated herein
`by reference. Tactile sensations can also be delivered to the
`
`50
`
`55
`
`60
`
`65
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`4
`input device 2 from a speaker included With an electronic
`device into Which the apparatus is placed, for example the
`speaker in a mobile telephone or in a personal computer.
`Although the embodiment shoWn in FIG. 1 includes one
`input device 2 and one actuator 6, other embodiments
`include a plurality of input devices, all in communication
`With a single actuator. Alternatively, an embodiment can
`include a plurality of actuators each in communication With
`at least one input device. Various arrangements of actuators
`in combination With input devices are suitable for use in the
`present invention. For example, US. patent application Ser.
`No. 09/263,263, ?led Jul. 26, 2001, published on Mar. 21,
`2002, as US. Patent Pub. No. US2002/0033795 illustrates
`actuators in combination With input devices that may be
`used in embodiments of the present invention. The entire
`disclosure of application Ser. No. 09/263,263, Pub. No.
`2002/0033795 is incorporated herein by reference.
`As mentioned, the actuator 6 is in communication With the
`input device 2. In the embodiment shoWn in FIG. 1, the
`actuator 6 is in communication With the input device 2
`through a cantilevered beam or lever arms 7 attached to the
`pivoting rocker, amplifying the e?‘ective forces of the actua
`tor 6 felt by the user. The tactile sensations generated by the
`actuator 6 propagate through the lever arm 7 to the input
`device 2. Suitable materials for the lever arm 7 are capable
`of transmitting the tactile sensations and can be, for
`example, metal. The lever arm 7 shoWn includes one or more
`bends 8 to ?t Within the electronic device in Which the
`apparatus 1 is disposed. Di?‘erent shapes of bends may be
`used to ?t Within the electronic device. In another embodi
`ment, the actuator 6 is mounted directly to the input device
`2 or to any component of the input device. Alternatively, the
`actuator 6 is mounted to the PCB 4 to Which the input device
`is attached, communicating tactile sensations to the input
`device through the PCB. In another embodiment, the actua
`tor is an existing eccentric mass motor as is used, for
`example, as a vibrating ringer in a pager or mobile tele
`phone.
`The vibrotactile actuator 6 can also be mounted to a
`portion of the case or housing of the electronic device in
`Which the apparatus 1 is disposed, communicating the tactile
`sensations to the entire electronic device. In one embodi
`ment, tWo actuators can be incorporated in the case or back
`of an electronic device, for example the case of a mobile
`phone in an area that contacts the user’s hand. This arrange
`ment e?cectively doubles the amplitude of the tactile sensa
`tion, and the user’s ?ngers do not tend to attenuate the tactile
`sensation.
`The apparatus 1 also includes a controller 9 in commu
`nication With the input device 2 to receive the input signals
`therefrom. The controller 9 can also receive additional
`information from the input device 2 including the position of
`the input device 2 and the amount of pressure applied to the
`input device 2. In one embodiment, the input signal includes
`information related to the amount of pressure applied to the
`input device 2, information related to the position of the
`input device 2, or a combination of information about
`pressure and position. In addition to being in communication
`With the input device 2, the controller 9 is in communication
`With the actuator 6 to produce a tactile sensation in the
`actuator 6 corresponding to the input or input signal received
`by the controller 9 from the input device 2.
`The controller 9 is located in a suitable location according
`to the needs of the device in Which the apparatus 1 is placed.
`In one embodiment, the controller 9 is attached to the PCB
`4 as shoWn in FIG. 1. Suitable controllers, include, for
`example, digital logical processors capable of processing
`
`SCEA Ex. 1006 Page 16
`
`

`
`US 7,336,260 B2
`
`5
`input, execute algorithms, and generate