(12) United States Patent
`Martin et al.
`
`(10) Patent No.:
`
`(45) Date of Patent:
`
`US 8,773,356 B2
`*Jul. 8, 2014
`
`US008773356B2
`
`(54) METHOD AND APPARATUS FOR
`PROVIDING TACTILE SENSATIONS
`
`(75)
`
`Inventors: Kenneth M. Martin, Los Gatos, CA
`(US); Steven P. Vassallo, Redwood City,
`CA (US); Alex S. Goldenberg, San
`Francisco, CA (US); Alexander Jasso,
`San Jose, CA (US); Kollin M. Tierling,
`Mflpitasa CA (US)
`Immersion Corporation, San Jose, CA
`(US)
`
`(73) Assignee:
`
`(58) Field of Classification Search
`USPC ................................ .. 345/163, 167—169, 173
`See application file for complete search history.
`
`(56)
`
`Referenees Cited
`
`U.S. PATENT DOCUMENTS
`_
`2/1961 Hirsch
`ll/1964 Hirsch
`11/E965 ‘Cutler
`Continued)
`
`2,972,140 A
`3,157,853 A
`3’220’121 A
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`pJatSerg 11s5:x1t)enbde((1) gar adjusted under 35
`~
`~
`~
`(
`) Y
`YS-
`
`EP
`EP
`
`FOREIGN PATENT DOCUMENTS
`0349086
`1/1990
`0817110
`1/1998
`
`This patent is subject to a terminal dis-
`claimer.
`
`(Continued)
`OTHER PUBLICATIONS
`
`(21) App]. NO‘: 13/362’113
`(22) Filed,
`Jam 31, 2012
`
`(65)
`
`Prior Publication Data
`US 2013/0027324 A1
`Jan. 31, 2013
`
`Related U-S-Application Data
`(63) Continuation of application No. 12/894,489, filed on
`Sep. 30, 2010, now Pat. No. 8,159,461, which is a
`continuation of application No. 11/693,117, filed on
`Mar‘. 29’ .2007; n0W.Patj N0‘ 7’808=488= Whlch 15 a
`IC\I0nmiua2t(1)%% 0 apgllcagon N0‘ 10/285>450> filed on
`0V‘
`’
`’now at‘
`0‘ 7’336’260‘
`(60) Provisional application No. 60/335,493, filed on Nov.
`1, 2001, provisional appficafion No. 50/399,883, filed
`on J11]. 31, 2002.
`
`(51)
`
`Int. Cl.
`3/033
`5/00
`(52) U.S. Cl.
`USPC ......................................... .. 345/163; 345/167
`
`(2013.01)
`(2006.01)
`
`Adelstein, “A Virtual Environment System for the Study of Human
`Arm Tremor,” Ph.D. Dissertation, Dept. of Mechanical Engineering,
`MIT, Jun. 1989.
`
`(Continued)
`
`Primary Examiner — Ricardo L Osorio
`(74) Attorney, Agent, or Firm —Kilpatrick Townsend &
`Stockton LLP
`
`(57)
`ABSTRACT
`Systems and methods for providing tactile sensations are
`disclosed. For example, one disclosed method includes the
`steps of outputting a display signal ‘configured to display a
`graPh1Ca10bJeCt0n at0uCh'5en51t1Ve 111131“ deV1Ce; TeCeiVi11g 3
`sensor signal from the touch-sensitive input device, the sensor
`signal indicating an object contacting the touch-sensitive
`input device; determining an interaction between the object
`contacting the touch-sensitive input device and the graphical
`objeietg and generating an actuator signal based at least in part
`ont e interaction.
`
`26 Claims, 11 Drawing Sheets
`
`INPUT DEVICE
`
`
`1
`
`
`
`INPUT SIGNAL
`
`Inut 1
`
`
`
`3
`
`< Pressure 1
`Pressure 1 <=
`--
`--
`Pressure 1
`Pressure 2
`Pressure 3
`X
`Sensation 7
`
`P
`Pressure -1-
`Y
`Sensation 8
`_'In ut 2E
`--
`Pressure 5
`Z
`Sensation 9
`--
`Pressure 1
`Menu 1
`Sensation 10
`
`Input 3
`——
`Pressure 2
`Menu 2
`Sensation 11
`——
`Pressure 3
`Menu 3
`Sensation 12
`--
`Pressure 1
`2
`Sensation 13
`--
`Pressure 2
`A
`Sensation 14
`--
`Pressure 3
`B
`Sensation 15
`—-
`Pressure 4
`C
`Sensation 16
`nput 4
`Position 1
`Pressure 1
`Function 1
`Sensation 17
`Input 5
`Position 2
`J_Pressure 2
`Function 2 L
`Sensation 18
`Ingut 5
`Position 3
`.
`Pressure 3
`Function 3
`Sensation 19
`
`Input 7
`7
`Location 1
`Pressure 1
`Function 1
`Sensation 20
`Location 2
`Pressure 2
`Function 2
`Sensation 21
`
`
`—— —— -- -—
`I? __
`——
`—-
`—-
`Sensation 22
`
`
`
`Sensation 23
`
`
`POSITION DATA
`
`
`Location 1
`Location 1
`Position 2
`--
`
`PRESSURE DATA
`FUNCTION
`TACTILE SENSATION
`
`Search
`Select
`On
`
`Sensation 1
`Sensation 2
`Sensation 3
`
`
`
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 1
`
`
`
`__
`
`
`
`4
`
`5
`
`6
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 1
`
`

`

`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3,497,668 A
`3,517,446 A
`3,623,064 A
`3,902,687 A
`3,903,614 A
`3,911,416 A
`4,127,752 A
`4,160,508 A
`4,236,325 A
`4,262,549 A
`4,311,980 A
`4,333,070 A
`4,362,408 A
`4,464,117 A
`4,484,191 A
`4,513,235 A
`4,581,491 A
`4,581,972 A
`4,599,070 A
`4,692,756 A
`4,708,656 A
`4,713,007 A
`4,725,817 A
`4,791,416 A
`4,794,392 A
`4,795,296 A
`4,798,919 A
`4,821,030 A
`4,823,106 A
`4,840,634 A
`4,885,565 A
`4,891,764 A
`4,930,770 A
`4,934,694 A
`4,982,918 A
`4,983,786 A
`5,019,761 A
`5,022,384 A
`5,022,407 A
`5,035,242 A
`5,038,089 A
`5,053,585 A
`5,078,152 A
`5,116,051 A
`5,165,897 A
`5,175,459 A
`5,182,557 A
`5,186,685 A
`5,212,473 A
`5223553 A
`5,237,327 A
`5,240,417 A
`5,241,308 A
`5,246,316 A
`5,271,290 A
`5,275,174 A
`5,283,970 A
`5,289,273 A
`5,299,810 A
`
`5,334,027 A
`5,355,148 A
`5390128 A
`
`2/1970 Hirsch
`6/1970 Corlyon et a1.
`11/1971 Kagen
`9/1975 Hrghrower
`9/1975 Diamond etal.
`10/1975 Feder
`11/1978 Lowthorp
`7/1979 Salsbury
`12/1980 Hall et a1.
`4/1981 Schwellenbach
`1/1982 Prudenziati
`6/1982 Barnes
`12/1982 Cordes et a1.
`8/1984 Forest
`11/1984 Vavra
`4/1985 Acklamet a1.
`4/1986 Boorhroyd
`4/1986 Hoshino
`7/1986 Hladky er a1.
`9/1987 Clark
`11/1987 DeVrieseta1.
`12/1987 Alban
`2/1988 Jay
`12/1988 Adler
`12/1988 Selrrrko
`1/1989 Wrjlborg
`1/1989 Surra
`4/1989 Barson
`4/1989 Pope
`6/1989 Muller
`12/1989 Embaeh
`1/1990 McIntosh
`6/1990 Baker
`6/1990 McIntosh
`1/1991 Kaye
`1/1991 Stevens
`5/1991 Kraft
`6/1991 Freels
`6/1991 Horch et a1.
`7/1991 Franklin et a1.
`8/1991 Szakaly
`10/1991 Yanrger
`1/1992 Bond
`5/1992 Monerref
`11/1992 Johnson
`12/1992 Danraler a1.
`1/1993 Lang
`2/1993 Grossmarm er a1.
`5/1993 Louis
`5/1993 Suzuki
`3/1993 Sajtoh
`8/1993 Smithson et a1.
`8/1993 Young
`9/1993 Smith
`12/ 1993 Fischer
`1/1994 Cook
`2/1994 Aigner
`2/1994 Lang
`4/1994 Pierce et al.
`
`8/1994 Wherlock
`10/1994 Anderson
`2/1995 RY1111
`
`5,402,499 A
`
`3/1995 Robinson
`
`5,437,607 A
`5,451,924 A
`5,461,711 A
`5,466,213 A
`5,489,812 A
`5,496,174 A
`5,514,150 A
`5,521,336 A
`
`8/1995 Tay1or
`9/1995 Massiniino
`10/1995 Wang
`11/1995 Hogan et a1.
`2/1996 Furuhata
`3/1996 G3-11191
`5/1996 Rostoker
`5/1996 Buchanan
`
`US 8,773,356 B2
`Page 2
`
`5,547,382 A
`5,575,761 A
`5,531,851 A
`5,684,722 A
`5,691,747 A
`5,709,219 A
`5,729,249 A
`5,766,016 A
`5,767,457 A
`5,785,630 A
`5,791,992 A
`5,844,392 A
`5,857,986 A
`5,887,995 A
`5,889,670 A
`5,889,672 A
`5,917,906 A
`5,943,044 A
`5,945,772 A
`5,977,867 A
`5,988,902 A
`6,059,506 A
`6,078,126 A
`6,097,964 A
`6,111,577 A
`6,118,435 A
`6,131,097 A
`6,160,489 A
`6,195,592 B1
`6,198,206 B1
`6,218,966 B1
`6,219,034 B1
`6,225,976 B1
`6,292,173 B1
`6,307,465 B1
`6,344,791 B1
`6,369,803 B2
`6,373,463 131
`6,374,255 B1
`6,388,655 131
`6,422,941 B1
`6,429,846 B2
`6,543,487 132
`6,597,347 131
`6,657,617 B2
`6,735,307 B1
`6,781,569 B1
`6,801,191 B2
`6,976,562 B1
`7,202,851 B2
`2002/0033795 A1
`2002/0128048 A1
`2002/0149561 A1
`2002/0171621 A1
`2002/0177471 A1
`2005/0099393 A1
`2008/0068350 A1
`
`8/1996 Yamasaki et al.
`11/1996 Hajianpour
`5/1997 Kramer
`11/1997 Thorner
`11/1997 A11r11r10
`1/1998 C11611
`3/1998 Yasutake
`6/1998 S1nc1a1r_ et a1.
`6/1998 G6rP1161r1e
`7/1998 B0b1cketa1.
`8/1998 Crump
`12/1998 Peurach
`1/1999 M0r1Y11S11
`3/1999 H016111111
`3/1999 501111161
`3/1999 Schuler
`6/1999 T110r111011_
`8/1999 Mart1ne111
`8/1999 M11°_11111<
`11/1999 13011111
`11/1999 Holehan
`5/2000 Krarrrer
`6/2000 R0111ns
`8/2000 N_u0V0
`8/2000 Z11_1_6S 61111,
`9/2000 Fuptaetal.
`10/2000 Peurah
`12/2000 Perryet 211.
`2/2001 Schuler
`3/2001 Saarmaa
`4/2001 Goodwln
`4/2001 E1b1ng et a1.
`5/2001 Yates
`_
`9/2001 Rambaldl et a1.
`10/2001 Kayamaetal.
`2/2002 Ar_mstr0_ng
`4/2002 Br1seb01seta1.
`4/2002 B6615
`4/2002 Peurah
`5/2002 1611118
`7/2002 Thorneret 211.
`8/2002 Rosenberg
`4/2003 1311211161
`7/2003 Y11S11111_1<6
`12/2003 Pa011n1eta1.
`5/2004 V01cker_s
`8/2004 Greg0_r10eta1.
`10/2004 Mu1<a1eta1.
`12/2005 Perret_, Jr. et a1.
`4/2007 Cunmngham et al.
`3/2002 Shah01an
`9/2002 Aaltonen
`10/2002 Fukumoto et al.
`11/2002 Johnson
`11/2002 Kaaresoja
`5/2005 Johnson
`3/2008 Rosenberg et a1.
`
`FOREIGN PATENT DOCUMENTS
`
`$3
`JP
`
`Jr,
`Jr,
`JP
`
`Jr,
`
`Jr,
`Jr,
`JP
`JP
`JP
`JP
`JP
`JP
`
`0r2_(1J§(3)2A61,2J
`H2_r85278
`
`H4_838r
`04_00737r
`H5-192449
`
`H7_24r47
`
`1 124834
`M85400
`2001—222379
`2001_265485
`2001-290572
`2001-296950
`2001-350592
`2002-259059
`
`$713317)
`7/1990
`
`r/1992
`8/1993
`8/1993
`
`r/1995
`
`r/1999
`3/1999
`8/2001
`9/2001
`10/2001
`10/2001
`12/2001
`9/2002
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 2
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 2
`
`

`

`US 8,773,356 B2
`Page 3
`
`(56)
`
`KR
`WO
`WO
`W0
`WO
`WO
`WO
`WO
`
`References Cited
`FOREIGN PATENT DOCUMENTS
`
`2001-0028369
`95/20787
`97/18546
`99/49443
`01/54109
`02/27645
`02/31807
`02/019110
`
`4/2001
`8/1995
`5/1997
`9/1999
`7/2001
`4/2002
`4/2002
`11/2002
`
`OTHER PUBLICATIONS
`
`Adelstein, “Design and Implementation of a Force Reflecting
`Manipulandum for Manual Control
`research,” DSC-vol. 42,
`Advances in Robotics, Edited by H. Kazerooni, pp. 1-12, 1992.
`Aukstakalnis et al., “Silicon Mirage: The Art and Science of Virtual
`Reality,” ISBN 0-938151-82-7, pp. 129-180, 1992.
`Baigrie, “Electric Control Loading—A Low Cost, High Performance
`Alternative,” Proceedings, pp. 247-254, Nov. 6-8, 1990.
`Bejczy,
`“Sensors, Controls,
`and Man-Machine Interface for
`Advanced Teleoperation,” Science, vol. 208, No. 4450, pp. 1327-
`1335, 1980.
`Bejczy, “Generalization of Bilateral Force-Reflecting Control of
`Manipulators,” Proceedings of Fourth CISM-IFToMM, Sep. 8-12,
`1981 .
`Bejczy, et al., “Universal Computer Control System (UCCS) for
`Space Telerobots,” CH2413-3/87/0000/0318501.00 1987 IEEE,
`1987.
`Bejczy et al., “A Laboratory Breadboard System for Dual-Arm
`Teleoperation,” SOAR ’89 Workshop, JSC, Houston, TX, Jul. 25-27,
`1989.
`Bliss, “Optical-to-Tactile Image Conversion for the Blind,” IEEE
`Transactions on Man-Machine Systems, vol. MMS-11, No. 1, Mar.
`1970.
`Brooks et al., “Hand Controllers for Teleoperation—A State-of-the-
`Art Technology Survey and Evaluation,” JPL Publication 85-11;
`NASA-CR-175890; N85-28559, pp. 1-84, Mar. 1, 1985.
`Burdea et al., “Distributed Virtual Force Feedback, Lecture Notes for
`Workshop on Force Display in Virtual Environments and its Appli-
`cation to Robotic Teleoperation,” 1993 IEEE International Confer-
`ence on Robotics and Automation, pp. 25-44, May 2, 1993.
`Cadler, “Design ofa Force-Feedback Touch-Introducing Actuator for
`Teleoperator Robot Control,” Bachelor of Science Thesis, MIT, Jun.
`23, 1983.
`Caldwell et al., “Enhanced Tactile Feedback (Tele-Traction) Using a
`Multi-Functional Sensory System,” 1050-4729/93, pp. 955-960,
`1993.
`Eberhardt et al., “OMAR—A Haptic display for speech perception
`by deaf and def-blind individuals,” IEEE Virtual Reality Annual
`International Symposium, Seattle, WA, Sep. 18-22, 1993.
`Eberhardt et al ., “Inducing Dynamic Haptic Perception by The Hand:
`System Description and Some Results,” DSC-vol. 55-1, Dynamic
`Systems and Control: vol. 1, ASME 1994.
`Fukumoto, “Active Click: Tactile Feedback for Touch Panels,” ACM
`CHI2001 Extended Abstracts, pp. 121-122, Apr. 2001.
`Gobel et al., “Tactile Feedback Applied to Computer Mice,” Interna-
`tional Journal of Human-Computer Interaction, vol. 7, No. 1, pp.
`1-24, 1995.
`Gotow et al, “Controlled Impedance Test Apparatus for Studying
`Human Interpretation of Kinesthetic Feedback,” WA11-11:00, pp.
`332-337.
`Howe, “A Force-Reflecting Teleoperated Hand System for the Study
`of Tactile Sensing in Precision Manipulation,” Proceedings of the
`1992 IEEE International Conference on Robotics and Automation,
`Nice, France, May 1992.
`IBM Technical Disclosure Bullein, “Mouse Ball-Actuating Device
`With Force and Tactile Feedback,” vol. 32, No. 9B, Feb. 1990.
`International Search Report filed Nov. 1, 2002 corresponding to
`PCT/US02/35016.
`
`Iwata, “Pen-based Haptic Virtual Environment,” 0-7803-1363-1/93
`IEEE, pp. 287-292, 1993.
`Jacobsen et al., “High Performance, Dextrous Telerobotic Manipu-
`lator With Force Reflection,” Intervention/ROV ’91 Conference &
`Exposition, Hollywood, Florida, May 21-23, 1991.
`Johnson,
`“Shape-Memory Alloy Tactile Feedback Actuator,”
`Armstrong Aerospace Medical Research Laboratory, AAMRL-TR-
`90-039, Aug. 1990.
`Jones et al., “A perceptual analysis of stiffness,” ISSN 0014-4819
`Springer
`International
`(Springer-VeItag)é Experimental Brain
`Research, vol. 79, No. 1, pp. 150-156, 1990.
`Kaczmarek et al., “Tactile Displays,” Virtual Environment Technolo-
`gies.
`Kontarinis et al., “Display of High-Frequency Tactile Information to
`Teleoperators,” Telemanipulator Technology and Space Telerobot-
`ics, Won S. Kim, Editor, Proc. SPIE vol. 2057, pp. 40-50, Sep. 7-9,
`1993.
`Kontarinis et al., “Tactile Display of Vibratory Information in
`Teleoperation and Virtual Environments,” Presence, 4(4):387-402,
`1995.
`Lake, “Cyberman from Logitech,” GameBytes, 1994.
`Marcus, “Touch Feedback in Surgery,” Proceedings ofVirtual Real-
`ity and Medicine The Cutting Edge, Sep. 8-11, 1994.
`McAffee, “Teleoperator Subsystem/Telerobot Demonstrator: Force
`Reflecting Hand Controller Equipment Manual,” JPL D-5172, pp.
`1-50, A1-A36, B1-B5, C1-C36, Jan. 1988.
`Minsky, “Computational Haptics: The Sandpaper System for Syn-
`thesizing Textue for a Force-Feedback Display,” Ph.D. Dissertation,
`MIT, Jun. 1995.
`Ouh-Young, “Force Display in Molecular Docking,” Order No.
`9034744, p. 1-369, 1990.
`Ouh-Young, “A Low-Co st Force Feedback Joystick and Its Use in PC
`Video Games,” IEEE Transactions on Consumer Electronics, vol. 41,
`No. 3, Aug. 1995.
`Ouhyoung et al., “The Development of a Low-Cost Force Feedback
`Joystick and Its Use in the Virtual Reality Environment,” Proceedings
`of the Third Pacific Conference on Computer Graphics and Applica-
`tions, Pacific Graphics ’95, Seoul, Korea, Aug. 21-24, 1995.
`Patrick et al., “Design and Testing of a Non-reactive, Fingertip,
`Tactile Display for Interaction with Remote Environments,” Coop-
`erative Intelligent Robotics in Space, Rui J. deFigueiredo et al.,
`Editor, Proc. SPIE vol. 1387, pp. 215-222, 1990.
`Pimentel et al., “Virtual Reality: through the new looking glass,” 2”’
`Edition; McGraw-Hill, ISBN 0-07-050167-X, pp. 41-202, 1994.
`Rabinowitz at al., “Multidimensional tactile displays: Identification
`of vibratory intensity, frequency, and contactor area,” Journal of the
`Accoustical Society ofAmerica, vol. 82, No. 4, Oct. 1987.
`Russo, “The Design and Implementation of a Three Degree of Free-
`dom Force Output Joystick,” MIT Libraries Archives Aug. 14, 1990,
`pp. 1-131, May 1990.
`Russo, “Controlling Dissipative Magnetic Particle Brakes in Force
`Reflective Devices,” DSC-vol. 42, Advances in Robotics, pp. 63-70,
`ASME 1992.
`Scannell, “Taking a Joystick Ride”, Computer Currents, Nov. 1994,
`Boston Edition, vol. 9 No. 11.
`Shimoga, “Finger Force and Touch Feedback Issues in Dexterous
`Telemanipulation,” Proceedings of Fourth Annual Conference on
`Intelligent Robotic Systems for Space Exploration, Rensselaer Poly-
`technic Institute, Sep. 30-Oct. 1, 1992.
`SMK Corporation, “Multi-Functional Touch Panel, Force-Feedback
`Type, Developed: A Touch Panel Providing a Clicking Feeling,”
`http://www.smk.cojp/whatsnew_e/628csc_e.html, Sep. 30, 2002.
`SMK Corporation, “Force Feedback Type Optical Touch Panel
`Developed,” SMK Corporation Website, Oct. 30, 2002.
`Snow et al., “Model-X Force-Reflecting-Hand-Controller,” NT Con-
`trol No. MPO-17851; JPL Case No. 5348, pp. 1-4, Jun. 15, 1989.
`Stanley et al., “Computer Simulation of Interacting Dynamic
`Mechanical Systems Using Distributed Memory Parallel Proces-
`sors,” DSV-vol. 42, Advances in Robotics, pp. 55-61, ASME 1992.
`Tadros, “Control System Design for a Three Degree of Freedom
`Virtual Environment Simulator Using Motor/Brake Pair Actuators,”
`MIT Archive © Massachusetts Institute ofTechnology, pp. 1-88, Feb.
`1990.
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 3
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 3
`
`

`

`US 8,773,356 B2
`Page 4
`
`(56)
`
`References Cited
`OTHER PUBLICATIONS
`
`Terry et al ., “Tactile Feedback in a Computer Mouse,” Proceedings of
`Fourteenth Annual Northeast Bioengineering Conference, Univer-
`sity ofNew Hampshire, Mar. 10-11, 1988.
`Wakiwaka, et al., “Influence of Mover Support Structure on Linear
`Oscillatory Actuator for Cellular Phones,” The Third International
`Symposium on Linear Drives for Industry Applications, 2001, p.
`260-263, Nagano, Japan.
`Wiker, “Teletouch Display Development: Phase 1 Report,” Technical
`Report 1230, Naval Ocean Systems Center, San Diego, Apr. 17, 1989.
`Notification of First Oflice Action mailed Sep. 5, 2007 for corre-
`sponding Chinese Application 02821854.X.
`Notice of Reasons for Rejection for Jan. 29, 2008 mailed Feb. 20,
`2008 for corresponding Japanese Patent Application No. 2003-
`540973.
`Notice of Reasons for Rejection of Sep. 11, 2007 mailed Sep. 11,
`2007 for corresponding Japanese Patent Application No. 2003-
`540973.
`United States Patent and Trademark Oflice, Oflice Action mailed
`Dec. 23, 2005 for corresponding U.S. Appl. No. 10/285,450.
`United States Patent and Trademark Oflice, Oflice Action mailed
`May 18, 2006 for corresponding U.S. Appl. No. 10/285,450.
`United States Patent and Trademark Oflice, Oflice Action mailed
`Nov. 15, 2006 for corresponding U.S. Appl. No. 10/285,450.
`United States Patent and Trademark Oflice, Oflice Action mailed Jun.
`1, 2007 for corresponding U.S. Appl. No. 10/285,450.
`European Supplemental Search Report mailed Jul. 1, 2008 for cor-
`responding European Patent Application No. 027739606.
`Notice of Preliminary Rejection mailed Mar. 28, 2009 for corre-
`sponding Korean Patent Application No. 10-2004-7006627.
`Oflice Action mailed Nov. 25, 2009 for corresponding Korean Patent
`Application No. 10-2009-7017838.
`
`Oflice Action mailed May 10, 2010 for corresponding Korean Patent
`Application No. 10-2009-7017838.
`Oflice Action mailed Nov. 25, 2009 for corresponding Korean Patent
`Application No. 10-2004-7006627.
`Oflice Action mailed Jul. 2, 2010 for corresponding Korean Patent
`Application No. 10-2009-7006555.
`Oflice Action mailed Jun. 19, 2009 for corresponding Chinese Appli-
`cation No. 200810008845.X.
`Oflice Action mailed Nov. 1, 2010 for corresponding Chinese Appli-
`cation No. 200810008845.X.
`Oflice Action mailed Nov. 23, 2010 for corresponding Chinese
`Application No. 02821854.X.
`Oflice Action mailed Mar. 5, 2009 for corresponding U.S. Appl. No.
`1 1/693, 1 17.
`Oflice Action mailed Jun. 24, 2009 for corresponding U.S. Appl. No.
`1 1/693, 1 17.
`Oflice Action mailed Dec. 29, 2009 for corresponding U.S. Appl. No.
`1 1/693, 1 17.
`Oflice Action mailed Jul. 7, 2011 for corresponding Chinese Appli-
`cation No. 200810008815.X.
`Oflice Action mailed Dec. 5, 2012 for corresponding Korean Patent
`Application No. 10-2011-7025 866.
`Oflice Action mailed Aug. 23, 201 1 for corresponding Korean Patent
`Application No. 10-2010-7006555.
`Oflice Action mailed Oct. 26, 2010 for corresponding Korean Patent
`Application No. 10-2009-7017838.
`Oflice Action mailed May 18, 2010 for corresponding Korean Patent
`Application No. 10-2004-7006627.
`Communication pursuant to Artilce 94(3) EPC mailed May 10, 2012
`for corresponding European Application No. 08007837.1.
`Notification of Reexamination mailed Mar. 9, 2012 for correspond-
`ing Chinese Application 02821854 .X.
`United States Patent and Trademark Oflice, Oflice Action mailed
`Mar. 2, 2011 for corresponding U.S. Appl. No. 12/894,489.
`United States Patent and Trademark Oflice, Oflice Action mailed
`Aug. 17, 2011 for corresponding U.S. Appl. No. 12/894,489.
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 4
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 4
`
`

`

`PPPPPPP C.
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 5
`
`

`

`U.S. Patent
`
`7,8S
`
`2B65
`
`47!66mWFm91M10Oa0MO4|-2,5111%8V5l.\..\\.5”u0J5
`1235KooUfCA1....|0224'beHbW0Wh2mx11o.\1S«MHW*
`
`W
`
`3:M30715
`
`Umm.01|1
`
`13
`
`5
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 6
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 6
`
`
`
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 3 of 11
`
`US 8,773,356 B2
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 7
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 7
`
`

`

`U.S. Patent
`
`Jul. 8 2014
`
`Sheet 4 of 11
`
`E
`
`<
`
`£2&2
`
`g2\\\\\\\2
`
`‘L
`
`
`
`._V.U_n_mm
`
` \
`\ON
`
`5
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 8
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 5 of 11
`
`US 8,773,356 B2
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 9
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 9
`
`

`

`U.S. Patent
`
`Jul. 8 2014
`
`Sheet 6 of 11
`
`9
`
`.mE
`
`5
`
`3,.
`
`a //
`
`E
`
`ed: «N
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 10
`
`
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 7 of 11
`
`US 8,773,356 B2
`
`40
`
`Input
`
`Devices
`
`Electronic
`.
`
`
`
`43
`
`39
`
`41
`
`Controller
`
`Display
`
`44
`
`Memory
`
`42
`
`
`
`Actuator
`
`45
`
`46
`
`F I G . 7
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 11
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 11
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 8 of 11
`
`US 8,773,356 B2
`
`Failure?
`
`Feedback
`lnfo.
`
`Data?
`
`
`
`Produce
`
`Tactile
`Sensation
`
`
`55
`
`Pressure
`
`
`
`
`53
`
`Obtain
`
`
`
`Pressure
`Data
`
`F|G.8
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 12
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 12
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 9 of 11
`
`US 8,773,356 B2
`
`M,..,§_§%2§_.§%
`
` l l I lmtag¢ l._ N n l..I lN._£_8n_IH l_8__._m§I l
`SSfimsmI I2_s_§EmI
`zoimzum5.55
`Ecozomcwme39:EE
`
`
`
`ad:
`
`2.5__%EII
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 13
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 13
`
`
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 10 of 11
`
`US 8,773,356 B2
`
`
` %%9cozomcomo9co_..om_._om1cozomcom<2sfimsmN
`
`
`R8_..$$mIIImcosasmllm=o8_m:<I58_§%m.5523 m5%.I
` l Im l l
`
` lIcozomcomN:82N2=m.aén_ImSac.¢ m ...I |N§_8n_II I_.s=_8._I _ zoimzmm55:2952.:
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 14
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 14
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jul. 8, 2014
`
`Sheet 11 of 11
`
`US 8,773,356 B2
`
`100
`
`\\
`
`108
`
`110
`
`112
`
`102
`
`Edit ew
`
`C’tr|+O
`
`'
`
`V
`
`Properties
`
`I
`Ctrl+P
`///////.1
`@M<_1i1 Recipient (as Attachment)
`.5: Microsoft EowerPoint
`
`114-
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 15
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 15
`
`

`

`1
`METHOD AND APPARATUS FOR
`PROVIDING TACTILE SENSATIONS
`
`CROSS-REFERENCES TO RELATED
`APPLICATION
`
`This application is a continuation of co-pending U.S.
`patent application Ser. No. 12/894,489, entitled “Method and
`Apparatus for Providing Tactile Sensations,” which is a con-
`tinuation ofU.S. patent application Ser. No. 11/693,117, filed
`Mar. 29, 2007, now U.S. Pat. No. 7,808,488, issued Oct. 5,
`2010, entitled “Method and Apparatus for Providing Tactile
`Sensations,” which is a continuation of U.S. patent applica-
`tion Ser. No. 10/285,450 filed Nov. 1, 2002, now U.S. Pat. No.
`7,336,260, issued Feb. 26, 2008, entitled “Method andAppa-
`ratus for Providing Tactile Sensations”, which claims priority
`to U.S. Provisional Application No. 60/335,493, filed Nov. 1,
`2001, and U.S. Provisional Application No. 60/399,883, filed
`Jul. 31, 2002, the entirety of all of which are hereby incorpo-
`rated by reference.
`
`FIELD OF THE INVENTION
`
`The present invention relates to methods and apparatus for
`providing tactile sensations.
`
`BACKGROUND
`
`Conventional electronic devices, such as mobile tele-
`phones and Personal Digital Assistants (PDAs),
`include
`visual displays. A user of such devices interacts with the
`visual display using any one of a number of input devices.
`Examples of such input devices include computer mice, joy-
`sticks, trackballs, steering wheels, stylus, tablets, pressure-
`sensitive spheres, scroll wheels, keyboards, and keypads. The
`user provides instructions, responses, and other input to the
`device using such input devices.
`In conventional mobile telephones and PDAs, confirma-
`tion of the input provided by the user is primarily limited to
`visual or audible confirmation. In some such devices, physi-
`cal
`feedback is provided by conventional mechanical
`switches in the form ofthe conventional mechanical feedback
`
`of switches, for example the switch closure force-displace-
`ment profile. Typically, in such devices, the mechanical feed-
`back provided by each button is identical. In addition, in such
`conventional devices, for those buttons that serve multiple
`functions, the mechanical feedback generally remains the
`same regardless of the current function of the button.
`In addition to providing extremely limited and rudimentary
`mechanical confirmation of button selection, conventional
`buttons as used, for example, in keypads for mobile tele-
`phones and PDAs, provide simple passive touch cues regard-
`ing the alignment ofkeys. Such cues include raised bumps on
`the center key of a telephone keypad or on the “F” and “G”
`keys of a keyboard that assist a user in orienting to the pattern
`of keys in the keypad and keyboard. Again, these physical
`queues are very limited, and users typically need to view a
`keypad or keypad for visual confirmation that the correct
`instructions or information is being entered.
`When a flat surface interface device is used, such as a
`touchpad for a computer or PDA, these simple mechanical
`cues are unavailable to the user. Often, touchpads are com-
`bined with flat-panel display screens that display one or more
`graphically generated buttons or softkeys. Normally, the soft-
`keys are visible through the touchpad. A user’s contact with
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 8,773,356 B2
`
`2
`
`the touchpad in an area defined by a softkey provides the
`electronic device having the touchpad with the input associ-
`ated with that softkey.
`The use of electronic devices using such conventional
`mechanical buttons and touchpad arrangements are particu-
`larly difficult to use in distracting environments or when the
`user is attempting to perform another task simultaneously
`withusing the electronic device. For example, ifthe other task
`involves operating a motor vehicle or heavy machinery, it
`may be difficult or impossible for a user to simultaneously use
`such an electronic device because such devices typically
`require the user to look at the device, at least briefly, when
`interacting with the device. In addition, electronic devices
`relying on softkeys can be difficult to read in bright light
`environments such as in bright sunlight and can contain very
`small fonts and graphics that are difficult to read and select.
`Some conventional touchpads include vibratory feedback
`to the user of the touchpad. U.S. Pat. No. 5,977,867 is one
`example. Such conventional systems and methods are lim-
`ited, though. They lack a full range of functionality assistance
`to a user interacting with an electronic device. Moreover, such
`systems and methods still require considerable visual atten-
`tion from the user.
`
`SUMMARY
`
`The present invention comprises products and processes
`for providing tactile sensations to input devices or electronic
`devices. Input devices include mechanical
`input devices
`(such as, for example, mechanical switches) and non-me-
`chanical input devices (such as, for example, touchpads).
`Tactile feedback is provided by using an actuator or other
`means in communication with the input device or electronic
`device. A controller may be employed to receive signals from
`the input devices and to control the actuator. Tactile feedback
`to an input device or electronic device may be provided in
`response to one or more events or situations. Such an event or
`situation may be any one designated. Examples of such events
`and situations include the level of pressure placed on an input
`device; the availability or lack of availability of a function
`associated with an input device; and the function, menu, or
`mode of operation associated with an input device’s activa-
`tion. A variety of feedback types and combinations may be
`selected. Mobile telephones and PDAs benefit from employ-
`ing such products and processes, but other devices benefit as
`well. The advantages offered by the various embodiments of
`the present invention may be understood by examining this
`specification.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a perspective view of an embodiment of an appa-
`ratus according to the present invention;
`FIG. 2 is a plan view of another embodiment of an appa-
`ratus according to the present invention;
`FIG. 3 is a plan view of an electronic device including an
`embodiment of the present invention;
`FIG. 4 is a schematic representations through line 4-4 of
`FIG. 3;
`FIG. 5 is a plan view of another electronic device including
`another embodiment of the present invention;
`FIG. 6 is a schematic representations through line 6-6 of
`FIG. 5;
`FIG. 7 is a block diagram illustrating an embodiment ofthe
`apparatus in an electronic device;
`FIG. 8 is a flow chart illustrating a method according to the
`present invention;
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 16
`
`APPLE INC.
`EXHIBIT 1101 - PAGE 16
`
`

`

`US 8,773,356 B2
`
`3
`FIG. 9 is a table illustrating a first set of data to be used in
`one embodiment of the present invention;
`FIG. 10 is a table illustrating a second set of data to be used
`in another embodiment of the present invention; and
`FIG. 11 shows a device for providing tactile sensations
`according to one embodiment of the present invention.
`
`DETAILED DESCRIPTION
`
`The present invention includes methods and systems for
`providing tactile sensations. One embodiment includes meth-
`ods and systems for providing tactile sensations to input
`devices, both mechanical and non-mechanical (for example
`soft-keys that are computer generated and displayed on a
`screen). Embodiments ofthe present invention can be utilized
`in wide variety of electronic devices including telephones,
`mobile telephones,
`remote controls, gamepads,
`joystick
`handles, automotive controls (radios, Compact Disc (CD)
`players, automobile functions, etc.), consumer electronics
`devices, Personal Digital Assistants (PDAs), personal com-
`puters, laptop computers, portable gaming devices, pagers,
`I-pagers, audio equipment, televisions, security or alarm sys-
`tems, Automated Teller Machines (ATM), calculators, home
`appliances, and white goods.
`FIG. 1 shows one embodiment of the present invention.
`The apparatus 1 shown in FIG. 1 includes an input device 2
`having multiple positions for communicating a plurality of
`input signals. The input device 2 can be any device capable of
`transmitting an input signal. In the embodiment shown, the
`input device 2 is a rocker-type switch. The rocker switch 2
`shown can pivot or rock between two positions in which the
`rocker switch contacts and activates one of two rubber
`
`switches 3 containing conductive pads. The use of rubber
`switches 3 provides the advantage of allowing the user to still
`feel a substantial vibration or force through the input device 2
`when the user had fully depressed the switch. Suitable rubber
`switches are available and known in the art. In other embodi-
`
`ments, the input device may include an analog switch, a force
`sending resistor, a strain gauge based sensor, a capacitative
`touch switch, a scroll wheel, a mini-joystick, a touchpad, a
`touch screen, a 3-way switch, a 4-way switch, a 5-way switch,
`or other input device. Each position of the input device 2
`corresponds to one of the input signals.
`The input device 2 and rubber switches 3 are mounted on a
`Printed Circuit Board (PCB) 4 in the embodiment shown to
`facilitate electrical communication between the input device
`2 and an electronic device (not shown). The PCB 4 can be
`custom shaped according to the device into which the appa-
`ratus 1 is placed. The PCB 4 also provides for secure mount-
`ing within the device by including, for example, a plurality of
`holes 5 to accept fasteners for securing to the electronic
`device. In another embodiment, the input device 2 can be
`directly connected or mounted in the electronic device.
`The apparatus 1 shown in FIG. 1 also includes a vibrotac-
`tile actuator 6 in communication with the input device 2.
`Preferably, the actuator 6 is configured to output a plurality of
`distinct tactile feedback sensations to the input device 2.
`Suitable tactile sensations include vibrations, for example,
`jolts and textures, and a plurality of distinct tactile sensations
`can be created by varying the frequency, amplitude and wave-
`form output by the actuator 6. The actuator 6 is selected to
`deliver the desired tactile sensations to the input device 2 The
`actuator 6 shown in FIG. 1 is a voice coil actuator. Other
`
`suitable actuators include, for example, piezo-electric actua-
`tors, eccentric mass actuators, moving magnet actuators, and
`friction brakes in contact with metal shafts. In addition, the
`actuator can include a flexure, for example an arrangement of
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`flexible material, coupled to the rotating shaft of a DC motor
`or step motor to transform the rotation of the motor shaft into
`vibrations or other haptic sensations. Various arrangements of
`a flexure coupled to a motor may be used as an actuator. For
`example, U.S. patent application Ser. No. 09/585,741, filed
`Jun. 2, 2000, illustrates suitable arrangements of flexures and
`motors for use as an actuator in embodiments of the present
`invention. The entire disclosure of the application Ser. No.
`09/585,741 is incorporated herein by reference. Tactile sen-
`sations can also be delivered to the input device 2 from a
`speaker included with an electronic device into which the
`apparatus is placed, for example the speaker in a mobile
`telephone or in a personal computer.
`Although the embodiment shown in FIG. 1 includes one
`input device 2 and one actuator 6, other embodiments include
`a plurality of input devices, all in communication with a
`single actuator. Alternatively, an embodiment can include a
`plurality of actuators each in communication with at least one
`input device. Various arrangements of actuators in combina-
`tion with input devices are suitable for use in the present
`invention. For example,