Ulllted States Patent [19]
`Rosenberg
`
`US006088019A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,088,019
`Jul. 11, 2000
`
`[54] LOW COST FORCE FEEDBACK DEVICE
`ACTUATOR FOR NON-PRIMARY
`
`FOREIGN PATENT DOCUMENTS
`OO85518A1 1/1983 European Pat. Off. .
`
`O626634A2 11/1994 European Pat. Off. .
`
`[75] Inventor: Louis B. Rosenberg, San Jose, Calif.
`
`
`
`gallrggeflupat' O?' """ 95/32459 11/1995 WIPO ............................. .. G06F 3/00
`
`[73] Assignee: Immersion Corporation, San Jose,
`Calif.
`
`97/2116()
`97/31333
`
`6/1997 WIPO _
`8/1997 WIPO .
`
`[21] Appl. No.: 09/103,281
`
`[22] Filed?
`
`Jun- 23! 1998
`
`7
`Int. Cl. ..................................................... .. G09G 5/00
`[51]
`[52] US. Cl. ............................................................ .. 345/156
`[58] Field of Search ................................... .. 345/156, 157,
`345/161, 163
`
`[56]
`
`References Cited
`
`U_S_ PATENT DOCUMENTS
`
`OTHER PUBLICATIONS
`
`Ramstein et al., “The Pantograph: A Large Workspace
`Haptic Device for a Multimodal Human—Computer Interac
`tion,” Computer—Human Interaction, CHI 1994, pp. 1—3.
`Buttolo et a1.’ “pen_based force Display for Precision
`Manipulation
`in
`Virtual
`Environments,”
`IEEE
`()_8186_7084_3, 1995, pp 217_224_
`
`(List continued on next page.)
`
`Primary Examiner—Richard A. Hjerpe
`Assistant Examiner—Ronald Laneau
`Attorney, Agent, or Firm—James R. Riegel
`
`3,875,488
`
`4/1975 Crocker et a1. ....................... .. 318/648
`
`[57]
`
`ABSTRACT
`
`,
`
`,
`
`3,919,691 11/1975 N011 ................................... .. 340/1725
`474367188
`3/1984 Jones
`188/378
`4,477,043 10/1984 Repperger _
`244/223
`4,604,016
`8/1986 Joyce . . . . . . . .
`. . . . . . .. 474/7
`4,782,327 11/1988 Kley et a1- -
`340/365
`4,794,384 12/1988 Jackson
`340/710
`' ' '
`g‘u
`' ' ' ' ' ' "63/1345;
`emens a e a.
`6/1989 LaBiche et a1‘ "
`364/709'11
`478397838
`8/1989 Meenen, Jr.
`.... .. 434/45
`4,861,269
`9/1989 Af?nito et aL
`__ 340/710
`478687549
`1/1990 Culver . . . . . . . . . . .
`. . . . .. 74/471
`4,896,554
`3/1990 Jones et a1_
`250/221
`4,906,843
`6/1990 Kley ...................................... .. 340/709
`4,935,728
`8/1990 Moncrief et a1. ..................... .. 364/578
`4,949,119
`4,961,038 10/1990 MaCMinn ~~~~ ~~
`318/696
`4’983’9O1
`1/1991 Lehmer """" "
`318/685
`gilceslsky et a1‘
`5,076,517 12/1991 Ferranti et a1. ....................... .. 244/228
`570957303
`3/1992 Clark et aL
`340/710
`571037404
`4/1992 McIntosh ____ __
`31856822
`5,107,080
`4/1992 Rosen ........................................ .. 200/6
`
`_
`_
`_
`Aforce feedback mterface and method 1nclud1ng an actuator
`in a non-primary axis or degree of freedom. The force
`feedback interface device is connected to a host computer
`that implements a host application program or graphical
`environment. The interface device includes a user manipu
`latable object, a sensor for detecting movement of the user
`-
`object, and an actuator to apply output forces to the user
`F’bJeCt' The aqua/Or outpllts a hnear force on the user Qblect
`1n non-prlmary hnear ax1s or degree of freedom that 1s not
`used to Control a graphical Object or entity implemented by
`the host computer, and movement in the non-primary degree
`of freedom is preferably not sensed by sensors. The axis
`extends through the user Object, and there are preferably [10
`other actuators in the device, thus alloWing the force feed
`back device to be very cost effective. Force sensations such
`as a jolt, vibration, a constant force, and a texture force can
`be Output on the user oblect Wfth the actuator‘ .The force
`sensat1ons can be output 1n a d1rect1on perpendlcular to a
`planar degree of freedom, radial to spherical degree of
`freedom, and/0r along a lengthwise axis of the user object.
`
`(List continued on next page.)
`
`45 Claims, 5 Drawing Sheets
`
`150
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 1
`
`

`
`6,088,019
`Page 2
`
`US. PATENT DOCUMENTS
`
`5,107,262
`5,116,180
`5,139,261
`
`5,146,566
`5,184,319
`5,185,561
`5,186,629
`5,189,355
`5,193,963
`571977003
`5,203,563
`
`4/1992 CadoZ et a1. ............................ .. 341/22
`5/1992 Fung et al. ..
`414/5
`8/1992 Openiano .... ..
`. 273/148
`
`9/1992 Hollis, Ji etal
`. 395/275
`2/1993 Kramer ................................. .. 364/806
`2/1993 Good et a1. ........................... .. 318/432
`2/1993 Rohen ______ __
`_434/114
`2/1993 Larkins et al. ........................ .. 318/685
`3/1993 McAffee et a1. .......................... .. 414/5
`3/1993 Moncrief et a1‘
`_ 364/410
`4/1993 Loper,III .............................. .. 273/148
`
`5,220,260
`5,223,776
`
`6/1993 Schuler ................................. .. 318/561
`6/1993 Radke et al.
`. 318/568.1
`
`9/1998 Chen ..................................... .. 345/157
`5,808,603
`5,823,876 10/1998 Unbehand ............................... .. 463/37
`5,825,308 10/1998 Rosenberg
`341/20
`5,831,408 11/1998 Jacobus et al. .
`. 318/568.11
`5,844,392 12/1998 Peurach et al. .
`. 318/568.17
`
`. . . . . .. 345/163
`3/1999 Redlich . . . . . . . . . . .
`5,877,748
`345/156
`3/1999 Rosenberg et al
`5,880,714
`3/1999 Schuleretal- ------------------------ -- 364/186
`5,889,670
`6/1999 Siddiqui ................................ .. 345/166
`5,912,661
`8/1999 Jekebe er e1~
`188/267-1
`5,944,151
`9/1999 Rosenberg er al- -------------- -- 395/200-33
`5,956,484
`9/1999 Rosenberg et al. ................... .. 345/161
`5,959,613
`5999869 11/1999 Kublca etal- ------------------------ -- 345/163
`
`OTHER PUBLICATIONS
`
`52353368 8/1993 Culver ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
`
`~ ~ ~ ~ ~ ~~ 74/471
`
`Hannaford et al., “Force—Feedback Cursor Control,” NASA
`
`,
`
`,
`
`Tech Briefs, vol. 13, No. 11, 1989, pp. 1—7.
`Rosenberg et al., “The use of force feedback to enhance
`.
`.
`,,
`.
`.
`.
`graphlcal user lntefacesa Stereoscoplc Dlsplays 8‘ Vmual
`Realny Sysrerns, 1996, PP- 243—248
`Munch et al., “Intelligent Control for Haptic Displays,”
`Eurographics ’96, vol. 15, No. 3, 1996, pp. 217—226.
`Rosenberg et alw “Commercially
`force feedback
`Controller for Individuals With Neuromotor Disabilities,”
`1A_r;n3strong Laboratory, AL/CF TR 1997 0016, 1996, pp.,
`
`_ _
`
`_
`
`l
`
`.......... ..
`
`.
`
`- 318/561
`5264568 11/1993 Gregory er al-
`434/29
`31994 Moncnef """ "
`434/45
`/1994 Fuller et al.
`3/1994 Paley ..................................... .. 345/163
`5,296,871
`5/1994 Venolia et a1. ........................ .. 345/163
`5,313,230
`5,354,162 10/1994 Burdea et a1, _
`414/5
`5,355,148 10/1994 Anderson .............................. .. 345/166
`Schnell et al. ........................ ..
`gl?nhan
`'
`4/1995 Katanics et a1‘
`. 273/438
`
`,
`
`,
`
`574057152
`
`5,414,337
`
`5/1995 Schuler . . . . . . . . . . . . .
`
`. . . .. 318/561
`
`_ 340/4071
`9/1995 Massimino et aL
`574517924
`5,457,479 10/1995 Cheng ................................... .. 345/163
`5,459,382 10/1995 Jacobus etal. .................. .. 318/568.11
`395/137
`5,471,571 11/1995 Smith et al
`5,473,235 12/1995 Lance et al.
`. 318/561
`1;;
`giiacrineita‘il'
`'
`
`"
`
`_
`
`_
`
`_
`
`Brooks, Jr. et al., “Pro]ect GROPE, Haptlc Displays for
`Seienri?e ViSna1iZan0n,”, cnrnpnrer Grannies, V01- 24, #4,
`1990,1111' 177—184
`Batter et al., “Grope—1: A computer Display to the sense of
`Feel,” Proc_ IFIP Congress, 1971, pp 759_763_
`GotoW et al., “Perception of Mechanical Properties at the
`
`5,512,919
`
`4/1996 Araki . . . . . . . . .
`
`. . . .. 345/156
`
`g/égigbg/lgchme Interface’ IEEE $125034’ 1987’ pp‘
`
`4/1996 Parker et al. .................... .. 364/167.01
`5,513,100
`6/1996 Gillick et al. ......................... .. 345/163
`5,530,455
`5,565,887 10/1996 McCambridge et al.
`. 345/145
`5,576,727 11/1996 Rosenberg et al.
`. 345/179
`
`'
`HnWe er 211» “Task Performance W/ a deXrrenS Teleeperared
`Hand System,” Free of SPIE, vol- 1833, 1992, pp- 1—9
`Atkinson et al., “Computing With Feelingj’, Comput. &
`
`
`
`5,583,407 12/1996 Yamaguchi 5589328 12/1996 Armstrong '
`
`
`
`. 318/551 ' 345/161
`
`
`
`Graphics, VOL 2, Minsky et al., “Feeling & SeeingzIssues in Force Display,”
`
`.345/161
`5,589,854 12/1996 Tsal .............. ..
`5,596,347
`1/1997 Robertson et al. ................... .. 345/145
`5,625,576
`4/1997 Massie et a1. ........................ .. 364/578
`576427469
`6/1997 Hannaford et aL
`395/99
`5,643,087
`7/1997 Marcus et a1. .......................... .. 463/38
`5,656,901
`8/1997 Kurita ................................... .. 318/436
`5,666,138
`9/1997 Culver
`- 345/161
`576667473
`9/1997 Wallace
`- 345/420
`$691547 11/1997 Arnene
`- 345/167
`5,691,898 11/1997 Rosenberg eta.
`. 364/190
`5,694,013 12/1997 Stewartetal.
`. 345/156
`5,709,219
`1/1998 Chen et a1. ........................... .. 128/782
`5,714,978
`2/1998 Yamanaka ............................. .. 345/157
`5,721,566
`2/1998 Rosenberg et al.
`. 345/161
`5,724,278
`3/1998 Chen et a1- ----- ~-
`- 345/156
`5a734a373
`3/1998 Rosenberg ct a1~
`- 345/161
`
`ACM089791_351—5,1990,PP-235—242>270
`.
`“
`.
`.
`.
`,,
`Ouh_Y°unga Mmga Force Dlsplay 1“ Molecular Dockmga
`Dissertation for University of N. Carolina, 1990, pp. 1—42.
`IWata, HirOO, “Pen—based Haptic Virtual Environment,”
`IEEE 0—7803—1363—1, 1993, pp. 287—292.
`Adelstein et al., “Design & Implementation of a Force
`Re?ecting Manipulandum for Manual Control Research,”
`1992, NASA—Ames Research Center and MIT, pp. 1—26.
`M11
`t 1 “D .
`f 4D
`fF d
`F
`_R _
`1 fnane 2“:
`eslgno a, egreeo ,ree 0m 0m 6
`?ectlng Manlpulandurn Wlth a SPeCl?ed Force/Torque
`WerkSpaee,” IEEE CH2969—4, 1991,1111 14884493
`Hannaford et al., “Performance Evaluation of a 6—Axis
`Generalized Force—Re?ecting Teleoperator,” IEEE Transac
`tions on Systems, Man, and Cybernetics, vol. 21, No. 3,
`
`4/1998 I‘I?SSCI et al. 5,736,978 2:323:33; Z1333 ?‘iiiieifiialj ........................ .1 iii/Z2? .
`
`
`
`5,755,577
`5/1998 Gillio .................................... .. 434/262
`5,757,358
`5/1998 Osga ..... ..
`.345/146
`5,760,764
`6/1998 Martinelli
`. 345/160
`5,767,839
`6/1998 Rosenberg -
`- 345/161
`5,771,037
`6/1998 Jackson .... ..
`. 345/157
`5’781’172
`7/1998 Engel et a1‘
`' 345/164
`5,790,108
`8/1998 Salcudean et al. ................... .. 345/184
`5,802,353
`9/1998 Avlla etal. ........................... .. 395/500
`5,805,140
`9/1998 Rosenberg et a1_
`_ 345/161
`5,805,165
`9/1998 Thorne, III et a1,
`_ 345/348
`5,808,601
`9/1998 Leah et al. ............................ .. 345/145
`
`Wraey rn, “eeaapaaer Simulare,e1 Vreaar & Taeare reeereaere
`as an Aldto Mampulator‘gevehlcle C°ntr°1>”MIT> 198L111’
`1—79
`Colgate et al., “Implementation of Stiff Virtual Walls in
`Force—Re?ecting Interfaces,” Northwestern University, IL,
`1993” pp 1_8_
`Hirota et al., “Development of Surface Display,” IEEE
`0_7803_1363_1 1993
`256_262
`a
`>PP~
`~
`Rosenberg, “Perceptual Design of a Virtual Rigid Surface
`Contact,” Armstrong Laboratory AL/CF—TR—1995—0029,
`1993, pp. 1—45.
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 2
`
`

`
`6,088,019
`Page 3
`
`IWata et al., “Arti?cial Reality W/ Force—Feedback: Devel
`opment of Desktop Virtual Space With Compact Master
`Manipulator,” Computer Graphics, vol. 24, No. 4, 1990, pp.
`165—170.
`Russo, “The Deisgn & Implementation of a 3—Degree—of
`—Freedom Force Ouput Joystick,” Dept. of Mech. Engineer
`ing, 1990, pp. 1—42.
`Rosenberg, L., “Virtual ?xtures as tools to enhance operator
`performance in telepresence environments,” SPIE Manipu
`lator Technology, 1993, pp. 1—12.
`Rosenberg et al., “Perceptual Decomposition of Virtual
`Haptic Surfaces,” Proc. IEEE Symposium on Research
`Frontiers in Virtual Reality, 1993, pp. 1—8.
`Ouh—Young et al., “Creating an Illusion of Feel: Control
`Issues in Force Display,” Univ. of N. Carolina, 1989, pp.
`1—14.
`Yokokoji et al., “What you can see is What you can feel,”
`IEEE 0—8186—7295—1, 1996, pp. 46—54.
`Rosenberg et al., “A Force Feedback Programming Primer,”
`Immersion Corp., 1997, pp. 1—176.
`Kilpatrick et al., “The Use of Kinesthetic Supplement in an
`Interactive Graphics System,” University of North Carolina,
`1976, pp. 1—172.
`Kelley et al., MagicMouse: Tactile and Kinesthetic Feed
`back in the Human—Computer Interface using an Electro
`magnetically Actuated Input/Output Device, Oct. 19, 1993
`University of British Columbia pp. 1—27.
`
`Kelley et al., On the Development of a Force—Feedback
`Mouse and its Integration into a graphical user Interface,
`Nov. 1994, Engineering Congress and Exhibition, pp. 1—8.
`Christophe Ramstein, Combining Haptic & Braille Tech
`nologies: Design Issues and Pilot Study, 96, Siggraph pp.
`37—44.
`Payette et al., Evaluation of a Force Feedback (Haptic)
`Computer Pointing Device in Zero Gravity, Oct. 17, 1996,
`ASME Dynamics Systems, vol. 58 pp. 547—553.
`Wiker et al., Development of Tactile Mice for Blind Access
`to Computers, Importance of Stimulation Locus, Object
`SiZe, and Vibrotactile Display Resolution, 1991, Human
`Factors Society Mtg., pp. 708—712.
`Ellis et al., Design & Evaluation of a High—Performance
`Prototype Planar Haptic Interface, Dec. 1993, Advances in
`Robotics, 55—64.
`Schmult et al., Application Areas for a Force—Feedback
`Joystick, 1993, Advances in Robotics, vol. 49, pp. 47—54.
`Adachi et al., Sensory Evaluation of Virtual Haptic Push
`—Buttons, 1994, SuZuki Motor Corp., pp. 1—7.
`Akamatsu et al., Multimodal Mouse: A Mouse—Type Device
`With Tactile and Force Display, 1994, Presence vol. 3, pp.
`73—80.
`Su et al., The Virtual Panel Architecture: A 3D Gesture
`FrameWork, University of Maryland, pp. 387—393.
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 3
`
`

`
`U.S. Patent
`
`Jul. 11,2000
`
`Sheet 1 0f5
`
`6,088,019
`
`HOST COMPUTER SYSTEM
`
`_RHAT_LO0ANHHP_IIR4_RKM1__R02_C___CE._EDI.1.O_.HS_MLMH.........1H8_HCN_.1.____.........................--HHNH_
` _nIn.DIHH4__._N..H_....._T_..........--.H................1O1H.........IH_HHH%RH%HHwR-._HHmHEHHMKwHHHm............asHRV:HmeasHHN6mm.Em.HSm_HCWYURHWPHWC0_.L.OP.RTPIu_l_Dunwu
`
`-_.....1..2,1RC_T_HHHH.,H%EHAHHHH....NmwH_0_H__EO_C..T2.HE.SHAOHUH.WCH_3.HWEYE_HAWHHH_UCMCHHBE.6HHHOWPW._DDH3E___E1SE..EE.RCIIIIIIL._O21..E.OA_._IIDDDH.FC.F.1H_HnDuHHEMHWR.4TCHHAHHWRHRHWHHWWEHW-1-SSHO_“H1.
`Il_llLIIIIII.»IIIIIIIIIIII4_4H_H_HH.41,,m%SMTHmH.111.............
`
`
`._...I|IIIIIIIIIIIIIIII
`
`-...L.HHmH.U.MHmm2EHHPHEH1.F“.N.
`................HH..........
`
`FIG. 1
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 4
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 4
`
`

`
`U.S. Patent
`
`Jul. 11,2000
`
`Sheet 2 0f5
`
`6,088,019
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 5
`
`

`
`U.S. Patent
`
`Jul. 11,2000
`
`Sheet 3 0f5
`
`6,088,019
`
`FIG. 3
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 6
`
`

`
`U.S. Patent
`
`Jul. 11,2000
`
`Sheet 4 0f5
`
`6,088,019
`
`9.3
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 7
`
`

`
`U.S. Patent
`
`Jul. 11,2000
`
`Sheet 5 0f5
`
`6,088,019
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 8
`
`

`
`1
`LOW COST FORCE FEEDBACK DEVICE
`WITH ACTUATOR FOR NON-PRIMARY
`AXIS
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to interface
`devices for allowing humans to interface With computer
`systems, and more particularly to computer interface devices
`that alloW the user to provide input to computer systems and
`alloW computer systems to provide force feedback to the
`user.
`A computer system in typical usage by a user displays a
`visual environment on a display output device. Using an
`interface device, the user can interact With the displayed
`environment to perform functions and tasks on the
`computer, such as playing a game, experiencing a simulation
`or virtual reality environment, using a computer aided
`design system, operating a graphical user interface (GUI),
`etc. Common human-computer interface devices used for
`such interaction include a joystick, mouse, trackball, steer
`ing Wheel, stylus, tablet, pressure-sensitive sphere, or the
`like, that is connected to the computer system controlling the
`displayed environment. Typically, the computer updates the
`environment in response to the user’s manipulation of a
`user-manipulatable physical object such as a joystick handle
`or mouse, and provides visual and audio feedback to the user
`utiliZing the display screen and audio speakers. The com
`puter senses the user’s manipulation of the user object
`through sensors provided on the interface device that send
`locative signals to the computer. For example, the computer
`displays a cursor or other graphical object in a graphical
`environment, Where the location of the cursor is responsive
`to the motion of the user object.
`In some interface devices, tactile and/or haptic feedback
`is also provided to the user, more generally knoWn as “force
`feedback.” These types of interface devices can provide
`physical sensations Which are felt by the user manipulating
`a user manipulatable object of the interface device. For
`example, the Force-FX joystick controller from CH
`Products, Inc. and Immersion Corporation may be connected
`to a computer and provides forces in the degrees of freedom
`of motion of the joystick to a user of the controller. One or
`more motors or other actuators are coupled to the joystick
`and are connected to the controlling computer system. The
`computer system controls forces on the joystick in conjunc
`tion and coordinated With displayed events and interactions
`by sending control signals or commands to the actuators.
`The computer system can thus convey physical force sen
`sations to the user in conjunction With other supplied feed
`back as the user is grasping or contacting the joystick or
`other object of the interface device. For example, When the
`user moves the manipulatable object and causes a displayed
`cursor to interact With a different displayed graphical object,
`the computer can issue a command that causes the actuator
`to output a force on the user object, conveying a feel
`sensation to the user. Other force feedback controllers
`include a force feedback mouse that provides forces in the
`degrees of freedom of motion of the mouse, and a steering
`Wheel controller outputting forces in the rotary degree of
`freedom of the Wheel.
`One problem With current force feedback controllers in
`the home consumer market is the high manufacturing cost of
`such devices, Which makes the devices expensive for the
`consumer. Alarge part of this manufacturing expense is due
`to the inclusion of multiple actuators and corresponding
`control electronics in the force feedback device. In addition,
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6,088,019
`
`2
`high quality transmission components such as linkages and
`bearings must be provided to accurately transmit forces from
`the actuators to the user manipulandum and to alloW accu
`rate sensing of the motion of the user object. These com
`ponents are complex and require greater precision in their
`manufacture than many of the other components in an
`interface device, and thus further add to the cost of the
`device. A need therefore exists for a force feedback device
`that is loWer in cost to manufacture yet offers the user force
`feedback to enhance the interaction With a computer appli
`cation.
`
`SUMMARY OF THE INVENTION
`
`The present invention is directed to a loW-cost force
`feedback interface Which provides a linear actuator along a
`non-primary axis or degree of freedom. This con?guration
`can provide a simpler, loWer cost force feedback device,
`especially When motion in the non-primary axis is not
`sensed and no other actuators are used.
`More speci?cally, the present invention relates to a force
`feedback interface device that is coupled to a host computer
`system Which implements a host application program. The
`interface device includes a user manipulatable object, such
`as a mouse or joystick, contacted by a user and movable in
`physical space in at least one primary degree of freedom. At
`least one sensor detects the movement of the user object in
`the degree of freedom and outputs sensor signals represen
`tative of the movement. An actuator is coupled to the user
`manipulatable object and applies a linear output force along
`a non-primary axis extending through the user manipulat
`able object, Where the force is output in a degree of freedom
`not sensed by the sensor. Preferably, there are no other
`actuators in the device. Force sensations such as a jolt,
`vibration, a constant force, and a texture force can be output
`on the user object With the actuator.
`In preferred embodiments, the actuator outputs the force
`directly on the user manipulatable object, such that no
`transmission system is required to be provided betWeen the
`actuator and the user manipulatable object, thus greatly
`reducing the cost of the device. In addition, the actuator can
`include a physical spring or other spring device for biasing
`said at least a portion of the user manipulatable object
`toWard an extended position. The actuator can take a variety
`of forms, such as a linear voice coil actuator, a linear
`solenoid, or a voice magnet. A microprocessor local to the
`interface device can be provided to receive host commands
`from the host computer and output force signals to the
`actuator for controlling the output force on the user object.
`The microprocessor can receive sensor signals from the
`sensors and report locative data to the host computer indica
`tive of the movement of the user object. Alternatively, a
`sensor can be coupled to the actuator to determine a position
`of the user manipulatable object in the degree of freedom of
`the actuator.
`In one embodiment in Which the user manipulatable
`object is moved in a planar degree of freedom, the output
`force of the actuator can be provided in a direction approxi
`mately perpendicular to the plane of motion. For example, in
`a mouse embodiment, the force is applied about perpendicu
`larly to the planar mouse Workspace and is applied to an
`entire portion of the mouse that is grasped or rested upon by
`the user’s hand. In a particular mouse embodiment, the
`actuator is coupled to a housing of the mouse and moves a
`portion of the housing in the perpendicular direction. Such
`a moveable portion of the housing can be a cover portion of
`the housing that is movably coupled to a base portion of the
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 9
`
`

`
`3
`housing, for example by a hinge, Where the cover portion is
`moved by the actuator With respect to the base portion. The
`output force can be correlated With a graphical representa
`tion displayed by the host computer, Where a position of the
`mouse in the planar Workspace corresponds With a position
`of a cursor displayed in the graphical representation. For
`example, a jolt force can be output When the mouse crosses
`a boundary of a WindoW or icon. Or, the output force can be
`correlated With an elevation of a portion of a 3-D graphical
`representation having different elevations on Which the
`cursor is displayed. In a different embodiment, the user
`manipulatable object can be a stylus; or a Wheel, such as a
`steering Wheel, that rotates in the single plane, and Where the
`axis extends approximately through a center of the Wheel.
`In a different embodiment, the user manipulatable object
`is moved in tWo sensed rotary degrees of freedom With
`respect to a ground, Where the degrees of freedom approxi
`mately de?ne a portion of a surface of a sphere. For
`example, the user manipulatable object can be at least a
`portion of a joystick handle that is typically moved in such
`rotary degrees of freedom. The actuator of the device applies
`an output force in a linear degree of freedom that is
`approximately radial to the sphere, Where preferably no
`force is output in the tWo primary sensed degrees of free
`dom. The force is applied along a lengthWise axis of the user
`manipulatable object.
`In another embodiment, the user manipulatable object is
`movable in physical space in a plurality of degrees of
`freedom With respect to a ground, and a linear actuator
`applies a linear output force only along a lengthWise axis of
`the user manipulatable object and not in the plurality of
`degrees of freedom. One such embodiment provides a stylus
`as a user manipulatable object, Where the sensor can be
`included in a tablet Which is contacted by the stylus. In one
`embodiment, the stylus includes a rigid tip for contact With
`the tablet, Where the actuator outputs a force to move a body
`portion of the stylus relative to a tip portion of the stylus. In
`a different stylus embodiment, the stylus includes a ball in
`a tip of the stylus, Where the ball rotates in place When the
`stylus is moved across a surface. The actuator can force a
`brake pad against the ball to output a resistive force on the
`stylus.
`The present invention advantageously provides a force
`feedback device that is signi?cantly loWer in cost than other
`types of force feedback devices and is thus quite suitable for
`home consumer applications. A single actuator can be pro
`vided that directly applies force to the user manipulatable
`object, thus saving cost by the elimination of multiple
`actuators and complex force transmission and control sys
`tems. The actuator does not output force in a main sensed
`degree of freedom of the device, thus alloWing sensors to
`read the position of the user object Without substantial
`interference from forces and also simplifying the control of
`output forces. Furthermore, the actuator of the present
`invention can provide a variety of different types of force
`sensations to enhance the user’s experience and interface
`With a computer application.
`These and other advantages of the present invention Will
`become apparent to those skilled in the art upon a reading of
`the folloWing speci?cation of the invention and a study of
`the several ?gures of the draWing.
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram of a system including a host
`computer and a force feedback interface device of the
`present invention;
`
`65
`
`6,088,019
`
`4
`FIG. 2 is a side elevational vieW of a linear voice coil
`actuator suitable for use With the present invention;
`FIG. 3 is a perspective vieW of a joystick embodiment of
`the force feedback device of the present invention;
`FIG. 4 is a side elevational vieW of a mouse embodiment
`of the force feedback device of the present invention;
`FIG. 5 is a perspective vieW of a steering Wheel embodi
`ment of the force feedback device of the present invention;
`FIG. 6 is a side elevational vieW of a stylus embodiment
`of the force feedback device of the present invention; and
`FIG. 7 is a side elevational vieW of a different stylus
`embodiment of the force feedback device of FIG. 6.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`FIG. 1 is a block diagram illustrating a force feedback
`interface system 10 of the present invention controlled by a
`host computer system. Interface system 10 includes a host
`computer system 12 and an interface device 14.
`Host computer system 12 is preferably a personal
`computer, such as a Pentium-class (IBM-compatible) PC or
`Macintosh personal computer, or a Workstation, such as a
`SUN or Silicon Graphics Workstation. For example, the host
`computer system can a personal computer Which operates
`under the WindoWs, MS-DOS, or Linux operating systems.
`Alternatively, host computer system 12 can be one of a
`variety of home video game systems commonly connected
`to a television set, such as systems available from Nintendo,
`Sega, or Sony. In other embodiments, home computer sys
`tem 12 can be a television “set top box” or a “netWork
`computer” Which can be used, for example, to provide
`interactive computer functions to users over netWorks, or
`other appliance having computer functions.
`In the described embodiment, host computer system 12
`implements a host application program With Which a user 22
`is interacting via peripherals and interface device 14. For
`example, the host application program can be a video game,
`Web broWser, scienti?c analysis program, operating system,
`graphical user interface, medical simulation, or other appli
`cation program that utiliZes force feedback. Typically, the
`host application provides images to be displayed on a
`display output device, as described beloW, and/or other
`feedback, such as auditory signals. The application program
`and host computer provide a graphical environment With
`Which the user may interact. For example, the graphical
`environment may display graphical objects, such as icons,
`WindoWs, or 3-D objects; or entities, such as a player
`controlled simulated vehicle or character.
`Host computer system 12 preferably includes a host
`microprocessor 16, a clock 18, a display screen 20, and an
`audio output device 21. The host computer also includes
`other Well knoWn components, such as random access
`memory (RAM), read-only memory (ROM), and input/
`output (I/O) electronics (not shoWn). Host microprocessor
`16 can include a variety of available microprocessors from
`Intel, AMD, Cyrix, Motorola, or other manufacturers.
`Microprocessor 16 can be single microprocessor chip, or can
`include multiple primary and/or co-processors. Micropro
`cessor preferably retrieves and stores instructions and other
`necessary data from RAM and ROM, as is Well knoWn to
`those skilled in the art. In the described embodiment, host
`computer system 12 can receive locative data or a sensor
`signal via a bus 24 from sensors of interface device 14 and
`other information. Microprocessor 16 can receive data from
`bus 24 using I/O electronics 21, and can use I/O electronics
`
`APPLE INC.
`EXHIBIT 1018 - PAGE 10
`
`

`
`6,088,019
`
`5
`to control other peripheral devices. Host computer system
`12 can also output a command to interface device 14 via bus
`24 to cause force feedback for the interface device. Clock 18
`is a standard clock crystal or equivalent component used by
`host computer system 12 to provide timing to electrical
`signals used by microprocessor 16 and other components of
`the computer system.
`Display screen 20 is coupled to host microprocessor 16 by
`suitable display drivers and can be used to display images
`generated by host computer system 12 or other computer
`systems. Display screen 20 can be a standard display screen,
`CRT, ?at-panel display, 3-D goggles, or any other visual
`interface. In a described embodiment, display screen 20
`displays images of a simulation, game environment, oper
`ating system application, etc. For example, images describ
`ing a point of vieW from a ?rst-person perspective can be
`displayed, as in a virtual reality simulation or game. Or,
`images describing a third-person isometric perspective of
`objects, backgrounds, etc., or a 2-D image of a graphical
`user interface can be displayed. User 22 of the host computer
`12 and interface device 14 can receive visual feedback by
`vieWing display screen 20. Herein, computer 12 may be
`referred as displaying computer or graphical “objects” or
`“entities”. These computer objects are not physical objects,
`but is a logical softWare unit collections of data and/or
`procedures that may be displayed as images by computer 12
`on display screen 20, as is Well knoWn to those skilled in the
`art.
`Audio output device 21, such as speakers, is preferably
`coupled to host microprocessor 16 via ampli?ers, ?lters, and
`other circuitry Well knoWn to those skilled in the art. Host
`processor 16 outputs signals to speakers 21 to provide sound
`output to user 22 When an “audio event” occurs during the
`implementation of the host application program. Other types
`of peripherals can also be coupled to host processor 16, such
`as storage devices (hard disk drive, CD ROM drive, ?oppy
`disk drive, etc.), printers, and other input and output devices.
`An interface device 14 is coupled to host computer system
`12 by a bi-directional bus 24. The bi-directional bus sends
`signals in either direction betWeen host computer system 12
`and the interface device. Herein, the term “bus” is intended
`to generically refer to an interface such as betWeen host
`computer 12 and microprocessor 26 Which typically
`includes one or more connecting Wires, Wireless connection,
`or other connections and that can be implemented in a
`variety of Ways. In the preferred embodiment, bus 24 is a
`serial interface bus providing data according to a serial
`communication protocol. An interface port of host computer
`system 12, such as an RS232 serial interface port, connects
`bus 24 to host computer system 12. Other standard serial
`communication protocols can also be used in the serial
`interface and bus 24, such as RS-422, Universal Serial Bus
`(USB), MIDI, or other protocols Well knoWn to those skilled
`in the art. For example, the USB standard provides a
`relatively high speed serial interface that can provide force
`feedback signals in the present invention With a high degree
`of realism. An advantage of the microprocessor-enabled
`local control of system 10 is that loW-bandWidth serial
`communication signal