Method and apparatus for providing tactile sensations. For
`one embodimenta first frequency at which to outputa tactile
`sensation is received. A second frequency higher than the
`(21) Appl. No.:—11/787,495
`first frequency is determined. The second frequencyis based
`on a frequency at which an inertial actuator outputs a second
`tactile sensation. A waveform havingthefirst frequency and
`a waveform having the second frequency is combined to
`produce a signal configured to cause a composite tactile
`sensation at the second frequency,
`the composite tactile
`sensation conveying the first frequency.
`
`Immersion Corporation
`
`Apr. 16, 2007
`
`Related U.S. Application Data
`
`|3
`
`as) United States
`a2) Patent Application Publication co) Pub. No.: US 2007/0285216 Al
`(43) Pub. Date: Dec. 13, 2007
`
`Tierling et al.
`
`US 20070285216A1
`
`(54) PROVIDING ENHANCED HAPTIC
`FEEDBACK EFFECTS
`
`(75)
`
`Inventors: Kollin M. Tierling, Campbell, CA
`(US); Adam C. Braun, Sunnyvale, CA
`(US); Alex S. Goldenberg, Mountain
`View, CA (US)
`
`Correspondence Address:
`IMMERSION -THELEN REID BROWN
`RAYSMAN & STEINER LLP
`P.O. BOX 640640
`SAN JOSE, CA 95164-0640 (US)
`
`continuation-in-part of application No. 09/669,029,
`filed on Sep. 25, 2000, now abandoned.
`
`(60) Provisional application No. 60/156,354, filed on Sep.
`28, 1999.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`HO4B 3/36
`(52) US. Che
`eeeeeeccececseeseeseeeeeessessessneaneenense 340/407.1
`
`(57)
`
`ABSTRACT
`
`(73) Assignee:
`
`(22)
`
`Filed:
`
`(63) Continuation of application No. 09/908,184, filed on
`Jul. 17, 2001, now Pat. No. 7,218,310, which is a
`
`HOST COMPUTER SYSTEM
`
`160
`
`HOST
`PROCESSOR
`
`AUDIO OUTPUT
`DEVICE
`
`164
`
`26
`
`|
`‘
`
`DISPLAY
`DEVICE
`
`we ne en de ee re ee ne eee rene
`
`meeebeeeeeeee
`
`By Bo
`where en
`
`{
`
`72!
`
`SENSORS
`
`i
`90}
`HOUSING
`OR
`MANIPULATION
`
`
`
`
`
`54
`
`HAPTIC FEEDBACK
`INTERFACE DEVICE
`
`170
`
`LOCAL
`MICROPROCESSOR
`
`480
`SENSOR
`INTERFACE
`
`178
`
`OTHER
`INPUT
`
`184
`
`176
`
`182
`
`ACTUATOR
`INTERFACE
`
`POWER
`SUPPLY
`
`—
`
`APPLE 1017
`
`'
`
`APPLE 1017
`
`1
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 1 of 15
`
`US 2007/0285216 Al
`
`2
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 2 of 15
`
`US 2007/0285216 Al
`
`
`pLLihaahahahanhanLLLLLELLELLLOhknbffehrdadLXLLLALLERLLLALPLELLLLLALLLLLLLLL)
`
`
`asaeae,
`
`
`
`ZOE2g
`
`¢Ola
`
`3
`
`
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 3 of 15
`
`US 2007/0285216 Al
`
`FIG.3
`
`4
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 4 of 15
`
`US 2007/0285216 Al
`
`HOST COMPUTER SYSTEM
`
`4
`
`' 162
`
`SYSTEM
`CLOCK
`
`AUDIO OUTPUT
`DEVICE
`
`!
`!
`!
`
`160
`
`HOST
`PROCESSOR
`
`164
`
`!
`26:
`
`DISPLAY
`DEVICE
`
`20
`
`:
`
`170
`
`LOCAL
`MICROPROCESSOR
`
`HAPTIC FEEDBACK
`INTERFACE DEVICE
`180
`SENSOR
`INTERFACE
`
`i2
`i
`A72 |
`!
`!
`
`[|eNSORS
`
`‘50 |
`
`!
`178
`HOUSING_|
`
`OR
`|
`CLOCK
`OTHER|
`484
`INPUT
`MANIPULATION |!
`A
`
`MEMORY
`
`476
`
`182
`
`ACTUATOR
`INTERFACE
`
`:
`|!
`ACTUATOR
`ASSEMBLY|;
`
`54
`
`!
`
`'
`
`POWER
`SUPPLY
`
`+186
`
`FIG. 4
`
`5
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 5 of 15
`
`US 2007/0285216 Al
`
`4—200
`
`FIG. 5A
`
`
`
`214y
`
`210
`
`214C
`224i
`
`FIG. 5B
`
`
`
`yo 222
`
`yo"
`
`220
`
`FIG. 5C
`
`6
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 6 of 15
`
`US 2007/0285216 Al
`
`
`
`FIG. 6
`
`fe
`
`7
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 7 of 15
`
`US 2007/0285216 Al
`
`300
`
`High and Low PassFilters
`
`302
`
`Gain
`
`0.8
`
`0.6
`
`0.4
`
`0.2
`
`304
`
`0
`
`50
`
`100
`
`150
`
`200
`
`250
`
`300
`
`350
`
`400
`
`Frequency (Hz)
`
`——17(1 + f/f)
`——1-1/(1+f/ fc)
`
`FIG. 8
`
`8
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 8 of 15
`
`US 2007/0285216 Al
`
`nf
`
`6Ola
`
`SuiteseeeeURENmeAATIFUTTTTETAUER
`pVNSWTttttyCCE
`¢di—ls (08s)
`8a——__L4a——9A—G3A—vAa—€5-----
`
`
`
`
`
`S}ayWpayeosunpeziewoN
`
`ay
`
`S'°0
`spmuubeyy,
`
`9
`
`
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 9 of 15
`
`US 2007/0285216 Al
`
`1.0
`
`0.60.8
`
`
`
`Time(sec)
`
`FIG.10
`
`N- 330 y
`
`0.4
`
`0.2
`
`apnyubep,
`
`10
`
`10
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 10 of 15
`
`US 2007/0285216 Al
`
`(99S)
`
`lLOla
`
`dul]
`
`apmuuBey)
`
`11
`
`11
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 11 of 15
`
`US 2007/0285216 Al
`
`clOls
`
`(98S)Olu
`
` gpmuBeyy
`
`12
`
`12
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 12 of 15
`
`US 2007/0285216 Al
`
`ebOld
`
`
`
` (99S)Sup [attheorca
`
`eer
`esaLaa
`
`
`
`Gb
`
`” spmuubey
`
`13
`
`13
`
`
`

`

`US 2007/0285216 Al
`
`
`
`oe
`
`$}oayqpeyeosunpeziewon
`
`Patent Application Publication Dec. 13,2007 Sheet 13 of 15
`
`(99S)OwlpftT|TeAUUM,
`OATNTAATCA,Th
` -
`vlOlas 8a—_
`
`
`243-—93———SA—vaA—€A--77°¢a——la—
`
`
`
`
`
`
`
`spmuubeyy
`
`14
`
`14
`
`
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 14 of 15
`
`US 2007/0285216 Al
`
`
`
`(99S)Swit
`
`GbOlA
`
`HARI Aaysodwoy
`Metteladlaladaltdadiat
`
`
`apmiuseyy
`
`15
`
`15
`
`

`

`Patent Application Publication Dec. 13,2007 Sheet 15 of 15
`
`US 2007/0285216 Al
`
` ybe i1Sulugeh=[Te]
`
`
`recwoul:teed‘Z6eageggeoseBE
` uoHenes‘weayoutUONEZEWON—JONIsiseuUAgspeyy
`
`
`
`QeMULIJaAug
`
`
`
`
`-O4©3||(ed+Jemjuls©oss||+)<)|-@<(x)AL cig
`(+)<7.zaaa]
`u(ed+JEm)ulsuciaae
`
`(zd+}M)ulsuzugiaea]
`
`(pd+We)uisupug4g
`(pd+ipm)uisurl
`
`(zd+l@m)uisuzig
`
`9“Sls
`
`
`
`peayeWy
`
`ooo
`
`oo9
`
`G)
`
`uLugb
`
`L
`
`
`
`uLUg-L
`
`16
`
`16
`
`
`
`
`
`
`
`

`

`US 2007/0285216 Al
`
`Dec. 13, 2007
`
`PROVIDING ENHANCED HAPTIC FEEDBACK
`EFFECTS
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application is a continuation of U.S. patent
`application Ser. No. 09/908,184,
`entitled,
`“Providing
`Enhanced Haptic Feedback Effects”, filed Jul. 17, 2001, by
`Kollin M. Tierling, Adam C. Braun and Alex F. Goldenberg.
`which is a continuation-in-part of co-pending U.S. patent
`application Ser. No. 09/669,029, entitled “Controlling Hap-
`tic Sensations for Vibrotactile Feedback Interface Devices,”
`filed Sep. 25, 2000 by Goldenberg et al., which claims the
`benefit of U.S. Provisional Application No. 60/156,354, filed
`Sep. 28, 1999, entitled, “Controlling Force Sensations for
`Vibrotactile Feedback Interface Devices,” and which are
`incorporated herein by reference in their entirety.
`BACKGROUND OF THE INVENTION
`
`[0002] Embodiments of the invention relate generally to
`interface devices for allowing humans to interface with
`computer systems, and more particularly to low-cost com-
`puter interface devices that allow the user to provide input
`to computer systems and allow computer systemsto provide
`tactile feedback to the user.
`
`[0003] A user can interact with an environment displayed
`by a computer to perform functions and tasks on the com-
`puter, such as playing a game, experiencing a simulation or
`virtual reality environment, using a computer aided design
`system, operating a graphical user interface (GUI), etc.
`Common human-computer interface devices used for such
`interaction include a mouse, joystick, trackball, gamepad,
`steering wheel,
`stylus,
`tablet, pressure-sensitive sphere,
`remote control, or the like. Typically, the computer updates
`the environmentin response to the user’s manipulation of a
`manipulandum or user object such as a joystick handle or
`mouse, and provides visual and audio feedback to the user.
`The computer senses the user’s manipulation of the user
`object using sensors provided on the interface device.
`
`In someinterface devices, haptic feedback is also
`[0004]
`provided to the user. These types of interface devices can
`provide physical sensations which are felt by the user
`manipulating the user object of the interface device. One or
`more motors or other actuators are coupled to the device
`housing or manipulandum and are connectedto the control-
`ling computer system. The computer system controls forces
`output by the actuators in conjunction and coordinated with
`displayed events. The computer system can thus convey
`physical force sensations to the user in conjunction with
`other supplied feedback as the user is grasping or contacting,
`the interface device or manipulatable object of the interface
`device.
`
`In many haptic feedback devices, the haptic feed-
`[0005]
`back takes the form of vibrations, jolts, or pulses output on
`the housing or manipulandum which are experienced by the
`user, referred to as “tactile” sensations herein. For example,
`many gamepad devices include a spinning eccentric mass
`that creates inertial vibrations on the housing or object.
`Other devices, such as the I-Feel Mouse from Logitech
`Corp., provide inertial vibrations using a linearly-moving
`mass. Still other devices may vibrate a housing or object by
`impacting or directly moving the housing or object with the
`actuator.
`
`[0006] One problem with current haptic feedback devices
`is that tactile sensations output to the user tend to be more
`effective in particular frequency ranges andless effective in
`other frequency ranges. For example, vibrations output on
`the housing by an inertial haptic feedback device often feel
`strong to the user at higher frequencies of vibration, but
`often feel less strong to the user at lower frequencies. In
`linearly-moving mass embodiments, for example, this is due
`to the inertial mass moving slower for lower frequency
`vibrations, so that the mass does not accelerate as much and
`causestactile sensationsthat feel less strong. In addition, the
`mass might be pressed against the limits of its range of
`motion during most of the cycle time of the vibration,
`providing less force to be felt by the user.
`
`[0007] Another problem with moving mass haptic feed-
`back devices is that combining commanded effects may
`cause the mass to oscillate about a point close to an end of
`travel of the mass. This may cause the massto hit the end of
`travel before it has completed the desired oscillation, thus
`clipping the output sensation and reducing its fidelity. In
`addition, output forces may be reduced in strength when the
`mass operates near an end oftravel due to a physical spring
`coupled to the mass, since such spring resistance is strongest
`near the end oftravel.
`
`SUMMARY OF THE INVENTION
`
`[0008] Embodiments of the invention include methodsfor
`providing tactile sensations. For one embodimenta first
`frequency at which to output a tactile sensation is received.
`A second frequency higher than the first frequency is deter-
`mined. The second frequency is based on a frequency at
`which an inertial actuator outputs a secondtactile sensation.
`A waveform having the first frequency and a waveform
`having the second frequency is combinedto produce a signal
`configured to cause a composite tactile sensation at the
`second frequency, the composite tactile sensation conveying
`the first frequency.
`
`[0009] These and other advantages of the present inven-
`tion will become apparent to those skilled in the art upon a
`reading of the following specification of the invention and a
`study of the several figures of the drawing.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG.1 is a perspective view of system including a
`[0010]
`haptic interface connected to a host computer in accordance
`with one embodimentof the invention;
`
`[0011] FIG. 2 is a side cross sectional view of the haptic
`interface device of FIG. 1;
`
`[0012] FIG. 3 is a perspective view of one embodiment of
`an actuator assembly suitable for use with various embodi-
`ments of the invention;
`
`[0013] FIG. 4 is a block diagram illustrating an embodi-
`mentof the haptic interface device and host computerfor use
`with embodiments of the invention;
`
`[0014] FIGS. 5a-5c are graphs illustrating a method for
`using resonance pulse bursts to provide stronger low fre-
`quency tactile sensations in accordance with one embodi-
`mentof the invention;
`
`[0015] FIGS. 6 and 7 are graphs illustrating another
`method for providing a strong low frequencytactile sensa-
`
`17
`
`17
`
`

`

`US 2007/0285216 Al
`
`Dec. 13, 2007
`
`tion by adding it with a higher frequency sensation in
`accordance with one embodiment of the invention;
`
`[0016] FIG. 8 is a graph illustrating high and low pass
`filters used in another method for providing strong low
`frequencytactile sensations or combiningtactile sensations
`in accordance with one embodimentof the invention;
`
`[0017] FIG. 9 is a graphillustrating a low frequency
`commanded waveform anda higher frequency waveform to
`be combined with the low frequency waveform in accor-
`dance with one embodiment of the invention;
`
`[0018] FIG. 10 is a graph illustrating a modulation enve-
`lope resulting from filtering the low frequency waveform of
`FIG. 9;
`
`[0019] FIG. 11 is a graph illustrating a waveform resulting
`from the sum of products method of one embodimentof the
`invention;
`
`[0020] FIG. 12 is a graph illustrating the waveform of
`FIG. 11 when using normalized terms;
`
`[0021] FIG. 13 is a graph illustrating the waveform of
`FIG. 12 after having been combined again with the higher
`frequency waveform of FIG. 9;
`
`[0022] FIG. 14 is a graph illustrating another example of
`a low frequency commanded wavelorm and a higher [re-
`quency waveform to be combined with the low frequency
`waveform;
`
`[0023] FIG. 15 is a graph illustrating the combined nor-
`malized result of the sum of products method using the
`waveforms of FIG. 14; and
`
`[0024] FIG. 16 is a block diagram illustrating the sum of
`products method for combining waveforms in accordance
`with one embodiment of the invention.
`
`DETAILED DESCRIPTION
`
`In the following description, numerous specific
`[0025]
`details are set forth. However, it is understood that embodi-
`ments of the invention may be practiced without these
`specific details.
`In other instances, well-known circuits,
`structures and techniques have not been shown in detail in
`order not to obscure the understanding of this description.
`
`the specification to “one
`[0026] Reference throughout
`embodiment” or “an embodiment” means that a particular
`feature, structure, or characteristic described in connection
`with the embodimentis includedin at least one embodiment
`of the present invention. Thus, the appearanceof the phrases
`“tn one embodiment” or “in an embodiment” in various
`places throughout the specification are not necessarily all
`referring to the same embodiment. Furthermore,the particu-
`lar features, structures, or characteristics may be combined
`in any suitable manner in one or more embodiments.
`
`inventive aspects lie in less than all
`[0027] Moreover,
`features of a single disclosed embodiment. Thus, the claims
`following the Detailed Description are hereby expressly
`incorporated into this Detailed Description, with each claim
`standing on its own as a separate embodiment of this
`invention.
`
`[0028] FIG. 1 is a perspective viewofa tactile feedback
`interface system 10, in accordance with one embodiment of
`the invention, capable of providing input to a host computer
`
`based on the user’s manipulation of the interface device and
`capable of providing tactile feedback to the user of the
`system based on events occurring in a program implemented
`by the host computer. System 10 includes a device 12 anda
`host computer 14. In the described embodimentof FIG.1,
`device 12 is a mouse, but can be other types of devices as
`discussed below.
`
`[0029] Device 12 is an object that is preferably grasped,
`gripped, or otherwise physically contacted and manipulated
`by a user. For example, a user can move mouse 12 to provide
`planar two-dimensional
`input
`to a computer system to
`correspondingly move a computer generated graphical
`object, such as a cursor or other image,
`in a graphical
`environment provided by computer 14 or to controla virtual
`character, vehicle, or other entity in a game or simulation.In
`other embodiments, a joystick, knob, wheel, button, or other
`user manipulandum may be moved. In addition, device 12
`preferably includes one or more buttons 16a and 16to
`allow the user to provide additional commands to the
`computer system. A scroll wheel, analog buttons or other
`inputs, or other controls can also be included.
`[0030] Device 12 preferably includes an actuator assem-
`bly which is operative to produce forces on the device 12
`and tactile sensations to the user. One example of an actuator
`assembly is described below with reference to FIG. 3.
`[0031]
`In some embodiments, such as a mouse, device 12
`rests on a ground surface 22 such as a tabletop or mousepad.
`Ausergrasps the mouse 12 and moves the mousein a planar
`workspace on the surface 22 as indicated by arrows 24.
`Device 12 can be a relative sensing device or an absolute
`sensing device. In other embodiments, such as a gamepad,
`the device 12 maybeheld by the user.
`[0032] Computer 14 is coupled to the device 12 by a bus
`20, which communicates signals between device 12 and
`computer 14 and mayalso, in some preferred embodiments,
`provide power to the device 12. Components such as the
`actuator assembly require power that can be supplied from
`a conventionalserial port or through an interface such as a
`USBorFirewire bus. In other embodiments, signals can be
`sent between device 12 and computer 14 by wireless trans-
`mission/reception. In some embodiments, the powerfor the
`actuator can be supplementedor solely supplied by a power
`storage device provided on the device, such as a capacitor or
`one or more batteries. Some such embodiments are dis-
`
`closed in U.S. Pat. No. 5,691,898, incorporated herein by
`reference.
`
`[0033] Host computer 14 can be a personal computer or
`workstation, such as a PC compatible computer or Macin-
`tosh personal computer, or a Sun or Silicon Graphics work-
`station. For example, the computer 14 can operate under the
`Windows™, MacOS, Unix, or MS-DOSoperating system.
`Alternatively, host computer system 14 can be one of a
`variety of home video game console systems commonly
`connected to a television set or other display, such as
`systems available from Nintendo, Sega, or Sony. In other
`embodiments, host computer system 14 can be a “set top
`box”or a “network-” or “internet-computer” which allows
`users to interact with a local or global network such as the
`Internet. Host computer preferably includes a host micro-
`processor,
`random access memory (RAM),
`read only
`memory (ROM),
`input/output
`(1/O) circuitry, and other
`components of computers well-knownto those skilled in the
`art.
`
`18
`
`18
`
`

`

`US 2007/0285216 Al
`
`Dec. 13, 2007
`
`[0034] Host computer 14 preferably implements a host
`application program with which a user is interacting via
`device 12 and other peripherals, if appropriate, and which
`may include force feedback functionality. For example, the
`host application program can be a video game, word pro-
`cessor or spreadsheet, Web page or browserthat implements
`HTML or VRMLinstructions, scientific analysis program,
`virtual reality training program or application, or other
`application program that utilizes input of device 12 and
`outputs force feedback commandsto the device 12. Herein,
`for simplicity, operating systems such as Windows™, MS-
`DOS, MacOS, Linux, Be,etc. are also referred to as “appli-
`cation programs.” In one preferred embodiment, an appli-
`cation program utilizes a graphical user interface (GUI) to
`present options to a user and receive input from the user.
`Herein, computer 14 maybereferred as providing a “graphi-
`cal environment,”, which can be a graphical user interface,
`game, simulation, or other visual environment. The com-
`puter displays “graphical objects” or “computer objects,”
`which are not physical objects, but are logical software unit
`collections of data and/or procedures that may be displayed
`as images by computer 14 on display screen 26, as is well
`known to those skilled in the art. A displayed cursor or a
`simulated cockpit of an aircraft might be considered a
`graphical object. The host application program checks for
`inpul signals received from the electronics and sensors of
`device 12, and outputs force values and/or commandsto be
`converted into forces output for device 12. Suitable software
`drivers which interface such simulation software with com-
`
`puter input/output (I/O) devices are available from Immer-
`sion Corporation of San Jose, Calif.
`
`[0035] Display device 26 can be includedin host computer
`14 and can be a standard display screen (LCD, CRT,flat
`panel, etc.), 3-D goggles, or any other visual output device.
`Typically, the host application provides images to be dis-
`played on display device 26 and/or other feedback, such as
`auditory signals. For example, display screen 26 can display
`images from a GUI.
`
`the device 12 can
`In alternative embodiments,
`[0036]
`instead be a different interface device or control device. For
`
`example, handheld devices are very suitable for the actuator
`assemblies described herein. A hand-held remote control
`device used to select functions of a television, video cassette
`recorder, sound stereo, internet or network computer(e.g.,
`Web-TV™). Furthermore, a gamepad controller for video
`games or computer games can be used with the tactile
`feedback components and methods described herein, where
`the user grips the gamepad housing while operating buttons,
`joysticks, dials, spheres, or other controls on the gamepad.
`Other interface devices may also make use of the actuator
`assemblies described herein. For example, a joystick handle
`can include an actuator assembly, where tactile sensations
`are output on the joystick handle as thesole tactile feedback
`or to supplement kinesthetic force feedback in the degrees of
`freedom of the joystick. Trackballs, steering wheels, sty-
`luses, rotary knobs,
`linear sliders, gun-shaped targeting
`devices, medical devices, spheres, grips, etc. can also make
`use of the actuator assemblies described herein to provide
`haptic sensations.
`
`[0037] FIG. 2 is a side cross-sectional view of one
`embodiment of device 12 of FIG. 1. Device 12 includes one
`
`or more actuator assemblies for imparting haptic feedback
`
`such as tactile sensations to the user of the device. The
`actuator assembly outputs forces on the device which the
`user is able to feel.
`
`[0038] Device 12 includes a housing 50, a sensing, system
`52, and an actuator assembly 54. Housing 50 is shapedtofit
`the user’s hand, in the example shown,like a standard mouse
`while the user moves the mouse in the planar degrees of
`freedom and manipulates the buttons 16. Other housing
`shapes can be provided in many different embodiments.
`
`[0039] Sensor 52 detects the position of the mousein its
`planar degrees of freedom, e.g. along the X and Y axes. In
`the example shown, sensor 52 includes a standard mouse
`ball 64 for providing directional
`input to the computer
`system. Ball 64 is a sphere that extends partially out the
`bottom surface of the mouse androlls in a direction corre-
`
`sponding to the motion of the mouse on a planar surface 22.
`The ball motion can be tracked by cylindrical rollers 60
`which are coupled to sensors, such as sensor 62, for detect-
`ing the motion of the cylinders corresponding to x and y
`motion of the mouse. Other types of mechanisms and/or
`electronics for detecting planar motion of the device 12 can
`be used in other embodiments. For example, an optical
`sensor can be used to detect motion of the device relative to
`the planar support surface by optically taking and storing a
`number of images of the surface and comparing those
`images over time to determine if the mouse has moved, as
`is well known in the art.
`
`[0040] Buttons 16 can be selected by the user as a “com-
`mand gesture” when the user wishes to input a command
`signal to the host computer 14. A scroll wheel can also be
`include to provide additional input to the host computer.
`
`[0041] Device 12 includes an actuator assembly 54, and
`the actuator assembly includes an actuator 66 and a flexure
`mechanism (“flexure”) 68 coupled to the actuator 66 by the
`flexure 68. In the described embodiment, the actuator 66 acts
`as an inertial mass, so that a separate inertial mass is not
`required; this is described in greater detail with respect to
`FIG. 3. The actuator acting as inertial mass is oscillated in
`a linear direction by the actuator 66, such as approximately
`along the z-axis 51 which is approximately perpendicular to
`the planar workspace of the mouse in the x- and y-axes. The
`actuator is coupled to the housing 50 of the mouse such that
`inertial forces caused by the motion ofthe inertial mass are
`applied to the housing of the mouse, thereby conveying
`haptic feedback such as tactile sensations to the user of the
`mouse whois contacting the housing. These type oftactile
`sensations are different from kinesthetic haptic sensations,
`which are caused by the user directly contacting a moved
`(forced) object without a compliant flexure between the user
`and object. The mouse device can apply inertial forces
`substantially along the z axis, orthogonalto the planar x and
`y axes of the mousecontroller.
`
`[0042] Many types of actuators can be used, such as DC
`motors, a moving magnet actuator, a stepper motor, a
`pneumatic/hydraulic actuator, a torquer (motor with limited
`angular range), shape memory alloy material (wire, plate,
`etc.), a piezo-electric actuator, etc. In other embodiments, a
`linear force can be output using a linearly moving element,
`as in a voice coil actuator or a linear moving-magnet
`actuator, which are suitable for high bandwidth actuation.
`These embodiments are described in greater detail in U.S.
`Pat. No. 6,211,861, which is incorporated herein by refer-
`
`19
`
`19
`
`

`

`US 2007/0285216 Al
`
`Dec. 13, 2007
`
`ence. A variety of tactile sensations can be outputto the user,
`many of which are described in greater detail in copending
`application Ser. No. 09/585,741,
`incorporated herein by
`reference.
`
`[0043] A flexible or semi-flexible surface can be provided
`between the mouse and the ground surface, such as a
`standard mouse pad, a layer of rubber provided on the
`underside of the mouse or between portions of the mouse
`housing, etc. This type of flexible surface increases the
`transmissibility of the inertial forces from the actuator to the
`housing. In handheld device embodiments such as game-
`pads, no such flexible surface or layer is required.
`
`[0044] FIG. 3 is a perspective view of one embodiment
`100 of the actuator assembly 50 for use in the remote control
`12. Actuator assembly 100 includes a grounded flexure 120
`and an actuator 110 coupled to the flexure 120. The flexure
`120 can be a single, unitary piece made of a material such
`as polypropylene plastic (“living hinge” material) or other
`flexible material. Flexure 120 can be groundedto the hous-
`ing of the device 12, for example, at portion 121.
`
`[0045] Actuator 110 is shown coupledto the flexure 120.
`The housingofthe actuatoris coupled to a receptacle portion
`122 of the flexure 120 which houses the actuator 110 as
`shown.A rotating shaft 124 of the actuator is coupled to the
`flexure 120 in a bore 125 of the flexure 120 andis rigidly
`coupled to a central rotating member 130. Therotating shaft
`124 of the actuator is rotated about an axis B which also
`rotates member 130 about axis B. Rotating member 130 is
`coupledto a first portion 132a of an angled member131 by
`a flex joint 134. The flex joint 134 preferably is made very
`thin in the dimensionit is to flex so that the flex joint 134
`will bend when the rotating portion 130 movesthe first
`portion 132a approximately linearly. Thefirst portion 132a
`is coupled to the grounded portion 140 of the flexure by a
`flex joint 138 and the first portion 132a@ is coupled to a
`second portion 1325 of the angled memberbyflex joint 142.
`The second portion 1324,in turn, is coupledat its other end
`to the receptacle portion 122 of the flexure by a flex joint
`144.
`
`[0046] The angled member 131 that includes first portion
`132a and second portion 1326 moves linearly along the
`X-axis as shownbyarrow 136. In actuality, the portions 1324
`and 1326 move only approximately linearly. When the
`flexure is in its origin position (rest position), the portions
`132a and 1326 are preferably angled as shown with respect
`to their lengthwise axes. This allows the rotating member
`130 to push or pull the angled member 131 along either
`direction as shown by arrow 136.
`
`[0048] By quickly changing the rotation direction of the
`actuator shaft 124, the actuator/receptacle can be made to
`oscillate along the z-axis and create a vibration on the
`housing with the actuator 110 acting as an inertial mass. In
`addition, the flex joints included in flexure 120, such as flex
`joint 152, act as spring membersto provide a restoring force
`towardthe origin position (rest position) of the actuator 110
`and receptacle portion 132. In some embodiments, the stops
`can be includedin the flexure 120 to limit the motion ofthe
`
`receptacle portion 122 and actuator 110 along the z-axis.
`
`[0049] Other embodiments can provide other types of
`actuator assemblies, such as an eccentric mass coupled to a
`rotating shaft of an actuator to provide rotational inertial
`tactile sensations to the housing. The eccentric mass can be
`unidirectionally driven or bidirectionally driven. Other types
`of actuator assemblies mayalso be used, as disclosed in U.S.
`Pat. No. 6,184,868, such as a linear voice coil actuator,
`solenoid, moving magnet actuator, etc.
`
`[0050] FIG. 4 is a block diagram illustrating one embodi-
`ment of a haptic feedback system suitable for use with
`embodiments of the invention including a local micropro-
`cessor and a host computer system.
`
`[0051] Host computer system 14 may include a host
`microprocessor 160, a clock 162, a display screen 26, and an
`audio output device 164. The host computer also includes
`other well known components, such as random access
`memory (RAM), read-only memory (ROM), and input/
`output (I/O) electronics (not shown). Display screen 26
`displays images of a game environment, operating system
`application, simulation, etc. Audio output device 164, such
`as speakers,
`is preferably coupled to host microprocessor
`160 provides sound output to user when an “audio event”
`occurs during the implementation of the host application
`program. Other types of peripherals can also be coupled to
`host processor 160, such as storage devices (hard disk drive,
`CD ROMdrive, floppy disk drive, etc.), printers, and other
`input and output devices.
`
`Theinterface device, such as mouse 12, is coupled
`[0052]
`to host computer system 14 by a bi-directional bus 20 The
`bi-directional bus sends signals in either direction between
`host computer system 14 and the interface device. Bus 20
`can be a serial
`interface bus, such as an RS232 serial
`interface, RS-422, Universal Serial Bus (USB), Firewire
`(1394), MIDI, or other protocols well knownto those skilled
`in theart; or a parallel bus or wireless link. For example, the
`USBstandard provides a relatively high speed interface that
`can also provide powerto the actuator of actuator assembly.
`
`[0047] The actuator 110 is operatedin onlyafraction ofits [0053] Device 12 can include a local microprocessor 170.
`
`Processor 170 is considered local
`to device 12, where
`rotational range when driving the rotating member 130 in
`twodirections, allowing high bandwidth operation and high
`“local” herein refers to processor 170 being a separate
`frequencies of pulses or vibrations to be output. To channel
`microprocessor from any processors in host computer sys-
`the compression orstretching of the flexure into the desired
`tem 14. Microprocessor 170 can be provided with software
`z-axis motion, a flex joint 152 is provided in the flexure
`instructions to wait for commandsor requests from com-
`portion betweenthe receptacle portion 122 and the grounded
`puter host 14, decode the commandorrequest, and handle/
`portion 140. The flex joint 152 allowsthe receptacle portion
`control input and output signals according to the command
`122 (as well as the actuator 110, rotating member 130, and
`or request. In addition, processor 170 can operate indepen-
`second portion 132) to move (approximately) linearly in
`dently of host computer 14 by reading sensorsignals and/or
`the z-axis in response to motion of the portions 132a and
`calculating appropriate forces from sensor signals,
`time
`1326. A flex joint 150 is provided in the first portion 132a
`signals, and stored or relayed instructions selected in accor-
`of the angled member 131 to allow the flexing about flex
`dance with a host command. Microprocessor 170 can
`joint 152 in the z-direction to more easily occur.
`include one microprocessor chip, multiple processors and/or
`
`20
`
`20
`
`

`

`US 2007/0285216 Al
`
`Dec. 13, 2007
`
`co-processor chips, and/or digital signal processor (DSP)
`capability, or can be implemented as digital
`logic, state
`machines, ASIC,etc.
`
`[0054] Microprocessor 170 can receive signals from sen-
`sor(s) 172 and provide signals to actuator assembly 54 in
`accordance with instructions provided by host computer 14
`over bus 20. For example, in a local control embodiment,
`host computer 14 provides high level supervisory commands
`to microprocessor 170 over bus 20, and microprocessor 170
`decodes the commands and manageslow level force control
`loops to sensors and the actuator in accordance with the high
`level commandsand independently of the host computer 14.
`This operation is described in greater detail in U.S. Pat. No.
`5,734,373,
`incorporated by reference herein. In the host
`control
`loop, force commands are output from the host
`computer to microprocessor