OA AA AAAA
`
`US 20050134561A1
`
`as) United States
`a2) Patent Application Publication co) Pub. No.: US 2005/0134561 Al
`
` Tierling etal. (43) Pub. Date: Jun. 23, 2005
`
`
`(54) SYSTEM AND METHOD FOR MAPPING
`INSTRUCTIONS ASSOCIATED WITH
`HAPTIC FEEDBACK
`
`(76)
`
`Inventors: Kollin M.Tierling, Milpitas, CA (US);
`Juan Manuel Cruz-Hernandez,
`Montreal (CA); Alex S. Goldenberg,
`San Francisco, CA (US); Danny A.
`Grant, Montreal (CA)
`Correspondence Address:
`COOLEY GODWARD LLP
`ATTN: PATENT GROUP
`11951 FREEDOM DRIVE,SUITE 1700
`ONE FREEDOM SQUARE- RESTON TOWN
`CENTER
`RESTON,VA 20190-5061 (US)
`
`(21) Appl. No.:
`
`11/001,190
`
`(22)
`
`Filed:
`
`Dec. 2, 2004
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/530,980,filed on Dec.
`22, 2003.
`
`Publication Classification
`
`Inte C0?eecccccccccceeeeesscccessssnnnseeceesnssnnnnensss G09G 5/00
`(SV)
`(52) US. Che ecceccesssscsseeseeseessesnseneesseenesnneeneees 345/156
`67)
`ABSTRACT
`An embodiment of the invention provides a system and
`method for mapping instructions associated with haptic
`feedback. An apparatus having a controller including an
`effect mapper is disclosed. The controller is configured to
`receive effect instructions from at least one application. The
`effect mapper is in communication with the controller, and
`is configured to produce multiple haptic instructions in
`response to at least a portion of the received effect instruc-
`tions. The haptic instructions are at least partially based on
`a physical characteristic of a haptic device.
`
`
`
`PROCESSOR SYSTEM~!10 INTERFACE DEVICE 120
`
`
`
`SENSOR
`
`PROCESSOR
`
`DEVICE
`
`APPLE 1008
`
`APPLE 1008
`
`1
`
`

`

`US 2005/0134561 Al
`
`[‘81g
`
`Patent Application Publication Jun. 23,2005 Sheet 1 of 12
`
`WOSsSADOUd oz1GOLAR
`
`FOVASaLNIOlt7W&LSASYOssHo0ud
`
`
`9CT
`
`YOSNGaS
`
`prtthewcecenas|UATIOULNOD|ewe
`anwences
`
`2
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 2 of 12
`
`US 2005/0134561 Al
`
`vO7
`
`(”‘TOULNOD
`
`SURLANVUVd
`
`Z‘S14
`
`921 '
`
`dATIOWLNOD
`
`t‘‘t'4!'''att1'J
`
`LAdNIYasn
`
`‘7Ot
`
`3
`
`
`

`

`Patent Application Publication Jun. 23, 2005 Sheet 3 of 12
`
`US 2005/0134561 Al
`
`pezisomuds
`suonesuessUOTIONIISUTsuOpONASU]SuOHOATSU]souvoraay
`
`
`
`
`
`ondeondeyyondeyoA
`
`NOLLVOITddV
`
`peullojsuely
`
`O€T
`
`onde:weay”_
`nee
`
`
`
`SUONOMNSU]suOnoOnNsU]
`
`:LOdsAdaOIF
`
`PAANIGINOD
`
`MAONENOUS
`
`/AOLOATAS
`
`UWeIeIGOW
`
`suogongsuy
`
`4
`
`
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 4 of 12
`
`US 2005/0134561 Al
`
`ondey
`
`SuUOTIONLQSU]
`
`Oc!
`
`YATIOWLNOO
`
`WAONANOAS
`
`LOgdsg
`
`PAANIGNOD
`suononnsuy
`
`poulrojsuely,
`
`109A
`
`WHddVANLoss
`
`DE‘314
`
`/AOLOATAS
`
`WHIGOW
`
`yay
`
`suOpOnysUy
`
`5
`
`
`
`
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 5 of 12
`
`US 2005/0134561 Al
`
`Fig. 4
`
`SmMaACGCEER
`
`
`6
`
`

`

`Patent Application Publication Jun. 23, 2005
`
`Sheet 6 of 12
`
`US 2005/0134561 Al
`
`
`
`
`Fig. 6A
`
`Fig. 6B
`
`7
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 7 of 12
`
`US 2005/0134561 Al
`
`SoCCONOSClA OrOOoCoCoCcOs mOOoCoCOCCOSO
`
` BmMACES<Q
`SCoCcooCCOD\;ls
`COCOCORCOff
`
`Fig. 6C
`
`8
`
`

`

`Patent Application Publication Jun. 23, 2005
`
`Sheet 8 of 12
`
`US 2005/0134561 Al
`
`Fig. 7
`
`
`
`9
`
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 9 of 12
`
`US 2005/0134561 Al
`
`My
`
`AAYYaBOLHPsysseun
`ETLINUUmanHUTTNAAT
`
`UaINNTritt
`
`NezyjsuLsoy
`
`6‘31
`
`{ows}:ou,
`
`Kelcncrioin,..|.
`
`10
`
`
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 10 of 12
`
`US 2005/0134561 Al
`
`
`
`eySodUny-
`
`OL14
`
`Sewcan ievMTT
`
`11
`
`11
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 11 of 12
`
`US 2005/0134561 Al
`
`ig.
`
`
`
`Fig.11B
`
`IIA
`
`12
`
`

`

`Patent Application Publication Jun. 23,2005 Sheet 12 of 12
`
`US 2005/0134561 Al
`
`50°
`
`400
`
`150
`
`200
`
`«250
`
`300
`
`a0"
`
`00:
`
`High and Low Pass Filters 9
`
`
`H
`6
`tT
`6
`wo
`vy
`ao
`0
`0
`0
`0
`0
`0
`1
`ao;
`9Q
`0
`0
`0
`0
`0
`wi;0
`=O
`0
`O Orn200
`@o;0
`9Q
`0
`6/0 0
`0
`0
`0
`@Q
`090
`t/0O 0
`0
`0
`1
`0
`41
`6:;0 0
`0
`0
`0
`41
`0
`n/j/0
`0
`0
`0
`0
`0
`1
`ow 2 0 0 0
`Fig. 13
`
`Fréquency: (Hz)
`
`Fig. 12
`
`13
`
`13
`
`

`

`US 2005/0134561 Al
`
`Jun. 23, 2005
`
`SYSTEM AND METHOD FOR MAPPING
`INSTRUCTIONS ASSOCIATED WITH HAPTIC
`FEEDBACK
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`[0001] This application claims the benefit under 35 U.S.C.
`119(e) of U.S. Provisional Application Ser. No. 60/530,980,
`entitled “System and Method for Mapping Instructions
`Associated with Haptic Feedback,” filed Dec. 22, 2003,
`which is incorporated herein by reference in its entirety.
`
`BACKGROUND
`
`[0002] Systems and methods of the invention relate gen-
`erally to mapping instructions associated with haptic feed-
`back. More specifically, systems and methods of the inven-
`tion relate to mapping drive signals to haptic sensations.
`
`[0003] Computer users often use interface devices to pro-
`vide information to a computers or other electronic devices.
`For example, with such interface devices, a user can interact
`with an environment displayed by a computer to perform
`functions and tasks on the computer, such as playing a game,
`experiencing a simulation or virtual reality environment,
`using a computer aided design system, operating a graphical
`user interface (GUI), or otherwise affecting processes or
`images depicted on an output device of the computer.
`Common human interface devices for computers or elec-
`tronic devices include, for example, a joystick, button,
`mouse, trackball, knob, steering wheel, stylus, tablet, pres-
`sure-sensitive ball, remote control, wireless phone, and
`stereo controls.
`
`In someinterface devices, feedback, such as force
`[0004]
`feedback, can also be providedto a user. This force feedback
`can also be referred to as haptic feedback or can be provided
`in the form of haptic sensations. Each of these interface
`devices, for example, includes one or more haptic devices,
`which are connected to a controlling processor, controller,
`and/or computer. Consequently, by a controlling processor,
`controller, and/or computer, haptic forces produced by the
`haptic device can be controlled in coordination with actions
`of the user and/or events associated with an audible envi-
`ronment, or a graphical or displayed environment by send-
`ing control signals or commandsto haptic feedback device.
`
`[0005] Multi-mode haptic devices that provide desirable
`performance have been developed. For example, U.S. appli-
`cation Ser. No. 10/301,809, the entire disclosure of which is
`incorporated herein by reference, discloses haptic feedback
`using a device having a rotary harmonic moving mass.
`Accordingly, additional systems and methods for mapping
`effects produced by multi-mode haptic devices, as well as
`single-mode haptic devices, are desirable.
`
`SUMMARY
`
`[0006] An embodimentof the invention provides a system
`and method for mapping instructions associated with haptic
`feedback. An apparatus having a controller including an
`effect mapper is disclosed. The controller is configured to
`receive effect instructions from at least one application. The
`effect mapper is configured to produce multiple haptic
`instructions in response to at least a portion of the received
`effect instructions. The haptic instructions are at least par-
`tially based on a physical characteristic of a haptic device.
`
`[0007] A methodis also disclosed that includes receiving
`an effect instruction and mapping the received effect instruc-
`tion to at least one haptic instruction based on an effect
`mapping schema. The mappingis basedat least partially on
`a pre-determined characteristic of a haptic device model. At
`least one haptic signal is transmitted to a haptic device, and
`is based on the at least one haptic instruction.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0008] FIG.1 is a block diagram of a system including a
`processor system and aninterface device, according to an
`embodimentof the invention.
`
`[0009] FIG.2 is a diagram illustrating a haptic device, a
`controller, and a sensor, according to an embodimentof the
`invention.
`
`[0010] FIG. 3A is a block diagram illustrating a haptic
`system, according to an embodimentof the invention.
`
`(0011] FIG. 3B is a block diagram illustrating the con-
`troller shown in FIG. 3A, according to an embodiment of
`the invention.
`
`[0012] FIG. 3C is a block diagram illustrating the con-
`troller shown in FIG. 3A, according to an embodiment of
`the invention.
`
`{0013] FIG. 4 is a diagram illustrating a transformation
`matrix, according to an embodimentof the invention.
`
`(0014] FIG.5 is a diagram illustrating transformation of
`effect
`instructions, according to an embodiment of the
`invention.
`
`[0015] FIG. 6A is a diagram illustrating an initiation
`transformation matrix, according to an embodimentof the
`invention.
`
`[0016] FIG. 6B is a diagram illustrating a sustaining
`transformation matrix, according to an embodimentof the
`invention.
`
`{0017] FIG. 6C is a diagram illustrating a termination
`transformation matrix, according to an embodimentof the
`invention.
`
`{0018] FIG.7 is a diagram illustrating a wavelet transfor-
`mation matrix, according to an embodimentof the invention.
`
`[0019] FIG. 8 is a diagram illustrating a pulse mode
`transformation matrix, according to an embodimentof the
`invention.
`
`[0020] FIG. 9 is a plot showing a desired output frequency
`and a resonant frequency of a haptic device, according to an
`embodimentof the invention.
`
`[0021] FIG.10 is a plot showing a modulated signal used
`to drive a haptic device according to an embodimentof the
`invention.
`
`[0022] FIG. 11A is a diagram illustrating a transformation
`matrix, according to an embodimentof the invention.
`
`[0023] FIG. 11B is a diagram illustrating a transformation
`matrix, according to an embodimentof the invention.
`
`[0024] FIG. 12 is a plot showing gains associated with
`blending functions of multiple operational modes, according
`to an embodiment of the invention.
`
`14
`
`14
`
`

`

`US 2005/0134561 Al
`
`Jun. 23, 2005
`
`FIG.13 is a diagram illustrating a transformation
`[0025]
`matrix associated with a blending according to an embodi-
`ment of the invention.
`
`DETAILED DESCRIPTION
`
`[0026] Systems and methods for mapping instructions
`associated with haptic feedback are described. More spe-
`cifically, an apparatus having a controller that includes an
`effect mapper according to an embodimentof the invention
`is described. The controller is configured to receive effect
`instructions from at least one application. The effect mapper
`is configured to produce haptic instructions in responseto at
`least a portion of the received effect instructions. The haptic
`instructions are at least partially based on a physical char-
`acteristic of a haptic device.
`
`second component, for example. When components are to
`be output in sequence, individual components can be used to
`mapthe start, the middle, and the end of a haptic feedback,
`for example.
`
`[0032] FIG. 1 is a block diagram of a system having a
`processor system and aninterface device, according to an
`embodimentof the invention. The system illustrated in FIG.
`1 includes a processor system 110 in communication with an
`interface device 120. The processor system 110 can be, for
`example, a commercially available personal computer or a
`less complex computing or processing device that is dedi-
`cated to performing one or more specific tasks. For example,
`the processor system 110 can be a terminal dedicated to
`providing an interactive virtual reality environment, such as
`a gaming system,or the like.
`
`[0033] The processor system 110 includes a processor 112,
`[0027] Feedback provided via a haptic device is some-
`times referred to as vibrotactile feedback or kinesthetic
`which according to one or more embodiments of the inven-
`tion, can be a commercially available microprocessor. Alter-
`feedback, and is referred to more generally herein as “haptic
`natively, the processor 112 can be an application-specific
`feedback.” Such haptic feedback can be provided,
`for
`integrated circuit (ASIC) or a combination of ASICs, which
`example, by way of a haptic device or an interface device
`is designed to achieve one or more specific functions, or
`including a haptic device. Sensations that can be perceived
`enable one or more specific devices or applications. In yet
`by a user of a haptic device providing haptic feedback are
`anotheralternative, the processor 112 can beacircuit analog
`generally referred to herein as “haptic sensations.”
`or digital, or a combination of multiple circuits.
`
`[0028] An “application” is software designed to fulfill
`specific needs. Such applications can include, for example,
`gaming software, haptic-device control software, etc., or
`portions thereof.
`
`[0029] Physical characteristics of a haptic device are mea-
`surable features of that haptic device or components ofthat
`haptic device. Physical characteristics can include,
`for
`example, mass, shape, acceleration, deceleration, resonant
`frequencies, harmonic modes, frequency response, perfor-
`mance at varying altitudes or orientations, and/or other
`measurable features of the haptic device.
`
`[0030] An effect instruction is an instruction that is con-
`figured to cause a haptic device to output a specific haptic
`sensation independent of physical characteristics of that
`haptic device. According to one or more embodiments of the
`invention, an effect mapper mapsreceivedeffect instructions
`to haptic instructions at least partially based on a physical
`characteristic of a haptic device. Thus, haptic instructions
`are tailored or modified according to a specific haptic device.
`When provided to that specific haptic device, these haptic
`instructions cause the haptic device to output haptic sensa-
`tions that better replicate the haptic sensations associated
`with the effect instructions. It should be noted that the term
`“instructions” as used herein connection with effect instruc-
`tions and haptic instructions can include,
`for cxample,
`signals, commands,or the like.
`
`[0031] Effect mappings, according to one or more embodi-
`ments of the invention, can alter or control effect compo-
`nents or parameters of a haptic sensation, such as amplitude,
`waveshape,
`frequency, phase, duration, or delay of the
`haptic feedback to be output by a haptic device. These effect
`components or parameters can form a portion of an overall
`haptic sensation to be output by way of a haptic device, for
`example, by way of synthesis or other combination. Thus,
`such haptic feedback components can be output simulta-
`neously or in sequence, for example. In cases where haptic
`feedback components are synthesized, the frequency of a
`first component can be modulated by the frequency of a
`
`the processor 112 can optionally
`[0034] Alternatively,
`include one or more individual sub-processors or coproces-
`sors. For example,
`the processor can include a graphics
`coprocessorthat is capable of rendering graphics, a control-
`ler that is capable of controlling one or more devices, a
`sensorthat is capable of receiving sensory input from one or
`more sensing devices, and so forth.
`
`[0035] The processor system 110 also includes a memory
`component 114. As shown in FIG. 1, the memory compo-
`nent 114 can include one or more types of memory. For
`example, the memory component 114 can include a read
`only memory (ROM) component 1144 and a random access
`memory (RAM) component 114b. The memory component
`114 can also include other types of memory notillustrated in
`FIG.1 that are suitable for storing data in a form retrievable
`by the processor 112. For example, electronically program-
`mable read only memory (EPROM),erasable electrically
`programmable
`read only memory (EEPROM),
`flash
`memory, as well as other suitable forms of memory can be
`included within the memory component114. The processor
`system 110 can also include a variety of other components,
`depending upon the desired functionality of the processor
`system 110.
`
`[0036] The processor 112 is in communication with the
`memory component 114, and can store data in the memory
`component 114 or retrieve data previously stored in the
`memory component 114. The components of the processor
`system 110 can communicate with devices external to the
`processor system 110 by way of an input/output (1/0)
`component 116.
`
`[0037] According one or more embodimentsof the inven-
`tion, the I/O component116 can includea variety of suitable
`communication interfaces. For example, the I/O component
`116 can include wired connections, such as standard serial
`ports, parallel ports, universal serial bus (USB) ports,
`S-video ports, large area network (LAN)ports, small com-
`puter system interface (SCSI) ports, and so forth. Addition-
`
`15
`
`15
`
`

`

`US 2005/0134561 Al
`
`Jun. 23, 2005
`
`for example,
`the I/O component 116 can include,
`ally,
`wireless connections, such as infrared ports, optical ports,
`Bluetooth wireless ports, wireless LAN ports, or the like.
`
`[0038] By way of the I/O component 116, the processor
`system 110 can communicate with other devices, such as
`interface devices 120. These interface devices 120 can be
`
`configured to provide haptic feedback. Each interface device
`120 can communicate with the processor system 110 by way
`of an I/O component 1164, which is similar to the I/O
`component 116 of the processor system 110 and can include
`any of the wired or wireless communicationsports described
`above in connection with that I/O component 116. Thus, the
`communications link between the I/O component 116 of the
`processor system 110 and the I/O component 116a of the
`interface device 120 can take a variety of forms, including,
`for example, wired communicationslinks, wireless commu-
`nications links (e.g., RF links), optical communications
`links, or other suitable links.
`
`[0039] The interface device 120 includes a number of
`components, such as a processor 122, a haptic device 124,
`and a sensor 126. As with the components of the processor
`system 110, the interface device 120 can include additional
`components. For example, the interface device can include
`additional duplicates of the components shown in FIG.1.
`Additionally,
`the interface device 120 can include other
`components not shown in the figure. For example, whereit
`is desirable to store data received by the interface device 120
`via I/O component 116a, a suitable memory component or
`buffer memory component can be used.
`
`[0040] The processor 122 of the interface device 120, can
`be similar to the processor 112 of the processor system 110,
`described above, or can be specifically designed (e.g., an
`ASIC) and/or programmed for
`the functionality of the
`interface device 120. As with the processor 112 of the
`processor system 110, the processor 122 of the interface
`device 120, can include a variety of sub-processors, which
`can, for example, be used in parallel.
`
`the interface device 120
`[0041] As discussed above,
`includes a haptic device 124, whichis used to provide tactile
`or haptic feedback to a user of the interface device 120.
`According to an embodimentof the invention, haptic feed-
`back can be provided by way of a physical object, such as
`a housing, a manipulandum,or the like. The haptic device
`124 can take a variety of forms, including one or more haptic
`devices that each have multiple operational modes associ-
`ated with multiple corresponding frequency ranges. Some
`examples of haptic device 124 configurations that can be
`used in accordance with one or more embodiments of the
`invention will be described in greater detail below. The
`examples of haptic devices 124 given below, however, are
`not intended to form an exhaustivelist of all types of haptic
`devices 124 that can be included in the interface device 120
`
`but are intended instead as examplesonly.
`
`the haptic device 124 can be a
`[0042] For example,
`harmonic (e.g., multi-mode) haptic device or a non-har-
`monic (e.g., single mode) haptic device. Harmonic haptic
`devices 124 have more than one operational mode, while
`non-harmonic haptic devices 124 can have, for example,
`only one operational mode. The operational mode associated
`with non-harmonic haptic devices, as well as with harmonic
`haptic devices operating in a single operational mode is
`termed the “unidirectional” operational mode. Harmonic
`
`haptic devices also have a “harmonic” operational mode,
`which corresponds to operation that use harmonics of the
`physical elements within the haptic device 124. For
`example,
`in cases where the haptic device 124 uses a
`rotational mass to provide vibrotactile haptic feedback, a
`spring and mass system, which has more than one harmonic
`mode, can be used in more than one corresponding opera-
`tional mode.
`
`[0043] The sensor 126 of the interface device 120 is
`configured to sense input from a user of the interface device
`120. For example,
`the sensor 126 can be used to sense
`manipulation or movement of a physical object, such as a
`manipulandum,of the interface device 120. The sensor 126
`can also be used to sense other forms of user input, such as
`pressure, speed, acceleration, torque, light, or other measur-
`able quantities. For example, the sensor 126 can incorporate
`a piezoelectric sensor to sense pressure, an inertial measure-
`ment unit IMU), such as an accelerometer, to sense various
`forms of motion, a photovoltaic sensor to sense changes in
`light levels, and/or other sensors.
`
`[0044] As shownin FIG.1, the various components ofthe
`interface device 120 are in communication with one another
`and with the components of the processor system 110 (via
`the I/O components 116, 116a). The processor 122 of the
`interface device 120, for example, can be used to control the
`haptic device 124 based on information received from the
`sensor 126. Similarly, the processor 112 of the processor
`system 110 can be used to control the haptic device 124 of
`the interface device 120 based on information received from
`the sensor 126 of the interface device 120;
`in such an
`embodiment, the processor 122 need not be present. Alter-
`natively, the processor 112 of the processor system 110 (also
`referred to as a “host processor”) can be used in coordination
`with the processor 122 of the interface device 120 (also
`referred to as a “local processor”) both to interpret data
`received from the sensor 126 andto control the haptic device
`124.
`
`[0045] The processor system 110 and the interface device
`120 can optionally use one or more controllers 1304, 130,
`130c, 130d (which can be referred to hereinafter as a
`controller 130, collectively, individually, or as a subset). As
`shown in FIG. 1, a controller or control method 130 can
`exist within the processor 112 of the processor system 110
`and/or the processor 122 of the interface device 120. Addi-
`tionally, a controller 130 can be a separate component
`connected to the other components of the processor system
`110 and/or the interface device 120 via a bus or other
`suitable connection.
`
`[0046] FIG. 2 is a diagram illustrating a haptic device
`controller, and a sensor, according to an embodimentof the
`invention. FIG. 2 also showsdifferent data values provided
`to the system.
`
`[0047] As shown in FIG. 2, user input 202 is provided
`(e.g., via the user interface device 120 shownin FIG.1), and
`is received by the sensor 126. The user input 202 can also
`optionally be provided to a controller 130 (e.g., by way of
`the sensor 126, or some other devices configured to accept
`and convey user input). The sensor 126 also receives infor-
`mation from the haptic device 124. For example, the sensor
`126 can sense the actual movements of the haptic device
`124, thereby sensing the tactile or haptic feedback output by
`the haptic device 124.
`
`16
`
`16
`
`

`

`US 2005/0134561 Al
`
`Jun. 23, 2005
`
`[0048] According to an arrangementof the system shown
`example, in cellular telephones, personal digital assistants
`in FIG. 2, the controller 130 receives data from the sensor
`(PDAs), and other handheld devices. When the application
`126 (shown byasolid line), and optionally receives user
`302, the controller 130, and the synthesizer/linearizer 304
`input 202 and control parameters 204 (shown by dotted
`are not collocated,
`the architecture of the system is a
`lines). Based on the data from the sensor 126, and any
`“distributed architecture system.” A distributed architecture
`received optional user input 202 and control parameters 204,
`system can be used, for example, in gaming systems, per-
`the controller 130 controls the tactile output or haptic
`sonal computing systems, and other systemsthat use periph-
`feedback of the haptic device 124. This is accomplished by
`eral or external interface devices 120 to access functionality
`a control signal that the controller 130 outputs to the haptic
`of a central processor system 110. Alternatively, when the
`device 124. The control signal can be based on a numberof
`controller 130 is located within a processor system 110, and
`parameters, including, for example, control parameters 204.
`the synthesizer/linearizer 304 is located within a remote
`Additionally, control parameters 204 and other parameters
`interface device 120, the architecture of the system can be
`that are used by the controller 130 to control the haptic
`referred to as a “hybrid architecture system.” Alternatively,
`device 124 can be stored in the memory component 114 of
`when the application 302, the controller 130, and the syn-
`thesizer/linearizer 304 are not collocated in the same device
`the processor system 110, or by another suitable memory
`component (e.g., a memory component of the interface
`that houses the haptic device 124, the architecture of the
`device 120).
`system can bereferred to as a “force-streaming architecture
`system.”
`
`[0049] FIG. 3A is a block diagram illustrating a haptic
`system, according to an embodimentof the invention. More
`specifically, FIG. 3A illustrates various elements of a haptic
`device used to generate haptic sensations.
`
`[0050] As shown in FIG.3A,an application 302 that can
`run on a processor system (e.g., the processor system 110
`shown in FIG. 1) or an interface device (e.g., the interface
`device 120 shownin FIG.1, is provided). The application
`302 communicates effect instructions to a controller 130.
`The controller 130, in turn, provides haptic instructions to a
`synthesizer/linearizer 304, which may beeither a synthe-
`sizer or a linearizer, or a combination of both (referred to
`herein as “synthesizer/linearizer”). The synthesizer/linear-
`izer 304 receives the haptic instructions and provides syn-
`thesized haptic signals to a haptic device 124. Upon receiv-
`ing synthesized haptic
`signals
`from the
`synthesizer/
`linearizer 304,
`the haptic device 124 produces haptic
`sensations that a user can perceive via an interface device
`120.
`
`Information can optionally be provided to the con-
`[0051]
`troller 130 and/or the synthesizer/linearizer 304 from the
`haptic device 124 in the form of one or more feedback
`signals (shown by dotted lines in FIG. 3A). In cases where
`feedback is provided to the controller 130, modified haptic
`instructions are provided from the controller 130 to the
`synthesizer/linearizer 304, and modified synthesized haptic
`signals are provided by the synthesizer/linearizer 304 to the
`haptic device 124. Alternatively, in the case where feedback
`is provided from the haptic device 124 to the synthesizer/
`linearizer 304, the synthesizer/linearizer 304 provides modi-
`fied synthesized haptic signals to the haptic device.
`
`location of the various elements
`[0052] The physical
`shown in FIG. 3A can be varied. For example, the appli-
`cation 302, the controller 130, the synthesizer/linearizer 304,
`and the haptic device 124 can all be collocated within an
`interface device 120, according to one or more embodiments
`of the invention. Alternatively, any combination of the
`components shown in FIG. 3A, with the exception of the
`haptic device 124 can form part of the processor system 110,
`which is in communication with the interface device 120,
`which includes the haptic device 124.
`
`[0053] When the application 302, the controller 130, and
`the synthesizer/linearizer 304 are collocated on the same
`device, the architecture can be referred to as an “embedded
`control system.” Such an architecture can be used,
`for
`
`[0054] FIGS. 3B and 3C illustrate the controller 130 in
`greaterdetail, according to one or more embodiments of the
`invention. The configuration illustrated in FIGS. 3B and 3C
`differ only in which component within the controller 130
`receives optional feedback from the haptic device 124.
`
`[0055] The controller 130 can use a variety of suitable
`techniques to convert effect instructions to haptic instruc-
`tions. For example, various duty-cycle-related control meth-
`ods for controlling a multi-mode haptic device can be used
`to determine appropriate drive signals applied to the haptic
`device 124. A description of duty-cycle-related control
`methods that can be used according to one or more embodi-
`ments of the invention can be found in U.S. application Ser.
`Nos. 09/669,029 and 09/675,995, the disclosures of which
`are incorporated herein by reference.
`
`[0056] Multiple blending, effect switching, and/or selec-
`tion algorithms can be implemented by wayof the controller
`130. For example, methods for controlling haptic devices
`having multiple operational modes are described in U.S.
`Application No. [Attorney Docket No. IMMR-093/00US]
`entitled System and Method for Controlling Haptic Devices
`Having Multiple Operational Modes,the entire disclosure of
`which is incorporated herein by reference in its entirety.
`Additionally, other suitable techniques can be implemented
`by way of the controller, such as envelope modulation, a
`sum-of-products technique, and time domain filtering. The
`sum-of-products techniques use a weighting factor, which
`can be applied to effect instructions by either the multipli-
`cation of effect amplitudes. Effects can also be gated to
`produce additional effects.
`
`[0057] The controller 130 can also use a switching tech-
`nique to switch between different frequency ranges. A
`switching technique can use various parameters, such as the
`period of a desired haptic sensation to control the output of
`the haptic device 124. Long periods(i.e., low frequencies) of
`desired haptic sensations that are below a threshold fre-
`quency can use a pulse mode mapping technique, such as
`techniques using a pulse mode transformation matrix (e.g.,
`the matrix described below in connection with FIG. 8).
`Desired haptic sensations having periods within a middle
`range (i.e., middle range frequencies), for example within
`about 20 to 30 Hz, can be mapped using direct current (DC)
`mapping used with single-mode haptic devices 124. Desired
`haptic sensations having shorter periods (i.e., higher fre-
`
`17
`
`17
`
`

`

`US 2005/0134561 Al
`
`Jun. 23, 2005
`
`quencies) can use direct mapping to produce the haptic
`instructions from the received effect instructions, according
`to one or more embodiments of the invention.
`
`[0058] The controller 130 can perform different operations
`on received effect instructions based upon the frequencies of
`the effect instructions and/or the frequencies of the desired
`haptic sensations to be output from the haptic device 124.
`According to one or more embodimentsof the invention that
`operate on effect
`instructions based on their frequency
`ranges, several
`frequency ranges can be defined, each
`defined frequency range having different mapping tech-
`niques. Examples of frequency ranges that can be used in
`mapping effect instructions for a multi-mode haptic device
`124 can include, for example, a low-frequency range(e.g.,
`below about 20 Hz), a unidirectional frequency range(e.g.,
`between about 20 to 30 Hz),a transitional frequency range
`above the unidirectional frequency range, but below a har-
`monic/resonant frequency range of the haptic device, a
`harmonic/resonant frequency range (i.e., a range of frequen-
`cies surrounding the harmonic/resonant frequency), a high-
`frequency range above the harmonic/resonant frequency of
`the haptic device 124.
`
`[0059] The controller 130 includes an effect mapper 306
`and an effect combiner/effect sequencer 308, which can
`include a combiner alone, a sequencer alone, or a combi-
`nation of both (collectively referred to herein as an “effect
`combiner/sequencer”). The effect mapper includes a selec-
`tor/modifier 310, which can include a selector alone, a
`modifier alone, or a combination of both (collectively
`referred to herein as a “selector/modifier’). The selector/
`modifier 310 can be used to gate and/or modify the received
`effect instructions to cause the controller 130 to properly
`render haptic instructions. Where used herein,
`the term
`“modify” includes various ways of modifying such as, for
`example, modulating.
`
`the controller 130
`[0060] The effect mapper 306 of
`receives effect instructions from the application 302. Based
`upon one or more predetermined mapping techniques, the
`effect mapper 306 mapsthe received effect instructions to
`produce haptic instructions. The mapping ofeffects can be
`accomplished in a variety of ways, several of which will be
`described in greater detail below. By way of one or more
`effect mapping techniques, the effect mapper 306 changes
`the form of the received effect
`instructions to provid