(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2008/0294984 A1
`Nov. 27, 2008
`Ramsay et al.
`(43) Pub. Date:
`
`US 20080294984A1
`
`(54)
`
`(75)
`
`(73)
`
`(21)
`(22)
`
`
`
`
`
`CUSTOMIZINGHAPTC EFFECTS ON AN
`ENDUSER DEVICE
`
`Inventors:
`
`Erin B. Ramsay, Montreal (CA);
`Robert W. Heubel, San Leandro,
`CA (US); Jason D. Fleming, San
`Jose, CA (US); Stephen D. Rank,
`San Jose, CA (US)
`Correspondence Address:
`WOMBLE CARLYLE SANDRIDGE & RICE,
`PLLC
`ATTN: PATENT DOCKETING 32ND FLOOR,
`P.O. BOX 7037
`ATLANTA, GA 30357-0037 (US)
`Assignee:
`
`Immersion Corporation, San Jose,
`CA (US)
`12/060,324
`
`Appl. No.:
`
`(60)
`
`(51)
`
`Related U.S. Application Data
`Provisional application No. 60/940,227, filed on May
`25, 2007.
`
`Publication Classification
`
`Int. C.
`(2006.01)
`G06F 3/0
`U.S. Cl. ........................................................ T15/702
`
`(52)
`(57)
`ABSTRACT
`Systems, methods, and associated Software are described
`herein for enabling a regular user of an end user device. Such
`as a cellular telephone, to customize parameters associated
`with haptic effects applied to the user by the end user device.
`In one implementation, among several, a method described
`herein includes enabling a user of an end user device to access
`software adapted to design or modify haptic effects of the end
`user device. The method further includes enabling the user to
`open a haptic track file and enter or modify parameters asso
`ciated with the haptic effects of the opened haptic track file.
`
`Filed:
`
`Apr. 1, 2008
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MEMORY
`INTERFACE
`
`
`
`
`
`
`
`HAPTIC
`TRACK
`EDITING
`MODULE
`
`
`
`
`
`
`
`
`
`
`
`TRACK/
`EVENT
`ASSOCIATION
`MODULE
`
`
`
`
`
`TEMPORARY
`STORAGE
`UNIT
`
`
`
`
`
`
`
`TRACK
`MIXING
`MODULE
`
`
`
`
`
`
`
`TEST
`PLAYBACK
`MODULE
`
`tO HAPTIC
`ACTUATING
`DEVICE 18
`
`1
`
`APPLE 1006
`
`

`

`Patent Application Publication
`
`Nov. 27, 2008 Sheet 1 of 5
`
`US 2008/0294984 A1
`
`INPUT/
`OUTPUT
`DEVICES
`
`
`
`PROCESSOR
`
`IO
`
`
`
`
`
`
`
`
`
`
`
`HAPTIC
`ACTUATING
`DEVICE
`
`MEMORY
`
`FIG. I.
`
`
`
`PROGRAMMABLE
`HAPTIC DESIGN
`SOFTWARE
`
`2
`
`

`

`Patent Application Publication
`
`Nov. 27, 2008 Sheet 2 of 5
`
`US 2008/0294984 A1
`
`30
`
`tO
`ROM
`
`
`
`
`
`
`
`
`
`
`
`HAPTIC
`TRACK
`EDITING
`MODULE
`
`40
`
`TRACK/
`EVENT
`ASSOCIATION
`MODULE
`
`MEMORY
`INTERFACE
`
`
`
`
`
`TEMPORARY
`STORAGE
`UNIT
`
`TRACK
`MIXING
`MODULE
`
`
`
`
`
`
`
`
`
`TEST
`PLAYBACK
`MODULE
`
`to HAPTIC
`ACTUATING
`DEVICE 18
`
`3
`
`

`

`Patent Application Publication
`
`Nov. 27, 2008 Sheet 3 of 5
`
`US 2008/0294984 A1
`
`50
`
`MAGNITUDE - K- t1
`10,000
`1
`
`DURATION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`8,000
`
`6,000
`
`4,000
`
`2,000
`
`O
`
`
`
`54
`
`
`
`
`
`56
`
`RUR ms
`1,000
`120ms
`
`10
`
`100
`
`10,000
`
`ENVELOPE
`
`Attack Time
`
`S
`
`Fade Time
`
`S
`
`58
`
`Attack Level
`
`Fade Level
`
`o
`
`---
`
`FIG. 4
`
`4
`
`

`

`Patent Application Publication
`
`60
`
`6.
`
`s2 1.1.1.
`
`6.
`
`66
`
`Nov. 27, 2008 Sheet 4 of 5
`
`US 2008/0294984 A1
`
`44
`
`FIG. 54
`
`FIG. 5B
`
`FIG. 5C
`
`FIG. 5D
`
`5
`
`

`

`Patent Application Publication
`
`Nov. 27, 2008 Sheet 5 of 5
`
`US 2008/0294984 A1
`
`
`
`70
`
`ENABLE USER OF END USER
`DEVICE TO ACCESS
`SOFTWARE FOR DESIGNING
`OR EDITING HAPTIC EFFECTS
`
`72
`
`74
`
`76
`
`ENABLE USER TO OPEN A
`NEW FILE, EXISTINGHAPTIC
`TRACK FILE, OR EXISTING
`MIXED TRACK FILE
`
`ENABLE USER TO ENTER
`ORMODIFY PARAMETERS
`OF THE OPENED FILE
`
`ENABLE USER TO STORE
`MODIFIED FILE
`
`FIG. 6
`
`6
`
`

`

`US 2008/0294984 A1
`
`Nov. 27, 2008
`
`CUSTOMIZINGHAPTC EFFECTS ON AN
`ENDUSER DEVICE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`0001. This application claims the benefit of U.S. Provi
`sional Application No. 60/940,227, filed May 25, 2007, the
`entire disclosure of which is incorporated by reference herein.
`
`TECHNICAL FIELD
`
`0002 The present disclosure generally relates to devices
`having haptic feedback capabilities for providing haptic
`effects to a user. More particularly, the present disclosure
`relates to enabling a user to customize the haptic effects on an
`end user device.
`
`BACKGROUND
`
`0003 Electronic device manufacturers strive to produce a
`rich interface for users. Conventional electronic devices typi
`cally use auditory and visual cues to provide feedback to a
`user. For example, cellphones are commonly equipped with
`auditory and visual output devices for indicating an incoming
`telephone call. The auditory cues, for instance, may include a
`series of tones, synthesized music, or digitally-recorded
`music. However, in Some instances, auditory and visual cues
`may not be useful to a cell phone user. For example, in a
`movie theater, a user will typically have the phone's audible
`ringer silenced and will have the phone inaplace where visual
`cues cannot be seen. In this case, kinesthetic and/or tactile
`feedback can provide a more desirable and more effective
`interface.
`0004 Some electronic devices are equipped to provide
`kinesthetic feedback (such as active and resistive force feed
`back) and/or tactile feedback (such as vibration, vibrotactile
`feedback, texture, heat, etc.) to a user. Generally, these types
`of feedback are collectively known as “haptic feedback'.
`Haptic feedback can provide cues that enhance and simplify
`the user interface. Specifically, haptic effects may be useful to
`alert the user to specific events or to provide realistic feedback
`to create greater sensory immersion within a simulated or
`virtual environment. However, further improvements in the
`field of haptics can still be made to enhance the user interface
`of end user devices.
`
`SUMMARY
`
`0005. The present disclosure describes systems and meth
`ods for allowing a user of an end user device to modify
`parameters associated with haptic effects that can be played
`on the end user device. Among several possible embodi
`ments, a method disclosed herein comprises enabling a user
`of an end user device to access Software adapted to design or
`modify haptic effects of the end user device. The method
`further includes enabling the user to open a haptic track file
`and enter or modify parameters associated with the haptic
`effects of the opened haptic track file.
`0006. Other features, advantages, and implementations of
`the present disclosure, not expressly disclosed herein, will be
`apparent to one of ordinary skill in the art upon examination
`of the following detailed description and accompanying
`
`drawings. It is intended that such implied features, advan
`tages, and implementations of the present disclosure be
`included herein.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0007. The components in the following figures are not
`necessarily drawn to scale. Instead, emphasis is placed upon
`clearly illustrating the general principles of the present dis
`closure. Reference characters designating corresponding
`components are repeated as necessary throughout the figures
`for the sake of consistency and clarity.
`0008 FIG. 1 is a block diagram illustrating an end user
`device according to one embodiment.
`0009 FIG. 2 is a block diagram illustrating the memory
`shown in FIG. 1 according to one embodiment.
`0010 FIG. 3 is a block diagram of the programmable
`haptic design software shown in FIG. 2 according to one
`embodiment.
`0011
`FIG. 4 is a graphical user interface associated with
`an end user device allowing changes to a haptic track accord
`ing to one embodiment.
`0012 FIGS. 5A-5D are graphs illustrating a number of
`examples of possible haptic tracks that can be created using
`the graphical user interface of FIG. 4.
`0013 FIG. 6 is a flow chart illustrating operations pro
`vided by a haptic design program according to one embodi
`ment.
`
`DETAILED DESCRIPTION
`0014 Some electronic devices are equipped with a haptic
`actuating device to provide haptic feedback to a user. This
`haptic feedback can be used to Supplement auditory and
`visual feedback to enhance how information is provided to
`the user. For years, users of electronic devices have been
`given the option to modify various characteristics of auditory
`and visual outputs. For example, cell phone users can easily
`change ring tones if desired. However, a user typically does
`not have the option on the device itself to modify character
`istics of haptic feedback. Normally, a professional haptics
`designer uses a computer system to create haptic effects or
`haptic tracks having parameters which are permanently set in
`the device during the manufacturing stage. The end user
`normally does not have the means to create or modify the
`haptic effect files directly on the device after the device is
`purchased.
`0015. However, in the present disclosure, systems and
`methods are described that allow a user to create and/or
`modify the parameters associated with haptic feedback on the
`target device itself through on-device means. In this respect,
`the power to create customized haptic effects is placed in the
`hands of a regular end user who may not be as skilled as a
`professional haptics designer. The end user can then create or
`modify the haptic effects to the user's liking using tools
`similar to those available to a professional haptic designer, but
`perhaps on a simplified scale. Software and systems can be
`embedded in the end user devices, according to the teachings
`herein, to lead a user through the process of entering and/or
`modify parameters and characteristics of haptic tracks.
`0016. With the ability to modify haptic tracks, a user can
`customize haptic effects to be associated with specific
`“events.” For example, with respect to cellphones, a user may
`wish to associate certain customized haptic effects with
`receiving a call from a particular caller or group of callers.
`
`7
`
`

`

`US 2008/0294984 A1
`
`Nov. 27, 2008
`
`Also, a user may wish to customize certain haptic effects to
`play during the output of specific auditory and/or visual cues.
`Other events associated with an end user device may include,
`for example, the pressing of a key on a keypad, a time
`sensitive message or alarm, receiving a phone call, receiving
`an e-mail, etc.
`0017. Furthermore, the customized haptic effects created
`by a user can be stored on the device for use by the user. Once
`saved, customized haptic effects can also be shared with
`others by transmitting the haptics files to another device. In
`this respect, other users can play the haptic effects on these
`other devices as desired. Customized haptic effects can also
`be transmitted to another device to initiate immediate play
`back of a haptic track on the receiving device. In this example,
`customized sensations can be applied to intended recipients
`when they receive (or make) calls from (or to) the author of
`the customized haptic effects. This would be the haptic
`equivalent of the audio feature known as “ringback tones' and
`“ringtones, whereby a telephone caller and receiver, respec
`tively, hear a specific Song or message instead of the standard
`ringing.
`0018. Although many examples described in the present
`disclosure relate to cell phones, it should be understood that
`the teachings of the present disclosure may also include other
`applications in which haptic effects can be applied to a user of
`an end user device. Other advantages will become apparent to
`one of ordinary skill in the art from an understanding of the
`present disclosure.
`0019 FIG. 1 is a block diagram of an embodiment of an
`end user device 10. At least part of end user device 10 includes
`regular electrical circuitry for performing tasks associated
`with the normal operation of end user device 10. For example,
`end user device 10 may be a mobile handset, cell phone,
`personal digital assistant (PDA), personal navigation device,
`tablet PC, hand-held video gaming device, game module,
`game console, touch screen, touch pad, virtual reality force
`feedback device, or other suitable electronic device. In these
`examples, end user device 10 may include circuitry for per
`forming the normal functions of these devices. The present
`disclosure, however, describes further components and
`related software of end user device 10 for allowing a user to
`modify haptic tracks according to the teachings herein.
`0020. In some embodiments, end user device 10 can be a
`haptic-enabled device that provides haptic feedback to the
`user during specific events. Regarding the aspect of providing
`haptic feedback, the user may be able to sense the haptic
`effects from the device when it is held in the user's hand,
`stored in a pocket of the user's clothing, stored in a purse,
`handbag, pouch, or other portable accessory, attached or
`clipped to the user's clothing, worn as an item of clothing, or
`held in any other Suitable manner adjacent to or near the user.
`In this regard, end user device 10 can be in direct or indirect
`contact with at least a portion of the user's body in order that
`the user can experience the sensation of haptic effects pro
`vided by the device.
`0021. In addition to the regular circuitry and functionality
`of end user device 10, additional elements reside in or are
`associated with end user device 10, as described herein, to
`enable the user to create and/or modify haptic effects. The
`customized haptic effects can be played on end user device 10
`itself, provided that end user device 10 is capable of providing
`haptic feedback, or can be played on a corresponding com
`patible device that is haptic-enabled.
`
`0022 End user device 10, according to the embodiment of
`FIG. 1, includes input/output devices 12, processor 14,
`memory 16, and haptic actuating device 18. End user device
`10 further comprises a bus 20 or other suitable communica
`tion interface allowing input/output devices 12, processor 14,
`memory 16, and haptic actuating device 18 to communicate
`with each other. In some embodiments, end user device 10
`may further include a transceiver, Such as for use in a cell
`phone or other wireless communication device. Such a trans
`ceiver can be used for communication within a cell phone
`network or within a short range network, Such as is associated
`with Bluetooth R compatible devices. In this case, newly cre
`ated or modified haptic effects can be transmitted to a device
`that has the capability to play the haptic effects. If end user
`device 10 is connected to another device via a mobile phone
`network, wide area network (WAN), local area network
`(LAN), Bluetooth R) transmission, infrared transmission, etc.,
`haptic files can be transmitted to these devices. In some
`embodiments, for instance, end user device 10 may use a
`streaming data transmission protocol for immediate playback
`on another user's device. When in a streaming mode, end user
`device 10 may stream music, video, and/or haptic data to
`other devices for an indefinite length of time.
`0023. In one embodiment, some elements shown in FIG.1
`can be positioned on different devices. For example, input/
`output devices 12, processor 14, and memory 16 can be used
`to allow a user to create or modify haptic effects. The created
`or modified haptic effects can be provided to or transmitted in
`any suitable manner to another device having haptic actuating
`device 18. In this case, the one device creates or modifies the
`haptic effects while the other device contains the haptic actu
`ating device 18 to be able to play the haptic effects for the user
`of the other device.
`0024. Input/output devices 12 consist of one or more input
`mechanisms, which may include any suitable combination of
`keyboards, keypads, cursor control devices, computer mice,
`Voice activation mechanisms, touch screen with respective
`stylus or other drafting instrument, touch pads, buttons, slid
`ers, microphones, digital cameras, etc. Input/output devices
`12 also consist of one or more output mechanisms, which may
`include any Suitable combination of computer monitors, dis
`play screens (such as touch screens), virtual reality display
`devices, speakers or other audio output devices, printers, etc.
`Some input/output devices 12 may be capable of functioning
`to both receive inputs and provide outputs.
`0025 Processor 14 may be a general-purpose or specific
`purpose processor or microcontroller. Generally, processor
`14 controls the functions of end user device 10 and can
`execute software stored in memory 16. Memory 16 may
`include internally fixed storage and/or removable storage
`media for storing information, data, and/or instructions.
`Memory 16 may include any suitable combination of volatile
`memory, Such as random access memory (RAM), dynamic
`RAM (DRAM), etc., and/or non-volatile memory, such as
`read only memory (ROM), electrically erasable program
`mable ROM (EEPROM), flash memory, etc. Memory 16 can
`also store a Software program for enabling an end user to
`create and/or modify haptic effects, the software program of
`which can be executed by processor 14.
`0026. Haptic actuating device 18 may include any suitable
`combination of electrical or electro-mechanical components
`for providing haptic effects to a user. In some embodiments,
`haptic actuating device 18 is configured as an eccentric rotat
`ing mass (ERM) actuator. Modifying the amplitude values
`
`8
`
`

`

`US 2008/0294984 A1
`
`Nov. 27, 2008
`
`applied to an ERM actuator results in a change of the speed of
`a motor associated with the ERM actuator. In other embodi
`ments, haptic actuating device 18 can be configured as a linear
`resonant actuator (LRA), whereby modifying the amplitude
`values applied to the LRA results in a change of vibration or
`pulse frequency. In still other embodiments, haptic actuating
`device 18 can be configured withina pressure sensing Surface,
`Such as a touch screen device, which is capable of sensing
`pressure applied by the user and also providing haptic feed
`back to the user. The pressure that is sensed by the pressure
`sensing Surface can be translated into data related to ampli
`tude parameters of a haptic effect as it is being modified.
`Regarding embodiments where haptic actuating device 18 is
`an accelerometer, the accelerometer can also be utilized as an
`input device for detecting a shaking motion from a user to
`record a certain vibration entry.
`0027 FIG. 2 is a block diagram of one embodiment of
`memory 16, which is shown in general in FIG.1. Memory 16
`in this implementation is configured to store programs, data,
`information, files, etc., including, for example, an operating
`system (O/S) 22, a graphical user interface (GUI) 24, RAM
`26 or other suitable volatile memory, and ROM 28 or other
`suitable non-volatile memory, such as EEPROM, flash
`memory, etc. Files associated with haptic effects and that are
`stored in memory 16 can include a particular file format that
`is compatible with or can be understood by haptic actuating
`device 18 or other haptic actuating devices.
`0028 ROM 28 may be configured to store a library of
`preset haptic tracks that can be used as a general starting point
`for creating new tracks. In this case, the preset tracks can be
`protected from being overwritten in order to be available
`whenever the user wishes to create a new file. ROM 28 can
`also store haptic track files created or modified by the user and
`can also store mixed track files, which include multiple haptic
`tracks combined together. The user is given the capability to
`assign new filenames to newly created or modified haptic
`track files and mixed track files. ROM 28 can further store
`files that include association information for associating cer
`tain track files with specific events related to end user device
`10.
`0029. As shown in FIG. 2, memory 16 also stores pro
`grammable haptic design software 30, which is configured to
`allow a user to create, modify, design, and customize haptic
`effects as desired. Using programmable haptic design soft
`ware 30, a user can design a track or sequence of haptic
`outputs for playback over any length of time. Using program
`mable haptic design software 30, the user is also able to
`associate certain haptic track files and mixed track files with
`specific events related to end user device 10. For example, a
`haptic track can be programmed to play when one or more
`specific events occur, Such as when an incoming cell phone
`call is received, or even when a call is received from a par
`ticular caller or phone number.
`0030 Certain parameters or characteristics of a haptic out
`put or haptic track can be created, entered, changed, modified,
`etc. For example, the modifiable parameters may include a
`frequency of vibration and amplitude of vibration. Also, the
`frequency and amplitude can be set to change over time. In
`addition, the user is able to combine or mix different haptic
`tracks together if desired to provide a fullerhaptic experience.
`In this way, a user is capable of programming or customizing
`a haptic track or track mix to one's liking.
`0031 While programming a haptic track or track mix, a
`user can enter a command to initiate a test playback of the
`
`haptic track or track mix in order that the user can experience
`upon demand how the creation may actually feel. This can be
`experienced, for example, using haptic actuating device 18
`(FIG. 1) located in end user device 10. The user may also have
`the option to run the haptic track or track mix in the test
`playback in Synchronization with audio and/or video outputs.
`0032. Programmable haptic design software 30 may be
`accessed by the user using any number of possible access
`techniques. In one example, the user may access program
`mable haptic design software 30 via GUI 24. In this case,
`programmable haptic design Software 30 may be opened
`from a menu option such as "Options”, “Tools', or “Tactile
`Sensations” from GUI 24. Other non-limiting techniques for
`accessing programmable haptic Software 30 include a Voice
`responsive system, a text interface, a Braille-type surface
`having either fixed or changeable components.
`0033. When initially accessed, programmable haptic
`design software 30 may present, via GUI 24, options to either
`“Create a New Track” or “Edit an Existing Track”. The term
`“track' used in the present disclosure can refer to parameters
`associated with haptic effects that can be played on a haptic
`enabled device. For instance, the parameters may include
`frequency and amplitude characteristics of a haptic output
`over a certain length of time.
`0034. According to one embodiment, when “Create a New
`Track” is selected, programmable haptic design software 30
`may provide the user with two options to either “Start with
`Track from Library” or "Start from Scratch'. As mentioned
`above, ROM 28 can store the library of tracks that can be used
`as a starting point for certain types of haptic effects. When
`“Start with Track from Library” is selected, a list of track files
`stored in the library are displayed for the user's selection. The
`track files in the library may be given filenames that represent
`understandable characteristics of the files. For the purpose of
`illustration, the files may be given names Such as "Regular.
`“Normal Alarm”, “Gentle”, “Rising Intensity”, “Long and
`Steady”, “Quick Pulse”, “Wake Up!”, “Loud and Obnox
`ious', among others.
`0035. When a new file or existing file is opened, the user
`has the opportunity to modify certain characteristics of the
`haptic track as desired. For example, the user may be able to
`set the amplitude or frequency of vibration of the haptic track.
`It may be desirable for the user to vary the amplitude over
`time. The amplitude can be changed gradually or in steps
`during a portion of or during the entirety of the haptic track.
`0036) Another aspect of the haptic track that can be modi
`fied by the user is the attack level, which corresponds to the
`initial amplitude of the signal. Attack time is another modi
`fiable parameter that corresponds to the time that it takes for
`the signal to change from the attack level to a center magni
`tude, which is another parameter that can be set. The center
`magnitude corresponds to the amplitude of the haptic signal
`during what may be the majority of the time of the haptic
`track. Another parameter is fade level, which corresponds to
`an amplitude at the end of the track. The time that it takes for
`the signal to change from the center magnitude to the fade
`level is referred to as fade time, which can also be modified.
`The user also can enter a certain duration, which is another
`modifiable parameter. In some embodiments, the signal
`remains at the center magnitude for a time equal to the dura
`tion minus the attack time and fade time. In other embodi
`ments, the duration may be associated with the time that the
`signal remains at the center magnitude.
`
`9
`
`

`

`US 2008/0294984 A1
`
`Nov. 27, 2008
`
`0037. The following description includes several embodi
`ments defining ways that the parameters of a haptic track can
`be modified. When the user accesses programmable haptic
`design software 30 via GUI 24 and opens a file to be edited,
`the user can continue to interact with programmable haptic
`design software 30 by input and output mechanisms of GUI
`24.
`0038. In a first example, specific values related to param
`eters of the haptic track can be entered via GUI 24. Regarding
`parameters such as attack level, attack time, magnitude, dura
`tion, fade level, and fade time, the user can enter values in
`units of Hertz, seconds, milliseconds, decibels, etc.. GUI 24
`may present an acceptable range of values for each of the
`parameters. Also, programmable haptic design software 30
`can monitor the entries to check for errors, which may be a
`result of setting values outside the acceptable range, conflict
`ing values among two or more parameters, etc.
`0039. Another example of how to modify haptic tracks is
`by presenting via GUI 24 a range of values that can be
`selected by the user. Alternatively, in place of values, descrip
`tive terms can be used that might be more easily understood
`by a non-skilled user. For example, Some descriptive terms
`may include the categories of "Starting Intensity' (attack
`level), “Intermediate Intensity” (amplitude or magnitude),
`and “Ending Intensity' (fade level). Other descriptive terms
`may include “Duration' (duration), “Start Time' (attack
`time), and "Ending Time' (fade time). Under the categories of
`Starting Intensity and Ending Intensity, descriptive terms
`such as "Strong, “Medium', and “Weak” may be presented
`to the user for selecting the desired attack level. Under the
`category of Intermediate Intensity, descriptive terms such as
`“High”, “Medium', and “Low' may be presented to the user
`for selecting the desired magnitude. Under the Duration cat
`egory, “Long”, “Medium', and “Short may be presented to
`the user. Under the Start Time and Ending Time categories,
`“Rise/Fade Quickly”, “Medium Rise/Fade', and “Rise/Fade
`Slowly” may be presented for selection of the desired rise or
`fade times. As suggested above, additional entry can be made
`to allow the user to alter the amplitude or pulse frequency
`during the haptic track. It should be understood that other
`descriptive terms can be used to describe in layman's terms
`the characteristics of a haptic track. Also, it should be under
`stood that the categories can be divided in more than three
`values or levels of selection.
`0040. In another example of how parameters can be
`entered, GUI 24 can display a graph showing some of the
`characteristics of the amplitude of a haptic output over time.
`GUI 24 may further enable the user to alter the graph as the
`user desires. For example, lines of the graph can be stretched,
`shortened, moved, slid, altered, etc., using various input
`device tools. In this embodiment, the user may use manipu
`lative input devices, such as a touch screen device and stylus.
`0041. Other visual presentations may be provided to the
`user to allow the user to modify parameters. For example, the
`amplitude of a haptic track can be represented by musical
`notes along a musical scale. Thus, musical notes higher on the
`scale can represent higher amplitude or higher pulse fre
`quency haptic output. Other embodiments for representing
`amplitude using musical notation include musical directions
`with respect to loudness. For example, ff (fortissimo) might
`be used for very loud or very high amplitude, f(forte) might
`be used for loud or high amplitude, mf (mezzo forte) for
`moderately loud or moderately high amplitude, m (meZZO)
`for moderate, mp (meZZO piano) for moderately soft or mod
`
`erately low amplitude, p (piano) for Soft or low amplitude, and
`pp (pianissimo) for very soft or very low amplitude. Duration
`parameters in this respect can also be represented by musical
`notation. For instance, musical notes may be shown as eighth
`notes, quarter notes, half notes, etc., to represent the duration
`of a haptic output.
`0042. After creation or modification of a haptic track file
`or mixed track file, the user can store the file using a new
`filename or can overwrite the original file, provided that the
`original file is not protected from overwriting. Particularly, as
`mentioned above, the files in the library can be set so that they
`cannot be overwritten. ROM 28 is configured to store any
`reasonable number of haptic track files and mixed track files
`according to the particular design.
`0043. Various logical instructions or commands may be
`included in programmable haptic design software 30. Pro
`grammable haptic design software 30 of the present disclo
`Sure can be implemented in hardware, software, firmware, or
`a combination thereof. When implemented in software or
`firmware, the software can be stored in memory 16 and
`executed by processor 14. When implemented in hardware,
`programmable haptic design software 30 can be imple
`mented, for example, using discrete logic circuitry, an appli
`cation specific integrated circuit (ASIC), a programmable
`gate array (PGA), a field programmable gate array (FPGA),
`etc., or any combination thereof.
`0044 FIG. 3 is a block diagram of an embodiment of
`programmable haptic design software 30 shown in FIG. 2. In
`this embodiment, programmable haptic design Software 30
`includes a memory interface 32, temporary storage unit 34.
`haptic track editing module 36, track mixing module 38,
`track/event association module 40, and test playback module
`42.
`0045 Memory interface 32 is configured to retrieve a copy
`of a file, either a haptic track file or a mixed track file, from
`ROM 28 (FIG. 2). Memory interface 32 places the copy of the
`retrieved file in temporary storage unit 34, where the file can
`be modified according to user's inputs. In this respect, tem
`porary storage unit 34 can simply contain RAM or other
`Volatile memory while the tracks are being programmed.
`After a user creates or modifies a file or files in temporary
`storage unit 34, memory interface 32 is configured to store a
`copy of the file in ROM 28 as instructed by the user. Also, the
`user may enter a unique filename for the file for later refer
`encing and accessing the file. Furthermore, each file can
`include information that associates the file with one or more
`specific events that may occur during normal use of end user
`device 10.
`0046 While files are opened in the temporary storage unit
`34, the user may use haptic track editing module 36 to edit the
`opened files. In addition, the user may use track mixing mod
`ule 38 to mix a combination of opened haptic tracks together.
`Haptic track editing module 36 allows the user to perform
`different types of editing of the haptic track by changing
`various parameters of the haptic track. The haptic tracks can
`be modified in any Suitable manner, Such as the ways
`described above. For example, haptic track editing module 36
`enables user to modify such features as the rise time, attack
`level, attack time, magnitude, duration, fade time, and other
`characteristics related to the waveform parameters with
`respect to time during which the haptic track is played. In
`addition, haptic track editing module 36 enables the user to
`
`10
`
`

`

`US 2008/0294984 A1
`
`Nov. 27, 2008
`
`enter one or more frequencies for the haptic track or even
`continually changing frequencies to be played along the
`length of the haptic track.
`0047 Temporary storage unit 34 is capable of storing
`more than one haptic track. Additionally, temporary storage
`unit 34 can even store multiple copies of the same haptic
`track, for example, if the user wishes to mix the same haptic
`track into a mixed track, Such as where a second occurrence of
`the haptic track is offset in time from a first occurrence, in
`order to provide a fuller haptic output. Track mixing module
`38 allows the user to combine haptic tracks to play simulta
`