·.: .. ·.·::.· ... ·.··
`
`·.;; ......
`
`0
`0
`"""'"
`
`-LL
`
`mi.0_u_
`
`-LL
`
`<(
`0
`"""'"
`
`-LL
`
`IMMR-ITC00001052
`
`\.....
`\\\“\.“\.$
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0101
`
`

`
`LL
`0
`"""'"
`
`w
`0
`
`"""'" -LL
`
`c
`0
`"""'"
`-LL
`
`IMMR-ITC00001053
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0102
`
`

`
`'\:·:·:·:·:-
`
`*m~~~
`~~~~t~t't~~~
`
`1101
`
`1103
`
`FIG. 11
`
`IMMR-ITC00001054
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0103
`
`

`
`.
`(!) -LL
`
`
`
`
`
`
`
`om?.0_n_mu..0_u_<N...9".
`
`.
`(!) -LL
`
`.
`(!) -LL
`
`IMMR-ITC00001055
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0104
`
`

`
`FIG.12E
`.
`(!) -L1.
`
`D(
`
`‘I
`1-
`
`.
`
`IMMR-ITC00001056
`
`(!)
`L1.
`L
`
`9L
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0105
`
`

`
`( START
`+
`
`/130 1
`
`RECEIVE A DEVICE SENSOR SIGNAL AT
`TIME T1
`
`+
`
`/130 3
`
`RECEIVE A GESTURE SIGNAL AT TIME T2
`
`+
`
`(130 5
`COMPARE THE DEVICE SENSOR SIGNAL
`WITH A HAPTIC EFFECT SIGNAL TO
`GENERATE A DEVICE SENSOR SIGNAL
`DIFFERENCE VECTOR
`
`+
`
`(130 7
`
`COMPARE THE GESTURE SIGNAL WITH A
`HAPTIC EFFECT SIGNAL TO GENERATE A
`GESTURE DIFFERENCE VECTOR
`
`+
`
`/130 9
`
`OPTIONALLY RECEIVE AN ANIMATION OR
`PHYSICAL MODEL DESCRIPTION
`
`+
`
`(
`
`131 1
`
`GENERATE AN INTERACTION
`PARAMETER USING THE GESTURE
`DIFFERENCE VECTOR, THE SIGNAL
`DIFFERENCE VECTOR AND ANIMATION
`OR PHYSICAL MODEL DESCRIPTION
`
`[131 3
`APPLY A DRIVE SIGNAL TO A HAPTIC
`OUTPUT DEVICE ACCORDING TO THE
`INTERACTION PARAMETER
`
`+
`
`+
`( END
`
`)
`
`FIG. 13
`
`IMMR-ITC00001057
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0106
`
`

`
`START
`
`+
`
`/140 1
`ENABLE A COMMUNICATION LINK
`BETWEEN FIRST AND SECOND DEVICES
`HAVING FIRST AND SECOND HAPTIC
`OUTPUT DEVICES
`
`+
`
`/140 3
`RECEIVE A FIRST GESTURE SIGNAL OR
`DEVICE SENSOR SIGNAL FROM THE
`FIRST DEVICE AND COMMUNICATE IT TO
`THE SECOND DEVICE
`
`+
`
`/140 5
`OPTIONALLY RECEIVE A SECOND
`GESTURE SIGNAL OR DEVICE SENSOR
`SIGNAL FROM THE SECOND DEVICE AND
`COMMUNICATE IT TO THE FIRST DEVICE
`
`+
`
`;-140 7
`
`GENERATE AN INTERACTION
`PARAMETER USING THE FIRST GESTURE
`OR SENSOR SIGNAL AND THE OPTIONAL
`SECOND GESTURE OR SENSOR SIGNAL
`
`+
`
`/140 9
`CONCURRENTLY APPLY A DRIVE SIGNAL
`TO THE FIRST AND SECOND HAPTIC
`OUTPUT DEVICES ACCORDING TO THE
`INTERACTION PARAMETER
`
`+
`( END )
`
`FIG. 14
`
`IMMR-ITC00001058
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0107
`
`

`
`( START
`+
`
`RECEIVE A GESTURE SIGNAL OR DEVICE
`SENSOR SIGNAL FROM A Fl RST DEVICE
`
`f150 1
`
`f150 3
`STORE OR ENCODE THE GESTURE OR
`SENSOR SIGNAL IN A DATA FILE ON THE
`FIRST DEVICE
`
`+
`
`+
`
`/150 5
`COMMUNICATE THE DATA FILE TO A
`SECOND DEVICE HAVING A HAPTIC
`OUTPUT DEVICE
`
`/150 7
`READ THE GESTURE OR SENSOR SIGNAL
`FROM THE DATA FILE ON THE SECOND
`DEVICE
`
`+
`
`+
`
`/150 9
`APPLY A DRIVE SIGNAL TO THE HAPTIC
`OUTPUT DEVICE ACCORDING TO THE
`SIGNAL
`
`+
`
`END
`
`FIG. 15
`
`IMMR-ITC00001059
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0108
`
`

`
`Docket No: IMM390C3
`
`UNITED STATES PATENT APPLICATION
`
`FOR
`
`INTERACTIVITY MODEL FOR SHARED FEEDBACK ON MOBILE
`DEVICES
`
`INVENTOR:
`
`DAVID BIRNBAUM
`CHRIS ULLRICH
`JASON SHORT
`RYAN DEVENISH
`
`PREPARED BY:
`
`IMMERSION CORPORATION
`
`ASSIGNEE: IMMERSION CORPORATION
`30 RIO ROBLES
`SAN JOSE, CA 95134
`
`IMMR-ITC00001060
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0109
`
`

`
`Patent
`Docket No. IMM390C3
`
`INTERACTIVITY MODEL FOR SHARED FEEDBACK ON MOBILE DEVICES
`
`CROSS-REFERENCE TO RELATED APPLICATIONS
`
`[0001]
`
`This application claims the benefit of priority under 35 USC § 120 to
`
`copending application 13/592,685, filed August 23, 2012, which claims the benefit of
`
`priority to application 13/472,709, filed May 16, 2012, which claims the benefit of
`
`priority to application 13/397, 142, filed February 1 5, 2012.
`
`FIELD OF THE INVENTION
`
`[0002]
`
`One embodiment is directed generally to a user interface for a device,
`
`and in particular to producing a dynamic haptic effect using multiple gesture signals
`
`and real or virtual device sensor signals.
`
`BACKGROUND INFORMATION
`
`[0003]
`
`Electronic device manufacturers strive to produce a rich interface for
`
`users. Conventional devices use visual and auditory cues to provide feedback to a
`
`user. In some interface devices, kinesthetic feedback (such as active and resistive
`
`force feedback) and/or tactile feedback (such as vibration, texture, and heat) is also
`
`provided to the user, more generally known collectively as "haptic feedback" or
`
`"haptic effects". Haptic feedback can provide cues that enhance and simplify the
`
`user interface. Specifically, vibration effects, or vibrotactile haptic effects, may be
`
`useful in providing cues to users of electronic devices to alert the user to specific
`
`events, or provide realistic feedback to create greater sensory immersion within a
`
`simulated or virtual environment.
`
`[0004]
`
`In order to generate vibration effects, many devices utilize some type of
`
`actuator or haptic output device. Known haptic output devices used for this purpose
`
`1
`
`IMMR-ITC00001061
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0110
`
`

`
`Patent
`Docket No. IMM390C3
`
`include an electromagnetic actuator such as an Eccentric Rotating Mass ("ERM") in
`
`which an eccentric mass is moved by a motor, a Linear Resonant Actuator ("LRA") in
`
`which a mass attached to a spring is driven back and forth, or a "smart material"
`
`such as piezoelectric, electro-active polymers or shape memory alloys. Haptic output
`
`devices also broadly include non-mechanical or non-vibratory devices such as those
`
`that use electrostatic friction (ESF), ultrasonic surface friction (USF), or those that
`
`induce acoustic radiation pressure with an ultrasonic haptic transducer, or thlose that
`
`use a haptic substrate and a flexible or deformable surface, or those that provide
`
`projected haptic output such as a puff of air using an air jet, and so on.
`
`[0005]
`
`Traditional architectures that provide haptic
`
`feedback only with
`
`triggered effects are available, and must be carefully designed to make sure the
`
`timing of the haptic feedback is correlated to user initiated gestures or system
`
`animations. However, because these user gestures and system animations have
`
`variable timing, the correlation to haptic feedback may be static and inconsistent and
`
`therefore less compelling to the user. Further, device sensor information is typically
`
`not used in combination with gestures to produce haptic feedback.
`
`[0006]
`
`Therefore, there is a need for an improved system of providing a
`
`dynamic haptic effect that includes multiple gesture signals and device sensor
`
`si<Qnals. There 'is a further need for providing concurrent haptic feedback to multiple
`
`devices which are connected via a communication link.
`
`2
`
`IMMR-ITC00001062
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0111
`
`

`
`Patent
`Docket No. IMM390C3
`
`SUMMARY OF THE INVENTION
`
`[0007]
`
`One embodiment is a system that produces a dynamic haptic effect
`
`and generates a drive signal that includes a gesture signal and a real or virtual
`
`device sensor signal. The haptic effect is modified dynamically based on both the
`
`gesture signal and the real or virtual device sensor signal such as from an
`
`accelerometer or gyroscope, or by a signal created from processing data such as still
`
`images, video or sound. The haptic effect may optionally be modified dynamically by
`
`using the gesture signal and the real or virtual device sensor signal and a physical
`
`model. The haptic effect may optionally be applied concurrently to multiple devices
`
`which are connected via a communication link. The haptic effect may optionally be
`
`encoded into a data file on a first device. The data file is then communicated to a
`
`second device and the haptic effect is read from the data file and applied to the
`
`second device.
`
`3
`
`IMMR-ITC00001063
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0112
`
`

`
`Patent
`Docket No. IMM390C3
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0008]
`
`FIG. 1 is a block diagram of a haptically-enabled system according to
`
`one embodiment of the present invention.
`
`[0009]
`
`FIG. 2 is a cut-away perspective view of an LRA implementation of a
`
`haptic actuator according to one embodiment of the present invention.
`
`[00010]
`
`FIG. 3 is a cut-away perspective view of an ERM implementation of a
`
`haptic actuator according to one embodiment of the present invention.
`
`[00011]
`
`FIGs. 4A-4C are views of a piezoelectric implementation of a haptic
`
`actuator according to one embodiment of the present invention.
`
`[00012]
`
`FIG. 5 is a view of a haptic device using electrostatic friction (ESF)
`
`according to one embodiment of the present invention.
`
`[00013]
`
`FIG. 6 is a view of a haptic device for inducing acoustic radiation
`
`pressure with an ultrasonic haptic transducer according to one embodiment of the
`
`present invention.
`
`[00014]
`
`FIG. 7 is a view of a haptic device using a haptic substrate and flexible
`
`or deformable surface according to one embodiment of the present invention .
`
`[00015]
`
`FIG. 8 is a view of a haptic device using ultrasonic surface friction (USF)
`
`according to one embodiment of the present invention.
`
`[00016]
`
`FIGs. 9A-9C are screen views of a user initiated dynamic haptic effect
`
`according to one embodiment of the present invention.
`
`[00017]
`
`FIGs. 1 OA-1 OF are screen views of encoding a haptic effect into a data
`
`file according to one embodiment of the present invention.
`
`[00018]
`
`FIG. 11
`
`is a screen view of a user initiated dynamic haptic effect
`
`according to one embodiment of the present invention.
`
`4
`
`IMMR-ITC00001064
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0113
`
`

`
`Patent
`Docket No. IMM390C3
`
`[00019]
`
`FIGs. 12A-12E are screen views of applying a haptic effect
`
`concurrently to multiple devices according to one embodiment of the present
`
`invention.
`
`[00020]
`
`FIG. 13 is a flow diagram for producing a dynamic haptic effect with a
`
`gesture signal and a device sensor signal according to one embodiment of the
`
`present invention.
`
`[00021]
`
`FIG. 14 is a flow diagram for concurrently applying a haptic effect to
`
`multiple devices according to one embodiment of the present invention.
`
`[00022]
`
`FIG. 15 is a flow diagram for encoding and applying a haptic effect
`
`using a data file according to one embodiment of the present invention.
`
`5
`
`IMMR-ITC00001065
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0114
`
`

`
`Patent
`Docket No. IMM390C3
`
`DETAILED DESCRIPTION
`
`[00023]
`
`As described below, a dynamic haptic effect refers to a haptic effect
`
`that evolves over time as it responds to one or more input parameters. Dynamic
`
`haptic effects are haptic or vibrotactile effects displayed on haptic devices to
`
`represent a change in state of a given input signal. The input signal can be a signal
`
`captured by sensors on the device with haptic feedback, such as position,
`
`acceleration, pressure, orientation, or proximity, or signals captured by other devices
`
`and sent to the haptic device to influence the generation of the haptic effect.
`
`[00024]
`
`A dynamic effect signal can be any type of signal, but does not
`
`necessarily have to be complex. For example, a dynamic effect signal may be a
`
`simple sine wave that has some property such as phase, frequency, or amplitude
`
`that is changing over time or reacting in real time according to a mapping schema
`
`which maps an input parameter onto a changing property of the effect signal. An
`
`input parameter may be any type of input capable of being provided by a device, and
`
`typically may be any type of signal such as a device sensor signal. A device sensor
`
`si,gnal may be generated by any means, and typically may be generated by capturing
`
`a user gesture with a device. Dynamic effects may be very useful for gesture
`
`interfaces, but the use of gestures or sensors are not necessarily required to create
`
`a dynamic signal.
`
`[00025]
`
`One common scenario that does not involve gestures directly is
`
`defining the dynamic haptic behavior of an animated widget. For example, when a
`
`user scrolls a list, it is not typically the haptification of the gesture that will feel most
`
`intuitive, but instead the motion of the widget in response to the gesture. In the scroll
`
`list example, gently sliding the list may generate a dynamic haptic feedback that
`
`changes according to the speed of the scrolling, but flinging the scroll bar may
`
`6
`
`IMMR-ITC00001066
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0115
`
`

`
`Patent
`Docket No. IMM390C3
`
`produce dynamic haptics even after the gesture has ended. This creates the illusion
`
`that the widget has some physical properties and it provides the user with
`
`information about the state of the widget such as its velocity or whether it is in motion.
`
`[00026]
`
`A gesture is any movement of the body that conveys meaning or user
`
`intent. It will be recognized that simple gestures may be combined to form more
`
`complex gestures. For example, bringing a finger into contact with a touch sensitive
`
`surface may be referred to as a "finger on" gesture, while removing a finger from a
`
`touch sensitive surface may be referred to as a separate "finger off" gesture. If the
`
`time between the "finger on" and "finger off" gestures is relatively short, the
`
`combined gesture may be referred to as "tapping"; if the time between the "finger on"
`
`and "finger off" gestures is relatively long, the combined gesture may be referred to
`
`as "long tapping"; if the distance between the two dimensional (x,y) positions of the
`
`"finger on" and "finger off" gestures is relatively large, the combined gesture may be
`
`referred to as "swiping"; if the distance between the two dimensional {x,y) positions
`
`of the "finger on" and "finger off" gestures is relatively small, the combined gesture
`
`may be referred to as "smearing", "smudging" or "flicking". Any number of two
`
`dimensional or three dimensional simple or complex gestures may be combined in
`
`any manner to form any number of other gestures, including, but not limited to,
`
`multiple finger contacts, palm or fist contact, or proximity to the device. A gesture can
`
`also be any form of hand movement recognized by a device having an
`
`accelerometer, gyroscope, or other motion sensor, and converted to electronic
`
`si·gnals. Such electronic signals can activate a dynamic effect, such as shaking
`
`virtual dice, where the sensor captures the user intent that generates a dynamic
`
`effect.
`
`7
`
`IMMR-ITC00001067
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0116
`
`

`
`Patent
`Docket No. IMM390C3
`
`[00027]
`
`FIG. 1 is a block diagram of a haptically-enabled system 10 according
`
`to one embodiment of the present invention. System 1 0 includes a touch sensitive
`
`surface 11 or other type of user interface mounted within a housing 15, and may
`
`include mechanical keys/buttons 13. Internal to system 10 is a haptic feedback
`
`system that generates vibrations on system 10. In one embodiment, the vibrations
`
`are generated on touch surface 11 .
`
`[00028]
`
`The haptic feedback system includes a processor 12. Coupled to
`
`processor 12 is a memory 20 and an actuator drive circuit 16, which is coupled to a
`
`haptic actuator 18. Processor 12 may be any type of general purpose processor, or
`
`could be a processor specifically designed to provide haptic effects, such as an
`
`application-specific integrated circuit ("ASIC"). Processor 12 may be the same
`
`processor that operates the entire system 1 o, or may be a separate processor.
`
`Processor 12 can decide what haptic effects are to be played and the order in which
`
`the effects are played based on high level parameters. In general, the high level
`
`parameters that define a particular haptic effect include magnitude, frequency and
`
`duration. Low level parameters such as streaming motor commands could also be
`
`used to determine a particular haptic effect. A haptic effect may be considered
`
`dynamic if it includes some variation of these parameters when the haptic effect is
`
`generated or a variation of these parameters based on a user's interaction.
`
`[00029]
`
`Processor 12 outputs the control signals to drive circuit 16 which
`
`includes electronic components and circuitry used to supply actuator 18 with the
`
`required electrical current and voltage to cause the desired haptic effects. System 1 0
`
`may include more than one actuator 18, and each actuator may include a separate
`
`drive circuit 16, all coupled to a common processor 12. Memory device 20 can be
`
`any type of storage device or computer-readable medium, such as random access
`
`8
`
`IMMR-ITC00001068
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0117
`
`

`
`Patent
`Docket No. IMM390C3
`
`memory (RAM) or read-only memory (ROM). Memory 20 stores instructions
`
`executed by processor 12. Among the instructions, memory 20 includes an actuator
`
`drive module 22 which are instructions that, when executed by processor 12,
`
`generate drive signals for actuator 18 while also determining feedback from actuator
`
`18 and adjusting the drive signals accordingly. The functionality of module 22 is
`
`discussed in more detail below. Memory 20 may also be located internal to
`
`processor 12, or any combination of internal and external memory.
`
`[00030]
`
`Touch surface 11 recognizes touches, and may also recognize the
`
`position and magnitude or pressure of touches on the surface. The data
`
`corresponding to the touches is sent to processor 12, or another processor within
`
`system 10, and processor 12 interprets the touches and in response generates
`
`haptic effect signals. Touch surface 11 may sense touches using any sensing
`
`technology, including capacitive sensing, resistive sensing, surface acoustic wave
`
`sensing, pressure sensing, optical sensing, etc. Touch surface 11 may sense multi-
`
`touch contacts and may be capable of distinguishing multiple touches that occur at
`
`the same time. Touch surface 11 may be a touchscreen that generates and displays
`
`images for the user to interact with, such as keys, dials, etc., or may be a touchpad
`
`with minimal or no images.
`
`[00031]
`
`System 10 may be a handheld device, such as a cellular telephone,
`
`PDA, computer tablet, gaming console, etc. or may be any other type of device that
`
`provides a user interface and includes a haptic effect system that includes one or
`
`more ERMs, LRAs, electrostatic or other types of actuators. The user interface may
`
`be a touch sensitive surface, or can be any other type of user interface such as a
`
`mouse, touchpad, mini-joystick, scroll wheel,
`
`trackball, game pads or game
`
`controllers, etc. In embodiments with more than one actuator, each actuator may
`
`9
`
`IMMR-ITC00001069
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0118
`
`

`
`Patent
`Docket No. IMM390C3
`
`have a different output capability in order to create a wide range of haptic effects on
`
`the device. Each actuator may be any type of haptic actuator or a single or
`
`multidimensional array of actuators.
`
`[00032]
`
`FIG. 2 is a cut-away side view of an LRA implementation of actuator 18
`
`in accordance to one embodiment. LRA 18 includes a casing 25, a magnet/mass 27,
`
`a linear spring 26, and an electric coil 28. Magnet 27 is mounted to casing 25 by
`
`spring 26. Coil 28 is mounted directly on the bottom of casing 25 underneath magnet
`
`27. LRA 18 is typical of any known LRA. In operation, when current flows through
`
`coil 28 a magnetic field forms around coil 28 which in interaction with the magnetic
`
`field of magnet 27 pushes or pulls on magnet 27. One current flow direction/polarity
`
`causes a push action and the other a pull action. Spring 26 controls the up and down
`
`movement of magnet 27 and has a deflected up position where it is compressed, a
`
`deflected down position where it is expanded, and a neutral or zero-crossing position
`
`where it is neither compressed or deflected and which is equal to its restiing state
`
`when no current is being applied to coil 28 and there is no movement/oscillation of
`
`magnet 27.
`
`[00033]
`
`For LRA 18, a mechanical quality factor or "0 factor" can be measured.
`
`In general, the mechanical Q factor is a dimensionless parameter that compares a
`
`time constant for decay of an oscillating physical system's amplitude to its oscillation
`
`period. The mechanical Q factor is significantly affected by mounting variations. The
`
`mechanical Q factor represents the ratio of the energy circulated between the mass
`
`and spring over the energy lost at every oscillation cycle. A low Q factor means that
`
`a large portion of the energy stored in the mass and spring is lost at every cycle. In
`
`general, a mini mum 0 factor occurs with system 10 is held firmly in a hand due to
`
`energy being absorbed by the tissues of the hand. The maximum Q factor generally
`
`10
`
`IMMR-ITC00001070
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0119
`
`

`
`Patent
`Docket No. IMM390C3
`
`occurs when system 1 0 is pressed against a hard and heavy surface that reflects all
`
`of the vibration energy back into LRA 18.
`
`[00034]
`
`In direct proportionality to the mechanical Q factor, the forces that
`
`occur between magnet/mass 27 and spring 26 at resonance are typically 1 0-1 00
`
`times larger than the force that coil 28 must produce to maintain the oscillation.
`
`Consequently, the resonant frequency of LRA 18 is mostly defined by the mass of
`
`magnet 27 and the compliance of spring 26. However, when an LRA is mou111ted to a
`
`floating device (i.e., system 10 held softly in a hand), the LR.A resonant frequency
`
`shifts up significantly. Further, significant frequency shifts can occur due to external
`
`factors affecting the apparent mounting weight of LRA 18 in system 10, such as a
`
`cell phone flipped open/closed or the phone held tightly.
`
`[00035]
`
`FIG. 3 is a cut-away perspective view of an ERM implementation of
`
`actuator 18 according to one embodiment of the present invention. ERM 18 includes
`
`a rotating mass 301 having an off-center weight 303 that rotates about an axis of
`
`rotation 305. In operation, any type of motor may be coupled to ERM 18 to cause
`
`rotation in one or both directions around the axis of rotation 305 in response to the
`
`amount and polarity of voltage applied to the motor. It will be recognized that an
`
`application of voltage in the same direction of rotation will have an acceleration effect
`
`and cause the ERM 18 to increase its rotational speed, and that an application of
`
`voltage in the opposite direction of rotation will have a braking effect and cause the
`
`E RM 18 to decmase or even reverse its rotational speed.
`
`[00036]
`
`One embodiment of the present invention provides haptic feedback by
`
`determining and modifying the angular speed of ERM 18. Angular speed is a scalar
`
`measure of rotation rate, and represents the magnitude of the vector quantity
`
`angular velocity. Angular speed or frequency w, in radians per second, correlates to
`
`11
`
`IMMR-ITC00001071
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0120
`
`

`
`Patent
`Docket No. IMM390C3
`
`frequency v in cycles per second, also called Hz, by a factor of 2rr. The drive signal
`
`includes a drive period where at least one drive pulse is applied to ERM 1.8, and a
`
`monitoring period where the back electromagnetic field ("EMF") of the rotating mass
`
`301 is received and used to determine the angular speed of ERM 18. In another
`
`embodiment, the drive period and the monitoring period are concurrent and the
`
`present invention dynamically determines the angular speed of ERM 18 during both
`
`the drive and monitoring periods.
`
`[00037]
`
`FIGs. 4A-4C are views of a piezoelectric implementation of a haptic
`
`actuator 18 according to one embodiment of the present invention. FIG. 4A shows a
`
`disk piezoelectric actuator that includes an electrode 401 , a piezo ceramics disk 403
`
`and a metal disk 405. As shown in FIG 48, when a voltage is applied to electrode
`
`401 , the piezoelectric actuator bends in response, going from a relaxed state 407 to
`
`a transformed state 409. When a voltage is applied, it is that bending of the actuator
`
`that creates the foundation of vibration. Alternatively, FIG. 4C shows a beam
`
`piezoelectric actuator that operates similarly to a disk piezoelectric actuator by going
`
`from a relaxed state 411 to a transformed state 413.
`
`[00038]
`
`FIG. 5 is a view of a haptic device using electrostatic friction (ESF)
`
`according to one embodiment of the present invention. Similar to the operational
`
`principles described by Makinen et al. in U.S. Pat. No. 7,982,588, the embodiment is
`
`based on the discovery that subcutaneous Pacinian corpuscles can be stimulated by
`
`means of a capacitive electrical coupling and an appropriately dimensioned control
`
`voltage, either without any mechanical stimulation of the Pacinian corpuscles or as
`
`an additional stimulation separate
`
`from such mechanical stimulation. An
`
`appropriately dimensioned high voltage is used as the control voltage. In the present
`
`context, a high voltage means such a voltage that direct galvanic contact must be
`
`12
`
`IMMR-ITC00001072
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0121
`
`

`
`Patent
`Docket No. IMM390C3
`
`prevented for rreasons of safety and/or user comfort. This results in a capacitive
`
`coupling between the Pacinian corpuscles and the apparatus causing the stimulation,
`
`wherein one side of the capacitive coupling is formed by at least one galvanically
`
`isolated electrode connected to the stimulating apparatus, while the other side, in
`
`close proximity to the electrode, is formed by the body member, preferably a finger,
`
`of the stimulation target, such as the user of the apparatus, and more specifically the
`
`subcutaneous Pacinian corpuscles.
`
`[00039]
`
`It likely that the invention is based on a controlled formation of an
`
`electric field between an active surface of the apparatus and the body member, such
`
`as a finger, approaching or touching it. The electric field tends to give rise to an
`
`opposite charg1e on the proximate finger. A local electric field and a capacitive
`
`coupling can be formed between the charges. The electric field directs a force on the
`
`charge of the finger tissue. By appropriately altering the electric field a force capable
`
`of moving the tissue may arise, whereby the sensory receptors sense such
`
`movement as vibration.
`
`[00040]
`
`As shown in FIG. 5, one or more conducting electrodes 501 are
`
`provided with an insulator. When a body member such as finger 505 is proximate to
`
`the conducting electrode 501 , the insulator prevents flow of direct current from the
`
`conducting electrode to the body member 505. A capacitive coupling field force 503
`
`over the insulator is formed between the conducting electrode 501 and the body
`
`member 505. The apparatus also comprises a high-voltage source for applying an
`
`electrical input to the one or more conducting electrodes, wherein the electrical input
`
`comprises a low-frequency component in a frequency range between 10 Hz and
`
`1 000 Hz. The capacitive coupling and electrical input are dimensioned to produce an
`
`13
`
`IMMR-ITC00001073
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0122
`
`

`
`Patent
`Docket No. IMM390C3
`
`electrosensory sensation which is produced independently of any mechanical
`
`vibration of the one or more conducting electrodes or insulators.
`
`[00041]
`
`FIG. 6 is a view of a haptic device for inducing acoustic radiation
`
`pressure with an ultrasonic haptic transducer similar to that described by Iwamoto et
`
`al., "Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound",
`
`Eurohaptics 2008, LNCS 5024, pp. 504-513. An airborne ultrasound transducer
`
`array 601 is designed to provide tactile feedback in three-dimensional (30) free
`
`space. The array radiates airborne ultrasound, and produces high-fidelity pressure
`
`fields onto the user's hands without the use of gloves or mechanical attachments.
`
`The method is based on a nonlinear phenomenon of ultrasound; acoustic radiation
`
`pressure. When an object interrupts the propagation of ultrasound, a pressure field is
`
`exerted on the surface of the object. This pressure is called acoustic radiatiol'l
`
`pressure. The acoustic radiation pressure P [Pa] is simply described as P = aE,
`
`where E [J=m3
`
`] is the energy density of the ultrasound and a is a constant ranging
`
`from 1 to 2 depending on the reflection properties of the surface of the object. The
`
`equation describes how the acoustic radiation pressure is proportional to the energy
`
`density of the ultrasound. The spatial distribution of the energy density of the
`
`ultrasound can be controlled by using the wave field synthesis techniques. With an
`
`ultrasound transducer array, various patterns of pressure field are produced in 30
`
`free space. Unlike air-jets, the spatial and temporal resolutions are quite fine. The
`
`spatial resolution is comparable to the wavelength of the ultrasound. The frequency
`
`characteristics are sufficiently fine up to 1 kHz.
`
`[00042]
`
`The airborne ultrasound can be applied directly onto the skin without
`
`the risk of the penetration. When the airborne ultrasound is applied on the surface of
`
`the skin, due to the large difference between the characteristic acoustic impedance
`
`14
`
`IMMR-ITC00001074
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0123
`
`

`
`Patent
`Docket No. IMM390C3
`
`of the air and that of the skin, about 99.9% of the incident acoustic energy is
`
`reflected on the surface of the skin. Hence, this tactile feedback system does not
`
`require the users to wear any clumsy gloves or mechanical attachments.
`
`[00043]
`
`FIG. 7 shows a three-dimensional (3D) diagram illustrating a haptic
`
`device 701 using a haptic substrate and a flexible surface in accordance with one
`
`embodiment of the present invention. Device 701 includes a flexible surface layer
`
`703, a haptic substrate 705, and a deforming mechanism 711. It should be noted
`
`that device 701 can be a user interface device, such as an interface for a cellular
`
`phone, a personal digital assistant ("PDA"), an automotive data input system, and so
`
`forth. It should be further noted that the underlying concept of the exemplary
`
`embodiment of the present invention would not change if one or more blocks (circuits
`
`or layers) were added to or removed from device 701.
`
`[00044]
`
`Fllexible surface layer 703, in one instance, is made of soft and/or
`
`elastic materialls such as silicone rubber, which is also known as polysiloxane. A
`
`function of the flexible surface layer 703 is to change its surface shape or texture
`
`upon contact with the physical pattern of haptic substrate 705. The physical pattern
`
`of haptic substrate 705 is variable as one or more of the local features 11 0-124 can
`
`be raised or lowered to present features to affect the surface of the flexible surface
`
`layer 703 upon contact. Once the physical pattern of haptic substrate 705 is
`
`determined, the texture of flexible surface layer 703 can change to confirm its
`
`surface texture to the physical pattern of haptic substrate 705. It should be note that
`
`the deformation of flexible surface layer 703 from one texture to another can be
`
`controlled by deforming mechanism 711. For example, when deforming mechanism
`
`711 is not activated, flexible surface layer 703 maintains its smooth configuration
`
`floating or sitting over haptic substrate 705. The surface configuration of flexible
`
`15
`
`IMMR-ITC00001075
`
`APPLE INC.
`EXHIBIT 1003 - PAGE 0124
`
`

`
`Patent
`Docket No. IMM390C3
`
`surface layer 703, however, deforms or changes from a smooth configuration to a
`
`coarse configuration when deforming mechanism 711 is activated and the haptic
`
`substrate 705 iis in contact with the flexible surface layer 703 so as to generate a
`
`similar pattern on the top surface of the flexible surface layer 703.
`
`[00045]
`
`Alternatively, flexible surface layer 703 is a flexible touch sensitive
`
`surface, which is capable of accepting user inputs. The flexible touch sensitive
`
`surface can be divided into multiple regions wherein each region of the flexible touch
`
`sensitive surface can accept an input when the region is being touched or depressed
`
`by a finger. In one embodiment, the flexible touch sensitive surface includes a
`
`sensor, which is capable of detecting a nearby finger and waking up or turning on the
`
`device. Flexible surface layer 703 may also include a flexible display, which is
`
`capable of deforming together with flexible surface layer 703. It should be noted that
`
`various flexible display technologies can be used to manufacture flexible displays,
`
`such as organic light-emitting diode (OLEO), organic, or polymer TFT (Thin Film
`
`Transistor).
`
`[00046]
`
`Haptic substrate 705 is a surface reconfigurable haptic device capable
`
`of changing its surface pattern in response to one or more pattern activatingJ signals.
`
`Haptic substrate 705 can also be referred to as a haptic mechanism, a haptic layer, a
`
`tactile element, and the like. Haptic substrate 705, in one embodiment, includes
`
`multiple tactile or haptic regions 707, 709, wherein each region can be indep