*— u.
`A D
`
`2'" ,__.
`‘,5 ___%‘
`
`PROVYSIONAL APPLICA TION C0 VER SHEET
`
`A
`3 -——.___ "3
`'
`|—_\ -—.— ‘
` ,g)y certify that this paper and the documents and/or fees referred to
`mttgched therein are being deposited with the United States Postal
`mike on November 1, 2001 in an envelope as “Express Mail Post
`Office to Addressee” service under 37 CFR § .10, Express Ma1lNo.
`EF266510225US, addressed to the Commissioner for Patents,
`Washington, DC 2
`31.
`
` James R. Rica
`
`Commissioner for Patents
`
`Box Provisional Patent Application
`Washington, DC 20231
`
`Attorney Docket No,;
`-
`_
`-
`Flrst Named Inventor‘ Kenneth M‘ Martln
`
`0 m
`2
`S: E‘
`3") %‘
`'
`,1‘;-9
`-—L'—g
`DS2 _-“§_"i—.;
`to g--‘——‘‘:‘‘53
`3% Ԥ,}
`«-4 E”
`"H E
`
`El Duplicate for fee processing
`
`Sir:
`
`This is a request for filing a PROVISIONAL APPLICATION under 37 CFR 1.53(c).
`
`LAST NAME
`
`FIRST NAME
`
`INVENTOR S /APPLICANT(S)
`
`RESIDENCE (CITY AND
`EITHER STATE OR FOREIGN
`COUNTRY)
`
`
`
`.
`
`Los Gatos, CA
`
`Redwood City, CA
`
`CUSTOMIZED TACTILE FEEDBACK IN MULTIPLE BUTTONS WITH A SINGLE VIBROTACTILE ACTUATOR
`
`TITLE OF INVENTION (280 characters max)
`
`(408) 467-1900
`
`CORRESPONDENCE ADDRESS
`
`HVIMERSION CORPORATION
`801 Fox Lane
`
`San Jose, CA 95131
`
`ENCLOSED APPLICATION PARTS check all that a l )
`
`Small Entity Statement XI
`
`Specification, Number of Pages
`
`K1
`
`Drawings Number of Pages
`
`8
`
`1
`
`1:]
`
`X]
`
`Other (specify): Cover Sheet
`
`E
`
`El
`
`A check is enclosed to cover the Provisional Filing Fee Amount of ($)l60.0()
`
`The commissioner is hereby authorized to charge any additional fees that may be required or credit any overpayment to
`Deposit Account No. 50-1815 (Order No. lMM147P).
`
`At least some of the inventions were made under a contract with an agency of the United States Government.
`
`[ENG
`
`[:]Yes, the name of the U.S. Government agency and the contract number are:
`
`Respectfully Submitted,
`
`SIGNATURE
`
`TYPED NAME
`
`Ja es R. Riegel
`
`“
`
`DATE 11/1/01
`
`REGISTRATION NO. 36,651
`
`PR 0 VISIONAL APPLICA TION FILING ONL Y
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 1
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 1
`
`

`

`CUSTOMIZED TACTILE FEEDBACK IN MULTIPLE BUTTONS
`WITH A SINGLE VIBROTACTILE ACTUATOR
`
`BY INVENTORS
`Kenneth M. Martin
`Steven P. Vassallo
`
`
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to computer interface devices that
`
`allow the user to experience haptic feedback.
`
`A user can interact with an environment displayed by a computer to perform
`
`functions and tasks on the computer, such as playing a game, experiencing a
`
`simulation or virtual reality environment, using a computer aided design system,
`
`operating a graphical user interface (GUI), etc. Common human—computer interface
`
`devices used for such interaction include a mouse, joystick, trackball, steering wheel,
`
`stylus, tablet, pressure-sensitive sphere, or the like, that is connected to the computer
`
`system controlling the displayed environment.
`
`Cellular phones, personal digital assistants (PDA’s), and other electronic
`
`devices are commonly used by a large number of people. The physical feedback
`
`provided in standard passive buttons on ce11—phones and PDA’s of the prior art is
`
`limited to the mechanical feedback of the switches, e.g., the switch closure force-
`
`displacement profile. As the same mechanical switch is usually used for each button,
`
`the buttons all feel the same when they are pressed. In addition, the physical feedback
`
`that the buttons provide is delivered only in the process of pressing the button. Some
`
`cellphones and PDA’s have raised bumps on the center key to help orient the user as
`
`to the center of the pattern, and some buttons are arranged in unique or characteristic
`
`ways to allow the user to detennine which button is which by feel without having to
`
`look at the buttons.
`
`In general, however, users typically must look at the dialpad on
`
`their cellphone or PDA to ensure that
`characters.
`
`they are entering the right numbers or
`
`In summary, passive buttons have a single characteristic feel generated by the
`
`mechanical design and do not have the ability to provide feedback to the user
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 2
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 2
`
`

`

`regarding displacement or position. This is similar for buttons or keypads provided
`on other electronic devices.
`
`In other embodiments, a flat touchpad used on a cell—phone or other electronic
`
`device for sensing a user’s touch can be integrated with an LCD or other flat panel
`
`display screen.
`
`In some embodiments, “soft” or graphical buttons (“softkeys”) are
`
`displayed on the screen and are receptive to the user’s touch to allow the command of
`
`device functions similar to normal mechanical buttons. However,
`
`the smooth
`
`touchpad/LCD module has no existing mechanism or method of providing the user
`
`any kind of feedback as to the graphical button they are about to press.
`
`Thus,
`
`the shortcomings of the conventional approach include: if the user
`
`attempts to select specific buttons in a distracting environment to perform a primary
`
`task or function of the device, the user’s attention will be severely divided between
`
`the primary task they are trying to complete, and the secondary task of pressing the
`
`correct buttons on the device. If the primary task involves looking at objects far from
`
`themselves (as, say, in a driving environment in a vehicle), then there is the additional
`
`challenge of needing to dramatically change the user’s focus point from somewhere
`
`far from themselves to the device he or she is manipulating in hand or closeby.
`
`In those devices including a touch—pad,
`
`the user does not get any tactile
`
`feedback as to which graphical button he or she is going to press.
`
`In fact, for the
`
`“soft—keys” in an integral
`
`touchpad/LCD screen,
`
`the user may press between
`
`displayed buttons, not realizing that his or her finger is bridging two ‘valid’ button
`
`locations, and leading in some cases to an undesired key being pressed and thus an
`
`undesired command sent to the electronic device.
`
`
`
`Docket No. IMM147P
`
`2
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 3
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 3
`
`

`

`SUMMARY OF INVENTION
`
`The present invention relates to providing tactile feedback for mechanical
`
`buttons used on electronic devices such as cell-phones, remote controls, and the like.
`
`The invention also relates to embodiments for providing tactile feedback for a
`
`touchpad having an integral display such as an LCD screen, and buttons displayed
`thereon.
`
`Haptic feedback interface devices can provide physical sensations which are
`
`felt by the user manipulating the interface device. One or more motors or other
`
`actuators are user in the device to output the haptic sensations.
`
`The present invention provides an actuator to output tactile sensations on a set
`
`of buttons or keys of an electronic device. The buttons preferably can detect multiple
`
`levels of pressure or user selection (variable or analog input deVices-- e. g., analog in,
`
`analog out) to determine whether the user is desiring to press a button to activate its
`
`function (heavier pressure), or is running his or her fingers over the buttons to locate a
`
`particular button (lighter pressure). Tactile sensations can be output if lighter pressure
`
`is detected to allow the user to locate a particular button more easily.
`
`In the touchpad/integrated display embodiments,
`
`the tactile effects of the
`
`present invention allow the user to press the desired key or button more accurately.
`
`For example, a confirming tactile sensation can confirm that a particular key will be
`
`pressed, and a different sensation can confirm the actual press/selection of the desired
`
`key.
`
`In one example, if the user ignored the absence of a confirming sensation over a
`
`valid button and tried to press down , a ‘not—valid’ tactile sensation can be output
`
`indicating to the user that his or her finger was bridging two valid button locations.
`
`
`
`Benefits of the tactile button confirmation with a single actuator include:
`
`0 Allows selections or entries to be made on a keypad with less user
`
`distraction
`
`0 Allows customized tactile responses in a keypad with a single actuator
`
`Docket No. IMMl47P
`
`3
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 4
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 4
`
`

`

`o Allows modes and other non—visible features in the controlled device to be
`
`more obvious and intuitive to a user
`
`0 Allows highly configurable displays like LCD’s to be used to present a
`
`changeable user interface to a user while still retaining a tactile feel.
`
`0 Allows a single button to potentially deliver multiple selections/characters
`
`based on how hard the user presses.
`
`
`
`Docket No. lMM147P
`
`4
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 5
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 5
`
`

`

`
`
`DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
`
`Haptic feedback technology, also called force feedback or tactile feedback
`technology, has been used in a variety of applications to enhance the realism of
`computer simulation environments. The present invention describes embodiments for
`providing tactile sensations associated with buttons, both mechanical and displayed
`on a screen. The present invention can be implemented for a variety of types of
`
`electronic devices, such as cellular telephones, remote controls, gamepads, joystick
`
`handles, automotive controls (for radios, CD players, automobile functions, etc.),
`
`consumer electronics devices, PDA’s, laptop computers, portable game devices, etc.
`
`One embodiment of a hand—held device including features of the present
`
`invention is shown in Fig. 1. A cell phone 10 can include a number of buttons 12 in a
`
`keypad 14. A display 16 can also be included to display various features and
`functions of the device 10. The present invention includes a single vibrotactile
`
`actuator 20 coupled to the buttons in the cellular phone device, where the actuator is
`
`coupled either to the buttons, to the printed circuit board (PCB) to which the buttons
`are coupled, or to the case or housing of the device. For example, Fig. 2 is a side
`elevational view of the buttons, a button PCB 22 to which the buttons are coupled,
`
`and a piezo-electric actuator 20 which is coupled to the PCB 22. When the piezo
`
`electric actuator is energized with a current, the motion or force form the piezo-
`
`electric actuator transfers to the PCB and to all the buttons 12 coupled thereto. One
`
`example of the control of a piezo-electric actuator is described in copending
`
`application no. 09/917,263, incorporated herein by reference in its entirety.
`
`The actuator 20 used can be a variety of types; for example, the actuator can
`
`be piezo-electric (some embodiments described in copending application no.
`09/917,263), a voice-coil, moving magnet actuator, using the integral speaker in a
`
`device that uses a spinning mass actuator, or a flexure coupled to a motor (as
`
`described in copending U.S. application no. 09/585,741,
`
`incorporated herein by
`
`reference in its entirety). To allow for simple construction and low cost, the actuator
`
`can output a force on all the buttons of the device simultaneously (or on all the
`
`buttons in a particular group on the device, e.g., buttons that are closely spaced in a
`
`matrix).
`
`Docket No. lMMl47P
`
`V
`
`5
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 6
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 6
`
`

`

`
`
`The present invention includes switches or buttons or a touchpad that has more
`resolution than a simple on—off switch closure; rather, it requires some notion of
`“levels” of pressure on the switch or touchpad (e.g., three or more levels of detection
`or pressure sensing as opposed to two levels in an on-off switch). Examples of these
`kinds of variable switches include the analog buttons found on Sony Playstation 2
`
`gamepads, capacitive touch switches, force sensing resistors, or strain-gauge based
`sensors.
`In the present invention, the user can move his or her finger(s) over the
`various buttons or switches, and when exerting a very light pressure, along the level
`
`of ‘touching’ a button (but not quite pressing it, e. g. a level of sensed pressure under a
`predetermined threshold pressure or level), the switch or sensor detects the light touch
`and the actuator is controlled to deliver in response a brief sensation to the various
`
`buttons, delivering a specific frequency or tactile sensation that signals that that
`particular button had been touched. As the user touches other buttons (e.g., in a
`matrix), other tactile effects unique to those other buttons can be delivered registering
`that touch. With a short amount of use, the user can quickly become used to these
`
`sensations and can identify and press buttons by touch alone. Because in this scenario
`
`it would be unusual for the user to be pressing or contacting more than one button at a
`
`time, the fact that the single actuator is outputting sensations on all of the buttons
`simultaneously is of little consequence. When the user presses more firmly, with a
`pressure or to a level that is greater than or further than the predetermined threshold
`level, then the button’s function is activated by the device (or the command associated
`with the button is delivered to the device’s controller) as if the user pressed the button
`
`normally.
`
`For example, if the user is lightly moving his finger over the 9 key, a high
`frequency vibration can be output on all the keys. When the user’s light finger
`pressure is detected at the 6 key, a lower-frequency vibration can be output, allowing
`the user to determine which key is presently under his finger solely through the sense
`
`of touch. The fact that the 6-key sensation is being output on all the keys is irrelevant
`
`since the user’s finger is only touching the 6 key.
`
`In an alternate, more expensive
`
`embodiment, each button can be associated with its own actuator so that a sensation is
`
`output on a particular key when the lighter or heavier touch is detected on that key.
`
`In addition to these ‘touch confirmations’, the actuator can also deliver other
`
`sensations, e. g. a confirmation click, pulse, or other tactile effect when a button had
`actually been pressed, confirming its selection. If the device has ‘modes’, e.g. a 1-
`button short-cut dialing feature on a cell-phone, then this usage mode could have
`
`associated with it a unique tactile sensation that informs the user when that feature has
`been activated and/or is active.
`
`Docket No. IIVIM l 47P
`
`I
`
`6
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 7
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 7
`
`

`

`
`
`
`
`
`
`A controller, such as a microprocessor, ASIC, state machines, or other
`
`processor, can be used to receive the sensed level of touch or pressure, and to control
`the actuator output. This controller can be the main processor of the device, or can be
`a separate dedicated processor for touch sensation control. A variety of tactile
`sensations can be output, in which the frequency and/or magnitude of vibrations or
`
`pulses can be varied.
`
`Another embodiment uses the pressure or level capability of analog buttons
`
`(or a touchpad) with tactile feedback to allow a single button to perform multiple
`fimctions based on how hard the user presses. One example is the telephone dialpad
`
`on which many of the number keys also correspond to 3 or 4 letters of the alphabet.
`
`In this example, the user can press a button down part of its full range of motion to a
`
`first level and receive a distinctive tactile feedback that corresponds to the first
`
`alphanumeric character associated with that button, which is input to the device. The
`
`user can press the button further down (e.g. without stopping at the first level) to a
`
`second, intermediate pressure level, and receive a different tactile cue as well as select
`
`the middle alphanumeric character associated with the button. Or, the user can press
`
`the button hard to the maximum ‘depth’ (third level) of the key, which selects the
`
`third alphanumeric character for input, and causes the corresponding tactile cue for
`
`that letter to be output on the button. This embodiment may include a feature of
`
`‘dwell to select’, i.e. holding a consistent pressure level would be equivalent to having
`
`selected, and again the confirmation of the selection could be signified by a
`
`distinctive, different tactile cue. Other embodiments may incorporate less or fewer
`levels of selection.
`
`The present invention can also be used with a touchpad/integrated flat display
`
`overlay, and the benefits may be even more compelling. A LCD (or other type of
`
`display) panel overlaid on a pressure-sensitive touchpad, and coupled to an actuator
`
`(such as actuator 20) can allow the user to move his or her finger over a software-
`
`generated grid of displayed buttons or keys which is provided in place of the
`
`mechanical buttons 12 shown in Fig. 1. (Some embodiments may provide such “soft”
`
`buttons on a display 16 in addition to a mechanical set of buttons 12 to allow more
`
`fiinctions to he commanded by the user—— such an embodiment can provide another
`
`dedicated actuator coupled to the display 16 in addition to the actuator coupled to the
`
`buttons 12, or a single actuator may output force on both buttons and display). Tactile
`
`feedback is output on the touchpad/display from the coupled actuator for a variety of
`
`interactions. For example, a tactile sensation can be output indicating when the user’s
`
`finger is over a valid key selection and when it is not (when the “lighter” pressure is
`
`sensed). The value of the tactile feedback in this embodiment is that it can make a
`
`Docket No. IMM l 47P
`
`7
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 8
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 8
`
`

`

`displayed key matrix significantly more useable in a consumer electronic device. For
`
`example, when the user’s finger is positioned between two displayed buttons so that it
`
`would not be known which button the user desired to select, no tactile sensation is
`
`output on the touchpad. When the user’s finger is moved to a position on the
`
`touchpad that is unambiguous as to which button would be activated if the user
`
`pressed harder, then a tactile pulse, vibration, or other “confirming sensation” can be
`
`output to indicate that the user’s finger is in a key-pressing position.
`
`In some
`
`embodiments, if the user ignored the absence of a confirming sensation over a valid
`
`button and tried to press down, a ‘not-valid’ tactile sensation can be output indicating
`
`to the user that his or her finger was bridging two valid button locations or was
`
`otherwise not in a correct position to press a button.
`
`Furthermore, different tactile sensations can distinguish different buttons, as in
`
`the mechanical button embodiment described above. When the user actually presses
`
`the key to initiate its function or send its command with a heavier pressure, then a
`
`different tactile sensation (or, alternatively, the same sensation) can be output on the
`
`touchpad to indicate this action has been successfully completed.
`
`The further Value in this embodiment is that the same software that allows the
`
`number, placement and purposes/functions of displayed buttons to change as the
`
`device mode or feature set changes, can also control the tactile feedback. While the
`
`clear LCD overlay as a display medium is emphasized here, other embodiments can
`
`employ a Thin Film Transistor or other flat displays such as are found in laptops and
`
`color personal digital assistants (PDA’s).
`
`Although handheld consumer products in the PDA/cell phone markets are
`
`described above, the present invention can be equally useful in automotive controls,
`
`home appliances, and white goods.
`
`In addition, the actuator 20 used for outputting tactile sensations based on
`
`sensed user touch of the buttons or a touchpad can also be used to output tactile
`
`sensations on the buttons,
`
`touchpad, or housing based on events or interactions
`
`performed on the device. For example, if a call is received on a cell phone, the
`
`buttons or housing can be vibrated, or if the player is playing a game or Viewing an
`
`animation displayed on a display screen of the device, tactile sensations can be output
`
`synchronized with events, interactions, and commands occurring during the game or
`animation.
`
`
`
`Docket No. IMMl47P
`
`'
`
`-
`
`8
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 9
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 9
`
`

`

`
`
`~\u§D. Ohscv
`
`fcozcaviffl /
`
`14
`
`
`
`WM;
`
`2'1, ox,
`24:
`
`99363-050
`‘
`
`o‘
`
` ’
`
`!‘N‘(T-5
`-»74-1
`"n~aL6c_." cams mm Luce
`
`E
`
`dd
`
`- Lgx
`
`\’7~*’
`
`Psi
`55wS¢§€
`
`<.AM.e?A.a ¢$ 55% (£475
`
`
`
`~"~ men.
`P \€&<= —
`
`U55
`\s To
`xi sbm:-rr».¢:ru..e
`
`5E\/15 !LAL./
`
`6rt7'u~z$ .
`
`Stwteui
`rk
`sarsdg
`"Tb
`
`;
`
`‘G
`EEO Am”
`f=«Ne£ilal12¢.’—.' /A-CTVATC-2
`
`KL
`
`3
`@’0/
`
`952
`A
`_. M asmg,
`-—n+e
`was Ac‘TvA"zu¢.
`-“rt-re
`5*n7<N
`L5
`usaa
`wc+e.c¢-s
`fixrrt/~1
`
`sn~mJ
`Pwr~JL.oc,
`<.:wu;é?n»o «s7~1c.e
`came
`55-
`P-/4.550 W-ad E. -Hw/
`Ptlfieo ¢.u.H2'L'7J
`l"’DL,(',ATl~G "70 THE
`Vt-e—e“z_
`"""‘v\«c~QL-: ‘gr’ 7;; ma?!-
`
`-mw Hnvé A~a-name >*am7wu -76 <:mm£M M2545’.
`{flax/(95:
`CLVLD
`"E-\L'-'
`\IM.~/ti
`£5
`75913"?
`0~JE
`H-C 7‘/é's7'¢(L
`oucwuwr
`smsmmws
`‘iv
`MVLTIPQE
`arr-rws, %s7zu.
`pub
`§ét~»\,eTg...,(.»
`wv ( <es/E
`rm flma Ewzwfs
`
`‘*
`
`~
`
`($3 \~3‘Za%D CF
`
`v5u...(;
`
`r:,.....,/::,e.nL
`
`QVTTUVS)
`
`a way
`
`coma
`
`a(_s'an-pg
`am“;
`on‘
`
`G7)-.%C~CK
`£ECtsf£&""-L
`
`APPLE INC.
`D3173! 5 4’-fl "‘/‘(W
`Date:
`C,/_
`flEXH|B|T1105—PAGE10
`
`Avuitbdsé
`H
`C Vacarwg
`5w\‘[u+
`F/#7.:/9%.
`$&~'5!-V—."”Y ‘W S’Zu.s.. A2550
`a
`F~/U... mm Cassy «E ~
`
`
`
`
`APPLE INC.
`EXHIBIT 1105 - PAGE 10
`
`