(12) United States Patent
`Rosenberg et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 6,429,846 B2
`*Aug. 6, 2002
`
`US006429846B2
`
`......... .. 345/156
`9/1999 Kuenzner et al.
`5,956,016 A *
`345/161
`6,005,551 A * 12/1999 Osborne et al.
`. . . .. 341/20
`6,061,004 A *
`5/2000 Rosenberg . . . . . . . .
`345/157
`6,100,874 A *
`8/2000 Schena et al.
`345/156
`6,111,577 A *
`8/2000 Zilles et al.
`345/163
`6,125,385 A *
`9/2000 Wies et al.
`707/102
`6,131,097 A * 10/2000 Peurach et al.
`,, 341/20
`5,147,422 A * 11/2000 13615011 eta1,
`345/157
`6,147,674 A * 11/2000 Rosenberg et al.
`709/203
`6,161,126 A * 12/2000 Wies et al.
`..... ..
`6,166,723 A * 12/2000 Schena et al.
`............ .. 345/184
`
`
`
`..
`
`.
`.
`* cued by exammer
`
`(54) HAPTIC FEEDBACK FOR TOUCHPADS AND
`OTHER TOUCH CONTROLS
`
`(75)
`
`Inventors: Louis B. Rosenberg, San Jose; James
`R. Riegel, Santa Clara, both Of CA
`(US)
`
`(73) Assignee:
`
`*
`
`,
`) N0t1C€¢
`
`(
`
`Immersion Corporation, San Jose, CA
`(US)
`,
`,
`,
`T1115, Patent {ssuéd 0H a C0Ht1H11€d Pf0S-
`ecution application filed under 37 CFR
`1.53(d), and is subject to the twenty year
`patent
`term provisions of 35 U.S.C.
`154(a)(2).
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`firlmary EExami’,1er_:3RiChafi Ejerpe
`Sslsmm xammer
`Ona,
`aneau
`,
`(7;1)fAt1t0rney, Agent, or FLrm—James R. Riegel; Paul M.
`T y au t
`
`(22)
`
`Filed:
`
`Jan. 19, 2000
`
`Related U_s_ Application Data
`
`(63)
`
`C0nt1nuat10n‘1n‘PaTt 0faPP11Cat1011 N0~ 09/467307; filed 011
`Dec. 17, 1999, which is a continuation—in—part of application
`No. 09/156,802, filed on Sep. 17, 1998, application No.
`09/487,737, which is a continuation.in-pait of application
`N°~ 09/253>182> filed 0“ F°b~ 18> 1999: and 3 Continuation‘
`in—part of application No. 09/103,281, filed on Jun. 23, 1998.
`
`(56)
`
`5,625,576 A *
`
`4/1997 Massie et al.
`
`............ .. 318/628
`
`43 Claims, 5 Drawing Sheets
`
`
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 1
`
`,
`,
`,
`Ahaptic feedback planar touch control used to provide input
`to a computer. A touch input device includes a planar touch
`surface that inputs a position signal to a processor of the
`computer based on a location of user contact on the touch
`f
`Th
`t
`-t~
`-
`d 1
`d
`5“ a°,‘°"
`5' ‘?°mP“ er Ca“ P051 1°“ a,°“rS°r 1“ a
`151’ *}Y‘°'
`graphical environment based at least in part on the position
`signal or perform a different function. At least one actuator
`.
`’
`.
`.
`is also coupled to the touch input device and outputs a force
`[Q prgvide a haptic Sehsafigh [0 the user Cghtacfihg the [Ouch
`surface. The touch input device can be a touchpad separate
`from the C01'nputer’S
`Screen, or can be a touch Screen.
`Output haptic sensations on the touch input device can
`include pulses, vibrations, and spatial textures. The touch
`input device can include multiple different regions to control
`different computer functions.
`
`Int. CL7 ................................................ ..
`(52) U-S- CL ...................... .. 345/156; 345/173; 345/179
`Of Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..
`345/:3?
`’
`’
`References Cited
`U.S. PATENT DOCUMENTS
`
`’
`
`’
`
`668807>
`’
`
`’
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 1
`
`

`
`U.S. Patent
`
`Aug. 6, 2002
`
`Sheet 1 of 5
`
`US 6,429,846 B2
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 2
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 2
`
`

`
`U.S. Patent
`
`Aug. 6, 2002
`
`Sheet 2 of 5
`
`US 6,429,846 B2
`
`PPPPPPPC.
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 3
`
`

`
`U.S. Patent
`
`Aug. 6, 2002
`
`Sheet 3 of 5
`
`US 6,429,846 B2
`
`
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 4
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 4
`
`

`
`U.S. Patent
`
`Aug. 6, 2002
`
`Sheet 4 of 5
`
`US 6,429,846 B2
`
`50\
`
`¢
`
`52
`
`16
`
`54
`
`52
`
`Z$’\56
`
`
`
`72a
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 5
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 5
`
`

`
`U.S. Patent
`
`Aug. 6, 2002
`
`Sheet 5 of 5
`
`US 6,429,846 B2
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 6
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 6
`
`

`
`US 6,429,846 B2
`
`1
`HAPTIC FEEDBACK FOR TOUCHPADS AND
`OTHER TOUCH CONTROLS
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation-in-part of co-pending
`patent application Ser. No. 09/467,309, entitled “Haptic
`Feedback for Directional Control Pads,” filed Dec. 17, 1999
`by Martin et al, and which is incorporated herein by refer-
`ence in its entirety, which is a continuation in part of
`application Ser. No. 09/156,802 filed Sep. 17, 1998 and a
`continuation in part of application Ser. No. 09/103,281 filed
`Jun. 23, 1998 and a continuation in part of application Ser.
`No. 09/253,132 filed Feb. 18, 1999.
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to the interfacing
`with computer and mechanical devices by a user, and more
`particularly to devices used to interface with computer
`system’s and electronic devices and which provide haptic
`feedback to the user.
`Humans interface with electronic and mechanical devices
`
`in a variety of applications, and the need for a more natural
`easy-to-use, and informative interface is a constant concern.
`In the context of the present invention, humans interface
`with computer devices for a variety of applications. One
`such application is interfacing with computer-generated
`environments such as games, simulations, and application
`programs. Computer input devices such as mice and track-
`balls are often used to control a cursor within a graphical
`environment and provide input in these applications.
`In some interface devices, force feedback or tactile feed-
`back is also provided to the user, collectively known herein
`as “haptic feedback.” For example, haptic versions of
`joysticks, mice, gamepads, steering wheels, or other types of
`devices can output forces to the user based on events or
`interactions occurring within the graphical environment,
`such as in a game or other application program.
`In portable computer or electronic devices, such as laptop
`computers, mice typically too large a workplace to be
`practical. As a result, more compact devices such as track-
`balls are often used. A more popular device for portable
`computers are “touchpads,” which are small rectangular,
`planar pads provided near the keyboard of the computer. The
`touchpads senses the location of a pointing object by any of
`a variety of sensing technologies, such as capacitive sensors
`or pressure sensors that detect pressure applied to the
`touchpad. The user contacts the touchpad most commonly
`with a fingertip and moves his or her finger on the pad to
`move a cursor displayed in the graphical environment. In
`other embodiments, the user can operate a stylus in con-
`junction with the touchpad by pressing the stylus tip on the
`touchpad and moving the stylus.
`One problem with existing touchpads is that there is no
`haptic feedback provided to the user. The user of a touchpad
`is therefore not able to experience haptic sensations that
`assist and inform the user of targeting and other control tasks
`within the graphical environment. The touchpads of the prior
`art also cannot take advantage of existing haptic-enabled
`software run on the portable computer.
`
`SUMMARY OF THE INVENTION
`
`The present invention is directed to a haptic feedback
`planar touch control used to provide input to a computer
`system. The control can be a touchpad provided on a
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`portable computer, or can be a touch screen found on a
`variety of devices. The haptic sensations output on the touch
`control enhance interactions and manipulations in a dis-
`played graphical environment or when controlling an elec-
`tronic device.
`
`More specifically, the present invention relates to a haptic
`feedback touch control for inputting signals to a computer
`and for outputting forces to a user of the touch control. The
`control includes a touch input device including an approxi-
`mately planar touch surface operative to input a position
`signal to a processor of said computer based on a location of
`user contact on the touch surface. The computer positions a
`cursor in a graphical environment displayed on a display
`device based at least in part on the position signal. At least
`one actuator is also coupled to the touch input device and
`outputs a force on the touch input device to provide a haptic
`sensation to the user contacting the touch surface. The
`actuator outputs the force based on force information output
`by the processor to the actuator.
`The touch input device can be a touchpad separate from
`a display screen of the computer, or can be included in a
`display screen of the computer as a touch screen. The touch
`input device can be integrated in a housing of the computer
`or handheld device, or provided in a housing that is separate
`from the computer. The user contacts the touch surface with
`a finger, a stylus, or other object. The force is preferably a
`linear force output approximately perpendicularly to a plane
`of the touch surface of the touch input device, and the
`actuator can include a piezo-electric actuator, a voice coil
`actuator, a pager motor, a solenoid, or other type of actuator.
`In one embodiment,
`the actuator is coupled between the
`touch input device and a grounded surface.
`In another
`embodiment,
`the actuator is coupled to an inertial mass,
`wherein said actuator outputs an inertial force on the touch
`input device approximately along an axis perpendicular to
`the planar touch surface. A touch device microprocessor
`separate from the main processor of the computer can
`receive force information from the host computer and pro-
`vide control signals based on the force information to
`control the actuator.
`
`The haptic sensations, such as a pulse, vibration, or spatial
`texture, are preferably output in accordance with an inter-
`action of a controlled cursor with a graphical object in the
`graphical environment. For example, a pulse can be output
`when the cursor is moved between menu elements in a
`
`menu, moved over said icon, or moved over a hyperlink. The
`touch input device can include multiple different regions,
`where at least one of the regions provides the position signal
`and at least one other region provides a signal that is used by
`the computer to control a different function, such as rate
`control function of a value or a button press. Different
`regions and borders between regions can be associated with
`different haptic sensations.
`The present
`invention advantageously provides haptic
`feedback to a planar touch control device of a computer,
`such as a touchpad or touch screen. The haptic feedback can
`assist and inform the user of interactions and events within
`
`a graphical user interface or other environment and ease
`cursor targeting tasks. Furthermore,
`the invention allows
`portable computer devices having such touch controls to
`take advantage of existing haptic feedback enabled software.
`The haptic touch devices disclosed herein are also
`inexpensive, compact and consume low power, allowing
`them to be easily incorporated into a wide variety of portable
`and desktop computers and electronic devices.
`These and other advantages of the present invention will
`become apparent to those skilled in the art upon a reading of
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 7
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 7
`
`

`
`US 6,429,846 B2
`
`3
`the following specification of the invention and a study of
`the several figures of the drawing.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a perspective view of a haptic touchpad of the
`present invention;
`FIG. 2 is a perspective view of a remote control device
`including the touchpad of the present invention;
`FIG. 3 is a perspective view of a first embodiment of the
`touchpad of the present invention including one or more
`actuators coupled to the underside of the touchpad;
`FIG. 4 is a side elevational view of a first embodiment of
`
`the present invention in which a piezo-electric actuator is
`directly coupled to the touchpad of the present invention;
`FIG. 5 is a side elevational view of a second embodiment
`
`of the touchpad of the present invention including a linear
`actuator;
`FIG. 6 is a side elevational view of a third embodiment of
`
`the touchpad of the present invention having an inertial
`mass;
`
`FIG. 7 is a top plan view of an example of a touchpad of
`the present invention having different control regions; and
`FIGS. 8a and 8b are top plan and side cross sectional
`views, respectively, of a touch screen embodiment of the
`present invention.
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`FIG. 1 is a perspective view of a portable computer 10
`including a haptic touchpad of the present invention. Com-
`puter 10 is preferably a portable or “laptop” computer that
`can be carried or otherwise transported by the user and may
`be powered by batteries or other portable energy source in
`addition to other more stationary power sources. Computer
`10 preferably runs; one or more host application programs
`with which a user is interacting via peripherals.
`Computer 10 may include the various input and output
`devices as shown, including a display device 12 for output-
`ting graphical images to the user, a keyboard 14 for provid-
`ing character or toggle input from the user to the computer,
`and a touchpad 16 of the present invention. Display device
`12 can be any of a variety of types of display devices;
`fiat-panel displays are most common on portable computers.
`Display device 12 can display a graphical environment 18
`based on application programs and/or operating systems that
`are running, such as a graphical user interface (GUI), that
`can include a cursor 20 that can be moved by user input, as
`well as windows 22, icons 24, and other graphical objects
`well known in GUI environments. Other devices may also
`be incorporated or coupled to the computer 10, such as
`storage devices (hard disk drive, DVD-ROM drive, etc.),
`network server or clients, game controllers, etc. In alternate
`embodiments, the computer 10 can take a wide variety of
`forms, including computing devices that rest on a tabletop or
`other surface, stand-up arcade game machines, other por-
`table devices or devices worn on the person, handheld or
`used with a single hand of the user, etc. For example, host
`computer 10 can be a video game console, personal
`computer, workstation, a television “set
`top box” or a
`“network computer”, or other computing or electronic
`device.
`
`Touchpad device 16 of the present invention preferably
`appears externally to be similar to the touchpads of the prior
`art. Pad 16 includes a planar, rectangular smooth surface that
`can be positioned below the keyboard 14 on the housing of
`
`4
`the computer 10, as shown, or may be positioned at other
`areas of the housing. When the user operates the computer
`10,
`the user may conveniently place a fingertip or other
`object on the touchpad 16 and move the fingertip to corre-
`spondingly move cursor 20 in the graphical environment 18.
`In operation, the touchpad 16 inputs coordinate data to the
`main microprocessor(s) of the computer 10 based on the
`sensed location of an object on (or near) the touchpad. As
`with many touchpads of the prior art, touchpad 16 can be
`capacitive, resistive, or use a different type of sensing. Some
`existing touchpad embodiments are disclosed, for example,
`in U.S. Pat. Nos. 5,521,336 and 5,943,044. Capacitive
`touchpads typically sense the location of an object on or near
`the surface of the touchpad based on capacitive coupling
`between capacitors in the touchpad and the object. Resistive
`touchpads are typically pressure-sensitive, detecting the
`pressure of a finger, stylus, or other object against the pad,
`where the pressure causes conductive layers,
`traces,
`switches, etc.
`in the pad to electrically connect. Some
`resistive or other types of touchpads can detect the amount
`of pressure applied by the user and can use the degree of
`pressure for proportional or variable input to the computer
`10. Resistive touchpads typically are at
`least partially
`deformable, so that when a pressure is applied to a particular
`location, the conductors at that location are brought into
`electrical contact. Such deformability can be useful in the
`present invention since it can potentially amplify the mag-
`nitude of output forces such as pulses or vibrations on the
`touchpad as used in the present invention. Forces can be
`amplified if a tuned compliant suspension is provided
`between an actuator and the object
`that
`is moved, as
`described in provisional application Ser. No. 60/157,206,
`incorporated herein by reference. Capacitive touchpads and
`other types of touchpads that do not require significant
`contact pressure may be better suited for the present inven-
`tion in many embodiments, since excessive pressure on the
`touchpad may in some cases interfere with the motion of the
`touchpad for haptic feedback. Other types of sensing tech-
`nologies can also be used in the touchpad. Herein, the term
`“touchpad” preferably includes the surface of the touchpad
`16 as well as any sensing apparatus included in the touchpad
`unit.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`Touchpad 16 preferably operates similarly to existing
`touchpads, where the speed of the fingertip on the touchpad
`correlates to the distance that the cursor is moved in the
`
`45
`
`graphical environment. For example, if the user moves his or
`her finger quickly across the pad, the cursor is moved a
`greater distance than if the user moves the fingertip more
`slowly. If the user’s finger reaches the edge of the touchpad
`before the cursor reaches a desired destination in that
`
`direction, then the user can simply move his or her finger off
`the touchpad, reposition the finger away from, the edge, and
`continue moving the cursor. This is an “indexing” function
`similar to lifting a mouse off a surface to change the offset
`between mouse position and cursor. Furthermore, many
`touchpads can be provided with particular regions that are
`each assigned to particular functions that can be unrelated to
`cursor positioning. Such an embodiment is described in
`greater detail below with respect to FIG. 7. In some embodi-
`ments the touchpad 16 may also allow a user to “tap” the
`touchpad (rapidly touch and remove the object from the pad)
`in a particular location to provide a command. For example,
`the user can tap or “double tap” the pad with a finger while
`the controlled cursor is over an icon to select that icon.
`
`In the present invention, the touchpad 16 is provided with
`the ability to output haptic feedback such as tactile sensa-
`tions to the user who is physically contacting the touchpad
`
`50
`
`55
`
`60
`
`65
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 8
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 8
`
`

`
`US 6,429,846 B2
`
`5
`16. Various embodiments detailing the structure of the haptic
`feedback touchpad are described in greater detail below.
`Preferably, the forces output on the touchpad are linear (or
`approximately linear) and oriented along the z-axis, approxi-
`mately perpendicular to the surface of the touchpad 16 and
`the top surface of computer 10. In a different embodiment,
`forces can be applied to the touchpad 16 to cause side-to-
`side (e.g., x-y) motion of the pad in the plane of its surface
`in addition to or instead of z-axis motion, although such
`motion is not preferred.
`Using one or more actuators coupled to the touchpad 16,
`a variety of haptic sensations can be output to the user who
`is contacting the pad. For example, jolts, vibrations (varying
`or constant amplitude), and textures can be output. Forces
`output on the pad can be at least in part based on the location
`of the finger on the pad or the state of a controlled object in
`the graphical environment of the host computer 10, and/or
`independent of finger position or object state. Such forces
`output on the touchpad 16 are considered “computer-
`controlled” since a microprocessor or other electronic con-
`troller is controlling the magnitude and/or direction of the
`force output of the actuator(s) using electronic signals.
`Preferably, the entire pad 16 is provided with haptic sensa-
`tions as a single unitary member; in other embodiments,
`individually-moving portions of the pad can each be pro-
`vided with its own haptic feedback actuator and related
`transmissions so that haptic sensations can be provided for
`only a particular portion. For example, some embodiments
`may include a touchpad having different portions that may
`be flexed or otherwise moved with respect to other portions
`of the pad.
`In other embodiments, the touchpad 16 can be provided in
`a separate housing that is connected to a port of the computer
`10 via a cable or via wireless transmission and which
`
`receives force information from and sends position infor-
`mation to the computer 10. For example, Universal Serial
`Bus (USB), Firewire, or a standard serial bus can connect
`such a touchpad to the computer 10. In such an embodiment,
`the computer 10 can be any desktop or stationary computer
`or device and need not be a portable device.
`One or more buttons 26 can also be provided on the
`housing of the computer 10 to be used in conjunction with
`the touchpad 16. The user’s hands have easy access to the
`buttons, each of which may be pressed by the user to provide
`a distinct input signal to the host computer 12. Typically,
`each button 26 corresponds to a similar button found on a
`mouse input device, so that a left button can be used to select
`a graphical object (click or double click), a right button can
`bring up a context menu, etc. In some embodiments, one or
`more of the buttons 26 can be provided with tactile feedback
`as described in copending patent application Ser. Nos.
`09/156,802 and 09/4671,309, entitled, “Haptic Feedback for
`Directional Control Pads,” filed Dec. 12, 1999, and both
`incorporated herein by reference. Other features of these
`disclosures may also be used with the present invention.
`Furthermore, in some embodiments, one or more move-
`able portions 28 of the housing of the computer device 10
`can be included which is contacted by the user when the user
`operates the touchpad 16 and which can provide haptic
`feedback. Having a moveable portion of a housing for haptic
`feedback is described in copending patent application Ser.
`No. 09/156,802 and application Ser. No. 09/103,281, both
`incorporated herein by reference. Thus, both the housing can
`provide haptic feedback (e.g., through the use of an eccentric
`rotating mass on a motor coupled to the housing) and the
`touchpad 16 can provide separate haptic feedback. This
`allows the host to control two different tactile sensations
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`simultaneously to the user; for example, a vibration of a low
`frequency can be conveyed through the housing to the user
`and a higher frequency vibration can be conveyed to the user
`through the touchpad 16. Each other button or other control
`provided with haptic feedback can also provide tactile
`feedback independently from, the other controls.
`The host application program(s) and/or operating system
`preferably displays graphical images of the environment on
`display device 12. The software and environment running on
`the host computer 12 may be of a wide variety. For example,
`the host application program can be a word processor,
`spreadsheet, video or computer game, drawing program,
`operating system, graphical user interface, simulation, Web
`page or browser that
`implements HTML or VRML
`instructions, scientific analysis program, virtual reality train-
`ing program or application, or other application program
`that utilizes input from the touchpad 16 and outputs force
`feedback commands to the touchpad 16. For example, many
`games and other application programs include force feed-
`back functionality and may communicate with the touchpad
`16 using a standard protocol/drivers such as I-Force®,
`FEELit®, or TouchsenseTM available from Immersion Cor-
`poration of San Jose, Calif.
`The touchpad 16 can include circuitry necessary to report
`control signals to the microprocessor of the host computer
`10 and to process command signals from the host’s micro-
`processor. For example, appropriate sensors (and related
`circuitry) are used to report the position of the user’s finger
`on the touchpad 16. The touchpad device also includes
`circuitry that receives signals from the host and outputs
`tactile sensations in accordance with the host signals using
`one or more actuators. In some embodiments, a separate,
`local microprocessor can be provided for the touchpad 16 to
`both report touchpad sensor data to the host and/or to carry
`out force commands received from the host, such commands
`including, for example,
`the type of haptic sensation and
`parameters describing the commanded haptic sensation.
`Alternatively, the touchpad microprocessor can simply pass
`streamed data from the main processor to the actuators. The
`term “force information” can include both commands/
`parameters and streamed data. Tlie touchpad microprocessor
`can implement haptic sensations independently after receiv-
`ing a host command by controlling the touchpad actuators;
`or,
`the host processor can maintain a greater degree of
`control over the haptic sensations by controlling the actua-
`tors more directly. In other embodiments, logic circuitry
`such as state machines provided for the touchpad 16 can
`handle haptic sensations as directed by the host main pro-
`cessor. Architectures and control methods that can be used
`
`for reading sensor signals and providing haptic feedback for
`a device are described in greater detail in U.S. Pat. No.
`5,734,373 and copending application Ser. Nos. 60/156,354,
`60,133,208, 09/376,649, and 60/160,401, all incorporated
`herein by reference.
`FIG. 2 is a perspective view of another embodiment of a
`device which can include the active touchpad 16 of the
`present invention. The device can be a handheld remote
`control device 30, which the user grasps in one hand and
`manipulates controls to access the functions of an electronic
`device or appliance remotely by a user (such as a television,
`video cassette recorder or DVD player, audio/video receiver,
`Internet or network computer connected to a television, etc.).
`For example, several buttons 32 can be included on the
`remote control device 30 to manipulate functions of the
`controlled apparatus. Atouchpad 16 can also be provided to
`allow the user to provide more sophisticated directional
`input. For example, a controlled apparatus may have a
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 9
`
`APPLE INC.
`EXHIBIT 1009 - PAGE 9
`
`

`
`US 6,429,846 B2
`
`7
`selection screen in which a cursor may be moved, and the
`touchpad 16 can be manipulated to control the cursor in two
`dimensions. The touchpad 16 includes the ability to output
`haptic sensations to the user as described herein, based on a
`controlled value or event. Fox example, a volume level
`passing a mid-point or reaching a maximum level can cause
`a pulse to be output to the touchpad and to the user.
`In one application,
`the controlled apparatus can be a
`computer system such as Web-TV from Microsoft Corp. or
`other computing device which displays a graphical user
`interface and/or web pages accessed over a network such as
`the Internet. The user can control the direction of the cursor
`
`by moving a finger (or other object) on the touchpad 16. The
`cursor can be used to select and/or manipulate icons,
`windows, menu items, graphical buttons, slider bars, scroll
`bars, or other graphical objects in a graphical user interface
`or desktop interface. The cursor can also be used to select
`and/or manipulate graphical objects on a web page, such as
`links, images, buttons, etc. Other force sensations associated
`with graphical objects are described below with reference to
`FIG. 7.
`
`FIG. 3 is a perspective view of a first embodiment 40 of
`a touchpad 16 of the present invention for providing haptic
`feedback to the user. In this embodiment, one or more
`piezoelectric actuators 42 are coupled to the underside of the
`touchpad 16. The piezoelectric actuator 42 is driven by
`suitable electronics, as is well known to those skilled in the
`art. In one embodiment, a single piezoelectric actuator 42 is
`positioned at or near the center of the touchpad 16, or off to
`one side if space constraints of the housing require such a
`position.
`In other embodiments, multiple piezoelectric
`actuators 42 can be positioned at different areas of the
`touchpad; the dashed lines show one configuration, where an
`actuator 42 is placed at each corner of the pad 16 and at the
`center of the pad.
`The piezoelectric actuators 42 can each output a small
`pulse, vibration, or texture sensation on the touchpad 16 and
`to the user if the user is contacting the touchpad. The entire
`touchpad 16 is preferably moved with the forces output by
`actuator(s) 42. Preferably, the forces output on the touchpad
`are linear (or approximately linear) and along the z-axis,
`approximately perpendicular to the surface of the touchpad
`16 and the top surface of computer 10.
`In a different
`embodiment, as mentioned above, forces can be applied to
`the touchpad 16 to cause side-to-side (e.g., x-y) motion of
`the pad in the plane of its surface in addition to or instead of
`z-axis motion. For example, one linear actuator can provide
`motion for the x-axis, and a second linear actuator can
`provide motion for the y-axis and/or the x-axis.
`The frequency of a vibration output by an actuator 42 can
`be varied by providing different control signals to an actua-
`tor 42. Furthermore, the magnitude of a pulse or vibration
`can be controlled based on the applied control signal. If
`multiple actuators 42 are provided, a stronger vibration can
`be imparted on the touchpad by activating two or more
`actuators simultaneously. Furthermore,
`if an actuator is
`positioned at an extreme end of the touchpad and is the only
`actuator that is activated, the user may experience a stronger
`vibration on the side of the touchpad having the actuator
`than on the opposite side of the touchpad. Different magni-
`tudes and localized effects can be obtained by activating
`some but not all of the actuators. Since the tip of a user’s
`finger that is touching the pad is fairly sensitive, the output
`forces do not have to be of a high magnitude for the haptic
`sensation to be effective and compelling.
`Besides using a finger to contact the touchpad, the user
`may also hold other objects that directly contact the touch-
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`pad. Any haptic sensations output on the pad can be trans-
`mitted through the held object
`to the user’s hand. For
`example,
`the user can hold a stylus having a point that
`contacts the touchpad 16 more precisely than a finger. Other
`objects may also be used. In some embodiments, specialized
`objects can be used to enhance the haptic sensations. For
`example, a stylus or other object having a flexible portion or
`compliance may be able to magnify at least some of the
`touchpad haptic sensations as experienced by the user.
`The piezo-electric actuators 42 have several advantages
`for the touchpad 16. These actuators can be made very thin
`and small, allowing their use in compact housings that are
`typical for portable electronic devices. They also require
`very low power, and are thus suitable for devices with
`limited power (e.g., powered by batteries). In some embodi-
`ments described herein, power for the actuators can be
`drawn off a bus connecting the computer to the touchpad (or
`touch screen). For example, if the touchpad 16 is provided
`in a separate housing, a Universal Serial Bus can connect the
`pad to the computer and provide power from the computer
`to the pad as well as data (e.g. streaming force data, force
`commands, etc.).
`FIG. 4 is a side elevational view of the embodiment 40 of
`
`the touchpad 16 of the present invention as shown in FIG.
`3. Touchpad 16 is directly coupled to a grounded piezo-
`electric actuator 42 which operates to produce a force on the
`touchpad 16 when an electrical signal is input to the actuator.
`Typically, a piezo-electric actuator includes two layers
`which can move relative to each other when a current is
`
`applied to the actuator; here, the grounded portion of the
`actuator remains stationary with respect to the surrounding
`housing 41 while the moving portion of the actuator and the
`touchpad move with respect to the housing 41. The opera-
`tion of piezo-electric actuators to output force based on an
`input electrical signal is well known to those skilled the art.
`The touchpad 16 can be coupled only to the actuator 42,
`or can be additionally coupled to the housing of the com-
`puter device at other locations besides the actuators 42.
`Preferably the other couplings are compliant connections,
`using a material or element such as a spring or foam. If such
`connections are not made compliant, then the touchpad 16
`itself preferably has some compliance to allow portions of
`the pad to move in response to actuator forces and to convey
`the haptic sensations to the user more effectively.
`Since the touchpad 16 is directly coupled to the actuator
`42, any produced forces are directly applied to the touchpad
`16. The electric signal preferably is obtained from