as) United States
`a2) Patent Application Publication co) Pub. No.: US 2006/0109256 A1
`(43) Pub. Date: May25, 2006
`
`Grantet al.
`
`US 20060109256A1
`
`(54) HAPTIC FEEDBACK FOR BUTTON AND
`SCROLLING ACTION SIMULATION IN
`TOUCH INPUT DEVICES
`
`(75)
`
`Inventors: Danny A. Grant, Montreal (CA);
`Pedro Gregorio, Verdun (CA); Robert
`W. Heubel, San Leandro, CA (US):
`Christophe Ramstein, San Francisco,
`CA (US)
`
`Correspondence Address:
`David B. Ritchie
`Thelen Reid & Priest LLP
`P.O. Box 640640
`San Jose, CA 95164-0640 (US)
`
`(73) Assignee:
`
`Immersion Corporation, a Delaware
`Corporation
`
`(21) Appl. No.:
`
`11/249,031
`
`(22)
`
`Filed:
`
`Oct. 11, 2005
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/616,648, filed on Oct.
`8, 2004.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`G09G 5/00
`(52) US. C1. eee cecssseceseessesceneneesssecsssensssenseneeneaee 345/173
`
`(57)
`
`ABSTRACT
`
`Aplanar touch control is used to provide input to a computer
`and haptic feedback is provided thereto. A touch control
`includes a touch input device with a planar touch surface that
`inputs a position signal to a processor associated with the
`computer based on a location of user implemented contact
`on the touch surface. The computer can position or modify
`a cursor or image in a displayed graphical environment
`based at least in part on the position signal, or perform a
`different function. At least one actuatoris also coupledto the
`touch input device and outputs a force to provide a haptic
`sensation to the user via the touch surface.
`
`APPLE 1007
`
`APPLE 1007
`
`1
`
`

`

`Patent Application Publication May 25,2006 Sheet 1 of 23
`
`US 2006/0109256 Al
`
`2
`
`

`

`Patent Application Publication May 25,2006 Sheet 2 of 23
`
`US 2006/0109256 Al
`
`
`
`3
`
`

`

`Patent Application Publication May 25,2006 Sheet 3 of 23
`
`US 2006/0109256 Al
`
`4
`
`

`

`Patent Application Publication May 25,2006 Sheet 4 of 23
`
`US 2006/0109256 Al
`
`"ON
`
`16
`
`
`
`5
`
`

`

`Patent Application Publication May 25,2006 Sheet 5 of 23
`
`US 2006/0109256 Al
`
`
`
`
`SSA———<5 l
`
`
`
`4 S
`
`6
`
`

`

`Patent Application Publication May 25,2006 Sheet 6 of 23
`
`US 2006/0109256 Al
`
`Force
`
`Position
`
`FIG. 9
`
`—— Button Down Force
`
`Characteristic Push Button Behavior
`
`— — Button Release Force
`
`
`
`PushButtonForce
`
`Push Button Displacement
`
`FIG.10.
`
`7
`
`

`

`Patent Application Publication May 25,2006 Sheet 7 of 23
`
`US 2006/0109256 Al
`
`Fié.
`
`it
`
`JLe
`LS
`
`Press Down
`
`Release
`
`Fig. 12
`
`1
`
`+
`
`PRESL Cowen
`
`| Reve ase
`
`Fu.
`
`(3
`
`8
`
`

`

`Patent Application Publication May 25,2006 Sheet 8 of 23
`
`US 2006/0109256 Al
`
`meshhay - Scroll bar
`
`eillx|
`Fema
`
`ES
`
`Online travel companies want your business and they're working hard to getit. It seems every few months,
`
`at least one of the major Canadian players —
`
`(travelocity.ca, expedia.ca, itrayel2000.com, exitnow.ca) — @
`
`FIG. 4
`
`9
`
`

`

`Patent Application Publication May 25, 2006 Sheet 9 of 23
`
`US 2006/0109256 Al
`
`IS!
`
`SENSE
`ComTAcT
`
`
`
`
`
`
`pov
`
`TOUCH SURFACE
`
`i
`:
`
`10
`
`

`

`Patent Application Publication May 25,2006 Sheet 10 of 23
`
`US 2006/0109256 Al
`
`(6 |
`
`
`
`
`sSociATe
`
`
`LocATioNn OF
`Caasom WITH
`ConTACT
`LocRhtiOw
`
`
`
`
`
`
`PRownidE |
`
`C mmensuRATE
`
`
`naptic. FEEDER
`
`
`
`162
`
`
`
`11
`
`

`

`Patent Application Publication May 25,2006 Sheet 11 of 23
`
`US 2006/0109256 Al
`
`Fle. 7
`
`
`
`12
`
`12
`
`

`

`Patent Application Publication May 25,2006 Sheet 12 of 23
`
`US 2006/0109256 Al
`
`
`
`
`
`13
`
`

`

`Patent Application Publication May 25, 2006 Sheet 13 of 23
`
`US 2006/0109256 Al
`
`
`
`(a
`
`
`Associ ATS
`
`LocATIoN OF
`
`
`} Cu@soR WT
`
`
`LOcnhTlos Ot.
`
`
`Cos TAC
`
`
`
`(A
`
`‘HAP TIC
`reDBACK
`
`
`—
`.
`|
`PREDETRAWED
`
`RELATION MET?
`
`
`
`
`
`KO HAPTIC
`FEEDS Ack
`
`14
`
`

`

`US 2006/0109256 Al
`
`Patent Application Publication May 25, 2006 Sheet 14 of 23
`
`FIG.20
`
`15
`
`

`

`ication May 25, 2006 Sheet 15
`
`of 23
`
`US 2006/0109256 Al
`
`FIG2)
`
`
`
`16
`
`

`

`Patent Application Publication May 25, 2006 Sheet 16 of 23
`
` US 2006/0109256 Al
`
`FIG,22
`
`17
`
`17
`
`

`

`122
`
`120
`
`400
`
`Patent Application Publication May 25,2006 Sheet 17 of 23
`
`US 2006/0109256 Al
`
`FIG,23
`
`18
`
`18
`
`

`

`US 2006/0109256 Al
`
`Patent Application Publication May 25, 2006 Sheet 18 of 23
`
`19
`
`19
`
`

`

`Patent Application Publication May 25,2006 Sheet 19 of 23
`
`US 2006/0109256 Al
`
`138
`
`FIG25
`
`20
`
`20
`
`

`

`
`
`21
`
`

`

`Patent Application Publication May 25, 2006 Sheet 21 of 23
`
`US 2006/0109256 Al
`
`quauins
`
`cL8
`
`|zoe
`
`ueIS
`
`
`
`
`
`wyjeubisjoRX}UOOSAIa0ayY
`
`eaowaYy
`
`+v‘Old
`
`OLg
`
`
`
`0}WINJa!0}SBdaIdBjOdMOj}y
`
`
`
`suomsodjeujbuo
`
`bs
`
`
`
`JOJENOES0}JUBLINGBPlADId
`
`“908
`
`22
`
`youbeu099|9MO}|\
`
`
`
`JoujeBo}eaow0}sjueuodWCo
`
`808
`
`
`
`9010}.
`
`
`
`
`
`oneubeweajoeiyeeseasoul
`
`
`
`
`
`9010)BuyseiqjsuieBe-
`
`
`
`jauedaAiSUSS-YONo}
`
`¥08
`
`22
`
`
`
`
`
`

`

`Patent Application Publication May 25, 2006 Sheet 22 of 23
`
`US 2006/0109256
`
`Al
`
`Z
`a
`%
`t
`L
`S
`. &
`-
`
`FIG2e
`
`@
`
`8
`®
`9°
`a
`©
`o
`2
`a
`
`23
`
`23
`
`

`

`Patent Application Publication May 25, 2006 Sheet 23 of 23
`
`US 2006/0109256 Al
`
`
`
`FIG. 30
`
`
`
`FiG. 3
`
`
`
`24
`
`24
`
`

`

`US 2006/0109256 Al
`
`May 25, 2006
`
`HAPTIC FEEDBACK FOR BUTTON AND
`SCROLLING ACTION SIMULATION IN TOUCH
`INPUT DEVICES
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims the benefit of U.S. Provi-
`sional Patent Application Ser. No. 60/616,648 filed Oct. 8,
`2004 in the name of the same inventors and commonly
`assigned herewith.
`
`[0002] This application may be considered to berelated to
`the following prior patents and patent applications: U.S.
`patent application Ser. No. 10/615,986, filed Jul. 10, 2003,
`whichis, in turn, a continuation of U.S. patent application
`Ser. No. 10/213,940,filed Aug. 6, 2002, whichis, in turn, a
`continuation of U.S. patent application Ser. No. 09/487,737,
`filed Jan. 19, 2000, now U.S. Pat. No. 6,429,846, whichis,
`in turn, a continuation-in-part of U.S. patent application Ser.
`No. 09/467,309, filed Dec. 17, 1999, now U.S. Pat. No.
`6,563,487, which is, in turn, a continuation-in-part of U.S.
`patent application Ser. No. 09/156,802, filed Sep. 17, 1998,
`now USS. Pat. No. 6,184,868, which is, in turn, a continu-
`ation-in-part of U.S. patent application Ser. No. 09/103,281,
`filed Jun. 23, 1998, now U.S. Pat. No. 6,088,019, which is,
`in turn, a continuation-in-part of U.S. patent application Ser.
`No. 09/253,132, filed Feb. 18, 1999, now U.S. Pat. No.
`6,243,078, all commonly assigned herewith. This applica-
`tion may also be considered to be related to U.S. patent
`application Ser. No. 09/917,263, filed Jul. 26, 2001, now
`USS. Pat. No. 6,822,635 (based on U.S. Provisional Patent
`Application Ser. No. 60/274,444, filed Mar. 9, 2001); U.S.
`patent application Ser. No. 10/213,354, filed Aug. 5, 2002,
`now abandoned; U.S. patent application Ser. No. 10/919,
`648, filed Aug. 17, 2004, now pending; U.S. patent appli-
`cation Ser. No. 10/919,798,filed Aug. 17, 2004, now pend-
`ing; PCT/US01/01486,filed Jan. 17, 2001; and PCT/US02/
`17102, filed Mar. 8, 2002. All of the foregoing U.S. patents
`and applications are hereby incorporated herein by reference
`as if set forth fully herein.
`
`BACKGROUND OF THE INVENTION
`
`[0003] The present invention relates generally to the inter-
`facing with computer and mechanical devices by a user, and
`moreparticularly to devices used to interface with computer
`systems and electronic devices and which provide haptic
`feedback to the user.
`
`[0004] Humansinterface with electronic and mechanical
`devices in a variety of applications, and the need for a more
`natural, easy-to-use, and informative interface is a constant
`concern. In the context of the present invention, humans
`interface with computer devices for a variety of applications.
`Onesuch application is interacting with computer-generated
`environments such as are found in, for example, games,
`simulations, and application programs.
`
`In someinterface devices, force feedbackortactile
`[0005]
`feedback is also provided to the user, collectively known
`herein as “haptic feedback.” For example, haptic versions of
`joysticks, mice, game pads, steering wheels, or other types
`of devices can output forces to the user based on events or
`interactions occurring within the computer-generated envi-
`ronment, such as a graphical environment found in a game,
`simulation or other application program.
`
`In portable computer or electronic devices, such as
`[0006]
`laptop computers, moveable mouse-type position encoding
`input device often require too large a workspace to be
`practical. As a result, more compact devices such as track-
`balls are often used. A more popular device for portable
`computers are “touchpads,” which are usually embodied as
`small rectangular, planar pads provided near the keyboard of
`the computer. Touchscreens are also used and becoming
`more popular. Touchpads do not
`incorporate an integral
`display device—touchscreens do. Such touch input devices
`sense the location of a pointing object (such as a user’s
`finger or an input stylus) by any of a variety of sensing
`technologies, such as capacitive sensors,
`infrared light
`beams, pressure sensors that detect pressure applied to the
`touch input device, and the like. In a common application the
`user contacts the touch input device with a fingertip and
`moveshis or her finger on the surface of the control to move
`a cursor displayed in the graphical environmentorto select
`a displayed element. In other applications, a stylus may be
`used instead of a finger.
`
`[0007] One problem with existing touch input devices is
`that there is no haptic feedback provided to the user. The
`user of a touchpadis therefore not able to experience haptic
`sensations that assist and inform the user of targeting and
`other control tasks within the graphical environment. The
`touch input devices of the prior art also cannot take advan-
`tage of existing haptic-enabled software run on the portable
`computer.
`
`SUMMARY OF THE INVENTION
`
`[0008] The present invention is directed to a haptic feed-
`back planar touch input device used to provide input to a
`computer system. The touch input device can be a touchpad
`provided on a portable computer, or it can be a touch screen
`found on a variety of devices, or it may be implemented with
`similar input devices. The haptic sensations output on the
`touch input device enhance interactions and manipulations
`in a displayed graphical environment or when using the
`touch input device to control an electronic device.
`
`[0009] Morespecifically, the present invention relates to a
`haptic feedback touch input device for inputting signals to a
`computer and for outputting forces to a user of the touch
`input device. The touch input device includes an approxi-
`mately planar (planar or near-planar) touch surface operative
`to input a position signal to a processor of said computer
`based on a location of user contact on the touch surface. The
`
`position signal may be used in a number of ways, for
`example, it may be used to position a cursor in a graphical
`environment displayed on a display device basedat least in
`part on the position signal.
`It may be used to rotate,
`reposition, enlarge and/or shrink an image of an object
`displayed on a display device based at least in part on the
`position signal. It may be used to provide other desired
`inputs to a computing device. These inputs may include
`scroll-inputs causing text or displayed images to move up,
`down,rightor left, to rotate, or to be madelarger or smaller
`in the graphical environment. At least one actuator is also
`coupled to the touch input device and outputs a force on the
`touch input device to provide a haptic sensation to the user
`contacting the touch surface. The actuator outputs the force
`based on force information output by the processor to the
`actuator. Most touch input devices also will
`include an
`ability to measure the relative pressure applied to the touch
`
`25
`
`25
`
`

`

`US 2006/0109256 Al
`
`May 25, 2006
`
`input device while touchingit and that relative pressure may
`also be used for control and may beusedatleast in part to
`create haptic output to the user.
`
`[0010] The touch input device can be a touchpad separate
`from a display screen of the computer, or can be included in
`a display screen of the computeras a touch screen. The touch
`input device can be integrated in a housing of the computer
`or handheld device, or provided in a housingthat is separate
`from the computer. The user contacts the touch surface with
`a finger, a stylus, or other object. The actuator can include
`a piezo-electric actuator, a voice coil actuator, a pager motor,
`a solenoid, or other type of actuator. In one embodiment, the
`actuator is coupled between the touch input device and a
`grounded surface. In another embodiment, the actuator is
`coupled to an inertial mass. The actuator may be coupled to
`cause relative movement between a display screen and a
`transparent touch input panel disposed over the display
`screen in a touch screen device. A touch device micropro-
`cessor which maybe separate from the main processorof the
`computer can receive force information from the host com-
`puter and provide control signals based on the force infor-
`mation to control the actuator.
`
`[0017] FIG. 2 is a perspective view of a remote control
`device including the touchpad of the present invention;
`
`[0018] FIG.3 is a perspective view ofa first embodiment
`of the touchpad of the present invention including one or
`more actuators coupled to the underside of the touchpad;
`
`[0019] FIG.4 is a side elevational view ofa first embodi-
`ment of the present
`invention in which a piezo-electric
`actuator is directly coupled to the touchpad of the present
`invention;
`
`[0020] FIG. 5 is a side elevational view of a second
`embodimentof the touchpadofthe present invention includ-
`ing a linear actuator;
`
`[0021] FIG.6 is a side elevational view of a third embodi-
`ment of the touchpad of the present invention having an
`inertial mass;
`
`[0022] FIG. 7 is a top plan view of an example of a
`touchpad of the present invention having different control
`regions;
`
`[0023] FIGS. 8A and 8B are top plan and side cross
`sectional views, respectively, of a touch screen embodiment
`of the present invention;
`
`[0025] FIG.10 is a plot of push button force versus push
`button displacementillustrating hysteresis in a typical push
`button;
`
`[0011] The haptic sensations, such as a pulse, vibration, or
`spatial texture, may be output in accordance with an inter-
`[0024] FIG.9 isaplot of force versusposition illustrating
`action between a user controlled location and a graphical
`a force profile typical of a conventional snap-type button;
`object in the graphical environment. The touch input device
`can include multiple different regions, where at least one of
`the regions providesthe position signal andat least one other
`region provides a signal that is used by the computer to
`control a different function, such as rate control function of
`a value or a button press. Different regions and borders
`between regions can be associated with different haptic
`sensations. Alternatively, rate control may be established
`through a magnitude of the touch force applied by the user.
`For example, more force could be used to increase the rate
`input and less force could be used to decrease it.
`
`[0026] FIG.11 is a plot illustrating a combined sawtooth
`waveform;
`
`[0027] FIG. 12 is a plotillustrating a single pulse wave-
`form in one direction on press down (left plot) followed by
`a single pulse in the opposite direction on press up (right
`plot);
`
`invention advantageously provides
`[0012] The present
`haptic feedback to a planar touch control device of a
`computer, such as a touchpad or touch screen. The haptic
`feedback can assist and inform the user of interactions and
`events within a graphical user interface or other environment
`and ease cursor targeting tasks. Furthermore, the invention
`allows portable computer devices having such touch con-
`trols to take advantage of existing haptic feedback enabled
`software. The haptic touch devices disclosed herein may
`also be produced so that they are inexpensive, compact and
`consumelow power, allowing them to be easily incorporated
`into a wide variety of portable and desktop computers and
`electronic devices.
`
`[0013] These and other advantages of the present inven-
`tion will become apparent to those skilled in the art upon a
`reading of the following specification of the invention and a
`study of the several figures of the drawing.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0014] The accompanying drawings, which are incorpo-
`rated into and constitute a part of this specification, illustrate
`one or more embodiments of the present invention and,
`together with the detailed description, serve to explain the
`principles and implementations of the invention.
`
`[0015]
`
`In the drawings:
`
`[0016] FIG. 1 is a perspective view ofa haptic touchpad
`of the present invention;
`
`[0028] FIG. 13 is a plot of a sawtooth type of single
`discontinuity waveform;
`
`[0029] FIG.14 is a diagram of a screen image showing a
`scroll bar;
`
`[0030] FIG. 15 is a flow diagram showing a method for
`simulating a button press using haptic feedback imparted
`through a touch surface;
`
`[0031] FIG. 16 is a flow diagram showing a method for
`providing haptic feedback representative of the extent to
`which an action triggered by manipulation of a cursor
`relative to a graphical object displayed on a display screen
`is occurring;
`
`[0032] FIG.17. is a perspective view of video poker game
`using a slider switch having haptic feedback;
`
`[0033] FIG. 18 is a flow diagram showing a method for
`providing haptic feedback in response to a manipulation of
`a graphical object;
`
`[0034] FIG. 19 is a flow diagram showing a method for
`providing haptic feedback representative of the relative
`location of a cursor and a graphical object displayed on a
`display screen;
`
`[0035] FIG. 20 is an elevational diagram illustrating an
`actuator for providing haptic effects in accordance with one
`embodimentof the present invention;
`
`26
`
`26
`
`

`

`US 2006/0109256 Al
`
`May 25, 2006
`
`FIG.21is an elevational diagram illustrating alter-
`[0036]
`native electromagnetic components for generating attractive
`magnetic force in an actuator in accordance with one
`embodiment of the present invention;
`
`FIG.22 is an elevational diagram of an alternative
`[0037]
`embodiment of an actuator in accordance with the present
`invention;
`
`[0038] FIG. 23 is an elevational diagram of another
`embodiment of an actuator in accordance with the present
`invention;
`
`[0039] FIG. 24 is an elevational diagram of a system
`employing an actuator in accordance with one embodiment
`of the present invention;
`
`[0040] FIG. 25 is an elevational diagram illustrating a
`second equilibrium position of an actuator in accordance
`with one embodiment of the present invention;
`
`FIG. 26 is a front perspective diagram of a system
`[0041]
`configured with a plurality of actuators in accordance with
`one embodiment of the present invention;
`
`FIG.27is a flow diagram illustrating a method for
`[0042]
`generating haptic effects in accordance with one embodi-
`ment of the present invention;
`
`[0043] FIG. 28 is a block diagram illustrating a system
`having an actuator in accordance with one embodiment of
`the present invention; and
`
`FIGS. 29, 30 and 31 are diagramsillustrating areas
`[0044]
`of a touch input device which may be used for particular
`inputs.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`invention are
`present
`the
`[0045] Embodiments of
`described herein in the context of a system of touch input
`devices with haptic feedback. Sometimesthese are referred
`to herein as touch control devices. Those of ordinary skill in
`the art will realize that the following detailed description of
`the present invention is illustrative only and is not intended
`to be in any waylimiting. Other embodimentsof the present
`invention will readily suggest themselves to such skilled
`persons having the benefit of this disclosure. Reference will
`now be madein detail to implementations of the present
`invention as illustrated in the accompanying drawings. The
`same reference indicators will be used throughout the draw-
`ings and the following detailed description to refer to the
`same or like parts.
`
`In the interest of clarity, not all of the routine
`[0046]
`features of the implementations described herein are shown
`and described. It will, of course, be appreciated that in the
`developmentof any such actual implementation, numerous
`implementation-specific decisions must be madein order to
`achieve the developer’s specific goals, such as compliance
`with application- and business-related constraints, and that
`these specific goals will vary from one implementation to
`another and from one developer to another. Moreover,it will
`be appreciated that such a development effort might be
`complex and time-consuming, but would nevertheless be a
`routine undertaking of engineering for those of ordinary skill
`in the art having the benefit of this disclosure.
`
`In accordance with the present invention, the com-
`[0047]
`ponents, process steps, and/or data structures may be imple-
`mented using various types of operating systems, computing
`platforms, computer programs, and/or general purpose
`machines. In addition, those of ordinary skill in the art will
`recognize that devices of a less general purpose nature, such
`as hardwired devices,
`field programmable gate arrays
`(FPGAs), application specific integrated circuits (ASICs), or
`the like, may also be used without departing from the scope
`and spirit of the inventive concepts disclosed herein.
`
`[0048] FIG.1 isa perspective view of a portable computer
`10 including a haptic touchpad of the present invention.
`Computer 10 is preferably a portable or “laptop” computer
`that can be carried or otherwise transported by the user and
`maybe poweredbybatteries or other portable energy source
`in addition to other more stationary power sources. Com-
`puter 10 preferably runs one or more host application
`programs with which a user is interacting via peripherals.
`Some display devices 12 for computers are display-only
`devices—in other cases the display devices incorporate a
`touch-sensitive surface and may themselves be used for
`touch input. Such screens are frequently seen in kiosks,
`automatic teller machines, automated vending machines of
`various types, and thelike.
`
`[0049] Computer 10 may include the various input and
`output devices as shown, including a display device 12 for
`outputting graphical images to the user, a keyboard 14 for
`providing character or toggle input from the user to the
`computer, and a touchpad 16 of the present
`invention.
`Display device 12 can be any of a variety of types of display
`devices; flat-panel displays are most common on portable
`computers. Display device 12 can display a graphical envi-
`ronment 18 based on application programs and/or operating
`systemsthat are running, such as a graphical user interface
`(GUD, that can include a cursor 20 that can be moved by
`user input, as well as windows 22,
`icons 24, and other
`graphical objects well known in GUI environments. Other
`devices mayalso be incorporated or coupled to the computer
`10, such as storage devices (hard disk drive, DVD-ROM
`drive, and the like), network server or clients, game con-
`trollers, and the like. In alternate embodiments, the com-
`puter 10 can take a wide variety of forms, including com-
`puting devices that rest on a tabletop or other surface,
`stand-up arcade game machines, automatic teller machines
`(ATMs),
`automatic vending machines, other portable
`devices or devices worn on the person, handheld or used
`with a single handofthe user, and the like. For example, host
`computer 10 can be a video game console, personal com-
`puter, workstation, a television “set top box”or a “network
`computer’, or other computing or electronic device.
`
`[0050] Touchpad device 16 of the present invention pref-
`erably appears externally to be similar to the touchpads of
`the prior art. Pad 16 includes a planar, rectangular smooth
`surface that can be positioned below the keyboard 14 on the
`housing of the computer 10, as shown, or may be positioned
`at other areas of the housing. When the user operates the
`computer 10, the user may conveniently place a fingertip or
`other object on the touchpad 16 and movethe fingertip to
`correspondingly move cursor 20 in the graphical environ-
`ment 18.
`
`In operation, the touchpad 16 inputs coordinate
`[0051]
`data to the main microprocessor(s) of the computer 10 based
`
`27
`
`27
`
`

`

`US 2006/0109256 Al
`
`May 25, 2006
`
`on the sensed location of an object on (or near) the touchpad.
`As with many touchpadsof the prior art, touchpad 16 can be
`capacitive, resistive, or use any appropriate type of sensing.
`Some existing touchpad embodiments are disclosed, for
`example, in U.S. Pat. Nos. 5,521,336 and 5,943,044. Capaci-
`tive touchpadstypically sense the location of an object on or
`near the surface of the touchpad based on capacitive cou-
`pling between capacitors in the touchpad and the object.
`Resistive touchpadsare typically pressure-sensitive, detect-
`ing the pressureofa finger, stylus, or other object against the
`pad, where the pressure causes conductive layers, traces,
`switches, and the like in the pad to electrically connect.
`Someresistive or other types of touchpads can detect the
`amount of pressure applied by the user and can use the
`degree of pressure for proportional or variable input to the
`computer 10. Resistive touchpads typically are at
`least
`partially deformable, so that when a pressure is applied to a
`particular location,
`the conductors at
`that
`location are
`brought into electrical contact. Such deformability can be
`useful
`in the present
`invention since it can potentially
`amplify the magnitude of output forces such as pulses or
`vibrations on the touchpad as used for haptic output in the
`present invention. Forces can be amplified if a tuned com-
`pliant suspension is provided between an actuator and the
`object that is moved, as described in U.S. Pat. No. 6,680,
`729, which is herebyincorporated herein by reference as if
`set forth fully herein. Capacitive touchpads and other types
`of touchpadsthat do not require significant contact pressure
`may be better suited for the present invention in many
`embodiments, since excessive pressure on the touchpad may
`in some cases interfere with the motion of the touchpad for
`haptic feedback. Other types of sensing technologies can
`also be used in the touchpad. Herein, the term “touchpad”
`preferably includesthe surface of the touchpad 16 as well as
`any sensing apparatus (including software and/or firmware
`associated therewith) included in the touchpad unit.
`[0052] Touchpad 16 preferably operates similarly to exist-
`ing touchpads, where the speed of the fingertip on the
`touchpad correlates to the distance that a cursor
`(for
`example)
`is moved in the graphical environment. For
`example, if the user moves his or her finger quickly across
`the pad, the cursor is moved a greater distance than if the
`user moves the fingertip more slowly. If the user’s finger
`reaches the edge of the touchpad before the cursor reaches
`a desired destination in that direction, then the user can
`simply movehis or her finger off the touchpad, reposition
`the finger away from the edge, and continue moving the
`cursor. This is an “indexing” function similar to lifting a
`mouse off a surface to change the offset between mouse
`position and cursor. Furthermore, many touchpads can be
`provided with particular regions that are each assigned to
`particular functions that can be unrelated to cursor position-
`ing. Such an embodimentis describedin greater detail below
`with respect to FIG. 7. In some embodiments the touchpad
`16 may also allow a user to “tap” the touchpad (rapidly
`touch and remove the object from the pad) in a particular
`location to provide a command. For example, the user can
`tap or “double tap” the pad with a finger while the controlled
`cursor is over an icon to select that icon.
`
`In the present invention, the touch input device
`[0053]
`(touchpad 16 or touch screen) is provided with the ability to
`output haptic feedback such as tactile sensations to the user
`whois physically contacting the touch input device. Various
`embodiments detailing the structure of the haptic feedback
`
`touch input device are described in greater detail below.
`Preferably, the forces output on the touch input device are
`linear (or approximately linear (near-linear) ) and oriented
`along the z-axis, perpendicular or approximately (near)
`perpendicular to the surface of the touch input device and a
`surface of computer 10. In a different embodiment, forces
`can be applied to the touch input device to cause side-to-side
`(e.g., X-y) motion of the touch input device in the plane of
`its surface in addition to or instead of z-axis motion,
`although such motion is not presently preferred.
`
`[0054] Using one or more actuators coupled to the touch
`input device, a variety of haptic sensations can be output to
`the user who is contacting the touch input device. For
`example, jolts, vibrations (varying or constant amplitude),
`and textures can be output. Forces output on the touch input
`device can be at least in part based on the location of the
`finger on the touch input device or the state of a controlled
`object in the graphical environmentof the host computer 10,
`and/or independentof finger position or object state. Such
`forces output on the touch input device are considered
`“computer-controlled” since a microprocessoror other elec-
`tronic controller is controlling the magnitude and/or direc-
`tion of the force output of the actuator(s) using electronic
`signals. Preferably, the entire touch input device is provided
`with haptic sensations as a single unitary member; in other
`embodiments, individually-moving portions of the pad can
`each be provided with its own haptic feedback actuator and
`related transmissions so that haptic sensations can be pro-
`vided for only a particular portion. For example, some
`embodiments may include a touch input device having
`different portions that may be flexed or otherwise moved
`with respect to other portions of the touch input device.
`
`In other embodiments, the touch input device can
`[0055]
`be provided in a separate housing that is electrically con-
`nected to a port of the computer 10 via wired or wireless
`means and whichreceives force information from and sends
`position information to the computer 10. For example, a
`number of well-known bus standards such as Universal
`Serial Bus (USB), FirewireEEE 1394), or a standard serial
`bus (RS-232) can connect such a touch input device to the
`computer 10. In such an embodiment, the computer 10 can
`be any desktop or stationary computer or device and need
`not be a portable device.
`
`[0056] One or more buttons 26 can also be provided on the
`housing of the computer 10 to be used in conjunction with
`the touch input device. The user’s hands have easy access to
`the buttons, each of which maybe pressed by the user to
`provide a distinct input signal to the host computer 12. In
`some cases, each button 26 corresponds to a similar button
`found on a more conventional mouse input device, so that a
`left button can be used to select a graphical object (click or
`double click), a right button can bring up a context menu,
`and the like. In other cases a larger plurality of context-
`sensitive physical buttons may be provided aboutthe periph-
`ery of the display with the current indicated function asso-
`ciated with a particular button displayed on the display. In
`some embodiments, one or more of the buttons 26 can be
`provided with tactile feedback as described in U.S. Pat. Nos.
`6,184,868 and 6,563,487. Other features of these disclosures
`mayalso be used with the present invention.
`
`[0057] Furthermore, in some embodiments, one or more
`moveable portions 28 of the housing of the computer device
`
`28
`
`28
`
`

`

`US 2006/0109256 Al
`
`May 25, 2006
`
`10 can be included which is contacted by the user when the
`user operates the touchpad 16 and which portions 28 can
`provide haptic feedback to the user. Structures having a
`moveable portion of a housing for haptic feedback are
`described in U.S. Pat. Nos. 6,184,868 and 6,088,019. Thus,
`both the housing can provide haptic feedback (e.g., through
`the use of an eccentric rotating mass on a motor coupled to
`the housing) and the touchpad 16 can provide separate
`haptic feedback, and the touch screen assembly can provide
`haptic feedback. This allows the host to control multiple
`different tactile sensations simultaneously to the user; for
`example, a vibration of a low frequency can be conveyed
`through the housing to the user and a higher frequency
`vibration can be conveyed to the user through the touchpad
`16. Each other button or other control provided with haptic
`feedback can also provide tactile feedback independently
`from the other controls, if desired.
`[0058] The host application program(s) and/or operating
`system preferably displays graphical images of the environ-
`ment on display device 12 (which may, in one embodiment,
`be a touch screen). The