,I
`
`I'
`
`l I l
`
`1 lIl
`
`\\
`
`PATENT NUI\4BER
`
`6697043
`
`\ r
`
`t{i
`
`,&*
`
`rt{g!3fii6dird*.+*=$k:
`
`ww
`
`_*al,--
`
`-_,.-,'- ,^
`
`.L
`l:lFE+$,- , .
`
`z t
`
`r
`
`tr
`)
`
`6U
`
`JO [
`
`rJ
`u)(')
`
`tJ>o
`
`a
`
`tt
`
`o -
`
`^
`
`f*04
`
`\4
`.9-
`
`U.S. UTILITY Patent Application
`PATENT DATE
`o.t.P.E.
`y'l -^
`-
`.w' r'r
`-
`^l
`/"t^;fro ,,'tl' (!o.^.
`FFR 2 4 ?$04
`
`-tbE
`
`'
`
`-wvv:
`
`APPLICATION NO.
`o9 / 5857 41
`
`CONT/PRIOR
`
`U
`
`cLAss
`3rt5
`
`i:i:.r' :i. l.': r:i:i1"1$rl"r'::t :i ;:it"r
`
`o.Fz
`'o
`=(L
`
`'sUBCLASS
`
`EXAMINER
`
`$"-,,p,y,;t i i' *P I l'*,.:,* '
`,' j* -#>
`-..i1
`
`j
`
`!i\ ,
`
`_,i
`
`i !"
`U'"
`il- ;,-
`
`APF r "'-
`
`/
`
`'...i.';:: i"i :i i:ri'i::. :i.':::'f"r:iri
`
`.
`
`PTO2040
`r2199
`
`ISSUING CLASSIFICATION
`cRoss REFERENCE(S)
`
`CLASS
`
`SUBGLASS (ONE SUBCLASS PER BLOCK)
`
`/
`
`oRtctNAL /
`CLASS l/ SuaCLaSS
`38s A rsz
`I G I/ 5 | oo
`
`TNTEFNATTOT$r- CUSSr FTCATTON
`
`G IJ
`
`{
`
`f-t TERMINAL
`I IDISCLAIMER
`
`f
`
`a
`
`I
`
`f ffte term of this patent
`subsequent to -_--
`has been disclaimed.
`i--i fne term of this patent shall
`nol exlenq oeyonq
`expl
`of U.S Patent. No.
`
`(date)
`
`qare
`
`DRAWTNGS
`
`Figs. Drwg.
`
`Print Fig.
`
`3t
`
`I
`
`, ^ i ^ "- i ^.
`{ul1b I uc
`(Date)'
`
`L-J Continued on lssue Slip InsidSufile Jacket
`
`Print Claim for O.G.
`
`,ff
`'g.{"s ALLowED
`rotalclf
`6f,
`Nor#E eF ALLowANcE MATLED
`, qIXWFY,
`#Jh- ?/ezftv
`rssuE FEE f;/
`' Amount Due
`# #/tu"-
`ln-f l-[n
`tf-?!#
`ISSUE BATCH NUMBER
`
`Date Paid
`
`(Primary Examiner)
`
`L- Tne terminal
`of
`this patent have been disclaimed.
`-months
`
`(Legal Instruments Examine0
`
`(D"ie)
`
`WARNING:
`The information disclosed herein may be restricted. Unauthorized disclosure may be prohibited by the United States.Code Title 35, Sections
`Possession outside the U.S. Patent & Trademark Office is restricted to authorized employees and contractors only. "'
`
`122, 181 and 368.
`
`FTLED wtrH: I OrSr< (CRF) [ rrCHe [,.l Co-norvr
`
`(Aftachod in pocket on right inslde flap)
`
`: Is
`
`It,fr:
`
`q\
`
`1
`
`:;
`
`FORMAT DRAIIIN6S
`
`(FACE)
`
`Form PTO-436A
`(Rev.6/99)
`
`[$$E$sFEEIlU Htt
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 1
`
`

`
`-'ilffiff{#4fififi{hilfiilffi
`
`T'oN
`
`!
`
`}ONTENTS
`,bived
`(lncl. G. of M.).
`or
`Date Mailed
`
`JUNiguu$4.
`
`INITIALS
`
`Date Received
`(lncl. C. of M.)
`or
`Date Mailed
`
`42.
`
`43.
`
`44.
`
`45.
`
`46.
`
`47.
`
`48.
`
`49.
`
`50.
`
`51.
`
`52.
`
`53.
`
`54.
`
`55.
`
`56.
`
`57.
`
`58.
`
`59.
`
`60.
`
`61.
`
`62.
`
`63.
`
`64.
`
`65.
`
`66.
`
`67
`
`68.
`
`69.
`
`71
`
`72.
`
`79.
`
`74.
`
`75.
`
`78.
`
`79.
`
`80.
`
`81
`
`82.
`
`Z.lZ^oz-
`--\:T'da
`z-(Lcn>_
`
`J6-eroa
`iLo-ofto?,
`ln.Ot4R,
`J/J!hv-
`rt l-a{a<
` t *a/1<
`0vat/-03
`ca-ff43
`414ou
`
`-r
`
`2.
`
`3.
`
`4.
`
`5.
`
`;t.
`'11
`"',\
`\
`di{'
`
`'a.. I
`
`1
`
`i 22.
`| 29.
`\ 24.
`25.
`
`l
`
`26.
`
`27.
`, 28.
`29.
`
`30.
`
`31.
`
`l
`
`32.
`, 33.
`34.
`i 35.
`36.
`
`37.
`
`38.
`
`39.'
`', 40.
`
`41.
`
`(LEFT OUTSIDE)
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 2
`
`

`
`PAIENT APPLICAIION SERIAL NO.
`
`U.S. DEPARTMEM OF COMMERCE
`PAIENT AND TRADEMARK OFFICE
`FEE RECORD SHEET
`
`\_\
`
`,{elK/€ffi,,M
`0trF"[lt$i'
`,S:F&*'lffLi-.
`GlrFffr{$f
`
`PTO-1556
`(s/87)
`
`'U.S. GPO: 1999-4 59.082/1 8l {4
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 3
`
`

`
`Page 1 of 1
`
`eomrElrt{ocr ror ltlaarl4a
`V\rcehlnartaf,e DC :02tlt
`ffi.u3pi9.9@v
`
`CONFIRMATION NO.7142
`
`CLASS
`345
`
`GROUP ART UNIT
`2674
`
`ATTORNEY
`DOCKET NO.
`tMM1P097
`
`*Jrrrr:rrrr .Srnames
`mAcrt|qr Arrlx>
`Thrujma*.*nlqGrrrcr
`
`I iltilil il|l ilil illlt ||il llllll lil lill llil llll lill lllil [ ilil
`Bib Data Sheel
`
`FILING DATE
`0610212000
`
`SERIAL NUMBER
`09/585,741
`
`RULE
`APPLTCA.NTS
`Erik J. Shahoian, San Leandro, CA;
`
`:
`
`,* CONTINUING DATA *it***t***r**************
`This appln claims benefit of 60/191 ,333 0312212000
`and claims benef it of 60/182,869 02116/2000
`and claims benefit of 60/172,9531212't11999
`
`'* FOREIGN APPLICATIONS ***i****************
`
`F REQUIRED, FOREIGN FILING LICENSE GRANTED
`'. 0811412000
`'eign Priority claimed
`
`USC 119 (a-d) conditions
`
`t r
`
`ified and
`(nowledoed
`qDDRESS
`JAMES R. RIEGEL
`MMERSION CORPORATION
`301 FOX LANE
`SAN JOSE ,CA 95131
`
`STATE OR
`COUNTRY
`CA
`
`SHEETS
`DRAWING
`23
`
`TOTALw NDEPENDENl
`
`CLAIMS
`A
`
`TITLE
`
`IAPTIC INTERFACE DEVICE AND ACTUATOR ASSEMBLY PROVIDING LINEAR HAPTIC SENSATIONS
`
`li ' ,ntt ru.,
`1.16 Fees ( Filinq )
`
`FtLtNG FEE*
`RECEIVED
`1 890
`
`:EES: Authority has been given in Paper
`tlo.
`to charge/credit DEPOSIT ACCOUNT
`rlo. _
`fo; following:
`f
`
`.', :1.17 Fees ( Processing Ext. of
`:ime )
`: i1.18 Fees ( lssue )
`
`-
`
`Other
`
`Credit
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 4
`
`

`
`66 - b!-* o a
`
`J"e?L.,9
`
`D >"r'A
`
`NT AND
`
`OFF
`
`I uf.l $tr
`.
`lszl
`CERTIFICATE oF EXPRESS MAILING
`I hercbycertt& that this paper and the documents andior fees refened to as attached therein are being deposited with the united states Postal
`Service on June 02, 200b in an envelope as'oExpress Mail Post Offrce to Addressee" service under 37 CFR $1.10, Mailing Label Number
`istant Commissioner for Patents, Washington, DC 20231.
`to
`8L618672286US,
`
`ot
`
`sA-l
`d
`UiE;
`;flB
`
`[g.i=
`
`1.53(b) in the name of inventor:
`
`PATENT APPLICATION TRANSMITTAL (37 C.F.R. $ 1.s3(b))
`
`Assistant Commissioner for Patents
`Box Patent Application
`Washington, DC 20231
`Sir: This is a request for filing a patent application under 37 C.F.R. $
`Erik J. Shahoian
`
`,
`
`FOT: HAPTIC INTERFACE DEVICE AND ACTUATOR ASSEMBLY PROVIDING LINEAR HAPTIC
`SENSATIONS
`
`Please find enclosed:
`
`53 Page(s) of Specification and Claims,
`01 Page of Abstract,
`23 Sheet(s) of informal Drawings,
`2 Pages Combined Declaration and Power of Attomey,
`Preliminary amendment.
`lnformation Disclosure Statement.
`Verified Statement that this filing is by a small entity.
`Assignment of the invention to Immersion Corporation.
`Assignment Recordation Cover Sheet and Assignment recording fee of $40.00.
`other:
`
`XXXX
`
`xX
`
`I
`
`ri*i
`
`t: ir ii
`
`ii ii i:
`
`ii ii i;
`
`:i""'
`
`:i,Ji
`
`ii{ !
`
`;it{:
`
`The fee has been calculated as shown below.
`Number of Without
`@q
`Extra Fee
`
`Present
`Extra
`
`FILING FEE
`TOTAL
`CLAIMS 66 - .20
`INDEP
`-
`CLAIMS 4
`[ ] Multiple Dependent Claim Pr6sent
`and Fee Not Previously Paid
`
`J .
`
`f
`
`3
`
`LARGE ENTITY
`OR RATE FEE
`$690.00
`
`Xl8 = $828.00
`
`X78 = $78.00
`
`s $
`
`1s96.00
`
`SMALL ENTITY
`RATE FEE
`$395.00 0R
`X09=$ OR
`X39=$ OR
`
`$
`
`:.
`
`46
`
`1
`
`TOTAL
`
`(Revised 2/98, Rule 53 Trans.)
`
`a
`
`Page 1 of2
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 5
`
`

`
`Enclosed is our Check No. 14163 - in the amount of $1636.00 to cover the filing fee,
`additional claim fee, and assignment recordation fee.
`If the required fees are missing or any additional fees are required to facilitate filing the
`enclosed application, please charge such fees or credit any overpayment to Deposit Account
`No. 50-0384 (Order No. IMM1P097). A copy of this sheet is enclosed.
`
`.f
`)
`Date: 6/ z/ oo
`
`P.O. Box 52037
`Palo Alto, CA 94303-0746
`Telephone: (408) 467-1900
`
`AttorneY Docket No. IMMIP097
`
`,gi
`
`Registration No. 36,651
`
`;u:.ii
`
`'bi, I
`n:$
`,-pl
`
`,
`
`'is',
`
`ti ii
`
`i!.i
`
`(Revised 2/98, Rule 53 Trans.)
`
`Page2 of 2
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 6
`
`

`
`BY INVENTOR
`ErikJ. Shahoian
`
`CROSS REFERENCE TO RELATED APPLICATIONS
`
`This
`
`091456,887,fr
`
`isa continuation-in-part of co-pending U.S. Patent Application No.
`1999, entitled, "Tactile Mouse Device,"
`7,
`
`and this
`filed December 21, I
`Using a Rotary
`with Rotary Actuator as
`"Abtuator Flexure Module,"
`
`claims the benefit of U.S. Provisional Applications No. 601172,953,
`, entitled, "Haptic Interface Device Providing Linear Tactile Sensations
`" No. 60/182,868, filed February 16, 2000, entitled, "Haptic Device
`ial Mass," and No. 601-,
`filed March 22, 2000, entitled,
`
`all of which are inco
`
`herein by reference in their entirety.
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to interface devices for allowing humans to
`interface with computer systems, and more particularly to low-cost computer interface devices
`that allow the user to provide input to computer systems and allow computer systems to provide
`haptic feedback to the user.
`
`A user can interact with an environment displayed by a computer to perform functions
`and tasks on the computer, such as playing a gune, experiencing a simulation or virtual reality
`\
`environment, using a computer aided design system, operating a graphical user interface (GUI),
`navigate web pages, etc. rCommon human-computer interface devices used for such interaction
`include a mouse, joyslick, trackball, gamepad, steering wheel, stylus, tablet, pressure-sensitive
`sphere, or the like, that is connected to the computer system controlling the displayed
`environment. Typically, the computer updates the environment in response to the user's
`manipulation of a physical manipulandum such as a joystick handle or mouse, and provides
`
`10
`
`\,X /
`ti-:tt /
`:t{}r
`-r--
`r*_-;: I
`liill
`,iisr
`Jl&l
`
`lwi I)
`
`20
`
`25
`
`30
`
`!l'*
`
`i t
`
`i*:l
`
`itffi
`
`iili :i
`li.,:Ji
`d:i
`i*i
`
`'1/?
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 7
`
`

`
`'I
`
`10
`
`ii i: :i
`
`rr'iiil
`
`jiili ls
`
`..i:
`
`ii i
`ii*,
`
`ii"i
`
`ili:i
`
`iii;ii )o
`ti :i
`;iF:r
`
`25
`
`30
`
`35
`
`visual and audio feedback to the user utilizing the display screen and audio speakers. The
`cbmputer senses the user's manipulation of the user object through sensors provided on the
`interface device that send locative signals to the computer. For example, the computer displays a
`cursor or other graphical object in a graphical environment, where the location of the cursor is
`responsive to the motion of the user object. In other applieations, interface devices such as
`remote controls allow a user to interface with the functions of an electronic device or appliance.
`
`In some interface devices, force (kinesthetic) feedback and/or tactile feedback is also
`provided to the user, more generally known collectively herein as "haptic feedback." These types
`of interface devices can provide physical sensations which are felt by the user manipulating a
`user manipulandum of the interface dgvice, such as a joystick handle, mouse, wheel, etc. One or
`more motors or other actuators are coupled to the joystick handle or mouse and are connected to
`the controlling computer system. The computer system controls forces on the joystick or mouse
`in conjunction and coordinated with displayed events and interactions by sending control signals
`or commands to the actuators. The computer system can thus convey physical force sensations to
`the user in conjunction with other supplied feedback as the user is grasping or contacting the
`interface device or manipulatable object of the interface device. For example, when the user
`rhoves the manipulatable object and causes a displayed cursor to interact with a different
`displayed graphical object, the computer can issue a command that causes the actuator to output a
`force on the physical object, conveying a feel sensation to the user.
`
`One problem with current haptic feedback controllers in the home consumer market is the
`high manufacturing cost of such devices, which makes the devices expensive for the consumer.
`A large part of this manufacturing expense is due to the inclusion of complex and multiple
`actuators and corresponding control electronics in the haptic feedback device. In addition, high
`quality mechanical and force transmission components such as linkages and bearings must be
`provided to accurately transmit forces from the actuators to the user manipulandum and to allow
`accurate sensing of the motion of the user object. These components are complex and require
`greater precision in their manufacture than many of the other components in an interface device,
`and thus further add to the cost of the device.
`
`'3ome
`
`low cost haptic devices exist, such as the vibrotactile gamepads for console game
`systems and personal cprnputers, e.g. the Sony DualShock or Nintendo Rumble Pack. These
`devices generate tactile sensations by including a motor having a rotating shaft and an inertial
`mass connected to'the shaft at an off-center point of the mass. The inertial mass is rotated
`around the motor shaft with respect to the interface device at various speeds. This can create
`sinusoidal force signals at various frequencies depending upon the current driven through the
`motor. The problem with such a methodology is slow response time because the spinning mass
`
`Docket No. IMMIP097
`
`4,
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 8
`
`

`
`rhust accelerate and decelerate over time to achieve the rotational velocity corresponding to a
`desired frequency output. Also, this implementation applies forces in a continually changing
`direction confined to a plane of rotation of the mass, providing a "wobble" sensation. This can
`be particularly disconcerting to the user at slow frequencies and, in many embodiments, may be
`unsuitable for use with devices like a mouse, which also provide input in a plane that may
`overlap with the plane in which forces are exerted.
`
`A need therefore exists for a haptic feedback device that is lower in cost to manufacture
`yet offers the user compelling haptic feedback to enhance the interaction with computer
`applications.
`
`10
`
`ii j,"ii
`rii, ;i
`;ii$r
`,iiii
`i:;r':i
`
`'ttrj
`
`ii...i,,
`il--"'
`
`i ;
`
`i,..,ri
`
`ii:: i
`,*r
`
`ii
`
`jj
`
`uI
`
`Docket No. IMMIP097
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 9
`
`

`
`SUMMARY OF THE INVENTION
`
`The present invention is directed toward an actuator assembly and an interface device
`including such an assembly that provides haptic sensations to a user. Inertial and/or contact
`forces are applied to a user with a low-cost actuator and mechanical structure, which allows a
`low-cost force feedback device to be produced.
`
`More particularly, a haptic feedback interface device of the present invention is coupled
`to a host computer implementing a host application program and is manipulated by a user. The
`interface device includes a housing that is physically contacted by the user, a sensor device
`detecting said manipulation of said interface device by the user, and an actuator assembly that
`provides output forces to the user as haptic sensations.
`
`In one embodiment, the actuator assembly includes an actuator that outputs a rotary force,
`and a flexure coupling the actuator to the device housing. The flexure is a unitary member and
`includes a plurality of flex joints allowing a portion of the flexure to be approximately linearly
`moved. The flexure converts the rotary force output by the aotuator to the linear motion, where
`the linear motion causes a force that is transmitted to the user. Preferably, the linear motion is
`provided approximately along an axis that is perpendicular to a planar workspace in which the
`interface device may be moved by the user. In some embodiments, a portion of the flexure is
`coupled to an inertial mass so that the inertial mass is linearly moved when the actuator outputs
`the rotary force, where an inertial force caused by the inertial mass is transmitted to the user
`through the housing.
`
`10
`
`rifii
`
`it ii"d
`jr;i
`
`;';' 15
`
`'i,ii
`
`ii-t
`
`ia lt i:
`
`rtI:::.t ,n
`
`;h::j:
`
`25
`
`In another embodiment, the actuator assembly includes an actuator which outputs a force,
`and a mechanism coupling the actuator to the device housing, where the mechanism allows the
`actuator to be moved with respect to the device housing. The actuator acts as an inertial mass
`when in motion to provide an inertial force that is transmitted to the user. The mechanism can be
`a flexure including at least one flex joint or a mechanism with bearings, and the actuator can
`output Frotary force. The actuator can approximately linearly move along a z-axis substantially
`perpendicular to an x-y.plane in which the user can move a manipulandum of the interface
`device. A method of the present invention similarly outputs a force from an actuator to move the
`30 actuator and provide liaptic sensations to the user of the interface device.
`
`In some embodiments, the mechanism or flexure is coupled to a moveable contact
`member which moves into physical contact with the user when said user is normally operating
`the interface device. For example, the contact member can include a cover portion that is at least
`
`4 f
`
`)
`
`Docket No. IMMIP097
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 10
`
`

`
`L-i
`
`10
`
`ii;
`
`tl ttrl
`il i: .i
`
`iii:..:l
`
`li;::i l5
`
`'u,'4
`
`irF,t
`
`itnsr
`
`ix
`
`ti'!'j
`
`!!i: :l
`,*F!i ii
`Iili ::iiti 20
`li,J
`ti ii
`
`a,portion of a top surface of the interface device. The actuator can be driven bi-directionally to
`provide an output force that produces pulse or vibration sensations to the user. The flexure can
`include at least one stop to prevent motion of an actuator shaft of the actuator past a desired
`fraction of a full revolution.
`
`Preferably, the interface device is a handheld interface device, such as a mouse, gamepad,
`or remote control device. The linear motion can be conelated with a graphical representation
`displayed by the host computer, where a position of a mouse in the planar workspace corresponds
`with a position of a cursor displayed in the graphical representation. The linear motion provides
`a pulse conelated with the interaction of a user-controlled cursor with a graphical object
`displayed in a graphical user interfacb. The linear motion can be included in a force sensation,
`such as a pulse, vibration, or texture force. Th-e actuator preferably outputs the forces in response
`to commands or signals received by the interface device from the host computer.
`
`The present invention advantageously provides a haptic feedback device that is
`significantly lower in cost than other types of haptic feedback devices and is thus well-suited for
`home consumer applications. One or more low-cost actuator assemblies of the present invention
`can Ue provided that apply a force in a particular degree of freedom, such as a Z-axis
`perpendicular to a support surface. A flexure is used is some embodiments to provide long-
`lasting and effective haptic sensations, and in some embodiments the actuator itself can be used
`as an inertial mass for inertial haptic sensations, saving cost and assembly time.
`
`These and other advantages of the present invention will become apparent to those skilled
`in the art upon a reading of the following specification of the invention and a study of the several
`figures of the drawing.
`
`Docket No. IMM|P097
`
`I
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 11
`
`

`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIGURE 1 is a perspective view of system including a haptic interface device of the
`present invention connected to a host computer;
`
`FIGURE 2 is a side cross sectional view of a mouse embodiment of the haptic interface
`device of Fig. I that provides inertial forces to the user;
`
`FIGURES 3a and 3b are pei'spective and side elevational views, respectively, of one
`embodiment of an actuator assembly suitable for use with the present invention;
`
`FIGURE 3c is a side elevational view of the actuator assembly of Figs. 3a and 3b in a
`flexed position;
`
`FIGURE 3d is a side elevational view of a second embodiment of the actuator assembly
`of the present invention providing a moving inertial mass;
`
`FIGURE 4 is a side cross sectional view of the mouse interface device of Fig. 2 that
`additionally provides contact forces to the user;
`
`FIGURES 5a-5c are perspective views of a third embodiment of the actuator assembly of
`the present invention;
`
`IIT, 10
`
`'\iJt
`ii;Rrli ii
`
`'li--
`llr
`
`,iii,
`
`I )
`
`it"r:i
`
`FIGURE 6 is a schematic diagram of a fourth embodiment of the actuator assembly of the
`present invention in which the actuator is moved as an inertial mass;
`
`20
`
`FIGURES 7a-7gare perspective views of a first embodiment of the actuator assembly of
`
`Fig.6;
`
`FIGURE 8 is a side cross sectional view of a mouse embodiment of the haptic interface
`devicdof Fig. 2 including a second embodiment of the actuator assembly of Fig. 6;
`
`FIGURES 9a-96and qc are perspective and top plan views of the actuator assembly used
`in the device of Fig.8;
`
`25
`
`FIGURE 10a is a perspective view of a flexure for use with a third embodiment of the
`actuator assembly of Fig. 6;
`
`Docket No, IMMIP097
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 12
`
`

`
`FIGURES 10b and 10c are perspective and top plan views of the third embodiment of the
`actuator assembly of Fig. 6;
`
`FIGURES 11a-1lb are exploded views of a fourth embodiment of the actuator assembly
`
`of Fig.
`
`FIGURES l2a and l}b are views of the actuator assembly shown in Figs. 1 1a-1 lb;
`
`FIGURE 13 is a block diagram illustrating an embodiment of the haptic interface device
`and host computer for use with the present invention; and
`
`FIGURE 14 is a representation of a graphical user interface with elements providing
`haptic feedback implemented by the present invention.
`
`ii*i l0
`;rfi
`I iiel
`
`rli:4r
`
`rlii ;i
`'*,rl
`
`ii l
`
`ll""1l
`
`riii4r
`'il: '
`iI-:i
`
`li ii"li
`
`Docket No. IMMIP097
`
`,{
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 13
`
`

`
`i.
`
`10
`
`iL#
`
`!!:i?iiili
`
`1 <
`
`:tr
`ll"*
`
`iir"ir:i:. 20
`
`tilli
`
`25
`
`30
`
`35
`
`DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
`
`FIGURE l isaperspectiveviewof ahapticfeedbackmouseinterfacesystem 10of the
`present invention capable of providing input to a host computer based on the user's manipulation
`of the mouse and capable of providing haptic feedback to the user of the mouse system based on
`events occurring in a program implemented by the host computer. Mouse system 10 includes a
`mouse 12 and a host cornputer 14. It should be noted that the term "mouse" as used herein,
`indicates an object generally shaped to be grasped or contacted by the user and moved within a
`substantially planar workspace (and additional degrees of freedom if available). Typically, a
`mouse is a smooth- or angular-shaped compact unit that snugly fits under a user's hand, fingers,
`and/or palm, but can also be implemented as a grip, finger cradle, cylinder, sphere, planar object,
`etc.
`
`Mouse 12 is an object that is preferably grasped or gripped and manipulated by a user.
`By "grasp," it is meant that users may releasably engage a portion of the object in some fashion,
`such as by hand, with their fingertips, etc. In the described embodiment, mouse 12 is shaped so
`that a user's fingers or hand may comfortably grasp the object and move it in the provided
`degrees of freedom in physical space. For example, a user can move mouse 12 to provide planar
`two-dimensional input to a computer system to correspondingly move a computer generated
`graphical object, such as a cursor or other image, in a graphical environment provided by
`computer 14 or to control a virtual character, vehicle, or other entity in a game or simulation. In
`addition, mouse 12 preferably includes one or more buttons 16a and 16b to allow the user to
`provide additional commands to the computer system. The mouse 12 may also include
`additional buttons. For example, a thumb button can be included on one side of the housing of
`mouse 12.
`
`Mouse 12 preferably includes an actuator assembly which is operative to produce forces
`on the mouse 12 and haptic sensations to the user. Several embodiments are described herein
`which provide different implementations of the actuator assembly, and which are described in
`greater detail below with reference to Figs.2-I2b.
`
`Mouse 12 rests glra ground surfape 22 wch as a tabletop or mousepad. A user grasps the
`mouse 12 and moves the mouse in a planar workspace on the surface 22 as indicated by arrows
`24. Mouse 12may be moved anywhere onthe ground surface 22,picked up and placed in a
`different location, etc. A frictional ball and roller assembly (not shown) can in some
`embodiments be provided on the underside of the mouse 12 to translate the planar motion of the
`mouse 12 into electrical position signals, which are sent to a host computer 14 over a bus 20 as is
`well known to those skilled.in the art. In other embodiments. different mechanisms and/or
`
`Docket No. IMMIP097
`
`,ri
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 14
`
`

`
`!-t
`
`(..i:
`
`iiI ii
`
`.',.;.
`
`,:. ii
`
`.-i.-..
`
`ti..i:.
`
`iT
`
`rill
`itI::
`
`illl r
`
`iiiqii
`ir....:
`
`ii'":i
`
`electronics can be used to convert mouse motion to position or motion signals received by the
`host computer, as described below. Mouse 12 is preferably a relative device, in which its sensor
`detect a change in position of the mouse, allowing the mouse to be moved over any surface at any
`location. An absolute mouse may also be used, in which the absolute position of the mouse is
`known with reference to a particular predefined workspace.
`
`Mouse 12 is coupled to the computer 14 by a bus 20, which communicates signals
`between mouse 12 and computer 14 and may also, in some prefened embodiments, provide
`power to the mouse 12. Components such as actuator assembly 18 require power that can be
`supplied from a conventional serial port or through an interface such as a USB or Firewire bus.
`In other embodiments, signals can'be sent between mouse 12 and computer 14 by wireless
`transmission/reception. ln some embodiments, the power for the actuator can be supplemented
`or solely supplied by a power storage device provided on the mouse, such as a capacitor or one or
`more batteries. Some embodiments of such are disclosed in Patent No. 5,691,898, incorporated
`herein by reference.
`
`Host computer 14 is preferably a personal computer or workstation, such as a PC
`compatible computer or Macintosh personal computer, or a Sun or Silicon Graphics workstation.
`For example, the comput'er 14 can operate under the WindowsrM , MacOS, Unix, or MS-DOS
`operating system. Alternatively, host computer system 14 can be one of a variety of home video
`game console systems commonly connected to a television set or other display, such as systems
`available from Nintendo, Sega, or Sony. In other embodiments, host computer system 14 can be
`a "set top box" which can be used, for example, to provide interactive television functions to
`users, or a o'network-" or "intemet-computer" which allows users to interact with a local or
`global network using standard connections and protocols such as used for the Internet and World
`Wide Web. Host computer preferably includes a host microprocessor, random access memory
`(RAM), read only memory GOM), input/output (I/O) circuitry, and other components of
`computers well-known to those skilled in the art.
`
`Host computer 14 preferably implements a host application progam with which a user is
`interacting via mouse 12 and other peripherals, if appropriate, and which may include force
`feedback functionality. For example, the host application program can be a video game, word
`processor or spreadsheet,'Web page or browser that implements HTML or VRML instructions,
`scientific analysis prpgram, virtual reality training program or application, or other application
`program that utilizes input of mouse 12 and outputs force feedback commands to the mouse 12.
`Herein, for simplicity, operating systems such as WindowsrM, MS-DOS, MacOS, Linux, Be, etc.
`are also referred to as "application programs." In one preferred embodiment, an application
`program utilizes a graphical user interface (GUI) to present options to a user and receive input
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`Docket No. IMMIP097
`
`i {-'
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 15
`
`

`
`from the user. Herein, computer 14 may be refened as providing a "graphical environment,
`'
`*tti.tt can be a graphical user interface, game, simulation, or other visual environment. The
`computer displays "graphical objects" or o'computer objects," which are not physical objects, but
`are logical software unit collections of data and/or procedures that may be displayed as images by
`5 computer 14 on display screen 26, asis well known to those skilled in the art. A displayed cursor
`or a simulated cockpit of an aircraft might be considered a gaphical object. The host application
`program checks for input signals received from the electronics and sensors of mouse 12, and
`outputs force values and/or commands to be converted into forces output for mouse 12. Suitable
`software drivers which interface such simulation software with computer inpuVoutput (I/O)
`l0 devices are available from Immersion, Corporation of San Jose, California.
`
`15
`
`Display device 26 can be included in host computer 14 and can be a standard display
`screen (LCD, CRT, flat panel, etc.), 3-D goggles, or any other visual output device. Typically,
`the host application provides images to be displayed on display device 26 andlor other feedback,
`such as auditory signals. For example, display screen 26 candisplay images from a GUI.
`As shown in Figure 1, the host computer may have its own o'host frame" 28 which is
`displayed on the display screen 26. In contrast, the mouse 12 has its own workspace or 'olocal
`frame" 30 in which the mouse 12 is moved. In a position control paradigm, the position (or
`change in position) of a user-controlled graphical object, such as a cursor, in host frame 28
`corresponds to a position (or change in position) of the mouse 12 in the local frame 30' The
`20 offset between the object in the host frame and the object in the local frame can be changed by
`the user by indexing, i.e., moving the mouse while no change in input is provided to the host
`computer, such as by lifting the mouse from a surface and placing it down at a different location.
`
`iiti i
`
`,!i::l
`jr.::
`
`i
`:L:;.
`!i'""
`
`,ii;ii
`
`r:rii'
`,;,,.,
`ii ii.i:
`
`rinli
`
`In alternative embodiments, the mouse 12 can instead be a different interface device or
`control device. For example, handheld devices are very suitable for the actuator assemblies
`25 described herein. A hand-held remote control device used to select functions of a television,
`video cassette recorder, sound stereo, internet or network computer (e.g., Web-TVrM)' or a
`gamepad controller for video games or computer games, can be used with the haptic feedback
`components described herein. Handheld devices are not constrained to a planar workspace like a
`mouse but can still benefit from the directed inertial sensations and contact forces described
`30 herein which, for exanple, can be output about perpendicularly to the device's housing surfaces.
`Other interface devices may also make use of the actuator assemblies described herein. For
`example, a joystick handle can include the actuator assembly, where haptic sensations are output
`on the joystick handle as the sole haptic feedback or to supplement kinesthetic force feedback in
`the degrees of freedom of the joystick. Trackballs, steering wheels, styluses, rotary knobs, linear
`
`Docket No. IMMLP097
`
`10
`
`tl
`
`APPLE INC.
`EXHIBIT 1016 - PAGE 16
`
`

`
`sliders, gun-shaped targeting devices, medical devices, grips, etc. can also make use of the
`actuator assemblies described herein to provide haptic sensations.
`
`FIGURE 2 is aside cross-sectional view of a first embodiment 40 of mouse 12 of Fig. 1.
`Mouse 40 includes one or more actuator assemblies for imparting haptic feedback such as tactile
`sensations to the user of the mouse. The actuator assembly outputs forces on the mouse 40
`which the user is able to feel. The embodiment of Fig. 2 is intended to provide inertial forces
`rather than contact forces; contact forces are described with respect to Fig. 4. In some
`embodiments, two or more actuator assemblies can provide inertial forces or contact forces, or
`one actuator assembly can provide inertial forces, while a different actuator assembly can provide
`contact forces.
`
`-sensing
`
`system 52, and an actuator assembly 54.
`Mouse 40 includes a housing 50, a
`Housing 50 is shaped to fit the usef's hand like a standard mouse while the user moves the mouse
`in the planar degrees of freedom and manipulates the buttons 16. Other housing shapes can be
`provided in many different embodiments.
`'
`Sensing system 52 detects the position of the mouse in its planar degrees of freedom, e.g.
`along the X and Y axes. In the described embodiment, sensing system 52 includes a standard
`mouse ball 54 for providing directional input to the computer system. Ball 45 is a sphere that
`extends partially out the bottom surface of the mouse and rolls in a direction corresponding to the
`motion of the mouse on a planar surface 22. For example, when the mouse 40 is moved in a
`direction indicated by arrow 56 (y direction), the ball rotates in place in a direction shown by
`zrrow 58. The ball motion can be tracked by a cylindrical roller 60 which is coupled to a sensor
`62 for detecting the motion of the mouse. A similar roller and sensor 28 can be used for the x-
`direction which is perpendicular to the y-axis.
`
`10
`
`1{
`
`rirrir
`
`rSrql
`rl#:
`!i nl
`'!"i
`
`,ri$
`ii ::
`it*'
`
`i!'"i
`
`riroi
`iili :!
`!i':r ZV
`^n
`if6,i
`It
`
`Other types of mechanisms and/or electronics for detecting planar motion of the mouse
`40 can be used in other embodiments. In some embodiments, high frequency tactile sensations
`can be applied by the actuator that cause a mouse ball 45 to slip with respect to the frictionally
`engaggd rollers. This is problematic, causing the mouse to be less accurate because of the tactile
`sensations. To remedy this problem, a more prefened embodiment employs the actuator
`assembly 54 within arr6ptical mouse that has no moving mouse ball component. A suitable
`optical mouse technology is made by Hewlett Packard of Palo Alto, CA and can be
`advantageously combined with the tactild sensation technologies described herein, where the
`optical sensor detects motion of the mouse relative to the planar support surface by optically
`taking and storin