UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`APPLE INC.,
`Petitioner
`
`v.
`
`IMMERSION CORPORATION
`Patent Owner
`
`U.S. Patent No. 8,773,356
`Filing Date: January 31, 2012
`Issue Date: July 8, 2014
`Title: Method and Apparatus for Providing Tactile Sensations
`
`
`
`Inter Partes Review No.: IPR2017-00897
`
`
`
`
`
`
`
`PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 8,773,356
`UNDER 35 U.S.C. §§ 311-319 AND 37 C.F.R. §§ 42.1-100, ET SEQ.
`
`
`
`
`
`WEST\275294320.2
`
`

`

`Table of Contents
`
`
`Page
`
`
`I.
`II.
`
`INTRODUCTION .......................................................................................... 1
`COMPLIANCE WITH FORMAL REQUIREMENTS ................................. 1
`A. Mandatory Notices Under 37 C.F.R. §§ 42.8(b)(1)-(4) ....................... 1
`1.
`Real Party-In-Interest ................................................................. 1
`2.
`Related Matters .......................................................................... 1
`3.
`Lead and Backup Counsel ......................................................... 1
`4.
`Service Information.................................................................... 2
`Proof of Service on the Patent Owner .................................................. 2
`B.
`Power of Attorney ................................................................................ 2
`C.
`Standing ................................................................................................ 2
`D.
`Fees ....................................................................................................... 2
`E.
`III. STATEMENT OF PRECISE RELIEF REQUESTED .................................. 2
`IV. THIS PETITION IS PROPER UNDER 35 U.S.C. §315(D) ......................... 3
`V.
`FULL STATEMENT OF REASONS FOR REQUESTED RELIEF ............ 4
`A.
`Summary of the ’356 Patent ................................................................. 4
`B.
`The ’356 Patent Prosecution History ................................................... 5
`C. No Claim of the ’356 Patent is Entitled to the Effective Filing
`Date of Any Provisional Application ................................................... 6
`Person of Ordinary Skill in the Art ...................................................... 8
`Claim Construction .............................................................................. 8
`Ground 1: Claims 1-3, 5, 7-13, 15, 17-23, 25-26 Are Obvious
`Under 35 U.S.C. §103(a) (pre-AIA) In Light Of Fukumoto ............... 9
`1.
`Fukumoto is §102(a) Prior Art ................................................. 10
`2.
`Overview of Fukumoto ............................................................ 10
`3.
`Detailed Analysis ..................................................................... 12
`G. Ground 2: Claims 8 and 18 Are Obvious Under 35 U.S.C. §
`103(a) (pre-AIA) In Light of Fukumoto In View Of Roysden .......... 46
`1.
`Roysden is Prior Art under 35 U.S.C. §102(b) ........................ 46
`
`D.
`E.
`F.
`
`
`
`
`
`i
`
`WEST\275294320.2
`
`

`

`Table of Contents
`(continued)
`
`Page
`
`
`
`Detailed analysis for claims 8 and 18 ...................................... 47
`2.
`H. Ground 3: Claims 1-5, 7-15, 17-26 Are Obvious Under 35
`U.S.C. § 103(a) (pre-AIA) In Light Of Tsuji ..................................... 48
`1.
`Tsuji is § 102(b) Prior Art ........................................................ 48
`2.
`Overview of Tsuji .................................................................... 48
`VI. CONCLUSION ............................................................................................. 95
`
`
`
`
`
`ii
`
`WEST\275294320.2
`
`

`

`
`
`EXHIBIT LIST
`
`Exhibit No. Description
`
`1101
`1102
`1103
`
`1104
`1105
`1106
`1107
`1108
`1109
`1110
`1111
`
`1112
`1113
`
`1114
`
`1115
`
`U.S. Patent No. 8,773,356 (“’356 patent”)
`Declaration of Dr. Richard Mihran
`Order 27 in Inv. No. 337-TA-990/1004 (ALJ’s Construction of
`Claim Terms in ’356 Patent) (“Order 27”)
`File History of U.S. Patent No. 8,773,356 (“’356 FH”)
`U.S. Provisional App. No. 60/335,493 (“First Provisional”)
`U.S. Provisional App. No. 60/399,883 (“Second Provisional”)
`International Pat. App. Pub. No. WO 2002/12991 A1
`Certified Translation of WO 2002/12991 A1 (“Fukumoto WO”)
`U.S. Pat. App. Pub. No. 2002/0149561 (“Fukumoto US”)
`Japanese Published Application No. H11-212725
`Certified Translation of Japanese Published Application No. H11-
`212725 (“Tsuji”)
`U.S. Pat. App. Pub. No. US2008/0068350 (“Rosenberg 350”).
`IBM Model M Keyboard Release, April 1986 (available at
`http://www-
`01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_
`ca/8/877/ENUSZG86-
`4008/index.html&lang=en&request_locale=en) (“IBM Release”)
`IBM Model M Keyboard Photograph
`https://upload.wikimedia.org/wikipedia/commons/4/48/IBM_Model
`_M.png
`U.S. Pat. No. 5,575,576 (“Roysden”)
`
`WEST\275294320.2
`
`i
`
`

`

`
`
`I.
`
`INTRODUCTION
`The claims of Immersion’s U.S. Patent No. 8,773,356 (the “’356 patent”)
`
`challenged in this Petition are invalid over the prior art.
`
`II. COMPLIANCE WITH FORMAL REQUIREMENTS
`A. Mandatory Notices Under 37 C.F.R. §§ 42.8(b)(1)-(4)
`1.
`Real Party-In-Interest
`Apple, Inc. is the real party-in-interest.
`
`2.
`Related Matters
`The ’356 patent is or was involved in the following actions: 1) Certain
`
`Mobile Electronic Devices Incorporating Haptics (Including Smartphones and
`
`Smartwatches) and Components Thereof, U.S. International Trade Commission
`
`Investigation No. 337-TA-990; 2) Immersion Corporation v. Apple Inc., et al.,
`
`Case No. 1:16-cv-00077 (D. Del.); 3) IPR2016-00807 (petition denied); and 4)
`
`IPR2016-01381 (instituted).
`
`Lead and Backup Counsel
`
`3.
`Lead counsel is James M. Heintz, Reg. No. 41,828, of DLA Piper LLP (US),
`
`11911 Freedom Drive, Suite 300; Reston, VA 20190; Apple-Immersion-
`
`IPRs@dlapiper.com, 703-773-4148 (phone), 703-773-5200 (fax). Backup counsel
`
`is Brian Erickson, Reg. No. 48,895, of DLA Piper LLP (US), 401 Congress
`
`Avenue, Suite 2500, Austin, TX 78701; brian.erickson@dlapiper.com, 512-457-
`
`7059 (phone).
`
`WEST\275294320.2
`
`1
`
`

`

`
`
`Service Information
`
`4.
`Please address all correspondence to the lead and back-up counsel as shown
`
`above. Apple consents to electronic service to lead and back-up counsel and to
`
`Apple-Immersion-IPRs@dlapiper.com.
`
`Proof of Service on the Patent Owner
`
`B.
`As identified in the attached Certificate of Service, a copy of this Petition in
`
`its entirety is being served to the Patent Owner’s attorney of record at the address
`
`listed in the USPTO’s records by overnight courier pursuant to 37 C.F.R. §42.6.
`
`Power of Attorney
`
`C.
`Powers of attorney are being filed with designation of counsel in accordance
`
`with 37 C.F.R. §41.10(b).
`
`Standing
`
`D.
`In accordance with 37 C.F.R. §42.104(a), Petitioner certifies that the ’356
`
`patent is available for inter partes review and that Petitioner is not barred or
`
`estopped from requesting an inter partes review challenging the patent claims on
`
`the grounds identified in this Petition.
`
`Fees
`
`E.
`The undersigned authorizes the Director to charge the fee specified by 37
`
`C.F.R. §42.15(a) and any additional fees that might be due in connection with this
`
`Petition to Deposit Account No. 50-3266.
`
`
`
`WEST\275294320.2
`
`2
`
`

`

`
`
`III. STATEMENT OF PRECISE RELIEF REQUESTED
`In accordance with 35 U.S.C. §311, Petitioner requests cancelation of claims
`
`1-26 of the ’356 patent in view of the following grounds:
`
`A. Ground 1: Claims 1-3, 5, 9-13, 15, 18-23, 25-26 Are Obvious Under
`35 U.S.C. § 103(a) (pre-AIA) In Light Of Fukumoto
`
`B. Ground 2: Claims 8 and 18 Are Obvious Under 35 U.S.C. § 103(a)
`(pre-AIA) In Light of Fukumoto In View Of Roysden
`
`C. Ground 3: Claims 1-7, 9-17, 19-26 Are Obvious Under 35 U.S.C. §
`103(a) (pre-AIA) In Light Of Tsuji
`IV. THIS PETITION IS PROPER UNDER 35 U.S.C. §315(d)
`The Board should reject any argument by Patent Owner that the Board
`
`should deny institution of this petition under 35 U.S.C. §315(d). First, the
`
`arguments in this Petition address additional claims and entirely different prior art
`
`than IPR2016-01381. See, e.g., IPR2016-00448, Paper 9 at 10 (instituting trial on
`
`second petition because it raised substantively different arguments). Second,
`
`Petitioner has not “overwhelmed Patent Owner with an unreasonable number of
`
`challenges of patentability” because this is only the second petition that Petitioner
`
`has filed regarding the ’356 patent. Id. Third, in order to conserve the resources of
`
`the Board and the private parties, Petitioner limited the petition in IPR2016-01381
`
`to the claims asserted in the ITC investigation. Unfortunately, the ITC action has
`
`not settled and the statutory bar provided under 35 U.S.C. § 315(b) is not limited to
`
`asserted claims. Accordingly, Petitioner files the instant Petition to challenge
`
`WEST\275294320.2
`
`3
`
`

`

`
`
`additional claims in light of the (currently stayed) District Court action. To avoid
`
`such costly litigation, this Petition aligns with the statutory goal of the America
`
`Invents Act (AIA) to provide lower-cost alternatives. The Board has permitted
`
`second petitions under similar circumstances. See e.g., IPR2015-00615, Paper 9 at
`
`24-25; IPR2015-00626, Paper 9 at 15-16.
`
`V.
`
`FULL STATEMENT OF REASONS FOR REQUESTED RELIEF
`A.
`The ’356 patent is directed to products and processes for providing tactile
`
`Summary of the ’356 Patent
`
`sensations, such as vibratory feedback, to input devices such as touchscreens on
`
`Automated Teller Machines (ATMs) and personal digital assistants (PDAs). Ex.
`
`1101, 2:27-29; 3:15-24. The ’356 patent acknowledges that touchscreen input
`
`devices are comprised of a display under a transparent touchpad. Id., 1:64-67.
`
`Tactile feedback is provided by an actuator connected to the input device. Id.,
`
`2:32-34. The ’356 patent also discloses the use of a controller that receives a
`
`signal from the input device and outputs a corresponding signal to the actuator to
`
`create a desired tactile sensation. Ex. 1101, 4:47-5:3. The ’356 patent further
`
`discloses a “table in which input signals are associated with various haptic signals”
`
`such as shown in Fig. 9 below (id., 7:63-8:17):
`
`WEST\275294320.2
`
`4
`
`

`

`
`
`
`
`The ’356 patent discloses that the controller, using “the data obtained from
`
`monitoring the input device, reads the tables and obtains the associated function
`
`and tactile feedback information.” Ex. 1101, 14:26-35; Ex. 1102, ¶77.
`
`The ’356 Patent Prosecution History
`
`B.
`The application for the ’356 patent was filed on January 31, 2012 and
`
`assigned serial number 13/362,113 (the “’113 application”). Ex. 1104, 2. The
`
`’113 application claimed priority to three earlier non-provisional applications and
`
`two provisional applications. Id., 11.
`
`In an office action dated 9/7/13, the twenty originally filed claims were
`
`rejected under 35 U.S.C. §102(b) as being anticipated by the publication of another
`
`WEST\275294320.2
`
`5
`
`

`

`
`
`application owned by Patent Owner, U.S. Pat. Pub. No. US2008/0068350 (Ex.
`
`1112, “Rosenberg 350”). Ex. 1104, 151-53.
`
`The applicants responded to this rejection by amending each claim to
`
`include a requirement that an actuator signal be generated based at least in part on
`
`“haptic effect data in a lookup table” (the “Lookup Table Limitation”). Ex. 1104,
`
`169-73. The applicants argued the Lookup Table Limitation “may be found in the
`
`specification as filed, such as in paragraphs [0043] and [0071] as well as in Figures
`
`9 and 10.” Id., 49. The applicants asserted that the amended claims were
`
`“patentable” because Rosenberg 350 did not disclose the Lookup Table Limitation.
`
`Id., 176. In response to applicants’ argument, the examiner issued a Notice of
`
`Allowance and the ’356 patent subsequently issued.
`
`C. No Claim of the ’356 Patent is Entitled to the Effective Filing Date
`of Any Provisional Application
`
`As established above, the claims of the ’356 patent were amended during
`
`prosecution to recite the Lookup Table Limitation. Ex. 1104, 169-73. Thus, in
`
`order for any claim of the ’356 patent to be entitled to claim priority to an earlier-
`
`filed provisional application, such an earlier filed application must provide 35
`
`U.S.C. §112 written description support for the claims as amended, including the
`
`Lookup Table Limitation. See New Railhead Mfg., L.L.C v. Vermeer Mfg. Co.,
`
`298 F.3d 1290, 1295 (Fed. Cir. 2002). In the remarks accompanying the
`
`amendment, Applicant stated that the subject matter of the Lookup Table
`
`WEST\275294320.2
`
`6
`
`

`

`
`
`Limitation was disclosed in the ’113 application “in paragraphs [0043] and [0071]
`
`as well as in Figures 9 and 10.” Ex. 1104, 49; Ex. 1102, ¶¶123-24.
`
`The ’356 patent claims priority to two provisional applications, U.S.
`
`Provisional App. No. 60/335,493 (Ex. 1105) and U.S. Provisional App. No.
`
`60/399,883 (Ex. 1106). Ex. 1101, 1:17-20. The provisional applications only
`
`name subsets of the five inventors named on the ’356 patent. Ex. 1105, 1; Ex.
`
`1106, 1. The provisional applications state that an unidentified “controller” will
`
`“control the actuator output” but do not mention the controller using haptic effect
`
`data in a lookup table to do so. Ex. 1105, 7; Ex. 1106, 7. The provisional
`
`applications lack any disclosure corresponding to paragraphs [0043], [0071],
`
`Figure 9 or Figure 10, which the applicants relied on during prosecution for
`
`support for the Lookup Table Limitation. Ex. 1104, 49; see generally Ex. 1106;
`
`Ex. 1102 ¶¶125-30.
`
`In its order instituting IPR, the Board agreed with Petitioner that no claim of
`
`the ’356 patent is entitled to the effective filing date of the First Provisional. See
`
`IPR2016-01381, Paper 7, 10-11. Patent Owner did not produce any evidence or
`
`make any assertion that any claim of the ’356 patent is entitled to a date of
`
`invention prior to November 1, 2002. To the extent Patent Owner does so for the
`
`first time in response to this Petition, the Board should resolve all factual
`
`inferences in favor of Petitioner in its institution decision so that this factual issue
`
`WEST\275294320.2
`
`7
`
`

`

`
`
`can be determined at trial on a complete record. See 37 C.F.R. §42.108(c). See
`
`also IPR2016-00908, Institution Decision, Paper 12, 23(“Where conflicting
`
`testimonial evidence creates a genuine issue of material fact, as it does here, the
`
`evidence must be viewed in the light most favorable to Petitioner at this stage of
`
`the proceeding.”).
`
`Person of Ordinary Skill in the Art
`
`D.
`A person of ordinary skill in the art (“POSITA”) at the time of the alleged
`
`invention of the ’356 patent would have had a Bachelors’ degree in computer
`
`science or a comparable field of study, plus approximately two to three years of
`
`professional experience with software engineering, haptics programming, or other
`
`relevant industry experience. Additional graduate education could substitute for
`
`professional experience and significant experience in the field could substitute for
`
`formal education. Ex. 1102 ¶131.
`
`E. Claim Construction
`“Haptic effect data” (claims 1, 10, 12, 20, 22, 25) should be construed to
`
`mean “data identifying or describing a tactile sensation.” This construction has
`
`been adopted in the pending ITC action based on Patent Owner’s request. Ex.
`
`1103, 22-26. The ALJ held this construction is consistent with the specification.
`
`Id., 24-25. This phrase should be construed at least as broadly here. Ex. 1102
`
`¶132.
`
`WEST\275294320.2
`
`8
`
`

`

`
`
`
`
`“Lookup tabble” (claimms 1, 11, 122, 21, 22, 226) shouldd be construued to meaan
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`“data structure in the form oof a table coontaining aassociationns betweenn interactioons
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and happtic effect ddata.” Ex. 1103, 26-330. This cconstructio
`
`
`
`
`
`
`
`
`
`
`
`n has beenn adopted i
`
`n
`
`
`
`consistentt with the
`
`
`
`the pendding ITC aaction. Thee ALJ heldd this consttruction is
`
`
`
`
`
`
`
`
`
`
`
`specificcation. Id.,, 29-30. Thhis phrase should be
`
`
`
`
`
`
`
`construed
`
`
`
`at least ass broadly hhere.
`
`
`
`
`
`Notablyy, Patent OOwner propposed an evven broadeer constructtion, whichh omitted tthe
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`phrase ““in the formm of a tablle.” Ex. 11103, 26; Exx. 1102 ¶1331.
`
`
`
`
`
`
`
`
`
`
`
`
`
`“Standard 1101-key keeyboard” (cclaims 8, 1
`
`
`
`
`
`
`
`8) should
`
`
`
`be construued to meann
`
`
`
`
`
`the stanndard 101 kkey IBM keyboard. TThis claimm phrase dooes not apppear in the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`specificcation. A pperson of oordinary skkill in the aart would uunderstand
`
`
`
`
`
`
`
`
`
`
`
`
`
`a “standarrd
`
`
`
`101-keyy keyboardd” to refer tto the standdard 101 kkey IBM keeyboard, wwhich was
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`introducced in 19866, because it becamee the standaard for PC
`
`
`
`
`
`keyboardss for many
`
`
`
`years. EExs. 1102 ¶134; 11133; 1114 (reeproduced
`
`
`
`
`
`
`
`
`
`below); 11115, 5:7-266.
`
`
`
`94320.2
`WEST\27529
`
`9
`
`
`
`

`

`
`
`
`F. Ground 1: Claims 1-3, 5, 7-13, 15, 17-23, 25-26 Are Obvious
`Under 35 U.S.C. §103(a) (pre-AIA) In Light Of Fukumoto
`1.
`WO 2002/12991 A1 (“Fukumoto WO”) was published in Japanese on
`
`Fukumoto is §102(a) Prior Art
`
`February 14, 2002 (Ex. 1107, certified translation Ex. 1108). It was subsequently
`
`published on October 17, 2002 in English as the corresponding U.S. national stage
`
`application, U.S. Pat. App. Pub. No. 2002/0149561 (“Fukumoto US”) (Ex. 1109).
`
`The Fukumoto publications are substantively identical. Ex. 1102, ¶136. Thus,
`
`citations to Fukumoto herein are in parallel to the certified translation of the
`
`Fukumoto WO publication and the Fukumoto US Application Publication, e.g., Ex.
`
`1108, 1 [page]; Ex. 1109, [0002] [paragraph]. Petitioner relies on Fukumoto WO
`
`as the basis for Grounds 1 and 2 because of its earlier publication date but both
`
`Fukumoto publications qualify as printed publications under 35 U.S.C. § 102(a)
`
`(pre-AIA) because they were published before the effective filing date of the ’356
`
`patent (November 1, 2001).
`
`2. Overview of Fukumoto
`Fukumoto is directed to devices and processes for providing vibratory
`
`feedback to input devices or electronic devices. Ex. 1108 & Ex. 1109, Abstract.
`
`Like the ’256 patent, expressly disclosed examples include ATMs and PDAs (as
`
`well as personal computers, mobile phones, electronic notebooks, mobile
`
`WEST\275294320.2
`
`10
`
`

`

`
`
`computers, wristwatches, electronic calculators, remote controller of an electronic
`
`device, and other various types of portable electronic devices). The vibration is
`
`generated by an actuator which can be a piezoelectric element (Ex. 1108, 48; Ex.
`
`1109, [0416]), a permanent magnet as a movable weight (Ex. 1108, 48; Ex. 1109,
`
`[0417]), an electrostatic type oscillatory actuator (Ex. 1108, 48; Ex. 1109, [0418]),
`
`or other suitable oscillatory actuators.
`
`Fukumoto discloses that a central processing unit (CPU) executes vibration
`
`control procedures upon detecting an operation input on a touch panel. Ex. 1108,
`
`2; Ex. 1109, [0149]. Fukumoto the CPU uses lookup tables to store waveform data
`
`corresponding to input touch buttons. In one embodiment, for each touch button
`
`displayed, the lookup table contains area or position data for the button as well as
`
`pressure-levels to determine which waveform signal to send to the actuator.
`
`WEST\275294320.2
`
`11
`
`

`

`
`
`
`
`Ex. 1108 & Ex. 1109, Fig. 46.
`
`3.
`
`Detailed Analysis
`
`Claim 12
`
`12.pre: a system, comprising:
`
`
`Fukumoto discloses “a system.” Fukumoto discloses various embodiments
`
`of such systems. For example, Fukumoto discloses a range of systems comprising
`
`an electronic device that provides a screen, an input device, and haptic feedback in
`
`the form of vibrations to the user of the device. Ex. 1108, 1; Ex. 1109, [0006].
`
`Fukumoto further discloses that the electronic device may comprise a
`
`handheld device such as a PDA:
`
`WEST\275294320.2
`
`12
`
`

`

`
`
`
`
`Ex. 1108 & Ex. 1109, Figure 1.
`
`Other embodiments of devices disclosed in Fukumoto include “for example
`
`a mobile phone, electronic notebook, mobile computer, wristwatch, electronic
`
`calculator, remote controller of an electronic device, and other various types of
`
`portable electronic devices.” Ex. 1108, 50; Ex. 1109, [0430]. Ex. 1102, ¶¶144-50.
`
`12.a:
`a touch sensitive input device configured to output a sensor signal indicating
`an object contacting the touch-sensitive input device
`
`
`Fukumoto discloses a touch sensitive input device configured to output a
`
`sensor signal indicating an object contacting the touch-sensitive input device. For
`
`example, one of the embodiments of the system disclosed in Fukumoto comprises
`
`a PDA having a touchscreen:
`
`WEST\275294320.2
`
`13
`
`

`

`
`
`Ex. 1108, 13 (describing Fig. 1 and PDA); Ex. 1109, [0146].
`
`Figure 2 of Fukumoto is a block diagram depicting the hardware
`
`configuration of PDA 10 that is shown in Figure 1:
`
`
`
`
`Ex. 1108 & Ex. 1109, Figure 2. Fukumoto describes how the touch-sensitive input
`
`device comprising touch panel 102 outputs a sensor signal indicating contact from
`
`WEST\275294320.2
`
`14
`
`

`

`
`
`an object, which Fukumoto calls a “touch signal:” “The touch panel 102 outputs a
`
`signal showing a touched position on the touch panel 102 (hereinafter called a
`
`‘touch signal’) to the CPU 113 in response to a touch operation.” Ex. 1108, 13;
`
`Ex. 1109, [0148], (emphasis added). Further the CPU 113 shown in Figure 2
`
`above detects the touch signal, “the CPU 113 determines whether a touch signal
`
`has been input from the touch panel 102 and whether a key operation signal has
`
`been input from the key input unit 111 (step S101).” Ex. 1108, 16; Ex. 1109,
`
`[0168], (emphasis added). Ex. 1102, ¶¶151-159.
`
`12.b:
`an actuator coupled to the touch-sensitive input device, the actuator
`configured to receive an actuator signal and output a haptic effect to the
`touch-sensitive surface based at least in part on the actuator signal;
`
`
`In the PDA embodiment shown in Figure 1 above, Fukumoto discloses an
`
`“oscillatory actuator 115” that is coupled to touchscreen in each of the
`
`embodiments disclosed. The oscillatory actuator 115 is shown, for example, in
`
`Figure 2 depicting the hardware configuration of the first PDA embodiment, and
`
`shown with accompanying description below:
`
`WEST\275294320.2
`
`15
`
`

`

`
`
`
`
`
`
`
`
`
`
`Ex. 11008 & Ex. 1109, Figure 2.
`
`
`
`FFigure 3 of Fukumotoo illustratess the locatiion of oscilllatory actuuator 115
`
`
`
`
`
`
`
`
`
`
`
`
`
`within tthe PDA 100, and is reeproduced below:
`
`
`
`
`
`
`
`e 3.
`Ex. 11008 & Ex. 1109, Figur
`
`
`
`
`
`94320.2
`WEST\27529
`
`16
`
`
`
`

`

`
`
`As can be observed in this figure, the internal components 121-123 of
`
`actuator 115 is surrounded by a case 115a, and the upper portion of this actuator
`
`case is in contact with the liquid crystal display panel 103a, which is in turn
`
`directly in contact with touch panel 102. Thus, Fukumoto discloses an actuator
`
`coupled to the touch-sensitive input device comprising touch panel 102. Ex. 1108,
`
`14; Ex. 1109, [0152]. Ex. 1102, ¶¶161-163.
`
`As shown in Figure 2 below, the oscillatory actuator 115 is configured to
`
`receive an actuator signal delivered via drive signal generation circuit 114:
`
`
`
`Ex. 1108 & Ex. 1109, Figure 2.
`
`Fukumoto describes how this actuator signal causes the oscillatory actuator
`
`115 to output a haptic effect comprising a vibration to the touch-sensitive surface
`
`based at least in part on the actuator signal. “The CPU 113 executes a program
`
`stored in the memory 112 to control the parts of the device interconnected through
`
`WEST\275294320.2
`
`17
`
`

`

`
`
`
`
`a bus 1116. This CPPU 113 exxecutes a viibration coontrol proc
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Upon deetection off an operatiion input ffrom the toouch panel
`
`
`
`
`
`
`
`essing 1 (ssee FIG. 5)).
`
`
`
`
`
`102 or anyy one of
`
`
`
`operatioon keys 104a to 104cc, it drives tthe oscillattory actuattor 115 thrrough the ddrive
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`signal ggeneration circuit 1144 to cause tthe touch ppanel 102 oor one of thhe operatioon
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`keys 1004a to 104cc to vibratee.” Ex. 11008, 13-14;
`
`
`
`
`
`
`
`Ex. 1109,
`
`[0149], (e
`
`mphasis
`
`
`
`added). See, also,, Ex. 1108,, 16-17 (CPPU 113 geenerates driive signal f
`from
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`waveforrm in memmory); Ex. 1109, [01770].
`
`
`
`
`
`TThe haptic eeffect that is output tto the surfaace of the ttouch-sens
`
`
`
`
`
`
`
`
`
`
`
`
`
`itive inputt
`
`
`
`device aas a vibratiion that is oorthogonall to the surrface. Thiss is depicteed graphicaally
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`in Figurre 3 in the region of ccontacting of the fingger:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex. 11008 & Ex. 1109, Figure 3, showinng haptic eeffect at coontacting fifinger.
`
`
`
`
`
`94320.2
`WEST\27529
`
`18
`
`

`

`
`
`Fukumoto explains that by coupling the actuator 115 to the touch-sensitive
`
`input device in this way, the received actuator signal causes the haptic effect to be
`
`output at the surface of the device. Ex. 1108, 17 (“it is possible to give the user by
`
`vibrational stimulus the feeling of pressing the touch buttons or operation keys
`
`when touching the touch panel 102 or when depressing thin operation keys 104a to
`
`104c”); Ex. 1109, [0175]. Ex. 1102, ¶¶164-169.
`
`12.c:
`a processor in communication with the sensor and the actuator, the processor
`configured to:
`
`
`Fukumoto discloses a processor in communication with the sensor and the
`
`actuator. As shown in Figure 2, “the PDA 10 has a touch panel 102, a display unit
`
`103, a key input unit 111, a memory 112, a CPU (central processing unit) 113, a
`
`drive signal generation circuit 114, and an oscillatory actuator 115. Ex. 1108, 13;
`
`Ex. 1109, [0147], (emphasis added).
`
`
`
`WEST\275294320.2
`
`19
`
`

`

`
`
`
`
`Ex. 1108 & Ex. 1109, Figure 2.
`
`Fukumoto explains that the processor comprising CPU 113 is in electrical
`
`communication with the sensor comprising touch panel 102, from which it receives
`
`sensor signals. CPU 113 is also in electrical communication with the oscillatory
`
`actuator 115, to which it sends drive signals: “The CPU 113 executes a program
`
`stored in the memory 112 to control the parts of the device interconnected through
`
`a bus 116. This CPU 113 executes a vibration control processing 1 (see FIG. 5).
`
`Upon detection of an operation input from the touch panel 102 or any one of
`
`operation keys 104a to 104c, it drives the oscillatory actuator 115 through the drive
`
`signal generation circuit 114 to cause the touch panel 102 or one of the operation
`
`keys 104a to 104c to vibrate.” Ex. 1108, 13-14; Ex. 1109, [0149], (emphasis
`
`added). Ex. 1102, ¶¶170-174.
`
`12.d:
`
`WEST\275294320.2
`
`20
`
`

`

`
`
`output a display signal configured to display a graphical object on the touch-
`sensitive input device;
`
`
`Fukumoto discloses the processor outputs a display signal configured to
`
`display a graphical object on the touch-sensitive input device.
`
`
`
`Ex. 1108 & Ex. 1109, Figure 2.
`
`
`Fukumoto discloses that the display signal output from CPU 113 is
`
`configured to display graphical objects on the touch-sensitive input device, such as
`
`softkeys that he refers to as “touch buttons.” Ex. 1108, 16; Ex. 1109, [0169].
`
`Examples of graphical objects displayed on the screen of PDA 10 are shown,
`
`for example, in Figures 32, 38-39 and 41 of Fukumoto. For example, Figure 32
`
`shows several graphical objects comprising touch buttons or softkeys being
`
`displayed on the screen of the PDA device:
`
`WEST\275294320.2
`
`21
`
`

`

`
`
`
`
`Ex. 1108 & Ex. 1109, Figure 32.
`
`
`Fukumoto identifies the graphical objects labeled A-G and displayed on the
`
`touch-sensitive device 102 as “touch buttons,” which a person of ordinary skill in
`
`the art would understand to be synonymous with softkeys, the term that is used in
`
`the ’356 patent: “FIG. 32 is a view of an example of the screen display of the PDA
`
`10 according to a first example of present embodiment. As shown in the figure, the
`
`display screen of the PDA 10 shows a plurality of touch buttons “A” to “G”. The
`
`touch panel 102 overlaid on the display screen detects a touch operation when a
`
`user touches a displayed touch button by his or her fingertip.” Ex. 1108, 28; Ex.
`
`1109, [0261], (emphasis added). Ex. 1102, ¶¶175-179.
`
`Another example of graphical icons being displayed on the PDA of
`
`Fukumoto are shown in Figures 38-39, which depict folder and “trash” icons,
`
`familiar to most users of computing devices.
`
`WEST\275294320.2
`
`22
`
`

`

`
`
`
`
`Ex. 11008, 29 (desccribing draagging iconns to trash
`
`
`
`
`
`
`
`
`
`icon); Ex. 1109, [02
`
`
`
`
`70], (emphhasis
`
`
`
`added). Further, FFigure 41 ddepicts a siimulated sllider contrrol using a
`
`
`
`
`
`
`
`
`
`
`
`
`
`can be ddragged aloong the scaale as showwn below:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex. 11008, 30 (desccribing draagging dispplayed “knnob”); Ex.
`
`
`
`
`
`¶¶180-1181.
`
`94320.2
`WEST\27529
`
`23
`
`
`
`“knob” thaat
`
`
`
`
`
`
`1109, [02777]. Ex. 11102,
`
`

`

`
`
`As shown above, Fukumoto discloses a processor configured to output a
`
`display signal configured to display a graphical object on the touch-sensitive input
`
`device. Ex. 1102, ¶¶182-186.
`
`12.e:
`receive the sensor signal from the touch-sensitive input device;
`
`
`Fukumoto discloses that the processor receives the sensor signal from the
`
`touch-sensitive input device. One of the embodiments of the system disclosed in
`
`Fukumoto comprises a PDA having a touchscreen, as shown in Figure 1, which is
`
`reproduced below:
`
`
`Ex. 1108, 13 (describing touch screen of Figure 1 above); Ex. 1109, [0146]; Ex.
`
`1102, ¶190.
`
`Figure 2 of Fukumoto is a block diagram depicting the hardware
`
`configuration of PDA 10 that is shown in Figure 1, and is reproduced below:
`
`WEST\275294320.2
`
`24
`
`

`

`
`
`
`
`Ex. 1108 & Ex. 1109, Figure 2.
`
`
`As shown above, CPU 113 is in electrical communication with touch panel
`
`102, from which it receives signal responsive to contact, or touch. “The touch
`
`panel 102 outputs a signal showing a touched position on the touch panel 102
`
`(hereinafter called a ‘touch signal’) to the CPU 113 in response to a touch
`
`operation.” Ex. 1108, 13; Ex. 1109, [0148], (emphasis added).
`
`Further, CPU 113 detects the touch signal as an “input,” meaning that it
`
`receives this signal: “As shown in the figure, first, the CPU 113 determines
`
`whether a touch signal has been input from the touch panel 102 and whether a key
`
`operation signal has been input from the key input unit 111 (step S101).” Ex.
`
`1108, 16; Ex. 1109, [0168], (emphasis added). Ex. 1102, ¶¶191-195.
`
`
`12.f:
`determine an interaction between the object contacting the touch-sensitive
`
`WEST\275294320.2
`
`25
`
`

`

`
`
`surface and the graphical object;
`
`
`Fukumoto discloses the processor determines an interaction between the
`
`object contacting the touch-sensitive surface and the graphical object.
`
`Fukumoto discloses that CPU 113 determines a positional interaction
`
`between the object contacting the touch panel 102 and a graphical object. “The
`
`touch panel 102 outputs a signal showing a touched position on the touch panel
`
`102 (hereinafter called a ‘touch signal’) to the CPU 113 in response to a touch
`
`operation.” Ex. 1108, 13; Ex. 1109, [0148]. Ex. 1102, ¶201.
`
`Fukumoto explains, in Figure 5, how the CPU then determines whether the
`
`contact position conveyed by this touch signal corresponds to the area on the
`
`screen of a displayed graphical object, such as a touch button.
`
`WEST\275294320.2
`
`26
`
`

`

`
`
`
`
`
`
`Ex. 11008 & Ex. 1109, Figure 5.
`
`
`
`
`“
`
`
`
`[T]he CPUU 113 deterrmines whether a touuch signal hhas been innput from tthe
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`touch paanel 102 annd whether a key opeeration siggnal has beeen input frrom the keyy
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`, 16; Ex. 1
`input unnit 111 (steep S101).” Ex. 1108
`
`
`
`
`
`
`
`
`
`109, [01688], (emphaasis added)
`
`
`
`.
`
`
`
`
`
`
`
`
`
`The CPU 113 deteermines a ppositional interactionn between tthe object
`
`
`
`
`
`
`
`contactingg the
`
`surface
`
`
`
`
`
`
`
`
`
`
`of the devvice and thee displayedd graphicall object, suuch as a touuch buttonn. If
`
`
`
`
`
`
`
`point off contact “ffalls outsidde the displlay areas oof the touchh buttons ddisplayed oon
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the dispplay screenn, the routinne does not proceed tto the proccessing of sstep S102.””
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`94320.2
`WEST\27529
`
`27
`
`

`

`
`
`Ex. 1108, 16, Ex. 1109, [0169]. If instead the point of contact falls within the
`
`display area of one of the touch buttons displayed on the display screen, the routine
`
`will proceed to the processing of step S102. As shown in Figure 5 above, in this
`
`event, the operation will proceed to read waveform data for the actuator signal at
`
`step S102, and use this to output an actuator signal to produce a haptic effect in
`
`response.
`
`Another example