UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`LG ELECTRONICS, INC. and LG ELECTRONICS U.S.A., INC.
`Petitioner
`
`v .
`
`GESTURE TECHNOLOGY PARTNERS LLC
`Patent Owner
`
`Case No. IPR2022-00093
`U.S. Patent No. 8,194,924
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,194,924
`
`EAST\185927510.2
`
`

`

`TABLE OF CONTENTS
`
`I.
`II.
`
`Page
`INTRODUCTION ........................................................................................... 1
`SUMMARY OF THE ’924 PATENT ............................................................. 1
`A.
`The ’924 Patent’s Alleged Invention .................................................... 1
`B.
`The ’924 Patent’s Prosecution............................................................... 4
`C. A Person Having Ordinary Skill in the Art ........................................... 5
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104 ............................ 5
`A.
`Standing Under 37 C.F.R. § 42.104(A) ................................................ 5
`B.
`Challenge Under 37 C.F.R. § 42.104(B) and Relief Requested ........... 6
`C.
`Claim Construction Under 37 C.F.R. § 42.104(B)(3) ........................... 7
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE ............................ 7
`A. Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35
`U.S.C. § 103 over Mann in view of Numazaki ..................................... 7
`1.
`Overview of Mann ...................................................................... 7
`2.
`Overview of Numazaki ............................................................. 13
`3. Motivation to Modify Mann in view of Numazaki .................. 18
`a.
`Claim 1 ............................................................................ 25
`b.
`Claim 2 ............................................................................ 40
`c.
`Claim 3 ............................................................................ 41
`d.
`Claim 4 ............................................................................ 42
`e.
`Claim 5 ............................................................................ 42
`f.
`Claim 6 ............................................................................ 42
`g.
`Claim 11 .......................................................................... 43
`h.
`Claim 14 .......................................................................... 46
`B. Ground 2: Claims 7, 8, 10, 12, and 13 are obvious under pre-AIA
`35 U.S.C. § 103 over Mann in view of Numazaki in view of
`Amir ..................................................................................................... 48
`1.
`Overview of Amir ..................................................................... 48
`
`

`

`TABLE OF CONTENTS
`(Continued)
`
`Page
`
`2. Motivation to Modify Mann in view of Numazaki and in
`view of Amir ............................................................................. 49
`a.
`Claim 7 ............................................................................ 52
`b.
`Claim 8 ............................................................................ 53
`c.
`Claim 10 .......................................................................... 54
`d.
`Claim 12 .......................................................................... 55
`e.
`Claim 13 .......................................................................... 57
`C. Ground 3: Claims 6 and 9 are obvious under pre-AIA 35 U.S.C.
`§ 103 over Mann in view of Numazaki in view of Aviv .................... 58
`1.
`Overview of Aviv...................................................................... 58
`2. Motivation to Modify Mann in view of Numazaki and in
`view of Aviv .............................................................................. 60
`a.
`Claim 6 ............................................................................ 63
`b.
`Claim 9 ............................................................................ 64
`V. DISCRETIONARY CONSIDERATIONS ................................................... 64
`A.
`The Fintiv Factors Favor Institution ................................................... 64
`1.
`The Fintiv factors strongly favor institution ............................. 65
`a. Whether the court granted a stay or evidence exists
`that one may be granted if a proceeding is instituted. .... 65
`Proximity of the court’s trial date to the Board’s
`projected statutory deadline for a final written
`decision. .......................................................................... 65
`Investment in the parallel proceeding by the court
`and the parties. ................................................................ 66
`Overlap between issues raised in the petition and in
`the parallel proceeding. ................................................... 67
`e. Whether the petitioner and the defendant in the
`parallel proceeding are the same party. .......................... 67
`Other circumstances that impact the Board’s
`exercise of discretion, including the merits. ................... 68
`
`f.
`
`EAST\185927510.2
`
`ii
`
`b.
`
`c.
`
`d.
`
`

`

`TABLE OF CONTENTS
`(Continued)
`
`2.
`
`Page
`The Fintiv Framework Should Be Overturned ......................... 68
`a.
`The Fintiv framework exceeds the Director’s
`authority .......................................................................... 69
`The Fintiv framework is arbitrary and capricious .......... 70
`The Fintiv framework was impermissibly adopted
`without notice-and-comment rulemaking....................... 71
`VI. CONCLUSION .............................................................................................. 72
`VII. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(A)(1) ...................... 73
`A.
`Real Party-In-Interest .......................................................................... 73
`B.
`Related Matters .................................................................................... 73
`C.
`Lead and Back-Up Counsel ................................................................. 73
`
`b.
`c.
`
`EAST\185927510.2
`
`iii
`
`

`

`I.
`
`INTRODUCTION
`Petitioner LG Electronics, Inc. and LG Electronics U.S.A. Inc. (“Petitioner”)
`
`requests an Inter Partes Review (“IPR”) of claims 1–14 (the “Challenged Claims”)
`
`of U.S. Patent No. 8,194,924 (“the ’924 Patent”). This petition is substantively the
`
`same as IPR2021-00923 (which is currently pending institution), and is being filed
`
`concurrently with a motion for joinder with respect to that proceeding.
`
`II.
`
`SUMMARY OF THE ’924 PATENT
`A.
`The ’924 Patent’s Alleged Invention
`The ’924 Patent describes computer devices that “optically sens[e] human
`
`input” using one or more cameras, contemplating applications in a “variety of
`
`fields such as computing, gaming, medicine, and education.” ’924 Patent (Ex.
`
`1001), 2:7– 23. A number of scenarios are described, including multiple cameras
`
`mounted in a fixed display as depicted in Fig. 1A below and a single camera
`
`mounted in a handheld device as depicted in Fig. 8B below:
`
`EAST\185927510.2
`
`1
`
`

`

`Id. at Fig. 1A, 3:19-56 (describing cameras 100, 101, and 144).
`
`Id. at Fig. 8B, 13:1-10 (describing camera 850).
`
`The Challenged Claims each require a handheld device containing two
`
`separate cameras that face in different directions such that they have distinct fields
`
`of view. Id. at Claim 1 (claiming a “handheld device comprising . . . a first camera
`
`[and] a second camera . . . wherein the first and second cameras include non-
`
`overlapping fields of view”). During prosecution, the claims were rejected under
`
`35 U.S.C. § 112, ¶ 1 for failing to satisfy the written description requirement. The
`
`Examiner noted that the specification failed to describe a handheld device with two
`
`cameras. ’924 File History (Ex. 1002), 109. In response, the applicant amended the
`
`specification, adding Fig. 18 (reproduced below) and its corresponding disclosure:
`
`EAST\185927510.2
`
`2
`
`

`

`Id. at Figure 18, 25:40-26:51 (describing the same); ’924 File History (Ex. 1002),
`
`122-129 (amending specification to add Fig. 18 and corresponding disclosure).
`
`As illustrated, a handheld computer 1901 with central processing unit (CPU)
`
`houses a camera 1902 that can be paired in stereo with another camera 1910 (on
`
`handheld unit), either of which may rotate about axis 1905 to view a user or aspect
`
`of that user like finger 1906. Id. at 25:40-43. When aimed at the user, the camera(s)
`
`can be used to obtain images and video images of a user’s fingers, hand, objects in
`
`the hand, gestures, and facial expressions. Id. at 25:50-63. Facing one or more of
`
`the cameras away from the user, they “can also be used to see gestures of others.”
`
`Id. at 26:25.
`
`EAST\185927510.2
`
`3
`
`

`

`The ’924 Patent’s Prosecution
`B.
`The Application that resulted in the ’924 Patent was filed on March 18,
`
`2011. The Application claims priority to provisional patent application No.
`
`60/142,777, filed July 8, 1999. Id. For purposes of this petition and without
`
`waiving its right to challenge priority in this or any other proceeding, Petitioner
`
`adopts July 8, 1999 as the invention date for the Challenged Claims.
`
`In a first non-final office action, Examiner rejected all thirty claims as failing
`
`to comply with the written description requirement of 35 U.S.C. § 112, first
`
`paragraph. ’924 File History (Ex. 1002), 109. Examiner noted “the claims call for a
`
`handheld device housing two cameras” but “[t]he closest explanation in the
`
`specification at paragraphs [0114] and [0123] discloses a handheld device (a cell
`
`phone) [that] contains one camera for imaging.” Id. In response, the Applicant
`
`identified subject matter contained in Applicant’s U.S. Patent 6,750,848, which
`
`was incorporated by reference. Id. at 128. Applicant sought to amend the
`
`specification to include Figure 18 and paragraphs 214.01–214.13 from this
`
`incorporated disclosure. Id. at 122–127.
`
`In a second non-final office action, the Examiner withdrew the written
`
`description rejection and the claims as obvious over U.S. Patent 6,788,336 of
`
`Silverbrook (“Silverbrook”) in view of a variety of secondary references, including
`
`U.S. Patent 5,845,006 of Sumi, et al. (“Sumi”), U.S. Patent 6,204,852 Kumar et al.
`
`EAST\185927510.2
`
`4
`
`

`

`(“Kumar”), U.S. Patent 5,491,507 of Umezawa et al. (“Umezawa”), and U.S.
`
`Patent 5,940,126 of Kimura (“Kimura”). Id. at 136-141.
`
`In response, Applicant canceled claims 1-23, amended claims 24-30, and
`
`added new claims 31-37 such that all pending claims required two cameras having
`
`non-overlapping fields of view. Id. at 157-162. Regarding claim 24 (the sole
`
`independent claim), which was rejected as obvious over Silverbrook in view of
`
`Kimura, Applicant argued neither prior art reference “perform a control function
`
`based on the output of first or second cameras having non-overlapping fields of
`
`view.” Id. at 160. A Notice of Allowance issued April 27, 2012. Id. at 166.
`
`A Person Having Ordinary Skill in the Art
`C.
`A person having ordinary skill in the art (“PHOSITA”) at the time of the
`
`’924 Patent would have had at least a bachelor’s degree in electrical engineering or
`
`equivalent with at least one year of experience in the field of human computer
`
`interaction. Additional education or experience might substitute for the above
`
`requirements. Bederson Dec. (Ex. 1003), ¶¶ 29-31.
`
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104
`A.
`Standing Under 37 C.F.R. § 42.104(A)
`Petitioner certifies that the ’924 Patent is available for IPR and that
`
`Petitioner is not barred or estopped from requesting an IPR challenging the claims
`
`of the ’924 Patent. Specifically, (1) Petitioner is not the owner of the ’924 Patent,
`
`(2) Petitioner has not filed a civil action challenging the validity of any claim of the
`5
`
`EAST\185927510.2
`
`

`

`’924 Patent, and (3) this Petition is filed less than one year after the Petitioner was
`
`served with a complaint alleging infringement of the ’924 Patent.
`
`Challenge Under 37 C.F.R. § 42.104(B) and Relief Requested
`B.
`In view of the prior art and evidence presented, claims 1–14 of the ’924
`
`Patent are unpatentable and should be cancelled. 37 C.F.R. § 42.104(b)(1). Further,
`
`based on the prior art references identified below, IPR of the Challenged Claims
`
`should be granted. 37 C.F.R. § 42.104(b)(2).
`
`Proposed Ground of Unpatentability
`Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35
`U.S.C. § 103 over Canadian Published Patent Application
`CA2,237,939A1 to Mann (“Mann”) in view of U.S. Patent
`6,144,366 to Numazaki. et al. (“Numazaki”)
`Ground 2: Claims 7, 8, 10, 12, and 13 are obvious under pre-
`AIA 35 U.S.C. § 103 over Mann in view of Numazaki and in
`further view of U.S. Patent 6,539,100 to Amir, et al. (“Amir”)
`Ground 3: Claims 6 and 9 are obvious under pre-AIA 35 U.S.C. §
`103 over Mann in view of Numazaki and in further view of U.S.
`Patent 5,666,157 to David G. Aviv (“Aviv”)
`
`Exhibits
`Ex. 1004,
`Ex. 1005
`
`Ex. 1004,
`Ex. 1005,
`Ex. 1006
`Ex. 1004,
`Ex. 1005,
`Ex. 1007
`
`Section IV identifies where each element of the Challenged Claims is found
`
`in the prior art. 37 C.F.R. § 42.104(b)(4). The exhibit numbers of the evidence
`
`relied upon to support the challenges are provided above and the relevance of the
`
`evidence to the challenges raised is provided in Section IV. 37 C.F.R. §
`
`42.104(b)(5). Exhibits 1001-1017 are also attached.
`
`EAST\185927510.2
`
`6
`
`

`

`Claim Construction Under 37 C.F.R. § 42.104(B)(3)
`C.
`In this proceeding, claims are interpreted under the same standard applied by
`
`Article III courts (i.e., the Phillips standard). See 37 C.F.R § 42.100(b); see also 83
`
`Fed. Reg. 197 (Oct. 11, 2018); Phillips v. AWH Corp., 415 F.3d 1303, 1312 (Fed.
`
`Cir. 2005) (en banc). Under this standard, words in a claim are given their plain
`
`meaning which is the meaning understood by a person of ordinary skill in the art in
`
`view of the patent and file history. Phillips, 415 F.3d 1303, 1212–13. For purposes
`
`of the proposed grounds below, Petitioner proposes no terms require express
`
`construction.
`
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`A.
`Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35
`U.S.C. § 103 over Mann in view of Numazaki
`1.
`Overview of Mann
`Canadian Published Patent Application CA2,237,939A1 to Steve Mann
`
`(“Mann”) (Ex. 1004) was filed on June 29, 1998, published on August 28, 1998,
`
`and is prior art to the ’924 Patent under at least 35 U.S.C. § 102(a) (pre-AIA).
`
`Mann was not cited or considered during prosecution of the ’924 Patent. ’924
`
`Patent (Ex. 1001).
`
`Mann discloses a “personal video annotation system.” Mann (Ex. 1004),
`
`Abstract. The invention “allows the user to take notes with pen and paper (or pen
`
`and screen) and continuously record video together with the written notes.” Id. at
`
`EAST\185927510.2
`
`7
`
`

`

`8.Figure 1 depicts an embodiment with a “personal digital assistant (PDA).” Id. at
`
`10. Mann refers to the PDA, synonymously, as a “hand-held device.” Id. at 12. The
`
`PDA includes a first camera 110, second camera 150, screen for notetaking 130,
`
`pen 140, and auxiliary screen 120 for displaying the view captured by the first
`
`camera 110. Id. at 11-12.
`
`Id. at Fig. 1 (annotated to identify the key components of Mann’s PDA).
`
`EAST\185927510.2
`
`8
`
`

`

`The PDA is connected, via wire 160 that may “run up the sleeve of the user”
`
`to a battery pack worn by that user and communications system 176 containing a
`
`“packet radio terminal node controller (high level data link controller with modem)
`
`and radio, which typically establishes an Internet connection by way of antenna
`
`178.” Id. at 12.
`
`Id. at Fig. 1 (annotated to illustrate PDA information transmission elements).
`
`In operation, Mann teaches the first camera 110 “captures a view of a person
`
`standing in front of the user of the PDA,” while the second camera 150 “may be
`
`used if the user wishes to make a video recording of himself/herself while
`
`recording another person with camera 110.” Id. at 11-12. The cameras are used for
`
`EAST\185927510.2
`
`9
`
`

`

`“simultaneously determining a quantity of light arriving from a plurality of
`
`directions and or at a plurality of locations,” which may take the form of a video,
`
`or “signal recorded from two cameras and several microphones arranged in an
`
`array.” Id. at 11. This ensures “both sides of [a] conversation may be
`
`simultaneously recorded by the two cameras” and “edited later, so that there could,
`
`for example, be a cut back and forth between the two cameras to follow the natural
`
`flow of the conversation.” Id. at 12. Mann intends the invention to be “useful as a
`
`new communications medium, in the context of collaborative photography,
`
`collaborative videography, and telepresence.” Id. at 16.
`
`Mann also taches a “wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch 300 houses cameras 310 and
`
`350 and auxiliary screen 320. Id. at 13.
`
`EAST\185927510.2
`
`10
`
`

`

`Id. at Fig. 3 (annotated to show similarity with PDA configuration). Users control
`
`Mann’s wristwatch by interacting with a clock face superimposed on the
`
`wristwatch display’s touch-based interface. Id. at 14-15.
`
`EAST\185927510.2
`
`11
`
`

`

`Id. at Fig. 4. Users can enter information like letters, symbols, and numbers using
`
`finger strokes in vector directions such as from top to bottom, or “downwards.” Id.
`
`at 15. These user instructions can also control the wristwatch such as starting or
`
`stopping specific operations:
`
`[t]he numbers may be assigned a secondary meaning (e.g., select “0”
`
`to stop recording, “4” to kill all processes and halt the processor, “7”
`
`to wake up the system from sleep mode, etc.”
`
`Id. at 14. These control operations may be “combined with the video recording
`
`made from the scene” and subsequently transmitted to a legal expert at a remote
`
`location. Id. at 12-15. Because Mann, like the ’924 Patent, discloses a portable
`
`camera system that may be controlled by human gesture input, Mann is in the same
`
`field of endeavor as the ’924 Patent. Compare Mann (Ex. 1004), 1 (describing a
`
`“new photographic or video means and apparatus typically comprising a hand-held
`
`portable electronic camera system with viewfinder means and electronic pen-based
`
`annotation means” in the form of a PDA and wristwatch with a touch-based
`
`interface screen), 11, 12, 14 (describing clock face gestures used to control device)
`
`with ’924 Patent (Ex 1001), Abstract, 25:40-41, 25:50-63 (describing a “[m]ethod
`
`and apparatus . . . to enable rapid TV camera and computer based sensing in . . .
`
`handheld devices” used to detect a user’s fingers and gestures)).
`
`EAST\185927510.2
`
`12
`
`

`

`Mann is therefore analogous art to the ’924 Patent. Bederson Dec. (Ex.
`
`1003), ¶¶ 35-38.
`
`Overview of Numazaki
`2.
`U.S. Patent No. 6,144,366 to Numazaki et al. (“Numazaki”) (Ex. 1005) was
`
`filed on October 17, 1997 and is prior art to the ’924 Patent under at least 35
`
`U.S.C. § 102(e) (pre-AIA). Numazaki was not cited or considered during
`
`prosecution of the ’924 Patent. ’924 Patent (Ex. 1001).
`
`Numazaki is generally directed to a method for detecting a gesture or the
`
`movement of a user’s hand. Numazaki (Ex. 1005), Abstract, 4:9-40. Numazaki
`
`purports to have improved upon prior methods by using a controlled light source to
`
`illuminate the target object (e.g., the user’s hand), a first camera unit (referred to
`
`by Numazaki as a “photo-detection unit”),1 and a second camera unit. Id. at 11:9-
`
`23. This arrangement is illustrated in Fig. 2 below:
`
`1 1 A PHOSITA would have considered Numazaki’s photo-detection units to be
`
`camera units. Bederson Dec. (Ex. 1003), ¶ 39 (explaining that Numazaki describes
`
`using CMOS or CCD sensor units at 15:24-16:19, which were two of the more
`
`common optical sensors used in camera units at the time).
`
`EAST\185927510.2
`
`13
`
`

`

`Id. at Fig. 2. A timing control unit is used to turn lighting unit 101 on (i.e.,
`
`illuminating the target object) when the first camera unit is active and off when the
`
`second camera unit is active. Id. at 11:20-32. The result of this light control is the
`
`first camera unit captures an image of the target object illuminated by both natural
`
`light and the lighting unit 101 and the second camera unit captures an image of the
`
`target object illuminated by only natural light. Id. at 11:33-39. The difference
`
`between the two images—obtained by difference calculation unit 111—represents
`
`the “reflected light from the object resulting from the light emitted by the lighting
`
`unit 101.” Id. at 11:43-51. This information is then used by feature data generation
`
`unit 103 to determine gestures, pointing, etc. of the target object that may be
`
`converted into commands executed by a computer. Id. at 10:57-66.
`
`In an eighth embodiment, Numazaki describes implementing this structure in
`
`“a compact portable information device equipped with the information input
`
`EAST\185927510.2
`
`14
`
`

`

`generation apparatus of the present invention, which is in a size that can be held by
`
`one hand.” Id. at 52:5-8. This PDA-style structure is illustrated in Fig. 78 below:
`
`Id. at Fig. 78. Numazaki explains that window 712 includes the light source
`
`and camera sensor, which is used to detect gestures performed by the user such that
`
`the “position of a cursor 714 on the screen can be controlled by moving a finger
`
`713 in front of this window 712.” Id. at 52:14-16. Numazaki also describes
`
`implementing the lighting and camera structure in a wristwatch such that “a cursor
`
`717 can be controlled by moving a finger 716” within an “operation space” where
`
`“[w]indows 718 and 719 are provided for the lighting unit and the photo-detection
`
`sensor unit.” Id. at 52:20-29. This arrangement is illustrated in Fig. 79 below:
`
`EAST\185927510.2
`
`15
`
`

`

`Id. at Figure 79 (annotated to emphasize orientation of light source 718 and
`
`camera 719 used to track a fingertip gesture to manipulate cursor 717). Numazaki
`
`orients the windows on the device in this way so that “it is possible to prevent the
`
`operating finger from obstructing the view of the display.” Id. at 52:30-32.
`
`Numazaki’s eighth embodiment incorporates “the information input
`
`generation apparatus of the present invention as described in the above
`
`embodiments.” Id. at 50:21-24. A PHOSITA would have understood that the
`
`referenced information input generation apparatus is that illustrated in Fig. 2 and
`
`described in the corresponding disclosure. Bederson Dec. (Ex. 1003), ¶ 42
`
`EAST\185927510.2
`
`16
`
`

`

`(explaining that Numazaki describes its controlled light and camera configuration
`
`as key to its invention and noting that Numazaki at 53:22-36 teaches that the eighth
`
`embodiment uses the precise image difference calculation taught by Fig. 2 and its
`
`corresponding disclosure).
`
`Because Numazaki, like the ’924 Patent, discloses a portable camera system
`
`that may be controlled by human gesture input, Numazaki is in the same field of
`
`endeavor as the ’924 Patent. Compare Numazaki (Ex. 1005), 10:11-13, 10:61-66,
`
`52:20-32 (describing an “information input scheme in which . . . light is irradiated
`
`onto a target object from a light source, and [that] reflected light . . . is captured as
`
`an image, so that information on this target object such as its shape, motion,
`
`distance, etc., can be obtained,” where the target object is a hand and the
`
`information obtained is a “gesture or a pointing according to the feature data
`
`extracted from the reflected light image of the hand” to “operate a computer” in a
`
`wrist watch embodiment) with ’924 Patent (Ex 1001), Abstract, 25:40-41, 25:50-
`
`63 (describing a “[m]ethod and apparatus . . . to enable rapid TV camera and
`
`computer based sensing in . . . handheld devices” used to detect a user’s fingers
`
`and gestures). Numazaki is therefore analogous art to the ’924 Patent. Bederson
`
`Dec. (Ex. 1003), ¶¶ 39-43.
`
`EAST\185927510.2
`
`17
`
`

`

`3. Motivation to Modify Mann in view of Numazaki
`Mann’s PDA and “wristwatch videotelephone” devices are specially
`
`designed for covert applications such as investigative journalism in which the
`
`subject does not know whether the camera is recording. Mann (Ex. 1004), 2-3, 11-
`
`12 (describing options to conceal operation from the subject and noting the PDA is
`
`well suited for “investigative journalism”). By positioning two cameras in roughly
`
`perpendicular directions, a wristwatch operator can “place his or her arm naturally
`
`upon [a] counter and aim [one] camera . . . at the official behind the counter
`
`without appearing unusual” while a second camera “may be included if the
`
`customer wishes to record himself/herself” so that “both sides of the conversation
`
`would be recorded.” Id. at 13, 12 (noting the PDA is also equipped such that “both
`
`sides of the conversation may be simultaneously recorded by the two cameras”).
`
`The two-camera arrangement in the wristwatch embodiment is illustrated in Fig. 3
`
`below:
`
`EAST\185927510.2
`
`18
`
`

`

`Id. at Fig. 3 (annotated to show the camera orientations and computer
`
`element). Next, the operator enters commands into a “touch sensitive clockface”
`
`using finger strokes, such as upwards from bottom to top to stop the recording. Id.
`
`at 14.
`
`EAST\185927510.2
`
`19
`
`

`

`Id. at Fig. 4 (annotated to show operator command entry of “0,” which stops a
`
`recording). Although the specific command entry process is not described with
`
`respect to the PDA, a PHOSITA would have understood that physical command
`
`entries, such as using the PDA stylus, would have been used to command functions
`
`such as starting or stopping the recording. Bederson Dec. (Ex. 1003), ¶ 45.
`
`For a number of reasons, rather than require physical gestures or physical
`
`inputs (e.g., using the PDA stylus), a PHOSITA would have been motivated to
`
`utilize Mann’s front-facing camera to recognize gestures performed in front of that
`
`camera pursuant to the teachings of Numazaki. First, a PHOSITA would have
`
`anticipated success in modifying Mann’s wristwatch based on the similarities in
`
`EAST\185927510.2
`
`20
`
`

`

`functionality and structure of the computer, cameras, and control functionality
`
`taught by Mann and Numazaki. Bederson Dec. (Ex. 1003), ¶ 46 (describing the
`
`similarities between Mann and Numazaki and concluding the modification would
`
`have been well within the skill set of a PHOSITA); see also KSR, 550 U.S. at 417
`
`(obvious to use known techniques to improve similar devices in the same way).
`
`Indeed, Mann already contemplates the requisite structure to capture gestures (i.e.,
`
`PDA and wristwatch devices with cameras facing that user), and it expressly
`
`describes receiving finger-based input to control the operation of the wristwatch.
`
`Mann (Ex. 1004), 11-12 (describing the PDA configuration with user-facing
`
`camera and PDA stylus for input), 13 (describing a two-camera system in the
`
`wristwatch embodiment for recording both the user and subject), 15 (describing a
`
`user entering commands on a touch sensitive watch face). Numazaki similarly
`
`describes PDA and wristwatch devices with cameras facing the user such that a
`
`user can perform gestures over the devices to exert control over the operation of
`
`the device. Numazaki at 52:5-32. A PHOSITA would have recognized that
`
`modifying Mann as proposed would have required a straightforward replacement
`
`of Mann’s native user-facing imaging with Numazaki’s gesture-recognition
`
`hardware. Bederson Dec. (Ex. 1003), ¶ 46 (noting both Mann and Numazaki
`
`describe user-facing imaging hardware mounted within PDA and wristwatch
`
`devices). With that hardware swap in place, a straightforward software update (i.e.,
`
`EAST\185927510.2
`
`21
`
`

`

`adding Numazaki’s gesture detection routines) would allow Mann’s devices to
`
`process no-touch gestures. Id.
`
`Second, a PHOSITA would have understood that allowing a user to perform
`
`no-touch gestures over the user-facing camera improves the covert nature of
`
`Mann’s devices. Bederson Dec. (Ex. 1003), ¶ 47. Indeed, physically touching a
`
`watch face or using a stylus to interact with the PDA in order to trigger a recording
`
`runs the risk of being noticed by the subject. Mann expressly teaches that
`
`remaining covert and avoiding attention from the subject are goals of its invention.
`
`Mann at 1-2 (explaining that traditional cameras “create[] a visual disturbance to
`
`others and attract[] considerable attention on account of the gesture of bringing the
`
`camera up to the eye” and describing the goal of the invention is to allow covert
`
`recordings or recordings for which the subject “cannot readily determine whether
`
`or not the apparatus is in use recording”). Allowing users to perform finger-based
`
`no-touch gestures over the user-facing camera in Mann’s devices would improve
`
`the covert nature of these devices—a PHOSITA would have understood that such
`
`gestures would draw much less attention than physically interacting with the watch
`
`face or using the PDA stylus. Bederson Dec. (Ex. 1003), ¶ 47.
`
`Finally, Numazaki provides express motivation to implement Mann’s device
`
`with no-touch gesture functionality. Namely, Numazaki teaches that, by orienting
`
`the light and camera windows on the device to capture gestures, “it is possible to
`
`EAST\185927510.2
`
`22
`
`

`

`prevent the operating finger from obstructing the view of the display.” Numazaki
`
`(Ex. 1005), 52:30-32. A PHOSITA would have understood that Mann’s native
`
`touch-based gesture control would have suffered from this precise downside—
`
`obstructing the user’s view of the watch face and PDA display when the user
`
`interacts with the device. Bederson Dec. (Ex. 1003), ¶ 48 (explaining that Mann’s
`
`touch gestures would necessarily obstruct a user’s view when gesturing). A
`
`PHOSITA would further have recognized that Mann’s native touch-base gestures
`
`on the wristwatch embodiment would inevitably cause the user’s fingers to touch
`
`the glass covering the user-facing camera. As seen in Fig. 3 below, Mann’s user-
`
`facing camera (350) is positioned on the watch face
`
`EAST\185927510.2
`
`23
`
`

`

`Mann (Ex. 1004), Fig. 3. Requiring a user to physically swipe a finger across the
`
`watch face in order to input a command would inevitably cause the user’s finger to
`
`traverse the glass covering the camera, causing the fidelity of that camera to
`
`decrease over time due to grease and grime from the user’s finger (or require
`
`regular cleanings). Bederson Dec. (Ex. 1003), ¶ 48. To avoid these downsides, a
`
`PHOSITA would have been motivated to supplant Mann’s touch-based gestures
`
`with Numazaki’s no-touch technique. Id. (describing the similarities between Mann
`
`and Numazaki and concluding the physical orientation recommended by Numazaki
`
`would have allowed Mann’s device to realize both the benefits discussed by
`
`EAST\185927510.2
`
`24
`
`

`

`Numazaki and those benefits that would have been apparent to a PHOSITA
`
`analyzing Mann’s native design); see also KSR, 550 U.S. at 418-19 (demonstrating
`
`a teaching, suggestion, or motivation to combine known elements in order to show
`
`that the combination is obvious).
`
`a.
`
`Claim 1
`i.
`1[P] A handheld device comprising:
`To the extent the preamble is limiting, Mann discloses a handheld device.
`
`Mann discloses a “new photographic or video means and apparatus typically
`
`comprising a hand-held portable electronic camera system with viewfinder
`
`means and electronic pen-based annotation means.” Mann (Ex. 1004), 1 (emphasis
`
`added). In Figure 1 below, Mann depicts an embodiment of a “personal digital
`
`assistant (PDA)”:
`
`EAST\185927510.2
`
`25
`
`

`

`Id. at Fig. 1 (annotated to illustrate the handheld portion of Mann’s handheld PDA
`
`embodiment), 10 (describing the same), 12 (referring to the PDA as a “hand-held
`
`device”).
`
`Mann also teaches a “wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch2 300 houses cameras 310 and
`
`350 and auxiliary screen 320, also described as a “viewfinder” that doubles as a
`
`computer screen display unit. Id. at page 10, 13. This wristwatch embodiment is
`
`illustrated in Fig. 3 below:
`
`2 A PHOSITA would have considered Mann’s wristwatch to be handheld.
`
`Bederson Dec. (Ex. 1003), ¶ 49 (explaining that Mann describes a PDA and
`
`wristwatch embodiment using the same configuration of hardware for
`
`manipulation by a user’s hands, that the term handheld connotes devices small
`
`enough to be held in one’s hand, but does not necessarily require the device is
`
`always operated while in a hand, and that, even if ope