`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`GOOGLE LLC
`Petitioner
`v.
`GESTURE TECHNOLOGY PARTNERS LLC
`Patent Owner
`_____________
`
`Case No. IPR2022-00361
`U.S. Patent No. 8,194,924
`_____________
`
`
`
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,194,924
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`TABLE OF CONTENTS
`
`
`
`INTRODUCTION
`
`103 over Mann in view of Numazaki
`
`U.S.C. § 103 over Mann in view of Numazaki in view of Amir
`
`over Mann in view of Numazaki in view of Aviv
`
`1
`I.
`1
`II. SUMMARY OF THE ’924 PATENT
`1
`A. The ’924 Patent’s Alleged Invention
`5
`B. The ’924 Patent’s Prosecution
`6
`C. A Person Having Ordinary Skill in the Art
`7
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104
`7
`A. Standing Under 37 C.F.R. § 42.104(A)
`7
`B. Challenge Under 37 C.F.R. § 42.104(B) and Relief Requested
`8
`C. Claim Construction Under 37 C.F.R. § 42.104(B)(3)
`8
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`A. Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35 U.S.C. §
`8
`B. Ground 2: Claims 7, 8, 10, 12, and 13 are obvious under pre-AIA 35
`56
`C. Ground 3: Claims 6 and 9 are obvious under pre-AIA 35 U.S.C. § 103
`66
`72
`V. DISCRETIONARY CONSIDERATIONS
`A. The Fintiv Factors Favor Institution
`72
`80
`VI. CONCLUSION
`81
`VII. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(A)(1)
`A. Real Party-In-Interest
`81
`B. Related Matters
`81
`C. Lead and Back-Up Counsel
`
`
`
`i
`
`82
`
`
`
`
`I.
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`INTRODUCTION
`Petitioner Google LLC (“Petitioner”) requests an Inter Partes Review
`
`(“IPR”) of claims 1-14 (the “Challenged Claims”) of U.S. Patent No. 8,194,924
`
`(“the ’924 Patent”). This petition is substantively the same as IPR2021-00923
`
`(which is instituted), and is being filed concurrently with a motion for joinder with
`
`respect to that proceeding.
`
`II.
`
`SUMMARY OF THE ’924 PATENT
`A. The ’924 Patent’s Alleged Invention
`The ’924 Patent describes computer devices that “optically sens[e] human
`
`input” using one or more cameras, contemplating applications in a “variety of
`
`fields such as computing, gaming, medicine, and education.” ’924 Patent (Ex.
`
`1001), 2:7-23. A number of scenarios are described, including multiple cameras
`
`mounted in a fixed display as depicted in Fig. 1A below and a single camera
`
`mounted in a handheld device as depicted in Fig. 8B below:
`
`
`
`1
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 1A, 3:19-56 (describing cameras 100, 101, and 144).
`
`Id. at Fig. 8B, 13:1-10 (describing camera 850).
`
`
`
`2
`
`
`
`
`
`
`
`The Challenged Claims each require a handheld device containing two
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`separate cameras that face in different directions such that they have distinct fields
`
`of view. Id. at Claim 1 (claiming a “handheld device comprising . . . a first camera
`
`[and] a second camera . . . wherein the first and second cameras include non-
`
`overlapping fields of view”). During prosecution, the claims were rejected under
`
`35 U.S.C. § 112, ¶ 1 for failing to satisfy the written description requirement. The
`
`Examiner noted that the specification failed to describe a handheld device with two
`
`cameras. ’924 File History (Ex. 1002), 109. In response, the applicant amended
`
`the specification, adding Fig. 18 (reproduced below) and its corresponding
`
`disclosure:
`
`
`
`3
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Figure 18, 25:40-26:51 (describing the same); Id. ’924 File History (Ex.
`
`1002), 122-129 (amending specification to add Fig. 18 and corresponding
`
`disclosure).
`
`As illustrated, a handheld computer 1901 with central processing unit (CPU)
`
`houses a camera 1902 that can be paired in stereo with another camera 1910 (on
`
`handheld unit), either of which may rotate about axis 1905 to view a user or aspect
`
`of that user like finger 1906. Id. at 25:40-43. When aimed at the user, the
`4
`
`
`
`
`
`
`camera(s) can be used to obtain images and video images of a user’s fingers, hand,
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`objects in the hand, gestures, and facial expressions. Id. at 25:50-63. Facing one
`
`or more of the cameras away from the user, they “can also be used to see gestures
`
`of others.” Id. at 26:25.
`
`B.
`The ’924 Patent’s Prosecution
`The Application that resulted in the ’924 Patent was filed on March 18,
`
`2011. The Application claims priority to provisional patent application No.
`
`60/142,777, filed July 8, 1999. Id. For purposes of this petition and without
`
`waiving its right to challenge priority in this or any other proceeding, Petitioner
`
`adopts July 8, 1999 as the invention date for the Challenged Claims.
`
`In a first non-final office action, Examiner rejected all thirty claims as failing
`
`to comply with the written description requirement of 35 U.S.C. § 112, first
`
`paragraph. ’924 File History (Ex. 1002), 109. Examiner noted “the claims call for
`
`a handheld device housing two cameras” but “[t]he closest explanation in the
`
`specification at paragraphs [0114] and [0123] discloses a handheld device (a cell
`
`phone) [that] contains one camera for imaging.” Id. In response, the Applicant
`
`identified subject matter contained in Applicant’s U.S. Patent 6,750,848, which
`
`was incorporated by reference. Id. at 128. Applicant sought to amend the
`
`specification to include Figure 18 and paragraphs 214.01-214.13 from this
`
`incorporated disclosure. Id. at 122-127.
`
`
`
`5
`
`
`
`In a second non-final office action, the Examiner withdrew the written
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`description rejection and the claims as obvious over U.S. Patent 6,788,336 of
`
`Silverbrook (“Silverbrook”) in view of a variety of secondary references, including
`
`U.S. Patent 5,845,006 of Sumi, et al. (“Sumi”), U.S. Patent 6,204,852 Kumar et al.
`
`(“Kumar”), U.S. Patent 5,491,507 of Umezawa et al. (“Umezawa”), and U.S.
`
`Patent 5,940,126 of Kimura (“Kimura”). Id. at 136-141.
`
`In response, Applicant canceled claims 1-23, amended claims 24-30, and
`
`added new claims 31-37 such that all pending claims required two cameras having
`
`non-overlapping fields of view. Id. at 157-162. Regarding claim 24 (the sole
`
`independent claim), which was rejected as obvious over Silverbrook in view of
`
`Kimura, Applicant argued neither prior art reference “perform a control function
`
`based on the output of first or second cameras having non-overlapping fields of
`
`view.” Id. at 160. A Notice of Allowance issued April 27, 2012. Id. at 166.
`
`C. A Person Having Ordinary Skill in the Art
`A person having ordinary skill in the art (“PHOSITA”) at the time of the
`
`’924 Patent would have had at least a bachelor’s degree in electrical engineering or
`
`equivalent with at least one year of experience in the field of human computer
`
`interaction. Additional education or experience might substitute for the above
`
`requirements. Bederson Dec. (Ex. 1003), ¶¶ 29-31.
`
`
`
`6
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104
`A.
`Standing Under 37 C.F.R. § 42.104(A)
`Petitioner certifies that the ’924 Patent is available for IPR and that
`
`Petitioner is not barred or estopped from requesting an IPR challenging the claims
`
`of the ’924 Patent. Specifically, (1) Petitioner is not the owner of the ’924 Patent,
`
`(2) Petitioner has not filed a civil action challenging the validity of any claim of the
`
`’924 Patent, and (3) Petitioner has not been served with a complaint more than one
`
`year ago alleging infringement of the ’924 patent .
`
`B. Challenge Under 37 C.F.R. § 42.104(B) and Relief Requested
`In view of the prior art and evidence presented, claims 1-14 of the ’924
`
`Patent are unpatentable and should be cancelled. 37 C.F.R. § 42.104(b)(1).
`
`Further, based on the prior art references identified below, IPR of the Challenged
`
`Claims should be granted. 37 C.F.R. § 42.104(b)(2).
`
`Proposed Ground of Unpatentability
`Ground 1: Claims 1-6, 11, and 14 are obvious under pre-
`AIA 35 U.S.C. § 103 over Canadian Published Patent
`Application CA2,237,939A1 to Mann (“Mann”) in view of
`U.S. Patent 6,144,366 to Numazaki. et al. (“Numazaki”)
`Ground 2: Claims 7, 8, 10, 12, and 13 are obvious under
`pre- AIA 35 U.S.C. § 103 over Mann in view of Numazaki
`and in further view of U.S. Patent 6,539,100 to Amir, et al.
`(“Amir”)
`
`Exhibits
`
`Ex. 1004,
`Ex. 1005
`
`Ex. 1004,
`Ex. 1005,
`Ex. 1006
`
`
`
`7
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`Ground 3: Claims 6 and 9 are obvious under pre-AIA 35
`U.S.C. § 103 over Mann in view of Numazaki and in further
`view of U.S. Patent 5,666,157 to David G. Aviv (“Aviv”)
`
`Ex. 1004,
`Ex. 1005,
`Ex. 1007
`
`Section IV identifies where each element of the Challenged Claims is found
`
`in the prior art. 37 C.F.R. § 42.104(b)(4). The exhibit numbers of the evidence
`
`relied upon to support the challenges are provided above and the relevance of the
`
`evidence to the challenges raised is provided in Section IV. 37 C.F.R. §
`
`42.104(b)(5). Exhibits 1001-1018 are also attached.
`
`C. Claim Construction Under 37 C.F.R. § 42.104(B)(3)
`In this proceeding, claims are interpreted under the same standard applied by
`
`Article III courts (i.e., the Phillips standard). See 37 C.F.R. § 42.100(b); see also
`
`83 Fed. Reg. 197 (Oct. 11, 2018); Phillips v. AWH Corp., 415 F.3d 1303, 1312
`
`(Fed. Cir. 2005) (en banc). Under this standard, words in a claim are given their
`
`plain meaning which is the meaning understood by a person of ordinary skill in the
`
`art in view of the patent and file history. Phillips, 415 F.3d 1303, 1212-13. For
`
`purposes of the proposed grounds below, Petitioner proposes no terms require
`
`express construction.
`
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`A. Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35
`U.S.C. § 103 over Mann in view of Numazaki
`Overview of Mann
`
`
`
`8
`
`
`
`Canadian Published Patent Application CA2,237,939A1 to Steve Mann
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`(“Mann”) (Ex. 1004) was filed on June 29, 1998, published on August 28, 1998,
`
`and is prior art to the ’924 Patent under at least 35 U.S.C. § 102(a) (pre-AIA).
`
`Mann was not cited or considered during prosecution of the ’924 Patent. ’924
`
`Patent (Ex. 1001).
`
`Mann discloses a "personal video annotation system.” Mann (Ex. 1004),
`
`Abstract. The invention “allows the user to take notes with pen and paper (or pen
`
`and screen) and continuously record video together with the written notes.” Id. at
`
`8. Figure 1 depicts an embodiment with a “personal digital assistant (PDA).” Id.
`
`at 10. Mann refers to the PDA, synonymously, as a “hand-held device.” Id. at 12.
`
`The PDA includes a first camera 110, second camera 150, screen for notetaking
`
`130, pen 140, and auxiliary screen 120 for displaying the view captured by the first
`
`camera 110. Id. at 11-12.
`
`
`
`9
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 1 (annotated to identify the key components of Mann's PDA).
`The PDA is connected, via wire 160 that may “run up the sleeve of the user”
`
`to a battery pack worn by that user and communications system 176 containing a
`
`“packet radio terminal node controller (high level data link controller with modem)
`10
`
`
`
`
`
`
`and radio, which typically establishes an Internet connection by way of antenna
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`178.” Id. at 12.
`
`Id. at Fig. 1 (annotated to illustrate PDA information transmission elements).
`
`
`
`
`
`11
`
`
`
`In operation, Mann teaches the first camera 110 “captures a view of a person
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`standing in front of the user of the PDA,” while the second camera 150 “may be
`
`used if the user wishes to make a video recording of himself/herself while
`
`recording another person with camera 110.” Id. at 11-12. The cameras are used
`
`for "simultaneously determining a quantity of light arriving from a plurality of
`
`directions and or at a plurality of locations,” which may take the form of a video,
`
`or "signal recorded from two cameras and several microphones arranged in an
`
`array.” Id. at 11. This ensures "both sides of [a] conversation may be
`
`simultaneously recorded by the two cameras” and "edited later, so that there could,
`
`for example, be a cut back and forth between the two cameras to follow the natural
`
`flow of the conversation.” Id. at 12. Mann intends the invention to be "useful as a
`
`new communications medium, in the context of collaborative photography,
`
`collaborative videography, and telepresence.” Id. at 16.
`
`Mann also taches a "wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch 300 houses cameras 310 and
`
`350 and auxiliary screen 320. Id. at 13.
`
`
`
`12
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`Id. at Fig. 3 (annotated to show similarity with PDA configuration). Users control
`
`Mann 's wristwatch by interacting with a clock face superimposed on the
`
`wristwatch display’s touch-based interface. Id. at 14-15.
`
`
`
`13
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 4. Users can enter information like letters, symbols, and numbers using
`
`finger strokes in vector directions such as from top to bottom, or “downwards.” Id.
`
`at 15. These user instructions can also control the wristwatch such as starting or
`
`stopping specific operations:
`
`
`
`14
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`[t]he numbers may be assigned a secondary meaning (e.g.,
`
`select “0” to stop recording, “4” to kill all processes and halt the
`
`processor, “7” to wake up the system from sleep mode, etc.”
`
`Id. at 14. These control operations may be “combined with the video recording
`
`made from the scene” and subsequently transmitted to a legal expert at a remote
`
`location. Id. at 12-15.
`
`Because Mann, like the ’924 Patent, discloses a portable camera system that
`
`may be controlled by human gesture input, Mann is in the same field of endeavor
`
`as the ’924 Patent. Compare Mann (Ex. 1004), 1 (describing a "new photographic
`
`or video means and apparatus typically comprising a hand-held portable electronic
`
`camera system with viewfinder means and electronic pen-based annotation means”
`
`in the form of a PDA and wristwatch with a touch-based interface screen), 11, 12,
`
`14 (describing clock face gestures used to control device) with ’924 Patent (Ex
`
`1001), Abstract, 25:40-41, 25:50-63 (describing a "[m]ethod and apparatus . . . to
`
`enable rapid TV camera and computer based sensing in . . . handheld devices” used
`
`to detect a user’s fingers and gestures)).
`
`Mann is therefore analogous art to the ’924 Patent. Bederson Dec. (Ex.
`
`1003), ¶¶ 35-38.
`
`Overview of Numazaki
`
`
`
`15
`
`
`
`U.S. Patent No. 6,144,366 to Numazaki et al. (“Numazaki”) (Ex. 1005) was
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`filed on October 17, 1997 and is prior art to the ’924 Patent under at least 35
`
`U.S.C. § 102(e) (pre-AIA). Numazaki was not cited or considered during
`
`prosecution of the ’924 Patent. ’924 Patent (Ex. 1001).
`
`Numazaki is generally directed to a method for detecting a gesture or the
`
`movement of a user’s hand. Numazaki (Ex. 1005), Abstract, 4:9-40. Numazaki
`
`purports to have improved upon prior methods by using a controlled light source to
`
`illuminate the target object (e.g., the user’s hand), a first camera unit (referred to
`
`by Numazaki as a “photo-detection unit”),1 and a second camera unit. Id. at 11:9-
`
`23. This arrangement is illustrated in Fig. 2 below:
`
`
`1 A PHOSITA would have considered Numazaki’s photo-detection units to be
`
`camera units. Bederson Dec. (Ex. 1003), ¶ 39 (explaining that Numazaki describes
`
`using CMOS or CCD sensor units at 15:24-16:19, which were two of the more
`
`common optical sensors used in camera units at the time).
`
`
`
`16
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 2. A timing control unit is used to turn lighting unit 101 on (i.e.,
`
`illuminating the target object) when the first camera unit is active and off when the
`
`second camera unit is active. Id. at 11:20-32. The result of this light control is the
`
`first camera unit captures an image of the target object illuminated by both natural
`
`light and the lighting unit 101 and the second camera unit captures an image of the
`
`target object illuminated by only natural light. Id. at 11:33-39. The difference
`
`between the two images—obtained by difference calculation unit 111—represents
`
`the “reflected light from the object resulting from the light emitted by the lighting
`
`unit 101.” Id. at 11:43-51. This information is then used by feature data
`
`generation unit 103 to determine gestures, pointing, etc. of the target object that
`
`may be converted into commands executed by a computer. Id. at 10:57-66.
`
`
`
`17
`
`
`
`In an eighth embodiment, Numazaki describes implementing this structure in
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`“a compact portable information device equipped with the information input
`
`generation apparatus of the present invention, which is in a size that can be held by
`
`one hand.” Id. at 52:5-8. This PDA-style structure is illustrated in Fig. 78 below:
`
`
`
`
`
`18
`
`
`
`
`Id. at Fig. 78. Numazaki explains that window 712 includes the light source and
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`camera sensor, which is used to detect gestures performed by the user such that the
`
`“position of a cursor 714 on the screen can be controlled by moving a finger 713 in
`
`front of this window 712.” Id. at 52:14-16. Numazaki also describes
`
`implementing the lighting and camera structure in a wristwatch such that “a cursor
`
`717 can be controlled by moving a finger 716” within an “operation space” where
`
`“[w]indows 718 and 719 are provided for the lighting unit and the photo-detection
`
`sensor unit.” Id. at 52:20-29. This arrangement is illustrated in Fig. 79 below:
`
`
`
`19
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`Id. at Figure 79 (annotated to emphasize orientation of light source 718 and camera
`
`719 used to track a fingertip gesture to manipulate cursor 717). Numazaki orients
`
`the windows on the device in this way so that “it is possible to prevent the
`
`operating finger from obstructing the view of the display.” Id. at 52:30-32.
`
`
`
`20
`
`
`
`Numazaki’s eighth embodiment incorporates “the information input
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`generation apparatus of the present invention as described in the above
`
`embodiments.” Id. at 50:21-24. A PHOSITA would have understood that the
`
`referenced information input generation apparatus is that illustrated in Fig. 2 and
`
`described in the corresponding disclosure. Bederson Dec. (Ex. 1003), ¶ 42
`
`(explaining that Numazaki describes its controlled light and camera configuration
`
`as key to its invention and noting that Numazaki at 53:22-36 teaches that the eighth
`
`embodiment uses the precise image difference calculation taught by Fig. 2 and its
`
`corresponding disclosure).
`
`Because Numazaki, like the ’924 Patent, discloses a portable camera system
`
`that may be controlled by human gesture input, Numazaki is in the same field of
`
`endeavor as the ’924 Patent. Compare Numazaki (Ex. 1005), 10:11-13, 10:61-66,
`
`52:20-32 (describing an “information input scheme in which . . . light is irradiated
`
`onto a target object from a light source, and [that] reflected light . . . is captured as
`
`an image, so that information on this target object such as its shape, motion,
`
`distance, etc., can be obtained,” where the target object is a hand and the
`
`information obtained is a “gesture or a pointing according to the feature data
`
`extracted from the reflected light image of the hand” to “operate a computer” in a
`
`wrist watch embodiment) with ’924 Patent (Ex 1001), Abstract, 25:40-41, 25:50-
`
`63 (describing a “[m]ethod and apparatus . . . to enable rapid TV camera and
`
`
`
`21
`
`
`
`
`computer based sensing in . . . handheld devices” used to detect a user’s fingers
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`and gestures). Numazaki is therefore analogous art to the ’924 Patent. Bederson
`
`Dec. (Ex. 1003), ¶¶ 39-43.
`
`Motivation to Modify Mann in view of Numazaki
`Mann’s PDA and “wristwatch videotelephone” devices are specially
`
`designed for covert applications such as investigative journalism in which the
`
`subject does not know whether the camera is recording. Mann (Ex. 1004), 2-3, 11-
`
`12 (describing options to conceal operation from the subject and noting the PDA is
`
`well suited for “investigative journalism”). By positioning two cameras in roughly
`
`perpendicular directions, a wristwatch operator can “place his or her arm naturally
`
`upon [a] counter and aim [one] camera . . . at the official behind the counter
`
`without appearing unusual” while a second camera “may be included if the
`
`customer wishes to record himself/herself” so that “both sides of the conversation
`
`would be recorded.” Id. at 13, 12 (noting the PDA is also equipped such that “both
`
`sides of the conversation may be simultaneously recorded by the two cameras”).
`
`The two-camera arrangement in the wristwatch embodiment is illustrated in Fig. 3
`
`below:
`
`
`
`22
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`Id. at Fig. 3 (annotated to show the camera orientations and computer element).
`
`Next, the operator enters commands into a “touch sensitive clockface” using finger
`
`strokes, such as upwards from bottom to top to stop the recording. Id. at 14.
`
`
`
`23
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`Id. at Fig. 4 (annotated to show operator command entry of “0,” which stops a
`
`recording). Although the specific command entry process is not described with
`
`respect to the PDA, a PHOSITA would have understood that physical command
`
`entries, such as using the PDA stylus, would have been used to command functions
`
`such as starting or stopping the recording. Bederson Dec. (Ex. 1003), ¶¶ 45.
`
`
`
`24
`
`
`
`For a number of reasons, rather than require physical gestures or physical
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`inputs (e.g., using the PDA stylus), a PHOSITA would have been motivated to
`
`utilize Mann’s front-facing camera to recognize gestures performed in front of that
`
`camera pursuant to the teachings of Numazaki. First, a PHOSITA would have
`
`anticipated success in modifying Mann’s wristwatch based on the similarities in
`
`functionality and structure of the computer, cameras, and control functionality
`
`taught by Mann and Numazaki. Bederson Dec. (Ex. 1003), ¶ 46 (describing the
`
`similarities between Mann and Numazaki and concluding the modification would
`
`have been well within the skill set of a PHOSITA); see also KSR, 550 U.S. at 417
`
`(obvious to use known techniques to improve similar devices in the same way).
`
`Indeed, Mann already contemplates the requisite structure to capture gestures (i.e.,
`
`PDA and wristwatch devices with cameras facing that user), and it expressly
`
`describes receiving finger-based input to control the operation of the wristwatch.
`
`Mann (Ex. 1004), 11-12 (describing the PDA configuration with user-facing
`
`camera and PDA stylus for input), 13 (describing a two-camera system in the
`
`wristwatch embodiment for recording both the user and subject), 15 (describing a
`
`user entering commands on a touch sensitive watch face). Numazaki similarly
`
`describes PDA and wristwatch devices with cameras facing the user such that a
`
`user can perform gestures over the devices to exert control over the operation of
`
`the device. Numazaki at 52:5-32. A PHOSITA would have recognized that
`
`
`
`25
`
`
`
`
`modifying Mann as proposed would have required a straightforward replacement
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`of Mann’s native user-facing imaging with Numazaki’s gesture-recognition
`
`hardware. Bederson Dec. (Ex. 1003), ¶ 46 (noting both Mann and Numazaki
`
`describe user-facing imaging hardware mounted within PDA and wristwatch
`
`devices). With that hardware swap in place, a straightforward software update
`
`(i.e., adding Numazaki’s gesture detection routines) would allow Mann's devices to
`
`process no-touch gestures. Id.
`
`Second, a PHOSITA would have understood that allowing a user to perform
`
`no-touch gestures over the user-facing camera improves the covert nature of
`
`Mann’s devices. Bederson Dec. (Ex. 1003), ¶ 47. Indeed, physically touching a
`
`watch face or using a stylus to interact with the PDA in order to trigger a recording
`
`runs the risk of being noticed by the subject. Mann expressly teaches that
`
`remaining covert and avoiding attention from the subject are goals of its invention.
`
`Mann at 1-2 (explaining that traditional cameras “create[] a visual disturbance to
`
`others and attract[] considerable attention on account of the gesture of bringing the
`
`camera up to the eye” and describing the goal of the invention is to allow covert
`
`recordings or recordings for which the subject “cannot readily determine whether
`
`or not the apparatus is in use recording”). Allowing users to perform finger-based
`
`no-touch gestures over the user-facing camera in Mann’s devices would improve
`
`the covert nature of these devices—a PHOSITA would have understood that such
`
`
`
`26
`
`
`
`
`gestures would draw much less attention than physically interacting with the watch
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`face or using the PDA stylus. Bederson Dec. (Ex. 1003), ¶ 47.
`
`Finally, Numazaki provides express motivation to implement Mann’s device
`
`with no-touch gesture functionality. Namely, Numazaki teaches that, by orienting
`
`the light and camera windows on the device to capture gestures, “it is possible to
`
`prevent the operating finger from obstructing the view of the display.” Numazaki
`
`(Ex. 1005), 52:30-32. A PHOSITA would have understood that Mann’s native
`
`touch-based gesture control would have suffered from this precise downside—
`
`obstructing the user’s view of the watch face and PDA display when the user
`
`interacts with the device. Bederson Dec. (Ex. 1003), ¶ 48 (explaining that Mann’s
`
`touch gestures would necessarily obstruct a user’s view when gesturing). A
`
`PHOSITA would further have recognized that Mann’s native touch-base gestures
`
`on the wristwatch embodiment would inevitably cause the user’s fingers to touch
`
`the glass covering the user-facing camera. As seen in Fig. 3 below, Mann’s user-
`
`facing camera (350) is positioned on the watch face:
`
`
`
`27
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`Mann (Ex. 1004), Fig. 3. Requiring a user to physically swipe a finger across the
`
`watch face in order to input a command would inevitably cause the user’s finger to
`
`traverse the glass covering the camera, causing the fidelity of that camera to
`
`decrease over time due to grease and grime from the user’s finger (or require
`
`regular cleanings). Bederson Dec. (Ex. 1003), ¶ 48. To avoid these downsides, a
`
`PHOSITA would have been motivated to supplant Mann's touch-based gestures
`
`
`
`28
`
`
`
`
`with Numazaki’s no-touch technique. Id. (describing the similarities between
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`Mann and Numazaki and concluding the physical orientation recommended by
`
`Numazaki would have allowed Mann's device to realize both the benefits discussed
`
`by Numazaki and those benefits that would have been apparent to a PHOSITA
`
`analyzing Mann’s native design); see also KSR, 550 U.S. at 418-19 (demonstrating
`
`a teaching, suggestion, or motivation to combine known elements in order to show
`
`that the combination is obvious).
`
`i.
`
`Claim 1
`
`1[P] A handheld device comprising:
`
`To the extent the preamble is limiting, Mann discloses a handheld device.
`
`Mann discloses a “new photographic or video means and apparatus typically
`
`comprising a hand-held portable electronic camera system with viewfinder
`
`means and electronic pen-based annotation means.” Mann (Ex. 1004), 1 (emphasis
`
`added). In Figure 1 below, Mann depicts an embodiment of a “personal digital
`
`assistant (PDA)”:
`
`
`
`29
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 1 (annotated to illustrate the handheld portion of Mann's handheld PDA
`
`embodiment), 10 (describing the same), 12 (referring to the PDA as a “hand-held
`
`
`
`device”).
`
`
`
`30
`
`
`
`Mann also teaches a “wristwatch embodiment 300 of the invention depicted
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch2 300 houses cameras 310 and
`
`350 and auxiliary screen 320, also described as a “viewfinder” that doubles as a
`
`computer screen display unit. Id. at page 10, 13. This wristwatch embodiment is
`
`illustrated in Fig. 3 below:
`
`
`2 A PHOSITA would have considered Mann’s wristwatch to be handheld. Bederson Dec. (Ex.
`
`1003), ¶ 49 (explaining that Mann describes a PDA and wristwatch embodiment using the same
`
`configuration of hardware for manipulation by a user’s hands, that the term handheld connotes
`
`devices small enough to be held in one’s hand, but does not necessarily require the device is
`
`always operated while in a hand, and that, even if operation in a hand were required, a watch
`
`may be removed from the wrist and held in a user’s hands while retaining the same functionality
`
`and structure).
`
`
`
`31
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 3 (annotated to show camera, viewfinder, and housing elements).
`
`l[a] a housing;
`
`In Figure 1, Mann depicts a PDA as a “hand-held device” configured to
`
`house several components, including a first camera 110, second camera 150,
`
`
`
`
`
`32
`
`
`
`
`screen for notetaking 130, pen 140, and auxiliary screen 120 for displaying the
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`view captured by the first camera 110. Id. at 11-12.
`
`Id. at Fig. 1 (annotated to illustrate the hardware components housed within
`
`Mann's handheld PDA embodiment). Mann discloses the inventions as “related to
`
`
`
`
`
`33
`
`
`
`
`the Personal Digital Assistant^ (PDA) such as the Palm Pilot. . . and the general
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`ideas of pen-based computing.” Id. at 3.
`
`Mann also teaches a “wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann's PDA embodiment, wristwatch 300 houses
`
`cameras 310 and 350 and auxiliary screen 320. Id. at page 13. Mann describes the
`
`wristwatch synonymously as a “housing worn on the wrist” and “housing.” Id. at
`
`20-21 (Claims 13-15). This wristwatch embodiment is illustrated in Fig. 3 below:
`
`
`
`34
`
`
`
`
`
`
`Id. at Fig. 3 (annotated to illustrate components housed within Mann's wristwatch
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`embodiment).
`
`l[b] a computer within the housing;
`In the PDA embodiment, Mann discloses a PDA connected to computer 174
`
`housed within a “body worn pack” 170 and connected by an “additional wire 160 .
`
`. . run up the sleeve of the user.” Id. at 12.
`
`
`
`35
`
`
`
`
`
`
`Id. at Fig. 1 (annotated to show PDA 100 wired connection to computer 174
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`housed within body worn pack 170). However, Mann also teaches reconfiguring
`
`this embodiment to situate the computer inside the PDA:
`
`The body-worn pack 170 would likely disappear in the final
`embodiment of the invention which would contain the
`functionality of the pack 170 inside the hand-held device
`100.”
`Id. at 12 (emphasis added). By embedding the computer within the PDA and
`
`discarding the wire-coupled configuration, Mann's invention provides users
`
`“convenience without a change of lifestyle” that former “wearable camera
`
`system[s]” requiring “special wiring of the body a-priori” could not. Id. at 2.
`
`Similarly, Mann's wristwatch embodiment depicts the same configuration,
`
`where “cabling 360 runs up the sleeve of the wearer of the watch, to an internet-
`
`connected body-worn computer system.” Id. at 13 (emphasis added).
`
`
`
`36
`
`
`
`
`
`IPR2022-00361
`U.S. Patent No. 8,194,924
`
`
`Id. at Fig. 3 (annotated to show wristwatch 300 cabled connection to computer 174
`
`housed within body worn pack 170). As with the PDA embodiment, Mann teaches
`
`that the cabling 360 “may be eliminated, so that the unit would be [] entirely self-
`
`contained.” Id. at 14 (emphasis added). The computer 174 is part of a system
`
`with processor and video recording capabilities. Id. at 21-22 (Claim 19).
`
`
`
`37
`
`
`
`
`l[c] a first camera oriented to view a user of the handheld device