`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`
`APPLE INC.
`Petitioner
`
`v.
`
`GESTURE TECHNOLOGY PARTNERS LLC
`Patent Owner
`____________
`
`
`Case No. IPR2021-00923
`U.S. Patent No. 8,194,924
`____________
`
`
`
`
`
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,194,924
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`TABLE OF CONTENTS
`
`INTRODUCTION ............................................................................... 1
`I.
`SUMMARY OF THE ’924 PATENT .................................................. 1
`II.
`THE ’924 PATENT’S ALLEGED INVENTION ..................................................... 1
`A.
`THE ’924 PATENT’S PROSECUTION ................................................................ 4
`B.
`C. A PERSON HAVING ORDINARY SKILL IN THE ART .......................................... 5
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104 ................. 5
`A.
`STANDING UNDER 37 C.F.R. § 42.104(A) ..................................................... 5
`B.
`CHALLENGE UNDER 37 C.F.R. § 42.104(B) AND RELIEF REQUESTED ............. 6
`C.
`CLAIM CONSTRUCTION UNDER 37 C.F.R. § 42.104(B)(3) .............................. 7
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE ................. 7
`A. GROUND 1: CLAIMS 1-6, 11, AND 14 ARE OBVIOUS UNDER PRE-AIA 35 U.S.C.
`§ 103 OVER MANN IN VIEW OF NUMAZAKI ....................................................... 7
`B. GROUND 2: CLAIMS 7, 8, 10, 12, AND 13 ARE OBVIOUS UNDER PRE-AIA 35
`U.S.C. § 103 OVER MANN IN VIEW OF NUMAZAKI IN VIEW OF AMIR .................49
`C. GROUND 3: CLAIMS 6 AND 9 ARE OBVIOUS UNDER PRE-AIA 35 U.S.C. § 103
`OVER MANN IN VIEW OF NUMAZAKI IN VIEW OF AVIV ......................................59
`V. DISCRETIONARY CONSIDERATIONS ......................................... 65
`THE FINTIV FACTORS FAVOR INSTITUTION....................................................65
`A.
`VI. CONCLUSION ................................................................................... 73
`VII. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(A)(1) .............. 75
`A.
`REAL PARTY-IN-INTEREST ...........................................................................75
`B.
`RELATED MATTERS .....................................................................................75
`C.
`LEAD AND BACK-UP COUNSEL ....................................................................75
`
`
`
`
`i
`
`
`
`I.
`
`INTRODUCTION
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`Petitioner Apple Inc. (“Petitioner”) requests an Inter Partes Review (“IPR”)
`
`of claims 1–14 (the “Challenged Claims”) of U.S. Patent No. 8,194,924 (“the ’924
`
`Patent”).
`
`II.
`
`SUMMARY OF THE ’924 PATENT
`
`A.
`
`The ’924 Patent’s Alleged Invention
`
`The ’924 Patent describes computer devices that “optically sens[e] human
`
`input” using one or more cameras, contemplating applications in a “variety of fields
`
`such as computing, gaming, medicine, and education.” ’924 Patent (Ex. 1001), 2:7–
`
`23. A number of scenarios are described, including multiple cameras mounted in a
`
`fixed display as depicted in Fig. 1A below and a single camera mounted in a
`
`handheld device as depicted in Fig. 8B below:
`
`Id. at Fig. 1A, 3:19-56 (describing cameras 100, 101, and 144).
`
`
`
`1
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 8B, 13:1-10 (describing camera 850).
`
`The Challenged Claims each require a handheld device containing two
`
`separate cameras that face in different directions such that they have distinct fields
`
`of view. Id. at Claim 1 (claiming a “handheld device comprising . . . a first camera
`
`[and] a second camera . . . wherein the first and second cameras include non-
`
`overlapping fields of view”). During prosecution, the claims were rejected under 35
`
`U.S.C. § 112, ¶ 1 for failing to satisfy the written description requirement. The
`
`Examiner noted that the specification failed to describe a handheld device with two
`
`cameras. ’924 File History (Ex. 1002), 109. In response, the applicant amended the
`
`specification, adding Fig. 18 (reproduced below) and its corresponding disclosure:
`
`2
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Figure 18, 25:40-26:51 (describing the same); ’924 File History (Ex. 1002),
`
`122-129 (amending specification to add Fig. 18 and corresponding disclosure).
`
`As illustrated, a handheld computer 1901 with central processing unit (CPU)
`
`houses a camera 1902 that can be paired in stereo with another camera 1910 (on
`
`handheld unit), either of which may rotate about axis 1905 to view a user or aspect
`
`of that user like finger 1906. Id. at 25:40-43. When aimed at the user, the camera(s)
`
`can be used to obtain images and video images of a user’s fingers, hand, objects in
`
`the hand, gestures, and facial expressions. Id. at 25:50-63. Facing one or more of the
`
`cameras away from the user, they “can also be used to see gestures of others.” Id. at
`
`26:25.
`
`3
`
`
`
`B.
`
`The ’924 Patent’s Prosecution
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`The Application that resulted in the ’924 Patent was filed on March 18, 2011.
`
`The Application claims priority to provisional patent application No. 60/142,777,
`
`filed July 8, 1999. Id. For purposes of this petition and without waiving its right to
`
`challenge priority in this or any other proceeding, Petitioner adopts July 8, 1999 as
`
`the invention date for the Challenged Claims.
`
`In a first non-final office action, Examiner rejected all thirty claims as failing
`
`to comply with the written description requirement of 35 U.S.C. § 112, first
`
`paragraph. ’924 File History (Ex. 1002), 109. Examiner noted “the claims call for a
`
`handheld device housing two cameras” but “[t]he closest explanation in the
`
`specification at paragraphs [0114] and [0123] discloses a handheld device (a cell
`
`phone) [that] contains one camera for imaging.” Id. In response, the Applicant
`
`identified subject matter contained in Applicant’s U.S. Patent 6,750,848, which was
`
`incorporated by reference. Id. at 128. Applicant sought to amend the specification to
`
`include Figure 18 and paragraphs 214.01–214.13 from this incorporated disclosure.
`
`Id. at 122–127.
`
`In a second non-final office action, the Examiner withdrew the written
`
`description rejection and the claims as obvious over U.S. Patent 6,788,336 of
`
`Silverbrook (“Silverbrook”) in view of a variety of secondary references, including
`
`U.S. Patent 5,845,006 of Sumi, et al. (“Sumi”), U.S. Patent 6,204,852 Kumar et al.
`
`4
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`(“Kumar”), U.S. Patent 5,491,507 of Umezawa et al. (“Umezawa”), and U.S. Patent
`
`5,940,126 of Kimura (“Kimura”). Id. at 136-141.
`
`In response, Applicant canceled claims 1-23, amended claims 24-30, and
`
`added new claims 31-37 such that all pending claims required two cameras having
`
`non-overlapping fields of view. Id. at 157-162. Regarding claim 24 (the sole
`
`independent claim), which was rejected as obvious over Silverbrook in view of
`
`Kimura, Applicant argued neither prior art reference “perform a control function
`
`based on the output of first or second cameras having non-overlapping fields of
`
`view.” Id. at 160. A Notice of Allowance issued April 27, 2012. Id. at 166.
`
`C. A Person Having Ordinary Skill in the Art
`
`A person having ordinary skill in the art (“PHOSITA”) at the time of the ’924
`
`Patent would have had at least a bachelor’s degree in electrical engineering or
`
`equivalent with at least one year of experience in the field of human computer
`
`interaction. Additional education or experience might substitute for the above
`
`requirements. Bederson Dec. (Ex. 1003), ¶¶ 29-31.
`
`III. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104
`
`A.
`
`Standing Under 37 C.F.R. § 42.104(A)
`
`Petitioner certifies that the ’924 Patent is available for IPR and that Petitioner
`
`is not barred or estopped from requesting an IPR challenging the claims of the ’924
`
`Patent. Specifically, (1) Petitioner is not the owner of the ’924 Patent, (2) Petitioner
`
`5
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`has not filed a civil action challenging the validity of any claim of the ’924 Patent,
`
`and (3) this Petition is filed less than one year after the Petitioner was served with a
`
`complaint alleging infringement of the ’924 Patent.
`
`B. Challenge Under 37 C.F.R. § 42.104(B) and Relief Requested
`
`In view of the prior art and evidence presented, claims 1–14 of the ’924 Patent
`
`are unpatentable and should be cancelled. 37 C.F.R. § 42.104(b)(1). Further, based
`
`on the prior art references identified below, IPR of the Challenged Claims should be
`
`granted. 37 C.F.R. § 42.104(b)(2).
`
`Exhibits
`Ex. 1004,
`Ex. 1005
`
`Proposed Ground of Unpatentability
`Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA
`35 U.S.C. § 103 over Canadian Published Patent Application
`CA2,237,939A1 to Mann (“Mann”) in view of U.S. Patent
`6,144,366 to Numazaki. et al. (“Numazaki”)
`Ground 2: Claims 7, 8, 10, 12, and 13 are obvious under pre-
`AIA 35 U.S.C. § 103 over Mann in view of Numazaki and in
`further view of U.S. Patent 6,539,100 to Amir, et al. (“Amir”)
`Ground 3: Claims 6 and 9 are obvious under pre-AIA 35 U.S.C.
`§ 103 over Mann in view of Numazaki and in further view of
`U.S. Patent 5,666,157 to David G. Aviv (“Aviv”)
`
`Section IV identifies where each element of the Challenged Claims is found
`
`Ex. 1004,
`Ex. 1005,
`Ex. 1006
`Ex. 1004,
`Ex. 1005,
`Ex. 1007
`
`in the prior art. 37 C.F.R. § 42.104(b)(4). The exhibit numbers of the evidence relied
`
`upon to support the challenges are provided above and the relevance of the evidence
`
`to the challenges raised is provided in Section IV. 37 C.F.R. § 42.104(b)(5). Exhibits
`
`1001-1017 are also attached.
`
`6
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`C. Claim Construction Under 37 C.F.R. § 42.104(B)(3)
`
`In this proceeding, claims are interpreted under the same standard applied by
`
`Article III courts (i.e., the Phillips standard). See 37 C.F.R § 42.100(b); see also 83
`
`Fed. Reg. 197 (Oct. 11, 2018); Phillips v. AWH Corp., 415 F.3d 1303, 1312 (Fed.
`
`Cir. 2005) (en banc). Under this standard, words in a claim are given their plain
`
`meaning which is the meaning understood by a person of ordinary skill in the art in
`
`view of the patent and file history. Phillips, 415 F.3d 1303, 1212–13. For purposes
`
`of the proposed grounds below, Petitioner proposes no terms require express
`
`construction.
`
`IV. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`
`A. Ground 1: Claims 1-6, 11, and 14 are obvious under pre-AIA 35
`U.S.C. § 103 over Mann in view of Numazaki
`Overview of Mann
`
`Canadian Published Patent Application CA2,237,939A1 to Steve Mann
`
`(“Mann”) (Ex. 1004) was filed on June 29, 1998, published on August 28, 1998, and
`
`is prior art to the ’924 Patent under at least 35 U.S.C. § 102(a) (pre-AIA). Mann was
`
`not cited or considered during prosecution of the ’924 Patent. ’924 Patent (Ex.
`
`1001).
`
`Mann discloses a “personal video annotation system.” Mann (Ex. 1004),
`
`Abstract. The invention “allows the user to take notes with pen and paper (or pen
`
`and screen) and continuously record video together with the written notes.” Id. at 8.
`
`7
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Figure 1 depicts an embodiment with a “personal digital assistant (PDA).” Id. at 10.
`
`Mann refers to the PDA, synonymously, as a “hand-held device.” Id. at 12. The PDA
`
`includes a first camera 110, second camera 150, screen for notetaking 130, pen 140,
`
`and auxiliary screen 120 for displaying the view captured by the first camera 110.
`
`Id. at 11-12.
`
`Id. at Fig. 1 (annotated to identify the key components of Mann’s PDA).
`
`
`
`8
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`The PDA is connected, via wire 160 that may “run up the sleeve of the user”
`
`to a battery pack worn by that user and communications system 176 containing a
`
`“packet radio terminal node controller (high level data link controller with modem)
`
`and radio, which typically establishes an Internet connection by way of antenna
`
`178.” Id. at 12.
`
`
`
`Id. at Fig. 1 (annotated to illustrate PDA information transmission elements).
`
`In operation, Mann teaches the first camera 110 “captures a view of a person
`
`standing in front of the user of the PDA,” while the second camera 150 “may be used
`9
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`if the user wishes to make a video recording of himself/herself while recording
`
`another person with camera 110.” Id. at 11-12. The cameras are used for
`
`“simultaneously determining a quantity of light arriving from a plurality of
`
`directions and or at a plurality of locations,” which may take the form of a video, or
`
`“signal recorded from two cameras and several microphones arranged in an array.”
`
`Id. at 11. This ensures “both sides of [a] conversation may be simultaneously
`
`recorded by the two cameras” and “edited later, so that there could, for example, be
`
`a cut back and forth between the two cameras to follow the natural flow of the
`
`conversation.” Id. at 12. Mann intends the invention to be “useful as a new
`
`communications medium, in the context of collaborative photography, collaborative
`
`videography, and telepresence.” Id. at 16.
`
`Mann also taches a “wristwatch embodiment 300 of the invention depicted in
`
`Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch 300 houses cameras 310 and 350
`
`and auxiliary screen 320. Id. at 13.
`
`10
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 3 (annotated to show similarity with PDA configuration). Users control
`
`Mann’s wristwatch by interacting with a clock face superimposed on the wristwatch
`
`display’s touch-based interface. Id. at 14-15.
`
`
`
`11
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 4. Users can enter information like letters, symbols, and numbers using
`
`finger strokes in vector directions such as from top to bottom, or “downwards.” Id.
`
`at 15. These user instructions can also control the wristwatch such as starting or
`
`stopping specific operations:
`
`[t]he numbers may be assigned a secondary meaning (e.g., select “0” to
`stop recording, “4” to kill all processes and halt the processor, “7” to
`wake up the system from sleep mode, etc.”
`
`Id. at 14. These control operations may be “combined with the video recording made
`
`from the scene” and subsequently transmitted to a legal expert at a remote location.
`
`Id. at 12-15.
`
`12
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Because Mann, like the ’924 Patent, discloses a portable camera system that
`
`may be controlled by human gesture input, Mann is in the same field of endeavor as
`
`the ’924 Patent. Compare Mann (Ex. 1004), 1 (describing a “new photographic or
`
`video means and apparatus typically comprising a hand-held portable electronic
`
`camera system with viewfinder means and electronic pen-based annotation means”
`
`in the form of a PDA and wristwatch with a touch-based interface screen), 11, 12,
`
`14 (describing clock face gestures used to control device) with ’924 Patent (Ex
`
`1001), Abstract, 25:40-41, 25:50-63 (describing a “[m]ethod and apparatus . . . to
`
`enable rapid TV camera and computer based sensing in . . . handheld devices” used
`
`to detect a user’s fingers and gestures)).
`
`Mann is therefore analogous art to the ’924 Patent. Bederson Dec. (Ex. 1003),
`
`¶¶ 35-38.
`
`Overview of Numazaki
`
`U.S. Patent No. 6,144,366 to Numazaki et al. (“Numazaki”) (Ex. 1005) was
`
`filed on October 17, 1997 and is prior art to the ’924 Patent under at least 35 U.S.C.
`
`§ 102(e) (pre-AIA). Numazaki was not cited or considered during prosecution of the
`
`’924 Patent. ’924 Patent (Ex. 1001).
`
`Numazaki is generally directed to a method for detecting a gesture or the
`
`movement of a user’s hand. Numazaki (Ex. 1005), Abstract, 4:9-40. Numazaki
`
`purports to have improved upon prior methods by using a controlled light source to
`
`13
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`illuminate the target object (e.g., the user’s hand), a first camera unit (referred to by
`
`Numazaki as a “photo-detection unit”),1 and a second camera unit. Id. at 11:9-23.
`
`This arrangement is illustrated in Fig. 2 below:
`
`
`
`Id. at Fig. 2. A timing control unit is used to turn lighting unit 101 on (i.e.,
`
`illuminating the target object) when the first camera unit is active and off when the
`
`second camera unit is active. Id. at 11:20-32. The result of this light control is the
`
`first camera unit captures an image of the target object illuminated by both natural
`
`light and the lighting unit 101 and the second camera unit captures an image of the
`
`
`
`1 A PHOSITA would have considered Numazaki’s photo-detection units to be
`
`camera units. Bederson Dec. (Ex. 1003), ¶ 39 (explaining that Numazaki describes
`
`using CMOS or CCD sensor units at 15:24-16:19, which were two of the more
`
`common optical sensors used in camera units at the time).
`
`14
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`target object illuminated by only natural light. Id. at 11:33-39. The difference
`
`between the two images—obtained by difference calculation unit 111—represents
`
`the “reflected light from the object resulting from the light emitted by the lighting
`
`unit 101.” Id. at 11:43-51. This information is then used by feature data generation
`
`unit 103 to determine gestures, pointing, etc. of the target object that may be
`
`converted into commands executed by a computer. Id. at 10:57-66.
`
`In an eighth embodiment, Numazaki describes implementing this structure in
`
`“a compact portable information device equipped with the information input
`
`generation apparatus of the present invention, which is in a size that can be held by
`
`one hand.” Id. at 52:5-8. This PDA-style structure is illustrated in Fig. 78 below:
`
`
`
`15
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Id. at Fig. 78. Numazaki explains that window 712 includes the light source and
`
`camera sensor, which is used to detect gestures performed by the user such that the
`
`“position of a cursor 714 on the screen can be controlled by moving a finger 713 in
`
`front of this window 712.” Id. at 52:14-16. Numazaki also describes implementing
`
`the lighting and camera structure in a wristwatch such that “a cursor 717 can be
`
`controlled by moving a finger 716” within an “operation space” where “[w]indows
`
`718 and 719 are provided for the lighting unit and the photo-detection sensor unit.”
`
`Id. at 52:20-29. This arrangement is illustrated in Fig. 79 below:
`
`
`
`16
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Id. at Figure 79 (annotated to emphasize orientation of light source 718 and camera
`
`719 used to track a fingertip gesture to manipulate cursor 717). Numazaki orients the
`
`windows on the device in this way so that “it is possible to prevent the operating
`
`finger from obstructing the view of the display.” Id. at 52:30-32.
`
`Numazaki’s eighth embodiment
`
`incorporates “the
`
`information
`
`input
`
`generation apparatus of the present invention as described in the above
`
`embodiments.” Id. at 50:21-24. A PHOSITA would have understood that the
`
`referenced information input generation apparatus is that illustrated in Fig. 2 and
`
`described in the corresponding disclosure. Bederson Dec. (Ex. 1003), ¶ 42
`
`(explaining that Numazaki describes its controlled light and camera configuration as
`
`key to its invention and noting that Numazaki at 53:22-36 teaches that the eighth
`
`embodiment uses the precise image difference calculation taught by Fig. 2 and its
`
`corresponding disclosure).
`
`Because Numazaki, like the ’924 Patent, discloses a portable camera system
`
`that may be controlled by human gesture input, Numazaki is in the same field of
`
`endeavor as the ’924 Patent. Compare Numazaki (Ex. 1005), 10:11-13, 10:61-66,
`
`52:20-32 (describing an “information input scheme in which . . . light is irradiated
`
`onto a target object from a light source, and [that] reflected light . . . is captured as
`
`an image, so that information on this target object such as its shape, motion, distance,
`
`etc., can be obtained,” where the target object is a hand and the information obtained
`
`17
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`is a “gesture or a pointing according to the feature data extracted from the reflected
`
`light image of the hand” to “operate a computer” in a wrist watch embodiment) with
`
`’924 Patent (Ex 1001), Abstract, 25:40-41, 25:50-63 (describing a “[m]ethod and
`
`apparatus . . . to enable rapid TV camera and computer based sensing in . . . handheld
`
`devices” used to detect a user’s fingers and gestures). Numazaki is therefore
`
`analogous art to the ’924 Patent. Bederson Dec. (Ex. 1003), ¶¶ 39-43.
`
`Motivation to Modify Mann in view of Numazaki
`
`Mann’s PDA and “wristwatch videotelephone” devices are specially designed
`
`for covert applications such as investigative journalism in which the subject does not
`
`know whether the camera is recording. Mann (Ex. 1004), 2-3, 11-12 (describing
`
`options to conceal operation from the subject and noting the PDA is well suited for
`
`“investigative journalism”). By positioning two cameras in roughly perpendicular
`
`directions, a wristwatch operator can “place his or her arm naturally upon [a] counter
`
`and aim [one] camera . . . at the official behind the counter without appearing
`
`unusual” while a second camera “may be included if the customer wishes to record
`
`himself/herself” so that “both sides of the conversation would be recorded.” Id. at
`
`13, 12 (noting the PDA is also equipped such that “both sides of the conversation
`
`may be simultaneously recorded by the two cameras”). The two-camera arrangement
`
`in the wristwatch embodiment is illustrated in Fig. 3 below:
`
`18
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 3 (annotated to show the camera orientations and computer element). Next,
`
`the operator enters commands into a “touch sensitive clockface” using finger strokes,
`
`such as upwards from bottom to top to stop the recording. Id. at 14.
`
`
`
`19
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 4 (annotated to show operator command entry of “0,” which stops a
`
`recording). Although the specific command entry process is not described with
`
`respect to the PDA, a PHOSITA would have understood that physical command
`
`entries, such as using the PDA stylus, would have been used to command functions
`
`such as starting or stopping the recording. Bederson Dec. (Ex. 1003), ¶ 45.
`
`For a number of reasons, rather than require physical gestures or physical
`
`inputs (e.g., using the PDA stylus), a PHOSITA would have been motivated to
`
`utilize Mann’s front-facing camera to recognize gestures performed in front of that
`
`camera pursuant to the teachings of Numazaki. First, a PHOSITA would have
`
`20
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`anticipated success in modifying Mann’s wristwatch based on the similarities in
`
`functionality and structure of the computer, cameras, and control functionality taught
`
`by Mann and Numazaki. Bederson Dec. (Ex. 1003), ¶ 46 (describing the similarities
`
`between Mann and Numazaki and concluding the modification would have been well
`
`within the skill set of a PHOSITA); see also KSR, 550 U.S. at 417 (obvious to use
`
`known techniques to improve similar devices in the same way). Indeed, Mann
`
`already contemplates the requisite structure to capture gestures (i.e., PDA and
`
`wristwatch devices with cameras facing that user), and it expressly describes
`
`receiving finger-based input to control the operation of the wristwatch. Mann (Ex.
`
`1004), 11-12 (describing the PDA configuration with user-facing camera and PDA
`
`stylus for input), 13 (describing a two-camera system in the wristwatch embodiment
`
`for recording both the user and subject), 15 (describing a user entering commands
`
`on a touch sensitive watch face). Numazaki similarly describes PDA and wristwatch
`
`devices with cameras facing the user such that a user can perform gestures over the
`
`devices to exert control over the operation of the device. Numazaki at 52:5-32. A
`
`PHOSITA would have recognized that modifying Mann as proposed would have
`
`required a straightforward replacement of Mann’s native user-facing imaging with
`
`Numazaki’s gesture-recognition hardware. Bederson Dec. (Ex. 1003), ¶ 46 (noting
`
`both Mann and Numazaki describe user-facing imaging hardware mounted within
`
`PDA and wristwatch devices). With that hardware swap in place, a straightforward
`
`21
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`software update (i.e., adding Numazaki’s gesture detection routines) would allow
`
`Mann’s devices to process no-touch gestures. Id.
`
`Second, a PHOSITA would have understood that allowing a user to perform
`
`no-touch gestures over the user-facing camera improves the covert nature of Mann’s
`
`devices. Bederson Dec. (Ex. 1003), ¶ 47. Indeed, physically touching a watch face
`
`or using a stylus to interact with the PDA in order to trigger a recording runs the risk
`
`of being noticed by the subject. Mann expressly teaches that remaining covert and
`
`avoiding attention from the subject are goals of its invention. Mann at 1-2
`
`(explaining that traditional cameras “create[] a visual disturbance to others and
`
`attract[] considerable attention on account of the gesture of bringing the camera up
`
`to the eye” and describing the goal of the invention is to allow covert recordings or
`
`recordings for which the subject “cannot readily determine whether or not the
`
`apparatus is in use recording”). Allowing users to perform finger-based no-touch
`
`gestures over the user-facing camera in Mann’s devices would improve the covert
`
`nature of these devices—a PHOSITA would have understood that such gestures
`
`would draw much less attention than physically interacting with the watch face or
`
`using the PDA stylus. Bederson Dec. (Ex. 1003), ¶ 47.
`
`Finally, Numazaki provides express motivation to implement Mann’s device
`
`with no-touch gesture functionality. Namely, Numazaki teaches that, by orienting
`
`the light and camera windows on the device to capture gestures, “it is possible to
`
`22
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`prevent the operating finger from obstructing the view of the display.” Numazaki
`
`(Ex. 1005), 52:30-32. A PHOSITA would have understood that Mann’s native
`
`touch-based gesture control would have suffered from this precise downside—
`
`obstructing the user’s view of the watch face and PDA display when the user
`
`interacts with the device. Bederson Dec. (Ex. 1003), ¶ 48 (explaining that Mann’s
`
`touch gestures would necessarily obstruct a user’s view when gesturing). A
`
`PHOSITA would further have recognized that Mann’s native touch-base gestures on
`
`the wristwatch embodiment would inevitably cause the user’s fingers to touch the
`
`glass covering the user-facing camera. As seen in Fig. 3 below, Mann’s user-facing
`
`camera (350) is positioned on the watch face:
`
`23
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Mann (Ex. 1004), Fig. 3. Requiring a user to physically swipe a finger across the
`
`watch face in order to input a command would inevitably cause the user’s finger to
`
`traverse the glass covering the camera, causing the fidelity of that camera to decrease
`
`over time due to grease and grime from the user’s finger (or require regular
`
`cleanings). Bederson Dec. (Ex. 1003), ¶ 48. To avoid these downsides, a PHOSITA
`
`would have been motivated to supplant Mann’s touch-based gestures with
`
`Numazaki’s no-touch technique. Id. (describing the similarities between Mann and
`
`Numazaki and concluding the physical orientation recommended by Numazaki
`
`would have allowed Mann’s device to realize both the benefits discussed by
`
`24
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Numazaki and those benefits that would have been apparent to a PHOSITA
`
`analyzing Mann’s native design); see also KSR, 550 U.S. at 418-19 (demonstrating
`
`a teaching, suggestion, or motivation to combine known elements in order to show
`
`that the combination is obvious).
`
`
`
`Claim 1
`
`1[P] A handheld device comprising:
`
`To the extent the preamble is limiting, Mann discloses a handheld device.
`
`Mann discloses a “new photographic or video means and apparatus typically
`
`comprising a hand-held portable electronic camera system with viewfinder
`
`means and electronic pen-based annotation means.” Mann (Ex. 1004), 1 (emphasis
`
`added). In Figure 1 below, Mann depicts an embodiment of a “personal digital
`
`assistant (PDA)”:
`
`25
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 1 (annotated to illustrate the handheld portion of Mann’s handheld PDA
`
`embodiment), 10 (describing the same), 12 (referring to the PDA as a “hand-held
`
`
`
`device”).
`
`26
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Mann also teaches a “wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA, wristwatch2 300 houses cameras 310 and
`
`350 and auxiliary screen 320, also described as a “viewfinder” that doubles as a
`
`computer screen display unit. Id. at page 10, 13. This wristwatch embodiment is
`
`illustrated in Fig. 3 below:
`
`
`
`2 A PHOSITA would have considered Mann’s wristwatch to be handheld. Bederson
`
`Dec. (Ex. 1003), ¶ 49 (explaining that Mann describes a PDA and wristwatch
`
`embodiment using the same configuration of hardware for manipulation by a user’s
`
`hands, that the term handheld connotes devices small enough to be held in one’s
`
`hand, but does not necessarily require the device is always operated while in a hand,
`
`and that, even if operation in a hand were required, a watch may be removed from
`
`the wrist and held in a user’s hands while retaining the same functionality and
`
`structure).
`
`27
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 3 (annotated to show camera, viewfinder, and housing elements).
`
`1[a] a housing;
`
`In Figure 1, Mann depicts a PDA as a “hand-held device” configured to house
`
`several components, including a first camera 110, second camera 150, screen for
`
`notetaking 130, pen 140, and auxiliary screen 120 for displaying the view captured
`
`by the first camera 110. Id. at 11-12.
`
`
`
`28
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 1 (annotated to illustrate the hardware components housed within Mann’s
`
`handheld PDA embodiment). Mann discloses the inventions as “related to the
`
`Personal Digital Assistant[] (PDA) such as the Palm Pilot . . . and the general ideas
`
`of pen-based computing.” Id. at 3.
`
`Mann also teaches a “wristwatch embodiment 300 of the invention depicted
`
`in Fig. 1.” Id. at 13. Like Mann’s PDA embodiment, wristwatch 300 houses cameras
`
`310 and 350 and auxiliary screen 320. Id. at page 13. Mann describes the wristwatch
`
`synonymously as a “housing worn on the wrist” and “housing.” Id. at 20-21 (Claims
`
`13-15). This wristwatch embodiment is illustrated in Fig. 3 below:
`
`29
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`Id. at Fig. 3 (annotated to illustrate components housed within Mann’s wristwatch
`
`
`
`embodiment).
`
`1[b] a computer within the housing;
`
`In the PDA embodiment, Mann discloses a PDA connected to computer 174
`
`housed within a “body worn pack” 170 and connected by an “additional wire 160 . .
`
`. run up the sleeve of the user.” Id. at 12.
`
`30
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`
`
`
`Id. at Fig. 1 (annotated to show PDA 100 wired connection to computer 174 housed
`
`within body worn pack 170). However, Mann also teaches reconfiguring this
`
`embodiment to situate the computer inside the PDA:
`
`The body-worn pack 170 would likely disappear in the final
`embodiment of the invention which would contain the functionality
`of the pack 170 inside the hand-held device 100.”
`
`Id. at 12 (emphasis added). By embedding the computer within the PDA and
`
`discarding the wire-coupled configuration, Mann’s invention provides users
`
`“convenience without a change of lifestyle” that former “wearable camera
`
`system[s]” requiring “special wiring of the body a-priori” could not. Id. at 2.
`
`31
`
`
`
`IPR2021-00923
`U.S. Patent No. 8,194,924
`Similarly, Mann’s wristwatch embodiment depicts the same configuration,
`
`where “cabling 360 runs up the sleeve of the wearer of the watch, to an internet-
`
`connected body-worn computer system.” Id. at 13 (emphasis added).
`
`
`
`Id. at Fig. 3 (annotated to show wristwatch 300 cabled connection to computer 174
`
`housed within body worn pack 170). As with the PDA embodiment, Mann teaches
`
`that the cabling 360 “may be eliminated, so that the unit would be [] entirely self-
`
`contained.” Id. at 14 (e