Trials@uspto.gov
`571-272-7822
`
`PUBLIC VERSION
`
`Paper 61
`Entered: April 9, 2024
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE, INC.,
`Petitioner,
`v.
`COREPHOTONICS LTD.,
`Patent Owner.
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`Before GREGG I. ANDERSON, JOHN F. HORVATH, and
`MONICA S. ULLAGADDI, Administrative Patent Judges.
`HORVATH, Administrative Patent Judge.
`
`JUDGMENT
`Final Written Decision on Remand
`Determining All Challenged Claims Unpatentable
`35 U.S.C. § 318(a)
`
`
`571-272-7822
`
`PUBLIC VERSION
`
`Paper 61
`Entered: April 9, 2024
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE, INC.,
`Petitioner,
`v.
`COREPHOTONICS LTD.,
`Patent Owner.
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`Before GREGG I. ANDERSON, JOHN F. HORVATH, and
`MONICA S. ULLAGADDI, Administrative Patent Judges.
`HORVATH, Administrative Patent Judge.
`
`JUDGMENT
`Final Written Decision on Remand
`Determining All Challenged Claims Unpatentable
`35 U.S.C. § 318(a)
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`INTRODUCTION
`I.
`A. Background and Summary
`Apple, Inc. (“Petitioner”) filed a Petition requesting inter partes
`review of claims 1–16, 18, 23–38, and 40 (“the challenged claims”) of
`U.S. Patent No. 10,225,479 B2 (Ex. 1001, “the ’479 patent”). Paper 3
`(“Pet.”), 9. Corephotonics Ltd. (“Patent Owner”) filed a Preliminary
`Response. Paper 8 (“Prelim. Resp.”). Upon consideration of the Petition
`and Preliminary Response, we instituted inter partes review of all
`challenged claims on all grounds raised. Paper 10 (“Dec. Inst.”).
`Patent Owner filed confidential (Paper 15) and public (Paper 39)
`versions of its Response to the Petition. See Paper 39 (“PO Resp.”). 1
`Petitioner filed confidential (Paper 24) and public (Paper 40) versions of a
`Reply. See Paper 40 (“Pet. Reply”). Patent Owner filed a Sur-Reply. See
`Paper 32 (“PO Sur-Reply”). An oral hearing was held on August 12, 2021,
`and the hearing transcript is included in the record. See Paper 49 (“Tr.”).
`We issued a Final Written Decision that found Petitioner had failed to
`demonstrate the challenged claims were unpatentable based on our
`construction of the term “a fused image with a point of view (POV) of the
`Wide camera,” which we construed to mean “a fused image having a Wide
`perspective POV and a Wide position POV.” Paper 51 (“Final Dec.”), 9–12.
`Although finding the case presented “a close issue of claim construction,”
`the Court of Appeals for the Federal Circuit construed the term to require a
`fused image that “maintain[s] Wide perspective point of view or Wide
`position point of view, but does not require both.” Apple Inc. v.
`
`
`1 Unless otherwise noted, we cite to the public versions of the papers in this
`proceeding.
`
`2
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Corephotonics Ltd., 81 F.4th 1353, 1357, 1359 (Fed. Cir. 2023)
`(“Corephotonics”). The Federal Circuit, therefore, vacated our Final
`Written Decision and remanded the case “for further proceedings in view of
`this claim construction.” Id. at 1359.
`This is a Final Written Decision on Remand under 35 U.S.C. § 318(a)
`and 37 C.F.R. § 42.73. For the reasons set forth below, we find Petitioner
`has shown by a preponderance of evidence that claims 1–16, 18, 23–38, and
`40 of the ’479 patent are unpatentable.
`B. Real Parties-in-Interest
`Petitioner and Patent Owner identify themselves, respectively, as the
`real parties-in-interest. Pet. 1; Paper 5, 1.
`C. Related Matters
`Petitioner and Patent Owner identify Corephotonics Ltd. v. Apple Inc.,
`5:19-cv-04809 (N.D. Cal.), as a district court proceeding that can affect or
`be affected by this proceeding, and Petitioner also identifies IPR2020-00906
`as an inter partes review that can affect or be affected by this proceeding.
`Pet. 1; Paper 5, 1. In addition, we note that the ’479 patent is part of a
`family of patents and patent applications that include at least U.S. Patent
`Nos. 9,185,291, 9,661,233, 10,015,408, and 10,326,942. Ex. 1001, code
`(63). Many of these patents were or currently are involved in inter partes
`review proceedings that could affect or be affected by a decision in this
`proceeding, including IPR2018-01348, IPR2020-00487, IPR2020-00488,
`IPR2020-00489, and IPR2020-00860.
`
`3
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`D. Evidence Relied Upon?
`
`Effective Date|Exhibit
`
`US 7,859,588B2|Dec. 28,2010
`Richard Szeliski, Computer Vision
`Algorithms andApplications, 468—503
`2011)
`(“Szeliski”
`
`2011
`
`1013
`
`
`Parulski, Konno, Stein
`
`Konno? JP 2013/106289A|May 30, 2013
`
`US 8,908,041 B2 Feb. 7, 20134|1973
`
`US 8,406,569 B2
`Segall
`Mar.26, 2013
`1024
`
`E.
`
`Instituted Grounds ofUnpatentability
`
`Weinstituted review on the following grounds:
`
`BUSCS
`
`18, 23, 32-36,|103(a) Parulski, Konno
`
`1, 10-14, 16,
`1
`
`38, 40
`2-4, 24-26
`5-9, 27-31
`15,37
`
`Parulski, Konno, Szeliski
`Parulski, Konno, Szeliski, Segall
`
`* Petitioner also relies upon the Declarations ofFredo Durand, Ph.D.(Exs.
`1003, 1038) and José Sasian, Ph.D. (Ex. 1021).
`> Konnoisa certified translation ofa Japanese Patent Application originally
`published in Japanese. See Ex. 1015, 34-59.
`* Petitioner identifies Stein as prior art under 35 U.S.C. § 102(a)(2) based on
`the February 7, 2013 filing date ofa provisionalapplication to which Stein
`claimspriority. See Pet.9. Patent Ownerdoesnotdispute this. See PO
`Resp. 1-47.
`
`4
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`II. ANALYSIS
`
`A. The ’479 Patent
`The ’479 patent is directed to “a dual-aperture zoom imaging system
`(also referred to simply as ‘digital camera’ or ‘camera’).” Ex. 1001, 6:21–
`23. Figure 1A, reproduced below, illustrates a dual-aperture zoom digital
`camera 100.
`
`Figure 1A is a “block diagram illustrating a dual-aperture zoom” digital
`camera 100. Id. at 5:64–65. Camera 100 includes a wide imaging
`subsystem consisting of wide lens 102, wide sensor 104, and wide image
`signal processor (“ISP”) 106, and a tele imaging subsystem consisting of tele
`lens 108, tele sensor 110, and tele ISP 112. Id. at 6:24–29.
`5
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Camera 100 also includes controller 114, which includes sensor
`control 116, user control 118, video processing module 126 and still
`processing module 128. Id. at 6:33–37. User control 118 controls various
`camera functions, including, operational mode 120, region of interest
`(“ROI”) 122, and zoom factor (“ZF”) 124, the latter of which allows a user
`“to choose a zoom factor.” Id. at 6:38–40, 6:50–51. Sensor control 116
`chooses “which of the [wide or tele] sensors is operational” based on the
`selected zoom factor. Id. at 6:41–45. ROI function 122 allows a user to
`“choose a region of interest,” i.e., a sub-region “on which both sub-cameras
`are focused.” Id. at 6:46–50.
`The dual lenses allow camera 100 to take an image having a depth-of-
`field (“DOF”) that is shallower than the DOF of the wide lens “by taking
`advantage of the longer focal length of the Tele lens.” Id. at 4:23–27. The
`image taken with the Tele lens can be enhanced “by fusing data from an
`image captured simultaneously with the Wide lens.” Id. at 4:27–30. For
`example, the Tele lens can focus “on a subject of the photo” and the Wide
`lens can focus on “a closer distance than the subject so that objects behind
`the subject appear very blurry.” Id. at 4:30–34. Then, a shallow depth-of-
`field image can be formed when “information from the out-of-focus blurred
`background in the Wide image is fused with the original Tele image
`background information, providing a blurrier background and even
`shallower DOF.” Id. at 4:34–38.
`The process for fusing images taken with the Wide and Tele lenses is
`shown in Figure 5 of the ’479 patent, which is reproduced below.
`
`6
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`Figure 5 is a flow chart depicting a method for acquiring a zoom image in a
`dual lens camera. Id. at 9:39–40. At step 502, separate images are captured
`
`7
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`by each of the Wide and Tele lenses. Id. at 9:40–44. At step 504, these
`images are aligned on an epipolar line. Id. at 9:46–47. At step 506, a
`registration map is generated. Id. at 9:47–49. At step 508, the registration
`map is used to resample the Tele image. Id. at 9:50–51. At step 510, Tele
`image pixel values are compared to Wide image pixel values, and if a
`significant difference is detected, the Wide image pixel values are chosen for
`the output image. Id. at 9:51–58. Finally, at step 512, a fused image is
`generated from the re-sampled Tele image and the Wide image. Id. at 9:58–
`60. The ’906 patent discloses that by “register[ing] Tele image pixels to a
`matching pixel set within the Wide image pixels, . . . the output image will
`retain the Wide POV” or point-of-view. Id. at 5:23–26.
`B. Illustrative Claims
`Of the challenged claims, claims 1 and 23 are independent and
`substantially similar in scope. Claim 1 recites a dual-aperture digital camera
`configured to generate a fused image from images taken with wide angle and
`telephoto lenses, and claim 23 recites a method for generating such a fused
`image using a dual-aperture digital camera. Compare Ex. 1001, 13:22–50,
`with id. at 15:49–67. The remaining challenged claims depend directly or
`indirectly from claims 1 or 23. Claim 1 is illustrative of the challenged
`claims and is reproduced below.
`1. A dual-aperture digital camera for imaging an object or
`scene, comprising:
`a) a Wide camera comprising a Wide lens and a Wide image
`sensor, the Wide camera having a respective field of view
`FOVW and being operative to provide a Wide image of the
`object or scene;
`b) a Tele camera comprising a Tele lens and a Tele image
`sensor, the Tele camera having a respective field of view
`
`8
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`FOVT narrower than FOVW and being operative to provide a
`Tele image of the object or scene, wherein the Tele lens has
`a respective effective focal length EFLT and total track
`length TTLT fulfilling the condition EFLT / TTLT > 1;
`c) a first autofocus (AF) mechanism coupled mechanically to,
`and used to perform an AF action on the Wide lens;
`d) a second AF mechanism coupled mechanically to, and used
`to perform an AF action on the Tele lens; and
`e) a camera controller operatively coupled to the first and
`second AF mechanisms and to the Wide and Tele image
`sensors and configured to control the AF mechanisms and to
`process the Wide and Tele images to create a fused image,
`wherein areas in the Tele image that are not focused are not
`combined with the Wide image to create the fused image
`and
`wherein the camera controller is further operative to output
`the fused image with a point of view (POV) of the Wide
`camera by mapping Tele image pixels to matching pixels
`within the Wide image.
`Id. at 13:22–50.
`C. Level of Ordinary Skill in the Art
`Petitioner identifies a person of ordinary skill in the art (“POSITA”) at
`the time of the invention as someone that would have had “a bachelor’s or
`the equivalent degree in electrical and/or computer engineering or a related
`field and 2-3 years of experience in imaging systems including image
`processing and lens design.” Pet. 6 (citing Ex. 1003 ¶ 13). In our Institution
`Decision, we adopted this description as our own. See Dec. Inst. 11–12.
`Neither party disputes that preliminary finding, which we maintain for
`purposes of this decision. See PO Resp. 3–4; Pet. Reply 1–27.
`
`9
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`D. Claim Construction
`In inter partes reviews, we interpret a claim “using the same claim
`construction standard that would be used to construe the claim in a civil
`action under 35 U.S.C. 282(b).” 37 C.F.R. § 42.100(b) (2019). Under this
`standard, a claim is construed “in accordance with the ordinary and
`customary meaning of such claim as understood by one of ordinary skill in
`the art and the prosecution history pertaining to the patent.” Id. Only claim
`terms which are in controversy need to be construed and only to the extent
`necessary to resolve the controversy. See Nidec Motor Corp. v. Zhongshan
`Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed. Cir. 2017).
`As noted above, in our Final Written Decision we construed the term
`“a fused image with a point of view (POV) of the Wide camera” to mean “a
`fused image having a Wide perspective POV and a Wide position POV.”
`Final Dec. 9–12. The Federal Circuit disagreed, finding “[t]he claim term
`requires only that the fused image maintain ‘a point of view of the Wide
`camera,’ i.e., only one of the disclosed types of Wide point of view” and that
`the fused image must “maintain Wide perspective point of view or Wide
`position point of view, but does not require both.” Corephotonics, 81 F.4th
`at 1359. We apply this construction in our Final Written Decision on
`Remand.
`E. Obviousness over Parulski and Konno
`Petitioner argues claims 1, 10–14, 16, 18, 23, 32–36, 38, and 40 are
`unpatentable as obvious over Parulski and Konno. Pet. 10–41; Pet. Reply 7–
`14, 18–27. Patent Owner disputes this. PO Resp. 26–31, 35–47; PO Sur-
`Reply 5–10. For the reasons discussed below, Petitioner establishes by a
`
`10
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`preponderance of evidence that claims 1, 10–14, 16, 18, 23, 32–36, 38, and
`40 are unpatentable as obvious over Parulski and Konno.
`1. Parulski
`Parulski discloses “a digital camera that uses multiple lenses and
`image sensors to provide an improved imaging capability.” Ex. 1005, 1:8–
`10. A schematic illustration of Parulski’s camera is shown in Figure 1,
`which is reproduced below.
`
`Figure 1 is “a block diagram . . . of a digital camera using a first zoom lens
`with a first image sensor, and a second zoom lens with a second image
`sensor.” Id. at 8:28–30.
`The camera includes “two imaging stages 1 and 2, both with zoom
`lenses 3 and 4.” Id. at 12:42–43. “[Z]oom lens 3 is controlled by a first lens
`focus adjuster, e.g., zoom and focus motors 5a, and provides an image to a
`11
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`first image sensor 12.” Id. at 12:47–49. “[Z]oom lens 4 is controlled by a
`second lens focus adjuster, e.g., zoom and focus motors 5b, and provides an
`image to a second image sensor 14.” Id. at 12:49–52. Each of zoom lenses
`3 and 4 can be “replaced with a fixed focal length lens.” Id. at 13:3–6.
`Image sensors 12 and 14 can “have a variety of aspect ratios” and “do not
`have to have the same specifications.” Id. at 13:26–32. “[C]ontrol processor
`and timing generator 40 [CPT 40] controls the first image sensor 12 . . . the
`second image sensor 14” and “the zoom and focus motors 5a and 5b.” Id. at
`13:37–42. Analog data from image sensors 12 and 14 are digitized by
`analog signal processors 22 and 24, respectively, and the digitized data is
`supplied to each of multiplexers 34 and 36. Id. at 13:48–59. CPT 40
`controls multiplexer 34 to select digitized data from either sensor 12 or 14 as
`an image signal and controls multiplexer 36 to select digitized data from the
`other of sensors 12 or 14 as an autofocus image signal. Id. at 14:1–5. Image
`processor 50 processes the digitized data from multiplexer 34 to produce a
`digital image and processes the digitized data from multiplexer 36 to
`calculate “focus detection signals that drive the first and second focus
`adjusters, that is, the zoom and focus motors 5a and 5b.” Id. at 14:15–16.
`Parulski’s dual-lens camera can be used to generate a distance or
`range map as illustrated in Figure 11, which is reproduced below.
`
`12
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`
`
`
`Figure 11 is a flow chart showing a method for processing images captured
`with a two-lens camera to generate a distance or range map. Id. at 19:49–51.
`At step 440, “a first autofocus image is captured with the lower focal length
`image capture stage,” e.g., lens 3 and image sensor 12. Id. at 20:1–3. At
`step 442, this image is “cropped and upsampled so that corresponding
`features in the two autofocus images span the same number of pixels.” Id. at
`20:3–6. At step 448, “a second autofocus image is captured with the higher
`focal length image capture stage,” e.g., lens 4 and image sensor 14. Id. at
`20:6–8. At step 480, “the second autofocus image is correlated with the
`cropped and upsampled image to determine the pixel offset between the
`images for different portions of the images.” Id. at 20:8–11. At step 482,
`these pixel offsets are “converted . . . to distances from the image capture
`device using the autofocus rangefinder calibration curve.” Id. at 20:11–14.
`
`13
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Finally, at step 484, a distance or range map is produced “showing the
`distances to different portions of the images.” Id. at 20:14–15.
`Parulski’s range map can be “used to modify the captured image
`signal or the output image for a variety of purposes,” including “to enable
`dynamic depth of field images by blurring of portions of the image that
`correspond to areas of the scene that lie outside of the desired depth of
`field.” Id. at 20:51–53, 20:63–65. For example, the range map can be used
`to modify a picture having a dog in the foreground, a field of flowers in the
`mid-ground, and a mountain range in the background. Id. at 21:7–17. “[I]f
`the user really wants to emphasize the dog more than the beautiful scenery,
`the range data can be used to isolate the mountains and the flowers, which
`can then be blurred.” Id. at 21:27–30.
`2. Konno
`Konno discloses “an imaging apparatus . . . [that] includes single-
`focus first and second imaging optical systems that face the same direction.”
`Ex. 1015 ¶ 7. Such a system is shown, for example, in Figure 21 of Konno,
`which is reproduced below.
`
`
`
`14
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Figure 21 of Konno is “a schematic view . . . of digital equipment [e.g., a
`digital camera] including first and second imaging optical units.” Id. ¶ 18.
`The digital camera includes optical units LU1 and LU2, which include
`“single-focus first and second imaging optical systems [i.e., lenses] LN1 and
`LN2 . . . for forming optical images” and “first and second imaging devices
`[i.e., sensors] SR1 and SR2 for converting the optical images . . . into
`electrical signals.” Id. ¶ 48. The camera also includes “a signal processing
`unit 1, a control unit 2, a memory 3, an operation unit 4, and a display unit
`5.” Id. ¶ 54. Control unit 2 “controls various functions including . . . a lens
`moving mechanism.” Id. “[T]he first and second imaging optical systems
`[i.e., lenses] LN1 and LN2 have different focus movements in the case of
`whole feeding.” Id. ¶ 50. Various characteristics of lenses LN1 and LN2
`(e.g., focal length, lens length, field of view) are disclosed in Table 1 of
`Konno. Id. ¶ 76.
`3. Reasons to Combine
`Petitioner argues that it would have been obvious to combine the
`teachings of Parulski and Konno because “Parulski does not provide lens
`prescription data for either the first [wide] or second [tele] fixed-focus lenses
`in its cell phone” camera. Pet. 16. Thus, Petitioner argues, a skilled artisan
`“would have looked to Konno which provides a fixed-focal length, dual-lens
`system designed for digital equipment like cell phones.” Id. at 16–17 (citing
`Ex. 1003 ¶ 57). Petitioner argues a person skilled in the art would have
`looked to Konno for lens prescription data because “Konno’s system offers
`fixed-focal length wide and telephoto lenses in a thin format for
`incorporation in a mobile device” and “Parulski teaches the importance of
`keeping the ‘z’ dimension (i.e., thickness) of its cell phone embodiment
`
`15
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`small.” Id. at 17; Ex. 1005, 24:20–27; Ex. 1015 ¶ 46. Patent Owner does
`not dispute these contentions. PO Resp. 26–35.
`We find Petitioner sets forth sufficient reasoning with rational
`underpinning to combine the teachings of Parulski and Konno. Parulski
`teaches a cell phone having a dual-lens camera and the need to have thin
`lenses, but fails to give lens prescription data for the two camera lenses.
`Konno discloses lens prescription data for a dual-lens camera utilizing two
`thin lenses. The combination, therefore, is one of familiar elements
`according to known methods to obtain predictable results or a substitution of
`one element for another known in the field to obtain a predictable result. See
`KSR Int’l Co. v. Teleflex, Inc., 550 U.S. 398, 416 (2007).
`4. Claim 1
`Claim 1 recites a dual-aperture digital camera that includes a Wide
`camera having a Wide FOV for providing a Wide image and a Tele camera
`having a narrower Tele FOV for providing a Tele image. Ex. 1001, 13:22–
`34. The Wide camera has a Wide lens and image sensor. Id. The Tele
`camera has a Tele lens and image sensor and an effective focal length (EFL)
`to total track length (TTL) ratio that is greater than 1. Id. The camera
`further includes a first autofocus mechanism to autofocus the Wide lens, a
`second autofocus mechanism to autofocus the Tele lens, and a controller to
`control the first and second autofocus mechanisms. Id. at 13:35–42.
`Petitioner argues the combination of Parulski and Konno teaches these
`limitations. See Pet. 19–25. Patent Owner does not argue otherwise. See
`PO Resp. 26–31; PO Sur-Reply 5–8.
`We have reviewed Petitioner’s contentions and agree that the
`combination of Parulski and Konno teaches the limitations identified above.
`
`16
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Figure 16A of Parulski discloses a dual-aperture camera that includes “a first
`fixed focal length lens 612 and a first image sensor 614, and a second fixed
`focal length lens 616 and a second image sensor 618,” where “[t]he first lens
`612 [is] preferably a fixed focal length wide angle lens . . . and the second
`lens 616 [is] preferably a fixed focal length telephoto lens.” Ex. 1005,
`23:28-40. Konno, similarly, teaches a dual-aperture camera having Wide
`and Tele lenses, where the Tele lens has an EFL/TTL ratio greater than 1
`and a FOV that is narrower than the Wide lens’s FOV. See Ex. 1015 ¶¶ 48,
`76 (Wide lens LN1 having a 76.18° FOV and Tele lens LN2 having
`narrower 55.52° FOV and EFL/TTL ratio of 5.51mm/4.91mm, which is
`greater than 1).
`Parulski further discloses that “both lenses 612 and 616 are adjustable
`focus lenses,” and that the output of tele lens 616 is used “to generate a
`focus detection signal for wide lens 612” or the output of wide lens 612 is
`used “to generate a focus detection signal for tele lens 616.” Id. at 23:62–
`24:4. Purulski also discloses an image processor 50 that applies “the focus
`detection signal . . . to the zoom and focus motors 5a and 5b of the selected
`imaging stage [i.e., Wide or Tele] in order to adjust the focus of the image
`providing the sensor output for the captured image signal.” Id. at 14:26–29,
`Fig. 1. Motor 5a is part of a “first lens focus adjuster” for a first image stage
`(i.e., Wide camera) and motor 5b is part of a “second lens focus adjuster” for
`a second image stage (i.e., Tele camera). Id. at 12:47–52. This disclosure
`supports Petitioner’s contention that a person skilled in the art “would have
`
`17
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`understood that each lens system includes a mechanically coupled autofocus
`mechanism” as required by claim 1. Pet. 24 (citing Ex. 1003, 44–45).5
`Claim 1 further requires the camera controller “to process the Wide
`and Tele images to create a fused image, wherein areas in the Tele image
`that are not focused are not combined with the Wide image to create the
`fused image and wherein the camera controller is further operative to output
`the fused image with a point of view (POV) of the Wide camera by mapping
`Tele image pixels to matching pixels within the Wide image.” Ex. 1001,
`13:43–50 (the “fused image” limitation).
`Petitioner argues Parulski teaches this limitation. See Pet. 26–30.
`First, Petitioner argues Parulski discloses modifying an image’s depth of
`field by capturing a first image with a first camera at a first focus, capturing
`a second image with a second camera at a second focus, and using the
`second image to enhance the depth of field of the first by combining them.
`Id. at 26–27 (citing Ex. 1005, 22:14–42, 28:45–57, Fig. 14). Next, Petitioner
`argues a person skilled in the art would have known from these disclosures
`that when the first image is taken with the Wide lens and the second with the
`Tele lens, “focused portions of the telephoto image are identified and
`combined with the wide image to broaden the wide image’s depth of field.”
`Id. at 27 (citing Ex. 1003, 48–49). Petitioner further argues a person skilled
`in the art would also have known that the portions of the Tele image that are
`combined with the Wide image are identified and extracted from the Tele
`image using Parulski’s range map, which is generated by matching Tele
`
`
`5 Petitioner cites to the pages of the Durand declaration rather than the
`paragraph numbers. We do as well.
`
`18
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`image pixels with Wide image pixels. Id. at 27–30 (citing Ex. 1005, 20:1–
`15, 20:50–59, Ex. 1003, 50–52). This is so, Petitioner argues, because:
`Parulski teaches using a range map “to improve object
`identification within the image by identifying the continuous
`boundaries of the object so the shape of the object can be
`define[d]” and “to enable object extraction from an image by
`identifying the continuous boundaries of the object so it can be
`segmented within the image.”
`Id. at 28 (quoting Ex. 1005, 20:50–59) (emphasis omitted). Finally,
`Petitioner argues that a person skilled in the art “would have understood that
`fusing portions of the telephoto image with the wide image . . . would have
`otherwise maintained the wide image, therefore outputting a fused image
`with the wide image’s field of view.” Id. at 29 (citing Ex. 1003 ¶¶ 50–51).
`Petitioner uses an example in Purulski to explain how a person skilled
`in the art would have understood Parulski’s teachings. Id. In the example,
`Purulski enhances the depth of field of an image containing a dog in the
`foreground, a field of flowers in the mid-ground, and a snow-capped
`mountain in the background so that “the dog is in focus, the mountains are in
`focus and so are those great flowers.” Ex. 1005, 21:9–13, 21:25–27.
`Petitioner argues that a person skilled in the art would have understood that:
`creating an enhanced image with both the mountains and the
`dog in focus would have meant that the pixel[s] corresponding
`to the dog from the telephoto image would have been identified
`by the range mapping process and then fused with the
`corresponding pixels in the wide image so that the dog would
`be sharpened in the wide image while maintaining the
`mountains in focus, thus broadening the wide image’s depth of
`field.
`Pet. 28 (citing Ex. 1003, 50).
`
`19
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`Patent Owner disputes that Parulski teaches the fused image
`limitation. PO Resp. 26–31; PO Sur-Reply 5–8. First, Patent Owner argues
`that Parulski does not teach the limitation when it is construed to require the
`fused image to have the wide camera’s position POV as well as its
`perspective POV. PO Resp. 29–31; PO Sur-Reply 5–6. We reject that
`argument in view of the Federal Circuit’s claim construction, which does not
`require the fused image to maintain both the position POV and perspective
`POV of the wide camera.
`Next, Patent Owner argues Petitioner’s combination improperly relies
`on hindsight to “combine cherry-picked portions of embodiments of Parulski
`to create a Frankenstein embodiment that Parulski neither disclosed nor
`preferred.” PO Resp. 26. Patent Owner argues that nothing in Parulski’s
`discussion of Figure 11 “describes using [a] range map as part of a system
`that outputs a ‘fused image’” and that a wide image’s depth of field can be
`enhanced “without directly incorporating image data from the tele image.”
`Id. at 26–27. Patent Owner further argues that nothing in Parulski’s
`discussion of Figures 14 and 26 “mentions using a range map” or
`“provide[s] any detail on how the two images are fused, if at all.” Id. at 28.
`Patent Owner argues Parulski’s discussion of Figure 14 “refers to use of an
`‘enhancement signal . . . to sharpen portions of the primary still image,’” but
`“does not specify that the enhancement signal actually contains image data
`from the secondary image, and . . . does not specify how the enhancement
`signal is used.” Id. at 28–29 (quoting Ex. 1005, 22:39–42). Patent Owner
`also argues that although Parulski’s discussion of Figure 26 “states that ‘the
`two images are combined into a modified image with a broadened depth of
`
`20
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`field,’” it fails to state “how the images are ‘combined.’” Id. at 29 (quoting
`Ex. 1005, 28:52–53).
`Petitioner replies that a person skilled in the art would have
`“recognized the applicability of using object identification and extraction to
`the last step in Fig. 14 (block 514) which teaches that the second image (e.g.,
`tele image) ‘is used to enhance the depth of field of the primary image.’”
`Pet. Reply 8 (quoting Ex. 1037 ¶ 17). Petitioner further argues that “[a]
`range map is the only method taught by Parulski that a [skilled artisan] could
`have used for identifying and extracting the portions of the tele image
`‘positioned near the secondary focus distance’ to ‘sharpen portions’ of the
`wide image.” Id. at 9.
`Patent Owner, in Sur-Reply, repeats its argument that “Petitioner
`relies on hindsight . . . to combine cherry-picked portions of embodiments of
`Parulski to create an embodiment that Parulski neither disclosed nor
`preferred.” PO Sur-Reply 6. 6 Patent Owner argues that Parulski’s example
`of enhancing the depth of field of an image of a dog in a field of flowers in
`front of a mountain does not motivate combining the teachings of Figures 11
`and 14, as Petitioner contends, because “this could be achieved using image
`
`
`6 In a conference call with the parties after remand from the Federal Circuit,
`Petitioner requested briefing on whether this argument was a proper sur-
`reply argument. See Paper 59, 2. We denied the request, indicating we
`would consider the properness of the argument in deciding the case on
`remand. Id. We find Patent Owner’s sur-reply arguments are proper
`because they are consistent with the arguments in Patent Owner’s Response
`and are responsive to Petitioner’s Reply argument that a person skilled in the
`art would have combined teachings from Parulski’s Figures 11 and 14. See
`PO Resp. 26–29; Pet. Reply 7–11; 37 C.F.R. § 42.23(b).
`21
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`data from the wide [lens], without any need for importing image data from
`the tele [lens].” Id. at 7.
`Upon considering the evidence and arguments presented, we agree
`with Petitioner that the combination of Parulski and Konno teaches the fused
`image limitation. In doing so, we reject Patent Owner’s argument that
`Petitioner’s contentions are improperly based on hindsight. Instead, we find
`Petitioner’s contentions are properly based on the teachings of Parulski and
`how a person skilled in the art would understand those teachings.
`In its discussion of Figure 14, Parulski teaches “enhancing the depth
`of field of an image by using images from both image capture stages,” i.e.,
`from the wide and tele lenses. Ex. 1005, 22:14–16. In its discussion of
`Figure 26, Parulski teaches there are “different types of [image]
`augmentation or modification,” including “a first type . . . depicted in
`connection with FIG. 14” in which “an image is captured from the primary
`capture unit [e.g. wide lens] . . . another image is captured from the scene
`analysis capture unit (the secondary image capture unit) [e.g., tele lens] . . .
`[and] the two images are combined into a modified image with a broadened
`depth of field.” Id. at 28:45–53 (emphasis added). Thus, Parulski teaches
`“creat[ing] a fused image” as required by claim 1.
`Parulski teaches its scene analysis capture unit determines an image’s
`“depth of field” and also identifies “different aspects of the scene including
`faces, grass, sunset, snow, etc.” Id. at 25:56–66. Parulski further teaches
`using a range map, which is generated by determining pixel offsets between
`first and second images (i.e., matching pixels in the two images), “to
`improve object identification within an image,” “to enable object extraction
`from an image,” and “to enable dynamic depth of field images by blurring
`
`22
`
`
`
`
`
`IPR2020-00905
`Patent 10,225,479 B2
`portions of [an] image that lie outside the desired depth of field.” Id. at
`20:1–15, 20:51–65, Fig. 11. We credit Dr. Durand’s testimony that a person
`skilled in the art would have understood these teachings to mean that a f
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
