Trials@uspto.gov
`571-272-7822
`
`Paper 34
`Entered: December 2, 2019
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE INC.,
`Petitioner,
`v.
`COREPHOTONICS, LTD.,
`Patent Owner.
`
`IPR2018-01133
`Patent 9,538,152 B2
`
`
`
`
`
`
`
`
`
`Before MARC S. HOFF, BRYAN F. MOORE, and MONICA S.
`ULLAGADDI, Administrative Patent Judges.
`MOORE, Administrative Patent Judge.
`
`
`
`JUDGMENT
`Final Written Decision
`Determining All Challenged Claims Unpatentable
`35 U.S.C. § 318(a)
`
`I.
`INTRODUCTION
`A. Background and Summary
`In this inter partes review, instituted pursuant to 35 U.S.C. § 314,
`Apple Inc. (“Petitioner”) challenge the patentability of claims 1–4 of U.S.
`Patent No. 9,538,152 B2 (“the ’152 patent,” Ex. 1001), owned by
`
`
`
`
`571-272-7822
`
`Paper 34
`Entered: December 2, 2019
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE INC.,
`Petitioner,
`v.
`COREPHOTONICS, LTD.,
`Patent Owner.
`
`IPR2018-01133
`Patent 9,538,152 B2
`
`
`
`
`
`
`
`
`
`Before MARC S. HOFF, BRYAN F. MOORE, and MONICA S.
`ULLAGADDI, Administrative Patent Judges.
`MOORE, Administrative Patent Judge.
`
`
`
`JUDGMENT
`Final Written Decision
`Determining All Challenged Claims Unpatentable
`35 U.S.C. § 318(a)
`
`I.
`INTRODUCTION
`A. Background and Summary
`In this inter partes review, instituted pursuant to 35 U.S.C. § 314,
`Apple Inc. (“Petitioner”) challenge the patentability of claims 1–4 of U.S.
`Patent No. 9,538,152 B2 (“the ’152 patent,” Ex. 1001), owned by
`
`
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`Corephotonics, Ltd. Paper 2 (“Pet.”). We have jurisdiction under 35 U.S.C.
`§ 6. This Final Written Decision, issued pursuant to 35 U.S.C. § 318(a) and
`37 C.F.R. § 42.73, addresses issues and arguments raised during trial. For
`the reasons discussed below, we determine that Petitioner has shown by a
`preponderance of the evidence that claims 1–4 of the ’152 patent are
`unpatentable.
`
`B. Procedural History
`On May 22, 2018, Petitioner filed a petition requesting an inter partes
`review of claims 1–4 of the ’152 patent pursuant to 35 U.S.C. §§ 311 et seq.
`Paper 2 (“Pet.”). Petitioner relies on the testimony of Dr. Oliver Cossairt.
`Ex. 1004. Patent Owner did not file a preliminary response.
`On December 4, 2018, we instituted an inter partes review of the
`challenged claims. Paper 8 (“Decision on Institution” or “Dec. on Inst.”).
`On March 28, 2019, Patent Owner filed a Corrected Patent Owner Response.
`Paper 15. Petitioner relies on the testimony of Dr. James Kosmach. Ex.
`2005. Ex. 2005. On June 3, 2019, Petitioner filed a Reply. Paper 19
`(“Reply”). A hearing was held on October 8, 2019. A transcript of the
`hearing has been entered into the record. Paper 32 (“Tr.”).
`C. Real Parties in Interest
`Petitioner indicates that Apple Inc. is the only real party in interest.
`Pet. 1. Patent Owner does not contest this indication.
`D. Related Matters
`A decision in this proceeding could affect or be affected by the
`following case pending in the United States District Court for the Northern
`District of California and involving the ’152 patent: Corephotonics, Ltd. v.
`Apple Inc., Case No. 5-17-cv-06457 (N.D. Cal.). Pet. 2; Paper 4, 2 (Patent
`Owner also asserts Corephotonics, Ltd. v. Apple Inc., Case No. 5:18-cv-
`
`2
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`02555 (N.D. Cal.) may affect, or be affected by, a decision in this
`proceeding).
`
`E. The ’152 Patent
`The ’152 patent is directed to “multi-aperture imaging (‘MAI’)
`systems with high color resolution and/or optical zoom.” Ex. 1001, 1:15–18.
`The ’152 patent states that while mechanical zoom solutions are common in
`digital still cameras, they are “typically too thick for most camera phones”
`and may result in “resolution compromise.” Id. at 1:35–43. In its
`background, the ’152 patent states that one of the known approaches is using
`a multi-aperture imaging (“MAI”) system, for example, a dual-aperture
`imaging system (“DAI”) including “two optical apertures which may be
`formed by one or two optical modules, and one or two image sensors” for
`“implementing zoom, as well as increasing the output resolution.” Id. at
`1:52–59.
`The Specification states that those known multi-aperture imaging
`systems “often trade-off functionalities and properties, for example zoom
`and color resolution, or image resolution and quality for camera module
`height,” and therefore, there was a need to have thin multi-aperture imaging
`systems that “produce an image with high resolution (and specifically high
`color resolution) together with zoom functionality.” Id. at 1:63–66, 1:67–
`2:3.
`
`As a solution to this problem, the’152 patent describes a dual aperture
`imaging system including a Wide sensor and a Tele sensor capturing a Wide
`image and a Tele image from two apertures, where color filter arrays may be
`used in the Wide sensor and Tele sensor. Id. at 2:34–65. The Wide image
`and Tele image may be fused to “output one fused (combined) output zoom
`
`3
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`image processed according to a user [zoom factor] ZF input request.” Id. at
`3:17–20.
`The ’152 patent describes a dual-aperture zoom imaging system 100
`including a Wide subset 104 and a Tele subset 106 each having a respective
`sensor. Id. at Figs. 1A, 1B. The ’152 patent explains that a processor 108
`“fuses . . . a Wide image obtained with the Wide subset and a Tele image
`obtained with the Tele subset, into a single fused output image according to
`a user-defined ‘applied’ ZF input or request.” Id. at 5:60–6:2. The ’152
`patent explains that an overlap area 110 of the Wide image and Tele image
`is illustrated on the Wide image in the figure. Id. at 4:62–64, 6:2–9.
`To obtain the output image, the ’152 patent teaches a registration
`process, which “chooses either the Wide image or the Tele image to be a
`primary image . . . based on the ZF chosen for the output image.” Id. at
`9:20–21, 31–33. The registration process “considers the primary image as
`the baseline image and registers the overlap area in an auxiliary image to it,”
`and the “output image point of view is determined according to the primary
`image point of view (camera angle).” Id. at 9:20–28.
`F. Illustrative Claims
`Independent claim 1, reproduced below, is illustrative of the claimed
`subject matter:
`1.
`A multi-aperture imaging system comprising:
`
`a first camera that provides a first image, the first camera
`having a first field of view (FOV1) and a first sensor with a first
`plurality of sensor pixels covered at least in part with a standard
`color filter array (CFA);
`a second camera that provides a second image, the second
`camera having a second field of view (FOV2) such that
`FOV2<FOV1 and a second sensor with a second plurality of
`
`4
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`sensor pixels being either Clear or covered with a standard CFA,
`the second image having an overlap area with the first image; and
`a processor configured to provide an output image from a
`point of view of the first camera based on a zoom factor (ZF)
`input that defines a respective field of view (FOVZF), the first
`image being a primary image and the second image being a non-
`primary image, wherein if FOV2<FOVZF<FOV1 then the point of
`view of the output image is that of the first camera, the processor
`further configured to register the overlap area of the second
`image as a non-primary image to the first image as primary
`image to obtain the output image.
`
`Ex. 1001, 12:60–13:13.
`
`
`G. Evidence
`Petitioner relies on the following references. Pet. 14–27.
`Name
`Reference
`Border
`US Patent Application Pub. No. 2008/0030592
`A1, filed Aug. 1, 2006, published Feb. 7, 2008.
`US Patent No. 7,859,588 B2, filed Mar. 9, 2007,
`issued Dec. 28, 2010
`
`Parulski
`
`Exhibit
`1006
`
`1007
`
`
`
`
`
`H. Prior Art and Asserted Grounds
`Petitioner asserts that claims 1–4 would have been unpatentable on the
`following grounds:
`Claim(s) Challenged
`1–4
`Pet. 12.
`
`Reference(s)/Basis
`Border and Parulski
`
`35 U.S.C. §
`103
`
`II. ANALYSIS
`A. Legal Standards
`A claim is unpatentable under 35 U.S.C. § 103(a) if the differences
`between the claimed subject matter and the prior art are such that the subject
`
`5
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`matter, as a whole, would have been obvious at the time the invention was
`made to a person having ordinary skill in the art to which said subject matter
`pertains. KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 406 (2007). The
`question of obviousness is resolved on the basis of underlying factual
`determinations including (1) the scope and content of the prior art; (2) any
`differences between the claimed subject matter and the prior art; (3) the level
`of skill in the art; and (4) where in evidence, so-called secondary
`considerations, including commercial success, long-felt but unsolved needs,
`failure of others, and unexpected results.1 Graham v. John Deere Co.,
`383 U.S. 1, 1718 (1966) (“the Graham factors”).
`B. Level of Ordinary Skill in the Art
`For an obviousness analysis, prior art references must be “considered
`together with the knowledge of one of ordinary skill in the pertinent art.” In
`re Paulsen, 30 F.3d 1475, 1480 (Fed. Cir. 1994) (quoting In re Samour, 571
`F.2d 559, 562 (CCPA 1978)). Moreover, “it is proper to take into account
`not only specific teachings of the reference but also the inferences which one
`skilled in the art would reasonably be expected to draw therefrom.” In re
`Preda, 401 F.2d 825, 826 (CCPA 1968). That is because an obviousness
`analysis “need not seek out precise teachings directed to the specific subject
`matter of the challenged claim, for a court can take account of the inferences
`and creative steps that a person of ordinary skill in the art would employ.”
`KSR, 550 U.S.at 418 (Fed. Cir. 2007); In re Translogic Tech., Inc., 504 F.3d
`1249, 1259 (Fed. Cir. 2007).
`
`
`1 Patent Owner does not put forth evidence it alleges tends to show
`secondary considerations of non-obviousness in its Patent Owner Response.
`
`6
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`Petitioner asserts a person of ordinary skill in the art of the subject
`matter of the ’152 patent at the time of the invention would have had a
`bachelor’s or the equivalent degree in computer science or
`electrical and/or computer engineering or a related field and 2-3
`years of experience in imaging systems including optics design
`and imaging processing [and] a person with less formal
`education but more experience, or more formal education but less
`experience, could have also met the relevant standard for a
`[person of ordinary skill in the art of the subject matter of the
`’152 patent at the time of the invention.]
`Pet. 9 (citing Ex. 1004 ¶ 19). We adopt Petitioner’s articulation of the level
`of skill and acknowledge that the level of ordinary skill in the art is also
`reflected by the prior art of record. See Okajima v. Bourdeau, 261 F.3d
`1350, 1355 (Fed. Cir. 2001); In re GPAC Inc., 57 F.3d 1573, 1579 (Fed. Cir.
`1995); In re Oelrich, 579 F.2d 86, 91 (CCPA 1978).
`C. Claim Construction
`In an inter partes review based on a petition filed prior to November
`13, 2018, claim terms in an unexpired patent are given their broadest
`reasonable construction in light of the specification of the patent in which
`they appear. See Trial Practice Guide, 77 Fed. Reg. at 48,766; 37 CFR
`§ 42.100(b). Consistent with the broadest reasonable construction, claim
`terms are presumed to have their ordinary and customary meaning as
`understood by a person of ordinary skill in the art in the context of the entire
`patent disclosure. In re Translogic Tech., 504 F.3d at 1257.
`Petitioner did not offer a construction for the term “point of view” in
`the Petition. Pet. 10–11. Patent Owner asserts that “point of view” should
`be construed as “camera angle.” PO Resp. 13. The Specification states that
`“[t]he output image point of view is determined according to the primary
`camera point of view (camera angle). Id. (citing Ex. 1001, 9:26–28).
`
`7
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`Nevertheless, Petitioner asserts “[t]o the extent the Board adopts Patent
`Owner’s proposed construction of ‘point of view’ as ‘camera angle,’ such a
`construction does not materially change Petitioner’s analysis.” Reply 1. We
`agree that this construction does not change the analysis in this case.
`Patent Owner, in its Sur-Reply, asserts its construction is more
`consistent with the extrinsic evidence and the “inventor’s usage of the term”
`than Petitioner’s construction of “viewpoint.” Sur-Reply 2. We do not
`agree that Petitioner’s contentions rely on a construction of “point of view”
`as “viewpoint.” Thus, we do not view this case as requiring a choice
`between two constructions.
`Patent Owner has not explained how construing “point of view” as
`“camera angle” serves to resolve any controversy regarding obviousness.
`Additionally, the claims recite “the point of view of the output image is that
`of the first camera.” Thus, a construction of camera angle is redundant to
`the words of the claim.
`In view of our analysis below, we determine that no claim terms
`require express construction. See Vivid Techs., Inc. v. Am. Sci. & Eng’g,
`Inc., 200 F.3d 795, 803 (Fed. Cir. 1999) (Only terms that are in controversy
`need to be construed, and only to the extent necessary to resolve the
`controversy).
`
`D. Obviousness over Border and Parulski
`Petitioner asserts that claims 1–4 are unpatentable under 35 U.S.C. §
`103 over Border (Ex. 1006) and Parulski (Ex. 1007). Pet. 14–70.
`Petitioner asserts that Border is a US patent application filed on
`August 1, 2006 and published on February 7, 2008 and Parulski was filed on
`March 9, 2007, published on September 11, 2008, and issued on December
`28, 2010. Pet. 12. Petitioner asserts further that these references are prior
`
`8
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`art to the ’152 patent under at least 35 U.S.C. § 102(b). Id. The ’152 patent
`is a National Phase application from PCT patent application
`PCT/IB2013/060356 filed November 23, 2013 which claims priority from
`US Provisional Application No. 61/730,570, filed November 28, 2012. Ex.
`1001. The ’152 patent issued on January 3, 2017. Id. We are persuaded
`that Border and Parulski are prior art.
`a) Overview of Border
`Border describes providing a digital camera with an extended zoom
`range without unduly increasing the size or cost of the digital camera “while
`providing good perceived image quality throughout the zoom range.” Ex.
`1006 ¶ 10. As shown in Figure 5 of Border, reproduced below, the
`processor of a digital camera includes an image compositor 202 to form a
`composite image 208 using the two images, wide image 204 and telephoto
`image 206 of the same scene, that are captured using lenses having different
`focal lengths. Id. ¶ 70.
`
`
`As seen in Figure 5, above, the image registration determiner 212
`determines the registration between the wide image 204 and the telephoto
`image 206, so that the two images are matched to “locate the high-resolution
`
`9
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`image accurately into the low-resolution image and then stitched into place
`so the edge between the two images in the composite image is not
`discernible.” Id. ¶ 29, Fig. 5. Border goes on to explain that in the context
`of Figure 5, telephoto image 206 captures a smaller portion of the scene, but
`with greater resolution than wide image 204. Id. ¶ 36.
`Border also describes that an image resampler 214 of the processor
`produces the composite image 208 based on a zoom amount Z specifying the
`desired relative zoom amount of the produced composite image 208. Id. ¶
`43. Specifically, Border explains that the composite image 208 is generated
`from the two images and that the resulting composite image is produced
`differently for different zoom amount values, such as Z=1, 1<Z<M, and
`Z=M, where M is the relative magnification ratio M of the telephoto image
`206 to the wide image 204. Id. ¶¶ 29, 44.
`2. Overview of Parulski
`Parulski “utilizes one of the images from a dual-lens camera as a
`secondary image that can be used to modify the other, primary image and
`thereby generate an enhanced primary image.” Ex. 1007, 7:32–35.
`Specifically, Parulski discloses that examples of the enhancement to the
`primary image include “to sharpen portions of the primary image,” “to
`modify the dynamic range of the primary image,” or “to replace portions of
`the primary image (areas of lower noise but with some motion blur) with
`corresponding portions of the secondary image (areas of higher noise but
`little or no motion blur) to obtain a modified image with relatively low noise
`and good sharpness.” Id. at 7:54–8:5, Fig. 26.
`Parulski describes determining the primary image and secondary
`image from two capture units of the digital camera based on a requested
`zoom position provided by a user. Id. at 27:8–24, Fig. 23. For example, if
`
`10
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`the requested zoom position is not within the zoom range of the current
`primary capture unit for providing a primary image, “the functions of the
`capture units are reversed” by switching the capture unit for providing a
`secondary image and the capture unit for providing the primary image. Id. at
`27:8–15.
`
`3. Analysis of Claims 1 and 3
`In discussing claim 3, Petitioner refers to its contentions for claim 1.
`Pet. 66–69. Patent Owner does not separately argue claim 3. PO Resp. 19.
`Therefore, our discussion of the limitations of claim 1 is applicable to claim
`3.
`
`Petitioner provides reasons to combine Border and Parulski including:
`(1) “Border and Parulski are analogous prior art and are in the same field of
`endeavor pertaining to a digital camera that uses multiple lenses and image
`sensors to provide an enhanced output image” (id. at 18), (2) “When
`evaluating the teachings of Border, a POSITA would naturally have
`considered the teachings of Parulski, which is a patent that has the same co-
`inventor (John N. Border) and the same assignee (Eastman Kodak Company,
`one of the top digital camera makers) as Border” (id. at 19), and (3)
`“Parulski explicitly provides that its image augmentation process of using a
`secondary image to modify a primary image ‘can also be applied in
`connection with image pairs having different resolutions[]’” and “refers to
`Border’s system as an example for such application, and explicitly
`incorporates Border by reference” (id. at 18, 20).
`a) Preamble
`The preamble of independent claim 1 recites, “[a] multi-aperture
`imaging system.” Petitioner has shown sufficiently that Border teaches this
`limitation of claim 1. Petitioner contends Border’s digital camera 10B
`
`11
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`teaches a multi-aperture imaging system. Pet. 20–21 (citing Ex. 1006 ¶¶ 36,
`58, 59, Fig. 1B; Ex. 1002 ¶¶ 63–65).
`Independent claim 1 further recites “a first camera that provides a first
`image.” Petitioner has shown sufficiently that Border teaches this limitation
`of claim 1. Petitioner contends that Border describes a digital camera 10B
`including two fixed focal length lenses 2 and 4 “each providing an image to
`a corresponding image sensor 12 and 14” (Ex. 1006 ¶ 58), as such, Border’s
`wide camera, including fixed focal length lens 2 and corresponding image
`sensor 12, corresponds to “a first camera that provides a first image” as
`recited in claim 1. Pet. 23–25 (citing Ex. 1006 ¶¶ 36, 58; Ex. 1002 ¶¶ 67–
`69).
`
`b) “the first camera having a first field of view (FOV1)”
`Independent claim 1 further recites “the first camera having a first
`field of view (FOV1).” Petitioner has shown sufficiently that Border teaches
`this limitation of claim 1. Petitioner contends that Border teaches that its
`fixed focal length lens 2 has a first field of view (FOV1) and describes that in
`an image capture device (e.g., digital camera 10B), “two or more lens
`systems are associated with a respective number of image sensors. The
`lenses have different focal lengths and different fields of view within the
`same scene wherein the field of view of the longer focal length lenses
`contains at least a portion of the field of view of the shorter focal length
`lens.” Pet. 25–26 (citing EX. 1006 ¶¶ 25, 31; Ex. 1002 ¶¶ 71–72).
`c) “a first sensor with a first plurality of sensor pixels . . .”
`Independent claim 1 further recites “[the first camera having] a first
`sensor with a first plurality of sensor pixels covered at least in part with a
`standard color filter array (CFA).” Petitioner has shown sufficiently that
`Border teaches this limitation of claim 1. Petitioner contends that Border
`
`12
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`teaches that its wide camera includes image sensor 12, which “includes an
`array of discrete light sensitive picture elements overlaid with a color filter
`array (CFA) pattern to produce color image data corresponding to the CFA
`pattern,” (Ex. 1006 ¶ 60) and Border teaches that its sensors 12 and 14 are
`“single-chip color Megapixel CCD sensors, using the well-known Bayer
`color filter pattern to capture color images.” (id. ¶ 32). Pet. (citing Ex. 1006
`¶¶ 32, 60; Ex. 1002 ¶¶ 74–78).
`d) “a second camera that provides a second image”
`Independent claim 1 further recites “a second camera that provides a
`second image.” Petitioner has shown sufficiently that Border teaches this
`limitation of claim 1. Petitioner contends that Border describes that “the
`image processor 50 of FIGS. 1A and 1B contains an image compositor 202
`that receives both the wide image 204 from the fixed focal length lens 2 and
`the telephoto image 206 from the zoom lens 3.” Pet. 28 (citing Ex. 1006 ¶
`36). Although some of the description in paragraph 36 of Border references
`a Figure 1A embodiment in which the telephoto image 206 is from the zoom
`lens 3, Petitioner contends that a person of ordinary skill in the art would
`have understood that in the corresponding example of Figure 1B, image
`processor 50 receives the telephoto image 206 from fixed focal length lens 4
`(Ex. 1006 ¶¶ 36, 58 Figs. 1B, 5; Ex. 1002 81). Pet. 28–30 (citing Ex. 1006,
`36, 58; Ex. 1002 ¶¶ 79–82).
`e) “the second camera having a second field of view (FOV2) such that
`FOV2<FOV1”
`Independent claim 1 further recites “the second camera having a
`second field of view (FOV2) such that FOV2<FOV1.” Petitioner has shown
`sufficiently that Border teaches this limitation of claim 1. Petitioner
`contends that Border teaches that its fixed focal length lens 4 has a second
`
`13
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`field of view (FOV2) such that FOV2<FOV1 because Border describes that in
`its image capture device, “two or more lens systems are associated with a
`respective number of image sensors. The lenses have different focal lengths
`and different fields of view within the same scene wherein the field of view
`of the longer focal length lenses contains at least a portion of the field of
`view of the shorter focal length lens.” Pet. 31 (citing Ex. 1006 ¶ 25). And,
`in Border’s digital camera 10B of Figure 1B, the fixed focal length lens 4 is
`the longer focal length lens and the fixed focal length lens 2 is the shorter
`focal length lens, for example, “[t]he two fixed focus lenses are selected to
`provide a substantial zoom range, for example, 3:1 wherein the focal length
`of the second fixed focal length lens 4 is 3× as long as the fixed focal length
`lens 2” (Ex. 1006 ¶ 58). Pet. 31–32. As such, according to Petitioner,
`because the FOV2 of the longer focal length lens 4 “contains at least a
`portion of” FOV1 of the shorter focal length lens 2, a POSITA would have
`understood that FOV2 is less than FOV1. Pet. 31–35 (with further detailed
`explanation, citing Ex. 1006 ¶¶ 25, 47, 58, Fig. 6; Ex. 1002 ¶ 85–90).
`f) “a second sensor with a second plurality of sensor pixels . . .”
`Independent claim 1 further recites “[the second camera having] a
`second sensor with a second plurality of sensor pixels, the second plurality
`of sensor pixels being either Clear or covered with a standard CFA.”
`Petitioner has shown sufficiently that Border teaches this limitation of claim
`1. Petitioner contends Border teaches that its tele camera includes an image
`sensor 14, which “includes an array of discrete light sensitive picture
`elements overlaid with a color filter array (CFA) pattern to produce color
`image data corresponding to the CFA pattern” (Ex. 1006 ¶ 60) and Border
`teaches that sensors 12 and 14 are “single-chip color Megapixel CCD
`sensors, using the well-known Bayer color filter pattern to capture color
`
`14
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`images” (id. ¶ 32). Pet. 36–37 (citing Ex. 1006 ¶¶ 32, 60; Ex. 1002 ¶¶ 91–
`95).
`g) “the second image having an overlap area with the first image”
`Independent claim 1 further recites “the second image having an
`overlap area with the first image.” Petitioner has shown sufficiently that
`Border teaches this limitation of claim 1. Petitioner contends that Border
`describes that in its image capture device, “[t]he lenses have different focal
`lengths and different fields of view within the same scene wherein the field
`of view of the longer focal length lenses contains at least a portion of the
`field of view of the shorter focal length lens” (Ex. 1006 ¶ 25) and a person
`of ordinary skill in the art would have understood that the region 220 in
`Figure 6 of Border below corresponds to the overlap area of the telephoto
`image 206 (e.g., the entire area of the telephoto image 206) with the wide
`image 204 (id. ¶ 32; Ex. 1002 ¶ 98). Pet. 37–39 (citing Ex. 1006 ¶¶ 25, 47;
`Ex. 1002 ¶¶ 96–100).
`h) “a processor configured to provide an output image from a point of view
`of the first camera . . .”
`Independent claim 1 further recites “a processor configured to provide
`an output image from a point of view of the first camera based on a zoom
`factor (ZF) input that defines a respective field of view (FOVZF).” Petitioner
`has shown sufficiently that Border teaches this limitation of claim 1.
`Petitioner contends Border teaches a processor configured to provide an
`output image from a point of view of the first camera based on a zoom factor
`(ZF) input that defines a respective field of view (FOVZF) and provides a
`detailed explanation of its contention. Pet. 39–48 (citing Ex. 1006 ¶¶ 28, 29,
`36–40, 42, 44, 48, 53, 58, 64, 66, Figs. 1B, 5; Ex. 1010 (Szeliski - as support
`
`15
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`for meaning of homography), 50–51, Fig. 2.12; Ex. 1008 (Jacobson - as
`support for meaning of homography), 5, 57–58; Ex. 1002 ¶¶ 101–115).
`i) “the first image being a primary image and the second image being a
`non-primary image”
`Independent claim 1 further recites “[processor configured to provide
`an output image from a point of view of the first camera based on a zoom
`factor (ZF) input that defines a respective field of view (FOVZF),] the first
`image being a primary image and the second image being a non-primary
`image.” Petitioner has shown sufficiently that the combination of Border
`and Parulski teach this limitation of claim 1. Petitioner contends Border in
`combination with Parulski renders obvious the limitation that that the
`processor is configured to provide an output image from a point of view of
`the first camera based on a zoom factor (ZF) input that defines a respective
`field of view (FOVZF), the first image being a primary image and the second
`image being a non-primary image and provides a detailed explanation of its
`contention. Pet. 39–48 (citing Ex. 1006 ¶¶ 44; Ex. 1007 (Parulski), 7:32–35,
`7:54–8:5, 23:28–40, 53–58, 27:8–15, 25–31, 28:33–40, 45–67, 29:51–67,
`15A, 15B, 16A, 16B, 23, 26; Ex. 1008 (Jacobson - as support for motivation
`to combine), 5, 57–58; Ex. 1010 (Szeliski - as support for motivation to
`combine), 50–51, Fig. 2.12; Ex. 1002 ¶¶ 116–123). For example, Petitioner
`contends that “[a]lthough Border does not expressly use ‘primary image’
`and ‘non-primary image’ labels, Parulski, in an analogous context, uses the
`labels ‘primary image’ and ‘secondary image’ to describe the roles of
`respective images used in forming a composite image.” Pet. 49.
`
`16
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`j) “wherein if FOV2<FOVZF<FOV1 then the point of view of the output
`image is that of the first camera”
`Independent claim 1 further recites “wherein if FOV2<FOVZF<FOV1
`then the point of view of the output image is that of the first camera.”
`Petitioner has shown sufficiently that Border teaches this limitation of claim
`1. Petitioner contends that Border teaches a zoom amount 210 Z that defines
`a respective field of view (FOVZF) (Ex. 1002 ¶ 125) and Border teaches
`providing a composite image 208 from a point of view of the wide camera
`(first camera) when the zoom amount 210 Z is between 1 and M, M being
`the relative magnification ratio of the telephoto image 206 to the wide image
`204 (id.), thus, because FOVZF defined by the zoom amount 210 Z between
`1 and M is between FOV1 and FOV2, Border teaches that if
`FOV2<FOVZF<FOV1 then the point of view of the output image is that of the
`first camera as claimed (id. at ¶ 126). Pet. 55–56 (citing Ex. 1002 ¶¶ 124–
`130).
`
`Patent Owner’s Contentions
`Patent Owner asserts that “Border fails to discuss any concept of
`creating an output image from the images of multiple cameras that is from
`the point of view of any specific camera.” PO Resp. 19–20. Patent Owner
`also asserts that Petitioner’s contentions are limited to the situation in which
`Z=1 and the composite image is the same as the wide angle image and no
`zoom is used. PO Resp. 25–26. According to Patent Owner, Petitioner’s
`contentions are not applicable to a situation in which FOV2<FOVZF<FOV1.
`Id. We disagree.
`According to the Petition, “when the zoom amount 210 Z is between 1
`and M, data from both the wide image 204 and the telephoto image 206 are
`used by the image resampler 214 to produce the composite image 208.” Pet.
`
`17
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`44. This contention is consistent with and within the range
`FOV2<FOVZF<FOV1 that Patent Owner asserts the Petition does not cover.
`Thus, we are not persuaded by Patent Owner’s argument.
`Patent Owner also asserts that the method by which Border combines
`images, i.e. “stitching,” does not result in a composite image from the point
`of view of a single camera. PO Resp. 20. According to Patent Owner,
`Petitioner admits “Border discusses prior art ‘image stitching’ techniques
`wherein ‘two images are matched to ‘locate the high resolution image
`accurately into the low-resolution image and then stitched into place so the
`edge between the two images in the composite image is not discernible.’”
`PO Resp. 20 (citing Pet. 44). This quote from the Petition, according to
`Patent Owner, means that Border only combines the two images at the edge
`between the two images and nowhere else.
`Patent Owner provides an annotated version of Fig. 6 of Border
`reproduced below.
`
`
`
`18
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`Annotated Figure 6, above, shows a first image, second image and
`combined image produced by Border. Ex. 1006, Fig. 6. Patent Owner
`asserts that the area outside the dashed-line border is from the perspective of
`the first image and the area inside the dashed-line border is from the
`perspective of the second image. PO Resp. 21.
`Petitioner refutes Patent Owner’s contention that “stitching” involves
`the straightforward combining of two images by using one image inside a
`border and another image outside the border without transposing either
`image so it is from the perspective of the other. Reply 3. Petitioner
`contends, in the Petition, that Border uses registration to “in the form of
`homography HTW that transforms the coordinates of the telephoto image
`206 to the wide image 204.” Id. (citing Pet. 46–47 (quoting Ex. 1006 ¶¶
`38–39). Petitioner relies on its declarant who testifies that registration in
`Border, which “transforms the coordinates of the telephoto image 206 to the
`wide image 204,” (Ex. 1006 ¶ 38) has the effect of making the telephoto
`portion of the composite image has the same point of view as the wide
`image 204. Reply 3–4 (citing Pet. 43–48 (citing Ex. 1004 [Coissart Decl.]
`¶ 114)).
`This assertion is supported by evidence in form of references that
`further explain that stitching can use registration that implements
`homography, which transforms an image to the point of view of another
`associated image. Ex. 1004 ¶ 114 (citing Ex. 1010 [Szeliski] Fig. 2.12, 50–
`51; Ex. 1008 [Jacobson] 5, 57–58). We credit Petitioner’s declarant on this
`contention. Patent Owner’s declarant admitted he did not have an
`understanding of stitching: “[Q.] So as you sit here today, you don’t know
`what image stitching includes and does not include because you’re not an
`expert in that area, correct? [A.] That – that’s fair to say, yes.” Ex. 1011,
`
`19
`
`
`
`IPR2018-01133
`Patent 9,538,152 B2
`164:9–13. Thus, we give Patent Owner’s declarant’s testimony on this issue
`less weight.
`Patent Owner suggests that Petitioner’s declarant is incorrect bec
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
