Trials@uspto.gov
`571-272-7822
`
`Paper 34
`Entered: December2, 2019
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLEINC.,
`Petitioner,
`
`V.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`
`IPR2018-01133
`Patent 9,538,152 B2
`
`Before MARC S. HOFF, BRYAN F. MOORE,and MONICAS.
`ULLAGADDI, Administrative Patent Judges.
`
`MOORE,Administrative Patent Judge.
`
`JUDGMENT
`Final Written Decision
`
`Determining All Challenged Claims Unpatentable
`35 USC. § 318(a)
`
`I.
`
`INTRODUCTION
`
`A. Background and Summary
`In this inter partes review, instituted pursuant to 35 U.S.C. § 314,
`Apple Inc. (“Petitioner”) challenge the patentability of claims 1-4 of U.S.
`Patent No. 9,538,152 B2 (“the ’152 patent,” Ex. 1001), owned by
`
`
`571-272-7822
`
`Paper 34
`Entered: December2, 2019
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLEINC.,
`Petitioner,
`
`V.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`
`IPR2018-01133
`Patent 9,538,152 B2
`
`Before MARC S. HOFF, BRYAN F. MOORE,and MONICAS.
`ULLAGADDI, Administrative Patent Judges.
`
`MOORE,Administrative Patent Judge.
`
`JUDGMENT
`Final Written Decision
`
`Determining All Challenged Claims Unpatentable
`35 USC. § 318(a)
`
`I.
`
`INTRODUCTION
`
`A. Background and Summary
`In this inter partes review, instituted pursuant to 35 U.S.C. § 314,
`Apple Inc. (“Petitioner”) challenge the patentability of claims 1-4 of U.S.
`Patent No. 9,538,152 B2 (“the ’152 patent,” Ex. 1001), owned by
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`Corephotonics, Ltd. Paper 2 (“Pet.”). We have jurisdiction under 35 U.S.C.
`§ 6. This Final Written Decision, issued pursuant to 35 U.S.C. § 318(a) and
`37 C.F.R. § 42.73, addresses issues and argumentsraised during trial. For
`the reasons discussed below, we determinethat Petitioner has shown by a
`
`preponderanceofthe evidence that claims 1-4 ofthe 152 patent are
`
`unpatentable.
`
`B. Procedural History
`
`On May22, 2018,Petitioner filed a petition requesting an inter partes
`review of claims 1-4 of the ’152 patent pursuant to 35 U.S.C. §§ 311 et seq.
`Paper2 (‘“Pet.”). Petitioner relies on the testimony of Dr. Oliver Cossairt.
`Ex. 1004. Patent Ownerdid notfile a preliminary response.:
`
`On December4, 2018, we instituted an inter partes review of the
`challenged claims. Paper 8 (“Decision onInstitution”or “Dec.on Inst.”).
`On March 28, 2019, Patent Ownerfiled a Corrected Patent Owner Response.
`Paper 15. Petitionerrelies on the testimony of Dr. James Kosmach. Ex.
`2005. Ex. 2005. On June 3, 2019, Petitionerfiled a Reply. Paper 19
`(“Reply”). A hearing was held on October 8, 2019. A transcript of the
`hearing has been enteredinto the record. Paper 32 (“Tr.”).
`
`C. Real Parties in Interest
`
`Petitioner indicates that Apple Inc. is the only real party in interest.
`
`Pet. 1. Patent Owner doesnot contest this indication.
`
`D. Related Matters
`
`A decisionin this proceeding could affect or be affected by the
`following case pending in the United States District Court for the Northern
`District of California and involving the 152 patent: Corephotonics, Ltd. v.
`Apple Inc., Case No. 5-17-cv-06457 (N.D.Cal.). Pet. 2; Paper4, 2 (Patent
`Owneralso asserts Corephotonics, Ltd. v. Apple Inc., Case No. 5:18-cv-
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`02555 (N.D.Cal.) mayaffect, or be affected by, a decisionin this
`
`proceeding).
`
`E. The ’152 Patent
`
`The ’152 patent is directed to “multi-aperture imaging (‘MAI’)
`systems with high color resolution and/or optical zoom.” Ex. 1001, 1:15-18.
`The °152 patent states that while mechanical zoom solutions are commonin
`digital still cameras, they are “typically too thick for most camera phones”
`and mayresult in “resolution compromise.” Jd. at 1:35—43. Inits
`background, the ’152 patent states that one of the known approachesis using
`a multi-aperture imaging (“MAI”) system, for example, a dual-aperture
`imaging system (“DAI”) including “two optical apertures which may be
`formed by one or two optical modules, and one or two image sensors” for
`“implementing zoom,as well as increasing the output resolution.” Id. at
`
`1:52—59.
`The Specification states that those known multi-aperture imaging
`systems “often trade-off functionalities and properties, for example zoom
`and color resolution, or image resolution and quality for camera module
`height,” and therefore, there was a needto havethin multi-aperture imaging
`systemsthat “produce an image with high resolution (and specifically high
`color resolution) together with zoom functionality.” Jd. at 1:63—66, 1:67—
`
`2:3.
`
`As a solution to this problem, the’152 patent describes a dual aperture
`imaging system including a Wide sensor and a Tele sensor capturing a Wide
`image and a Tele image from twoapertures, wherecolorfilter arrays may be
`used in the Wide sensor and Tele sensor. Jd. at 2:34-65. The Wide image
`
`and Tele image may be fused to “output one fused (combined) output zoom
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`image processed accordingto a user [zoom factor] ZF input request.” Jd.at
`
`3:17-20.
`
`The ’152 patent describes a dual-aperture zoom imaging system 100
`including a Wide subset 104 and a Tele subset 106 each having a respective
`sensor. Jd. at Figs. 1A, 1B. The ’152 patent explains that a processor 108
`“fuses ...a Wide image obtained with the Wide subset and a Tele image
`obtained with the Tele subset, into a single fused output image according to
`a user-defined ‘applied’ ZF input or request.” Jd. at 5:60-6:2. The °152
`patent explains that an overlap area 110 of the Wide imageand Tele image
`is illustrated on the Wide imagein the figure. Id. at 4:62—-64, 6:2-9.
`To obtain the output image, the ’152 patent teaches a registration
`process, which “chooseseither the Wide imageor the Tele image to be a
`primary image .. . based on the ZF chosenfor the output image.” Jd.at
`9:20-21, 31-33. The registration process “considers the primary image as
`the baseline image andregisters the overlap area in an auxiliary imagetoit,”
`and the “output image point of view is determined according to the primary
`image point of view (camera angle).” Jd. at 9:20—28.
`F.
`Illustrative Claims
`Independent claim 1, reproduced below,is illustrative of the claimed
`subject matter:
`1.
`A multi-aperture imaging system comprising:
`a first camera that providesa first image, the first camera
`having a first field of view (FOV)) andafirst sensor withafirst
`plurality of sensor pixels covered at least in part with a standard
`colorfilter array (CFA);
`a second camerathat provides a second image, the second
`camera having a second field of view (FOV2) such that
`FOV2<FOV) and a second sensor with a second plurality of
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`sensor pixels being either Clear or covered with a standard CFA,
`the second image having an overlap area with the first image; and
`a processor configured to provide an output image from a
`point of view of the first camera based on a zoom factor (ZF)
`input that defines a respective field of view (FOVzr), the first
`image being a primary image and the second imagebeing a non-
`primary image, wherein if FOV2<FOVzr<FOV| thenthe pointof
`view of the output imageis that ofthe first camera, the processor
`further configured to register the overlap area of the second
`image as a non-primary image to the first image as primary
`image to obtain the output image.
`|
`
`Ex. 1001, 12:60-13:13.
`
`G. Evidence
`
`Petitioner relies on the following references. Pet. 14-27.
`
`
`
`
`
`Border
`
`
`
`US Patent Application Pub. No. 2008/0030592
`Al, filed Aug. 1, 2006, published Feb. 7, 2008.
`Parulski US Patent No. 7,859,588 B2, filed Mar. 9, 2007,|1007
`
`
`issued Dec. 28, 2010
`
`
`
`
`1006
`
`
`
`
`
`H. Prior Art and Asserted Grounds
`
`Petitioner asserts that claims 1-4 would have been unpatentable on the
`
`following grounds:
`Claim(s) Challenged
`
`35 U.S.C. §
`
`Reference(s)/Basis
`Borderand Parulski
`
`Pet. 12.
`
`e
`
`II. ANALYSIS
`
`A. Legal Standards
`A claim is unpatentable under 35 U.S.C. § 103(a) if the differences
`between the claimed subject matter and theprior art are such that the subject
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`matter, as a whole, would have been obviousat the time the invention was
`madeto a person having ordinary skill in the art to which said subject matter
`pertains. KSR Int’l Co.v. Teleflex Inc., 550 U.S. 398, 406 (2007). The
`question of obviousnessis resolved on the basis of underlying factual
`determinations including (1) the scope and contentof the priorart; (2) any
`differences between the claimed subject matter and theprior art; (3) the level
`of skill in the art; and (4) where in evidence, so-called secondary
`considerations, including commercial success, long-felt but unsolved needs,
`failure of others, and unexpected results.! Graham v. John Deere Co.,
`
`383 U.S. 1, 17-18 (1966)(“the Graham factors”).
`
`B. Level of Ordinary Skill in the Art
`For an obviousness analysis, prior art references must be “considered
`together with the knowledgeof one of ordinaryskill in the pertinentart.” Jn
`re Paulsen, 30 F.3d 1475, 1480 (Fed. Cir. 1994) (quoting Jn re Samour, 571
`F.2d 559, 562 (CCPA 1978)). Moreover,“it is proper to take into account
`not only specific teachings of the reference but also the inferences which one
`skilled in the art would reasonably be expected to draw therefrom.” Jn re
`Preda, 401 F.2d 825, 826 (CCPA 1968). That is because an obviousness
`analysis “need not seek out precise teachings directed to the specific subject
`matter of the challenged claim, for a court can take accountofthe inferences
`and creative steps that a person ofordinary skill in the art would employ.”
`KSR, 550 U.S.at 418 (Fed. Cir. 2007); In re Translogic Tech., Inc., 504 F.3d
`
`1249, 1259 (Fed. Cir. 2007).
`
`| Patent Owner doesnot put forth evidenceit alleges tends to show
`secondary considerations of non-obviousnessin its Patent Owner Response.
`
`6
`
`
`
`IPR2018-01133
`Patent 9, 938, 152 B2
`
`Petitioner asserts a person ofordinary skill in the art of the subject
`matter of the ?152 patentat the time of the invention would have had a
`bachelor’s or the equivalent degree in computer science or
`electrical and/or computer engineering or a related field and 2-3
`years of experience in imaging systems including optics design
`and imaging processing [and] a person with less
`formal
`education but more experience, or more formal educationbutless
`experience, could have also met the relevant standard for a
`[person of ordinaryskill in the art of the subject matter of the
`’152 patentat the time of the invention.]
`Pet. 9 (citing Ex. 1004 J 19). We adopt Petitioner’s articulation ofthe level
`of skill and acknowledgethat the level of ordinary skill in the art is also
`reflected by the priorart of record. See Okajima v. Bourdeau, 261 F.3d
`1350, 1355 (Fed.Cir. 2001); In re GPAC Inc., 57 F.3d 1573, 1579 (Fed.Cir.
`1995); In re Oelrich, 579 F.2d 86, 91 (CCPA 1978).
`
`C. Claim Construction
`
`In an inter partes review based onapetition filed prior to November
`13, 2018, claim terms in an unexpired patent are given their broadest
`reasonable constructionin light of the specification of the patent in which
`they appear. See Trial Practice Guide, 77 Fed. Reg. at 48,766; 37 CFR
`§ 42.100(b). Consistent with the broadest reasonable construction, claim
`terms are presumed to have their ordinary and customary meaningas
`understood by a person ofordinary skill in the art in the context ofthe entire
`patent disclosure. Jn re Translogic Tech., 504 F.3d at 1257.
`Petitioner did not offer a construction for the term “point of view”in
`
`the Petition. Pet. 10-11. Patent Ownerasserts that “point of view” should
`be construed as “camera angle.” PO Resp. 13. The Specification states that
`“[t]he output image point of view is determined according to the primary
`camera point of view (camera angle). Jd. (citing Ex. 1001, 9:26—28).
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`Nevertheless, Petitioner asserts “[t]o the extent the Board adopts Patent
`Owner’s proposed construction of ‘point of view’ as ‘camera angle,’ such a
`construction does not materially change Petitioner’s analysis.” Reply 1. We
`agree that this construction does not change the analysisin this case.
`Patent Owner, in its Sur-Reply, asserts its construction is more
`
`consistent with the extrinsic evidence and the “inventor’s usage of the term”
`
`than Petitioner’s construction of “viewpoint.” Sur-Reply 2. We do not
`agree that Petitioner’s contentionsrely on a construction of “point of view”
`as “viewpoint.” Thus, we do not view this case as requiring a choice
`
`between twoconstructions.
`
`Patent Ownerhas not explained how construing “point of view”as
`“camera angle” servesto resolve any controversy regarding obviousness.
`Additionally, the claimsrecite “the point of view of the output imageis that
`of the first camera.” Thus, a construction of camera angle is redundantto
`
`the words of the claim.
`
`In view of our analysis below, we determine that no claim terms
`require express construction. See Vivid Techs., Inc. v. Am. Sci. & Eng’g,
`Inc., 200 F.3d 795, 803 (Fed. Cir. 1999) (Only terms that are in controversy
`need to be construed, and only to the extent necessary to resolve the
`
`controversy).
`
`D. Obviousness over Border and Parulski
`
`Petitioner asserts that claims 1-4 are unpatentable under 35 U.S.C. §
`
`103 over Border (Ex. 1006) and Parulski (Ex. 1007). Pet. 14-70.
`Petitioner asserts that Border is a US patent application filed on
`August 1, 2006 and published on February 7, 2008 and Parulski was filed on
`March 9, 2007, published on September 11, 2008, and issued on December
`28, 2010. Pet. 12. Petitioner asserts further that these references are prior
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`art to the °152 patent underat least 35 U.S.C. § 102(b). Jd. The ’152 patent
`
`is a National Phase application from PCT patent application
`PCT/IB2013/060356 filed November 23, 2013 which claimspriority from
`
`US Provisional Application No. 61/730,570, filed November 28, 2012. Ex.
`1001. The °152 patent issued on January 3, 2017. Jd. Weare persuaded
`that Border and Parulskiare priorart.
`a) Overview ofBorder
`Borderdescribes providing a digital camera with an extended zoom
`range without unduly increasing the size or cost of the digital camera “while
`providing good perceived image quality throughout the zoom range.” Ex.
`1006 4 10. As shownin Figure 5 of Border, reproduced below,the
`processorof a digital camera includes an image compositor 202 to form a
`composite image 208 using the two images, wide image 204 and telephoto
`image 206 of the same scene,that are captured using lenses having different
`
`focal lengths. Jd. § 70.
`
`IMAGE
`
`206
`
`TELEPHOTO
`IMAGE
`
`
`
`ZOOM
`AMOUNT
`
` 202
`
`
`> IMAGE
`
`
`i DETERMINER
`IMAGE 210
`
`
`| RESAMPLER P|
`
`REGISTRATION
`
`IMAGE
`
`COMPOSITE
`
`208
`
`
`
`FIG. 5
`
`As seenin Figure 5, above, the image registration determiner 212
`determinesthe registration between the wide image 204 andthe telephoto
`image 206, so that the two images are matchedto “locate the high-resolution
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`image accurately into the low-resolution image and then stitchedinto place
`so the edge between the two images in the composite image is not
`discernible.” : Jd. § 29, Fig. 5. Border goes on to explain that in the context
`of Figure 5, telephoto image 206 captures a smaller portion of the scene, but
`with greater resolution than wide image 204. Id. { 36.
`Borderalso describes that an image resampler 214 of the processor
`produces the composite image 208 based on a zoom amount Z specifying the
`desired relative zoom amountof the produced composite image 208. Jd.
`43. Specifically, Border explains that the composite image 208 is generated
`from the two images andthatthe resulting composite image is produced
`differently for different zoom amountvalues, such as Z=1, 1<Z<M, and
`Z=M,where M is the relative magnification ratio M of the telephoto image
`
`206 to the wide image 204. Id. J] 29, 44.
`
`2. Overview ofParulski
`
`Parulski “utilizes one of the images from a dual-lens camera as a
`secondary image that can be used to modify the other, primary image and
`thereby generate an enhanced primary image.” Ex. 1007, 7:32-35.
`Specifically, Parulski discloses that examples of the enhancementto the
`primary imageinclude “to sharpen portions ofthe primary image,”“to
`modify the dynamic rangeofthe primary image,” or “to replace portions of
`the primary image(areas of lower noise but with some motion blur) with
`correspondingportions of the secondary image (areas of higher noise but
`little or no motion blur) to obtain a modified image with relatively low noise
`and good sharpness.” Jd. at 7:54-8:5, Fig. 26.
`Parulski describes determining the primary image and secondary
`image from twocaptureunits of the digital camera based on a requested
`zoom position provided by a user. Jd. at 27:8-24, Fig. 23. For example,if
`
`10
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`the requested zoom position is not within the zoom range of the current
`primary capture unit for providing a primary image, “the functions of the
`capture units are reversed” by switching the capture unit for providing a
`secondary image and the capture unit for providing the primary image. Jd.at
`
`27:8-15.
`
`3. Analysis of Claims I and 3
`In discussing claim 3, Petitioner refers to its contentions for claim 1.
`Pet. 66-69. Patent Owner does not separately argue claim 3. PO Resp. 19.
`Therefore, our discussionof the limitations of claim 1 is applicable to claim
`
`3.
`
`Petitioner provides reasons to combine Border and Parulski including:
`(1) “Border and Parulski are analogousprior art and are in the samefield of
`endeavorpertainingto a digital camera that uses multiple lenses and image
`sensors to provide an enhanced output image”(id. at 18), (2) “When
`evaluating the teachings of Border, a POSITA wouldnaturally have
`considered the teachings of Parulski, whichis a patent that has the same co-
`inventor (John N. Border) and the same assignee (Eastman Kodak Company,
`one of the top digital camera makers) as Border”(id. at 19), and (3)
`“Parulski explicitly provides that its image augmentation process of using a
`secondary image to modify a primary image‘can also be applied in
`connection with image pairs having different resolutions[]’” and “refers to
`Border’s system as an example for such application, and explicitly
`incorporates Border by reference”(id. at 18, 20).
`a) Preamble
`The preamble ofindependentclaim 1 recites, “[a] multi-aperture
`imaging system.” Petitioner has shownsufficiently that Border teaches this
`limitation of claim 1. Petitioner contends Border’s digital camera 10B
`
`11
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`teaches a multi-aperture imaging system. Pet. 20-21 (citing Ex. 1006 If 36,
`
`58, 59, Fig. 1B; Ex. 1002 §{ 63-65).
`Independentclaim 1 further recites “a first camera that providesa first
`image.” Petitioner has shownsufficiently that Border teachesthis limitation
`of claim 1. Petitioner contends that Border describes a digital camera 10B
`
`including two fixed focal length lenses 2 and 4 “each providing an imageto
`a corresponding image sensor 12 and 14” (Ex. 1006 58), as such, Border’s
`wide camera, including fixed focal length lens 2 and corresponding image
`sensor 12, correspondsto “a first camera that providesa first image” as
`recited in claim 1. Pet. 23-25 (citing Ex. 1006 {J 36, 58; Ex. 1002 4] 67—
`
`69).
`b) “thefirst camera havingafirstfield ofview (FOV:)”
`
`Independentclaim 1 further recites “the first camera havinga first
`field of view (FOV)).” Petitioner has shownsufficiently that Border teaches
`this limitation of claim 1. Petitioner contends that Border teachesthatits
`
`fixed focal length lens 2 hasa first field of view (FOV:) and describes that in
`an image capture device (e.g., digital camera 10B), “two or more lens
`systems are associated with a respective numberof imagesensors. The
`lenses have different focal lengths and different fields of view within the
`same scene wherein the field of view of the longer focal length lenses
`contains at least a portion ofthe field of view of the shorter focal length
`lens.” Pet. 25—26(citing EX. 1006 §{ 25, 31; Ex. 1002 {fj 71-72).
`c)
`“afirst sensor with afirst plurality ofsensor pixels...”
`Independent claim 1 furtherrecites “[the first camera having]a first
`sensor with a first plurality of sensor pixels coveredat least in part with a
`standardcolorfilter array (CFA).” Petitioner has shownsufficiently that
`Border teachesthis limitation of claim 1. Petitioner contends that Border
`
`12
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`teaches that its wide camera includes image sensor 12, which “includes an
`
`array ofdiscrete light sensitive picture elements overlaid with a colorfilter
`array (CFA) pattern to produce color image data corresponding to the CFA
`pattern,” (Ex. 1006 4 60) and Border teaches thatits sensors 12 and 14 are
`“single-chip color Megapixel CCD sensors, using the well-known Bayer
`colorfilter pattern to capture color images.” (id. ] 32). Pet. (citing Ex. 1006
`
`q7 32, 60; Ex. 1002 9] 74-78).
`
`“asecond camera that provides a second image”
`d)
`Independent claim 1 further recites “a second camerathatprovides a
`second image.” Petitioner has shownsufficiently that Border teachesthis
`limitation of claim 1. Petitioner contends that Border describes that “the
`
`image processor 50 of FIGS. 1A and 1B contains an image compositor 202
`that receives both the wide image 204 from the fixed focal length lens 2 and
`the telephoto image 206 from the zoomlens 3.” Pet. 28 (citing Ex. 1006
`36). Although someofthe description in paragraph 36 of Border references
`a Figure 1A embodimentin which the telephoto image 206 is from the zoom
`lens 3, Petitioner contendsthat a person of ordinary skill in the art would
`have understoodthat in the corresponding example of Figure 1B, image
`processor 50 receivesthe telephoto image 206 from fixed focal length lens 4
`(Ex. 1006 J{ 36, 58 Figs. 1B, 5; Ex. 1002 81). Pet. 28-30 (citing Ex. 1006,
`36, 58; Ex. 1002 F¥ 79-82).
`e)
`“the second camera having a secondfield ofview (FOV2) such that
`FOV2<FOV,”
`
`Independentclaim 1 further recites “the second camera having a
`second field of view (FOV2) such that FOV2<FOV).” Petitioner has shown
`sufficiently that Border teachesthis limitation of claim 1. Petitioner
`contends that Border teachesthatits fixed focal length lens 4 has a second
`
`13
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`field of view (FOV2) such that FOV2<FOV) because Border describesthat in
`its image capture device, “two or more lens systems are associated with a
`respective numberof image sensors. The lenses have different focal lengths
`and different fields of view within the same scene wherein thefield of view
`
`of the longer focal length lenses containsat least a portion ofthe field of
`view of the shorter focal length lens.”Pet. 31 (citing Ex. 1006 { 25). And,
`in Border’s digital camera 10B of Figure 1B,the fixed focal length lens4 is
`the longer focal length lens andthe fixed focal length lens2 is the shorter
`focal length lens, for example, “[t]he two fixed focuslenses are selected to
`provide a substantial zoom range, for example, 3:1 wherein the focal length
`of the second fixed focal length lens 4 is 3x as long asthe fixed focal length
`lens 2” (Ex. 1006 § 58). Pet. 31-32. As such, accordingto Petitioner,
`because the FOV?of the longer focal length lens 4 “containsat least a
`portion of? FOV) ofthe shorter focal length lens 2, a POSITA would have
`understood that FOV?is less than FOV). Pet. 31-35 (with further detailed
`
`explanation,citing Ex. 1006 J 25, 47, 58, Fig. 6; Ex. 1002 { 85-90).
`f)
`“asecond sensor with a secondplurality ofsensor pixels...”
`Independentclaim 1 further recites “[the second camera having] a
`second sensor with a secondplurality of.sensorpixels, the second plurality
`of sensorpixels being either Clear or covered with a standard CFA.”
`Petitioner has shownsufficiently that Border teaches this limitation of claim
`1. Petitioner contends Borderteachesthatits tele camera includes an image
`
`sensor 14, which “includes anarray of discrete light sensitive picture
`elements overlaid with a color filter array (CFA) pattern to produce color
`image data corresponding to the CFA pattern” (Ex. 1006 § 60) and Border
`teaches that sensors 12 and 14 are “single-chip color Megapixel CCD
`sensors, using the well-known Bayercolorfilter pattern to capture color
`
`14
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`images” (id. J 32). Pet. 36-37 (citing Ex. 1006 J 32, 60; Ex. 1002 {J 91-
`
`95).
`
`g)
`
`“the second image having an overlap area with thefirst image”
`Independent claim 1 further recites “the second image having an
`overlap area with the first image.” Petitioner has shownsufficiently that
`Border teaches this limitation of claim 1. Petitioner contends that Border
`
`describes that in its image capture device, “[t]he lenses have different focal
`lengths and different fields of view within the same scene whereinthefield
`of view of the longer focal length lenses containsat least a portion ofthe
`field of view of the shorter focal length lens” (Ex. 1006 § 25) and a person
`of ordinary skill in the art would have understood that the region 220 in
`Figure 6 of Border below correspondsto the overlap area of the telephoto
`image 206 (e.g., the entire area of the telephoto image 206) with the wide
`image 204(id. 4 32; Ex. 1002 4 98). Pet. 37-39 (citing Ex. 1006 {if 25, 47;
`
`Ex. 1002 Yj 96-100).
`h)
`“a processor configured to provide an output image from a pointofview
`of the first camera...”
`Independentclaim 1 furtherrecites “‘a processor configured to provide
`an output image from a pointof view of the first camera based on a zoom
`factor (ZF) input that defines a respective field of view (FOVzr).” Petitioner
`has shownsufficiently that Border teachesthis limitation of claim 1.
`Petitioner contends Borderteaches a processor configured to provide an
`output image from a point of view ofthe first camera based on a zoom factor
`(ZF) input that defines a respective field of view (FOVzr) and provides a
`detailed explanation ofits contention. Pet. 39-48 (citing Ex. 1006 4§ 28, 29,
`36-40, 42, 44, 48, 53, 58, 64, 66, Figs. 1B, 5; Ex. 1010 (Szeliski - as support
`
`15
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`for meaning of homography), 50-51, Fig. 2.12; Ex. 1008 (Jacobson- as
`support for meaning of homography), 5, 57-58; Ex. 1002 {J 101-115).
`i)
`“thefirst image being a primary image and the second image being a
`non-primary image”
`Independentclaim 1 further recites “[processor configured to provide
`an output image from a point of view ofthe first camera based on a zoom
`factor (ZF) input that defines a respective field of view (FOVzr),] the first
`image being a primary image and the second image being a non-primary
`image.” Petitioner has shownsufficiently that the combination of Border
`and Parulski teach this limitation of claim 1. Petitioner contends Border in
`
`combination with Parulski renders obviousthe limitation that that the
`
`processoris configured to provide an output image fromapoint of view of
`the first camera based on a zoom factor (ZF) input that defines a respective
`field of view (FOVzr), the first image being a primary image and the second
`image being a non-primary image andprovides a detailed explanationofits
`contention. Pet. 39-48 (citing Ex. 1006 JJ 44; Ex. 1007 (Parulski), 7:32—-35,
`
`7:54-8:5, 23:28-40, 53-58, 27:8-15, 25-31, 28:33-40, 45-67, 29:51-67,
`15A, 15B, 16A, 16B, 23, 26; Ex. 1008 (Jacobson- as support for motivation
`to combine), 5, 57-58; Ex. 1010 (Szeliski - as support for motivation to
`combine), 50-51, Fig. 2.12; Ex. 1002 J 116-123). For example,Petitioner
`contendsthat“[a]lthough Border does not expressly use ‘primary image’
`and ‘non-primary image’ labels, Parulski, in an analogous context, uses the
`labels ‘primary image’ and ‘secondary image’ to describe the roles of
`respective images used in forming a composite image.” Pet. 49.
`
`16
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`j)
`
`“wherein ifFOV2<FOVzr<FOV;then the point ofview ofthe output
`imageis that ofthefirst camera”
`Independentclaim 1 further recites “wherein if FOV2<FOVzr<FOV)
`then the point of view of the output imageis that of the first camera.”
`Petitioner has shownsufficiently that Border teachesthis limitation of claim
`
`1. Petitioner contends that Border teaches a zoom amount 210 Z that defines
`
`a respectivefield of view (FOVzr) (Ex. 1002 { 125) and Border teaches
`providing a composite image 208 from a point of view of the wide camera
`(first camera) when the zoom amount 210 Z is between 1 and M, M being
`the relative magnification ratio of the telephoto image 206 to the wide image
`204 (id.), thus, because FOVzr defined by the zoom amount 210 Z between
`1 and M is between FOV) and FOV2, Border teachesthatif
`
`FOV2<FOVzr<FOV; then the point of view of the output imageis that of the
`
`first camera as claimed (id. at 126). Pet. 55—56 (citing Ex. 1002 {J 124—
`
`130).
`
`Patent Owner’s Contentions
`
`Patent Ownerasserts that “Border fails to discuss any concept of
`
`creating an output image from the images of multiple camerasthat is from
`the point of view of any specific camera.” PO Resp. 19-20. Patent Owner
`also asserts that Petitioner’s contentions are limited to the situation in which
`
`Z=1 and the composite image is the same as the wide angle image and no
`zoom is used. PO Resp. 25-26. According to Patent Owner, Petitioner’s
`contentions are not applicable to a situation in which FOV2<FOVzr<FOVi.
`
`Id. We disagree.
`Accordingto the Petition, “when the zoom amount 210 Z is between 1
`and M,data from both the wide image 204 andthe telephoto image 206 are
`used by the image resampler 214 to produce the composite image 208.” Pet.
`
`17
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`44. This contention is consistent with and within the range
`
`FOV2<FOVzr<FOV\ that Patent Ownerasserts the Petition does not cover.
`
`Thus, we are not persuaded by Patent Owner’s argument.
`
`Patent Owneralso asserts that the method by which Border combines
`
`images,i.e. “stitching,” does notresult in a composite image from the point
`of view of a single camera. PO Resp. 20. According to Patent Owner,
`Petitioner admits “Border discusses priorart ‘image stitching’ techniques
`
`wherein ‘two images are matched to‘locate the high resolution image
`accurately into the low-resolution image andthen stitched into placeso the
`edge between the two imagesin the composite image is not discernible.”
`PO Resp.20 (citing Pet. 44). This quote from the Petition, according to
`Patent Owner, means that Border only combines the two imagesat the edge
`
`between the two images and nowhereelse.
`Patent Ownerprovides an annotated version of Fig. 6 of Border
`
`d
`
`=
`
`|
`
`|
`
`\
`
`204
`
`208
`
`reproduced below.
`
`‘o
`
`FirstImage
`
`Second
`
`Image
`
`an overlap
`area with the
`first image
`
`
`
`FIG. 6
`
`(APPL-1006), Border, Fig. 6, annotated
`
`18
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`Annotated Figure 6, above, showsa first image, second image and
`
`combined image produced by Border. Ex. 1006, Fig. 6. Patent Owner
`
`asserts that the area outside the dashed-line border is from the perspective of
`
`the first image and the area inside the dashed-line borderis from the
`
`perspective of the second image. PO Resp. 21.
`Petitioner refutes Patent Owner’s contention that “stitching” involves
`
`the straightforward combining of two images by using one imageinside a
`border and another image outside the border without transposing either
`image soit is from the perspective of the other. Reply 3. Petitioner
`contends, in the Petition, that Border usesregistration to “in the form of
`
`homography Hrwthat transformsthe coordinates of the telephoto image
`206 to the wide image 204.” Jd. (citing Pet. 46-47 (quoting Ex. 1006 {if
`38-39). Petitioner relies on its declarant whotestifies that registration in
`Border, which “transformsthe coordinates of the telephoto image 206to the
`
`wide image 204,” (Ex. 1006 4 38) has the effect of making the telephoto
`portion of the composite image has the samepoint of view as the wide
`image 204. Reply 3-4 (citing Pet. 43-48 (citing Ex. 1004 [Coissart Decl.]
`
`q 114)).
`This assertion is supported by evidence in form of references that
`
`further explain that stitching can use registration that implements
`homography, which transforms an image to the point of view of another
`associated image. Ex. 1004 4 114 (citing Ex. 1010 [Szeliski] Fig. 2.12, 50-
`51; Ex. 1008 [Jacobson] 5, 57-58). We credit Petitioner’s declarant on this
`contention. Patent Owner’s declarant admitted he did not have an
`
`understandingofstitching: “[Q.] So as you sit here today, you don’t know
`what imagestitching includes and does not include because you’re not an
`expert in that area, correct? [A.] That — that’s fair to say, yes.” Ex. 1011,
`
`19
`
`
`
`IPR2018-01133
`Patent 9, 938, 152 B2
`
`164:9-13. Thus, we give Patent Owner’s declarant’s testimony onthis issue
`
`less weight.
`Patent Ownersuggests that Petitioner’s declarant is incorrect because
`the homography of Border cannot take into account image transformations
`in which features are occluded in one of the two images to be combined. PO
`
`Resp. 22. Patent Owner further asserts “the shape and perspective of objects
`captured in the images, whereoneportion of an object would be occludedin
`the image captured by one sensorbutnot included in the image captured by
`another sensor in the system.” Jd.. Patent Ownerincludes two images from
`Figure 4.24 of the Jacobson textbook (Ex. 1008) to show “occlusions.”
`
`alliaiyMe=we ee ee eee eter, ee
`
`2|tnl| ZRHh . ai|
`
`
`
`
`
`20
`
`
`
`IPR2018-01133
`Patent 9, 538, 152 B2
`
`
`
`Jo
`
`m
`iv
`—
`
`ne a
`e a
`Woo Wee oe
`
`EOMoe
`
`
`Figure 4.24 Perspective o
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
