UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`
`APPLE INC.,
`Petitioner
`
`v.
`
`COREPHOTONICS, LTD.
`Patent Owner
`
`_______________
`
`IPR2018-01133
`U.S. Patent No. 9,538,152
`_______________
`
`PETITION FOR INTER PARTES REVIEW
`UNDER 35 U.S.C. § 312 AND 37 C.F.R. § 42.104
`
` 
`
`
`
`
`
`
`
`
`
`

`

`
`
`
`IPR2018-01133 Petition
`
`Inter Partes Review of 9,538,152
`
`TABLE OF CONTENTS
`
`INTRODUCTION ........................................................................................... 1 
`I. 
`II.  MANDATORY NOTICES ............................................................................. 1 
`A. 
`Real Party-in-Interest ............................................................................ 1 
`B. 
`Related Matters ...................................................................................... 2 
`C. 
`Lead and Back-up Counsel and Service Information ........................... 2 
`III.  GROUNDS FOR STANDING ........................................................................ 2 
`IV.  OVERVIEW OF THE ’152 PATENT ............................................................ 3 
`A. 
`Summary of the ’152 Patent .................................................................. 3 
`B. 
`Prosecution History of the ’152 Patent ................................................. 7 
`LEVEL OF ORDINARY SKILL IN THE ART ............................................. 9 
`V. 
`VI.  CLAIM CONSTRUCTION .......................................................................... 10 
`A. “standard color filter array (CFA)” (claims 1 and 3) .................... 10 
`VII.  REQUESTED RELIEF ................................................................................. 11 
`VIII.  IDENTIFICATION OF HOW THE CLAIMS ARE UNPATENTABLE .... 11 
`A. 
`Challenged Claims .............................................................................. 11 
`B. 
`Statutory Grounds for Challenges ....................................................... 12 
`C. 
`Discretionary Denial is Not Warranted ............................................... 12 
`D.  Note Regarding Page Citations and Emphasis .................................... 13 
`IDENTIFICATION OF HOW THE CLAIMS ARE UNPATENTABLE .... 14 
`A.  Ground 1: Claims 1-4 are unpatentable under 35 U.S.C. § 103(a) over
`Border and Parulski ............................................................................. 14 
`1. 
`Summary of Border ................................................................... 14 
`2. 
`Summary of Parulski ................................................................. 16 
`3. 
`Reasons to Combine Border and Parulski ................................ 18 
`4.  
`Claim 1 ...................................................................................... 20 
`5. 
`Claim 2 ...................................................................................... 63 
`6. 
`Claim 3 ...................................................................................... 66 
`
`IX.  
`
`
`
`- i -
`
`

`

`
`
`
`IPR2018-01133 Petition
`
`Inter Partes Review of 9,538,152
`
`Claim 4 ...................................................................................... 69 
`7. 
`CONCLUSION ............................................................................................. 70 
`X. 
`XI.  CERTIFICATE OF WORD COUNT ........................................................... 71 
`CERTIFICATE OF SERVICE ........................................................................... 72 
`
`
`
`
`
`
`
`
`- ii -
`
`

`

`
`
`
`IPR2018-01133 Petition
`
`Inter Partes Review of 9,538,152
`
`PETITIONER’S EXHIBIT LIST
`
`May 22, 2018
`
`APPL-1001
`APPL-1002
`APPL-1003
`
`APPL-1004
`APPL-1005
`APPL-1006
`
`APPL-1007
`APPL-1008
`
`U.S. Patent No. 9,538,152 to Shabtay et al. (the “’152 Patent”)
`Prosecution File History of the ’152 Patent (the “’823 App”)
`Prosecution File History of U.S. Provisional App. No.
`61/730,570 (the “’570 App”)
`Declaration of Dr. Oliver Cossairt (“Cossairt”)
`Curriculum Vitae of Dr. Oliver Cossairt
`U.S. Patent Application Publication No. 2008/0030592 to
`Border et. al (“Border”)
`U.S. Patent No. 7,859,588 (“Parulski”)
`Ralph E. Jacobson et al., The Manual of Photography:
`photographic and digital imaging, 9th Edition, 2000
`(“Jacobson”)
`APPL-1009 Michael Langford et al., Langford’s Advanced Photography,
`7th Edition, 2008 (“Langford”)
`Richard Szeliski, Computer Vision: Algorithms and
`Applications, 2011 (“Szeliski”)
`
`APPL-1010
`
`
`
`
`
`- iii -
`
`

`

`
`
`I.
`
`INTRODUCTION
`
`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`U.S. Patent No. 9,538,152 (the “’152 Patent,” APPL-1001) is generally
`
`directed to “multi-aperture imaging (‘MAI’) systems ... with high color resolution
`
`and/or optical zoom.” (APPL-1001), ’152 Patent, 1:15-18. The claims of the ’152
`
`Patent are directed to a multi-aperture imaging system using 1) two cameras
`
`having respective field of views (FOVs) and image sensors with color filter arrays
`
`to provide two images and 2) a processor providing an output image based on a
`
`relationship between a zoom factor input and the FOVs of the two cameras. As
`
`shown in this Petition, these concepts in a digital camera that uses multiple lenses
`
`and image sensors were known in the art before the priority date of the ’152 patent.
`
`This Petition, along with the cited evidence, demonstrates that claims 1-4 of
`
`the ’152 Patent are rendered obvious under (pre-AIA) 35 U.S.C. § 103(a). Apple
`
`Inc. (“Petitioner”) therefore respectfully requests these claims be held unpatentable
`
`and cancelled.
`
`II. MANDATORY NOTICES
`
`A. Real Party-in-Interest
`
`The real party-in-interest is Apple Inc.
`
`
`
`- 1 -
`
`

`

`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`B. Related Matters
`
`As of the filing date of this Petition and to the best knowledge of the
`
`petitioner, the ’152 Patent has been asserted in Corephotonics, Ltd. v. Apple Inc.,
`
`Case No. 5-17-cv-06457 (N.D. Cal. filed Nov. 6, 2017).
`
`C.
`
`Lead and Back-up Counsel and Service Information
`
`Lead Counsel
`David W. O’Brien
`HAYNES AND BOONE, LLP
`600 Congress Ave. Suite 1300
`Austin, TX 78701
`
`Back-up Counsel
`Andrew S. Ehmke
`HAYNES AND BOONE, LLP
`2323 Victory Ave. Suite 700
`Dallas, TX 75219
`
`512-867-8457
`Phone:
`512-867-8644
`Fax:
`
`david.obrien.ipr@haynesboone.com
`USPTO Reg. No. 40,107
`
`214-651-5116
`Phone:
`214-200-0853
`Fax:
`andy.ehmke.ipr@haynesboone.com
`USPTO Reg. No. 50,271
`
`Hong Shi
`HAYNES AND BOONE, LLP
`600 Congress Ave. Suite 1300
`Austin, TX 78701
`
`512-867-8440
`Phone:
`512-867-8644
`Fax:
`
`hong.shi.ipr@haynesboone.com
`USPTO Reg. No. 69,009
`
`Please address all correspondence to lead and back-up counsel. Petitioner
`
`consents to electronic service.
`
`III. GROUNDS FOR STANDING
`
`Pursuant to 37 C.F.R. § 42.104(a), Petitioner certifies that the ’152 Patent is
`
`available for inter partes review and that Petitioner is not barred or estopped from
`
`requesting an inter partes review challenging the patent claims on the ground
`
`identified in this Petition.
`
`- 2 -
`
`

`

`
`
`
`
`IV. OVERVIEW OF THE ’152 PATENT
`
`A.
`
`Summary of the ’152 Patent
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`The ’152 Patent is directed to “multi-aperture imaging (‘MAI’) systems ...
`
`with high color resolution and/or optical zoom.” (APPL-1001), ’152 Patent, 1:15-
`
`18; (APPL-1004), Cossairt, ¶27. The ’152 Patent recognizes that while mechanical
`
`zoom solutions are common in digital still cameras, they are “typically too thick
`
`for most camera phones” and may result in “resolution compromise.” ’152 Patent,
`
`1:35-43; (APPL-1004), Cossairt, ¶27. In its background, the ’152 Patent
`
`acknowledges that one of the known approaches is using a multi-aperture imaging
`
`(“MAI”) system, for example, a dual-aperture imaging system (“DAI”) including
`
`“two optical apertures which may be formed by one or two optical modules, and
`
`one or two image sensors” for “implementing zoom, as well as increasing the
`
`output resolution.” (APPL-1001),’152 Patent, 1:52-59; (APPL-1004), Cossairt,
`
`¶27; (APPL-1004), Cossairt, ¶28.
`
`However, the ’152 Patent alleges that those known multi-aperture imaging
`
`systems “often trade-off functionalities and properties, for example zoom and color
`
`resolution, or image resolution and quality for camera module height,” and
`
`therefore there was a need to have thin multi-aperture imaging systems that
`
`“produce an image with high resolution (and specifically high color resolution)
`
`
`
`- 3 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`together with zoom functionality.” (APPL-1001),’152 Patent, 1:63-66, 1:67-2:3;
`
`(APPL-1004), Cossairt, ¶28.
`
`As an alleged solution to this problem, the’152 Patent describes a dual-
`
`aperture imaging system including a Wide sensor and a Tele sensor capturing a
`
`Wide image and a Tele image from two apertures, where color filter arrays may be
`
`used in the Wide sensor and Tele sensor. (APPL-1001), ’152 Patent, 2:34-65;
`
`(APPL-1004), Cossairt, ¶29. The Wide image and Tele image may be fused to
`
`“output one fused (combined) output zoom image processed according to a user
`
`[zoom factor] ZF input request.” (APPL-1001), ’152 Patent, 3:17-20; (APPL-
`
`1004), Cossairt, ¶29.
`
`Figure 1A of the ’152 Patent below illustrates a dual-aperture zoom imaging
`
`system 100 including a Wide subset 104 and a Tele subset 106 each having a
`
`respective sensor, and Figure 1B of the ’152 Patent illustrates the Wide image and
`
`Tele image obtained by the Wide subset 104 and a Tele subset 106. (APPL-1004),
`
`Cossairt, ¶30.
`
`
`
`- 4 -
`
`

`

`
`
`
`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`
`
`
`
`(APPL-1001), ’152 Patent, Figs. 1A and 1B
`
`In describing Figure 1A, the ’152 Patent explains that a processor 108 “fuses
`
`a Wide image obtained with the Wide subset and a Tele image obtained with the
`
`
`
`- 5 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`Tele subset, into a single fused output image according to a user-defined ‘applied’
`
`ZF input or request.” (APPL-1001), ’152 Patent, 5:60-6:2; (APPL-1004), Cossairt,
`
`¶31. In describing Figure 1B, the ’152 Patent explains that an overlap area 110 of
`
`the Wide image and Tele image is illustrated on the Wide image in the figure.
`
`(APPL-1001), ’152 Patent, 4:62-64, 6:2-9; (APPL-1004), Cossairt, ¶31. Figure 1B
`
`also illustrates, by way of exemplary images, a larger field of view (FOV) for the
`
`Wide image and a smaller FOV for the corresponding Tele image. (APPL-1004),
`
`Cossairt, ¶31.
`
`To obtain the output image, the ’152 Patent teaches a registration process,
`
`which “chooses either the Wide image or the Tele image to be a primary image”
`
`“based on the ZF chosen for the output image.” (APPL-1001), ’152 Patent, 9:20-
`
`21, 31-33; (APPL-1004), Cossairt, ¶32. The registration process “considers the
`
`primary image as the baseline image and registers the overlap area in an auxiliary
`
`image to it,” and the “output image point of view is determined according to the
`
`primary image point of view (camera angle).” (APPL-1001), ’152 Patent, 9:20-28’
`
`(APPL-1004), Cossairt, ¶32.
`
`As demonstrated in detail below, however, and confirmed in testimonies of
`
`Dr. Oliver Cossairt (APPL-1004), it was well-known in the art, before the earliest
`
`claimed priority date of the ’512 Patent, to provide a multi-aperture imaging
`
`system providing a high resolution output image together with zoom functionality
`
`
`
`- 6 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`using 1) two cameras having respective field of views (FOVs) and image sensors
`
`with color filter arrays to provide two images and 2) a processor providing an
`
`output image using the two images based on the relationship between a zoom
`
`factor input and the FOVs of the two cameras. (APPL-1004), Cossairt, ¶32
`
`B.
`
`Prosecution History of the ’152 Patent
`
`The ’152 Patent issued from U.S. Patent Application No. 14/386,823 (“’823
`
`App”), which is a National Phase application from PCT patent application
`
`PCT/IB2013/060356 filed November 23, 2013, which claims priority from US
`
`Provisional Application No. 61,730,570, filed November 28, 2012. (APPL-1002),
`
`’823 App, 210; (APPL-1003), ’570 App.
`
`During prosecution of the ’823 App, a single non-final Office Action was
`
`issued on May 17, 2016, rejecting claims 1, 4, 10, 13, and 44-61 “under pre-AIA
`
`35 U.S.C. 103(a) as being unpatentable over Dagher, US 2011/0064327 in view of
`
`Koskinen, US 8179457.” (APPL-1002), ’823 App, 48. Regarding claim 44 which
`
`ultimately issued as claim 1, the Examiner stated that Dagher discloses elements
`
`(a), (b), and (c) of claim 44. (APPL-1002), ’823 App, 52-53.
`
`On August 10, 2016, the Applicant filed an Amendment to cancel claims 52
`
`and 61. (APPL-1002), ’823 App, 36. Regarding claim 44, the Applicant argued
`
`that Dagher “does not teach an output image being related to a zoom factor and a
`
`point of view” as recited in element (c) because Dagher’s output image “is always
`
`
`
`- 7 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`based on the Wide camera point of view.” (APPL-1002), ’823 App, 39
`
`(emphasis original).
`
`On November 10, 2016, the Applicant had a telephonic interview with the
`
`Examiner and discussed references Border, US 2008/0218612 and Williams, US
`
`2013/0141525. (APPL-1002), ’823 App, 17. The Applicant “agreed to make
`
`amendments to claims 1, 44 & 53 to include the relationship of the zoom factor to
`
`the first and second field of view that dictates at least the switching of the point of
`
`view, via Examiner’s Amendment in order to issue the case.” (APPL-1002), ’823
`
`App, 17.
`
`On November 25, 2016, a notice of allowance issued. (APPL-1002), ’823
`
`App, 7. In the Examiner’s Amendment, claims 1, 4, 10, 13, 45-51, and 54-60 were
`
`deleted. (APPL-1002), ’823 App, 12. Claims 44 and 53, which were ultimately
`
`issued as claims 1 and 3, were amended. (APPL-1002), ’823 App, 12-14. New
`
`claims 62 and 63, which ultimately issued as claims 2 and 4 respectively, were
`
`added. (APPL-1002), ’823 App, 14.
`
`The Examiner’s statement of reasons for allowance provided, “The prior art
`
`teaches switching the point of view from different cameras or sensors based on an
`
`adjusted zoom level and also fusing images. It does not explicitly teach or suggest
`
`a relationship of the zoom factor to a first and second FOV that dictates which
`
`
`
`- 8 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`corresponding image is used as the primary image when the images are fused, in
`
`conjunction with other elements.” (APPL-1002), ’823 App, 15.
`
`The ’512 Patent issued on January 3, 2017. Claims 44, 53, 62, and 63 were
`
`issued as claims 1, 3, 2, and 4 respectively. Note that neither the actual language
`
`of independent claim 1 nor that of independent claim 3 can be fully reconciled with
`
`the Examiner’s expressed reasons for allowance or with the apparent concession
`
`of Applicant’s agreement to make amendments to secure allowance.
`
`V.
`
`LEVEL OF ORDINARY SKILL IN THE ART
`
`The level of ordinary skill in the art may be reflected by the prior art of
`
`record. See Okajima v. Bourdeau, 261 F.3d 1350, 1355 (Fed. Cir. 2001). Here, a
`
`Person of Ordinary Skill in the Art (“POSITA”) at the time of the claimed
`
`invention would have a bachelor’s or the equivalent degree in computer science or
`
`electrical and/or computer engineering or a related field and 2-3 years of
`
`experience in imaging systems including optics design and imaging processing.
`
`(APPL-1004), Cossairt, ¶19. Furthermore, a person with less formal education but
`
`more experience, or more formal education but less experience, could have also
`
`met the relevant standard for a POSITA. (APPL-1004), Cossairt, ¶19. However,
`
`Petitioner does not imply that a person having an extraordinary level of skill should
`
`be regarded as a POSITA.
`
`
`
`- 9 -
`
`

`

`
`
`VI. CLAIM CONSTRUCTION
`
`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`This Petition presents claim analysis in a manner that is consistent with a
`
`claim term’s plain and ordinary meaning in light of the specification. See 37
`
`C.F.R. § 42.100(b). Accordingly, claim terms are given their ordinary and
`
`accustomed meaning as would be understood by a person of ordinary skill in the
`
`art in the context of the entire disclosure. In re Translogic Tech., Inc., 504 F.3d
`
`1249, 1257 (Fed. Cir. 2007) (citing Phillips v. AWH Corp., 415 F.3d 1303 (Fed.
`
`Cir. 2005) (en banc)). For terms not addressed below, Petitioner submits that no
`
`specific construction is necessary for this proceeding.1
`
`A. “standard color filter array (CFA)” (claims 1 and 3)
`
`In the context of the ’512 Patent, a POSITA would have understood “standard
`
`color filter array (CFA)” to mean “a color filter array including a RGB (Bayer)
`
`pattern, RGBE, CYYM, CYGM, RGBW#l, RGBW#2, or RGBW#3.” (APPL-1004),
`
`Cossairt, ¶¶44-47.
`
`The ’152 Patent expressly defines the term “standard CFA” to mean a color
`
`filter array including “a RGB (Bayer) pattern or a non-Bayer pattern such as RGBE,
`
`CYYM, CYGM, RGBW#l, RGBW#2 or RGBW#3.” (APPL-1004), Cossairt, ¶46.
`
`
`
`1 Petitioner does not concede that any term not construed herein meets the statutory
`
`requirements of 35 U.S.C. § 112.
`
`
`
`- 10 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`Specifically, the ’152 Patent provides, “A ‘standard CFA’ may include a RGB
`
`(Bayer) pattern or a non-Bayer pattern such as RGBE, CYYM, CYGM, RGBW#l,
`
`RGBW#2 or RGBW#3. Thus, reference may be made to ‘standard Bayer or
`
`‘standard non-Bayer’ patterns or filters.” ’152 Patent, 2: 43-47; (APPL-1004),
`
`Cossairt, ¶46. As contrasted with “standard CFA,” the ’153 Patent provides, “[a]s
`
`used herein, ‘non-standard CFA’ refers to a CFA that is different in its pattern
`
`that [sic] CFAs listed above as ‘standard’.” ’152 Patent, 2: 48-49; (APPL-1004),
`
`Cossairt, ¶46.
`
`Accordingly, in the context of the ’152 Patent, a POSITA would have
`
`understood “standard color filter array (CFA)” to mean “a color filter array including
`
`a RGB (Bayer) pattern, RGBE, CYYM, CYGM, RGBW#l, RGBW#2, or RGBW#3.”
`
`(APPL-1004), Cossairt, ¶47.
`
`VII. REQUESTED RELIEF
`
`Petitioner requests that the Board institute inter partes review of claims 1-4
`
`of the ’152 Patent and cancel each of those claims as unpatentable.
`
`VIII. IDENTIFICATION OF HOW THE CLAIMS ARE UNPATENTABLE
`
`A. Challenged Claims
`
`Claims 1-4 of the ’152 Patent are challenged in this petition.
`
`
`
`- 11 -
`
`

`

`
`
`
`
`
`Statutory Grounds for Challenges
`
`B.
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`Grounds
`Ground 1
`
`Claims
`1-4
`
`Basis
`Obvious under 35 U.S.C. § 103(a) over U.S. Patent
`Application Publication No. 2008/0030592 to
`Border et al. (“Border”) in view of U.S. Patent No.
`7,859,588 to Parulski, et al. (“Parulski”)
`
`
`Border is a US patent application filed on August 1, 2006 and published on
`
`February 7, 2008. Parulski was filed on March 9, 2007, published on September
`
`11, 2008, and issued on December 28, 2010. These references are prior art to the
`
`’512 Patent under at least 35 U.S.C. § 102(b).
`
`C. Discretionary Denial is Not Warranted
`
`The Board’s discretionary determination of whether to institute review is
`
`guided by 35 U.S.C. § 325(d) and its precedent under § 314(a). Under the present
`
`circumstances, the Board should not exercise its discretion to deny institution on
`
`the ground.
`
`Although Border was cited in an information disclosure statement (IDS)
`
`during prosecution, it was never substantively discussed, and thus inter partes
`
`review in light of Border-based grounds is appropriate. APPL-1002, 94. See
`
`Limelight Networks, Inc. v. Mass. Inst. of Tech., IPR2017-00249, Paper 9, at 7
`
`(May 18, 2017) (instituting despite a § 325(d) challenge where reference was never
`
`substantively discussed by Examiner). Furthermore, Border is combined with
`
`
`
`- 12 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`Parulski, which was not cited during prosecution. Thus, the primary reference,
`
`Border, was never substantively discussed, and Petitioner’s specific ground, based
`
`on proposed combinations of Border and Parulski, has not yet been considered by
`
`the Office. Additionally, Petitioner’s expert declaration, which provides evidence
`
`as to how a POSITA would understand the teachings of Border has not yet been
`
`considered by the Office. Finally, as noted above (see IV.B, supra at 9), the
`
`prosecution history raises serious questions as to the relation between claims that
`
`the Office intended to allow, even vis-à-vis the art that the Examiner appears to
`
`have substantively discussed. In short, institution of trial is warranted based on
`
`Petitioner’s challenges, and § 325(d) does not provide a basis for discretionary
`
`denial.
`
`D. Note Regarding Page Citations and Emphasis
`
`For exhibits that include suitable page, column, or paragraph numbers in
`
`their original publication, Petitioner’s citations are to those original page, column,
`
`or paragraph numbers and not to the page numbers added for compliance with
`
`37 CFR 42.63(d)(2)(ii). Also, the following analysis may bold or italicize
`
`quotations and add color or colored arrow annotations to the figures from these
`
`exhibits for the sake of emphasis.
`
`
`
`- 13 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`IX. IDENTIFICATION OF HOW THE CLAIMS ARE UNPATENTABLE
`
`A. Ground 1: Claims 1-4 are unpatentable under 35 U.S.C. § 103(a)
`over Border and Parulski
`
`1.
`
`Summary of Border
`
`U.S. Patent Application Publication No. 2008/0030592 to Border et al.
`
`(“Border”) is directed to “a digital camera that uses multiple lenses and image sensors
`
`to provide an extended zoom range and the method used to produce a digital image
`
`that combines the multiple images produced by the digital camera.” (APPL-1006),
`
`Border, Abstract, [0002]; (APPL-1004), Cossairt, ¶48. The face of Border lists John
`
`N. Border of Walworth, NY as an inventor, and lists Eastman Kodak Company of
`
`Rochester NY as the assignee. (APPL-1006), Border, 1; (APPL-1004), Cossairt, ¶48.
`
`Border recognizes that “[s]mall camera size and a large zoom range are two
`
`very important features of digital cameras.” (APPL-1006), Border, [0004]; (APPL-
`
`1004), Cossairt, ¶49. The zoom range may include “optical zoom which is provided
`
`by variable focal length lenses and digital zoom which is provided by a magnification
`
`of the digital image after capture.” (APPL-1006), Border, [0004]; (APPL-1004),
`
`Cossairt, ¶49. While digital zoom is fast and simple, the resulting decrease in
`
`resolution may produce a perceived decrease in image quality. (APPL-1006), Border,
`
`[0006]; (APPL-1004), Cossairt, ¶49.
`
`Border describes providing a digital camera with an extended zoom range
`
`without unduly increasing the size or cost of the digital camera “while providing good
`
`
`
`- 14 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`perceived image quality throughout the zoom range.” (APPL-1006), Border, [0010];
`
`(APPL-1004), Cossairt, ¶50. As shown in Figure 5 of Border below, the processor of
`
`a digital camera includes an image compositor 202 to form a composite image 208
`
`using the two images, wide image 204 and telephoto image 206 of the same scene,
`
`that are captured using lenses having different focal lengths. (APPL-1006), Border,
`
`[0070]; (APPL-1004), Cossairt, ¶51. The image registration determiner 212
`
`determines the registration between the wide image 204 and the telephoto image 206,
`
`so that the two images are matched to “locate the high-resolution image accurately
`
`into the low-resolution image and then stitched into place so the edge between the two
`
`images in the composite image is not discernible.” (APPL-1006), Border, [0029].
`
`Border goes on to explain that in the context of Figure 5, telephoto image 206
`
`captures a smaller portion of the scene, but with greater resolution than wide image
`
`204. (APPL-1006), Border, [0036]; (APPL-1004), Cossairt, ¶51.
`
`
`
`- 15 -
`
`

`

`
`
`
`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`
`
`(APPL-1006), Border, Fig. 5
`
`Border also describes that an image resampler 214 of the processor produces
`
`the composite image 208 based on a zoom amount Z specifying the desired relative
`
`zoom amount of the produced composite image 208. (APPL-1006), Border, [0043];
`
`(APPL-1004), Cossairt, ¶52. Specifically, Border explains that the composite image
`
`208 is generated from the two images and that the resulting composite image is
`
`produced differently for different zoom amount values, such as Z=1, 1<Z<M, and
`
`Z=M, where M is the relative magnification ratio M of the telephoto image 206 to the
`
`wide image 204. (APPL-1006), Border, [0029], [0044]; (APPL-1004), Cossairt, ¶52.
`
`2.
`
`Summary of Parulski
`
`Parulski is titled “Method and Apparatus for Operating a Dual Lens Camera to
`
`
`
`- 16 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`Augment an Image,” and discloses “a digital camera that uses multiple lenses and
`
`image sensors to provide an improved imaging capability.” (APPL-1007), Parulski,
`
`Title, 1:8-10; (APPL-1004), Cossairt, ¶53. As with Border, the face of Parulski lists
`
`John N. Border of Walworth, NY as an inventor and lists Eastman Kodak Company
`
`of Rochester NY as the assignee. (APPL-1004), Cossairt, ¶53.
`
`Parulski “utilizes one of the images from a dual-lens camera as a secondary
`
`image that can be used to modify the other, primary image and thereby generate an
`
`enhanced primary image.” (APPL-1007), Parulski, 7:32-35; (APPL-1004), Cossairt,
`
`¶54. Specifically, Parulski discloses examples of the enhancement to the primary
`
`image include “to sharpen portions of the primary image,” “to modify the dynamic
`
`range of the primary image,” or “to replace portions of the primary image (areas of
`
`lower noise but with some motion blur) with corresponding portions of the secondary
`
`image (areas of higher noise but little or no motion blur) to obtain a modified image
`
`with relatively low noise and good sharpness.” (APPL-1007), Parulski, Fig. 26, 7:54-
`
`8:5; (APPL-1004), Cossairt, ¶54.
`
`Parulski describes determining the primary image and secondary image from
`
`two capture units of the digital camera based on a requested zoom position provided
`
`by a user. (APPL-1007), Parulski, Fig. 23, 27:8-24; (APPL-1004), Cossairt, ¶55. For
`
`example, if the requested zoom position is not within the zoom range of the current
`
`primary capture unit for providing a primary image, “the functions of the capture units
`
`
`
`- 17 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`are reversed” by switching the capture unit for providing a secondary image and the
`
`capture unit for providing the primary image. (APPL-1007), Parulski, 27:8-15;
`
`(APPL-1004), Cossairt, ¶55.
`
`Parulski incorporates Border by reference, and provides that its image
`
`augmentation process may be applied to “image pairs having different resolutions” as
`
`described in Border, including “a first wide angle digital image of a scene and a
`
`second telephoto digital image of a portion of substantially the same scene” that are
`
`captured by two capture stages as described in Border. (APPL-1007), Parulski, 29:51-
`
`58; (APPL-1004), Cossairt, ¶56.
`
`3.
`
`Reasons to Combine Border and Parulski
`
`A POSITA would have been motivated to combine the teachings of Parulski
`
`that a primary image may be modified using a secondary image to generate an
`
`enhanced primary image with Border’s teachings of a digital imaging system using
`
`multiple lenses and multiple sensors to provide an extended zoom range and good
`
`image quality. (APPL-1004), Cossairt, ¶¶57-61.
`
`First, Border and Parulski are analogous prior art and are in the same field of
`
`endeavor pertaining to a digital camera that uses multiple lenses and image sensors to
`
`provide an enhanced output image. (APPL-1004), Cossairt, ¶58. Border discloses “a
`
`digital camera that uses multiple lenses and image sensors to provide an extended
`
`zoom range,” which improves perceived image quality throughout the zoom range by
`
`
`
`- 18 -
`
`

`

`IPR2018-01133 Petition
`
`
`Inter Partes Review of 9,538,152
`
`
`“forming a composite image with portions of the image from the short focal length
`
`lens and portions of the image from the longer focal length lens.” (APPL-1006),
`
`Border, [0002], [0014]-[0015]; (APPL-1004), Cossairt, ¶58. Similarly, Parulski
`
`discusses using “one of the images from a dual-lens camera as a secondary image ... to
`
`modify the other, primary image and thereby generating an enhanced primary image.”
`
`(APPL-1007), Parulski, 7:32-35; (APPL-1004), Cossairt, ¶58. “[A]ny need or
`
`problem known in the field of endeavor at the time of the invention and addressed
`
`by the patent can provide a reason for combining the elements in the manner
`
`claimed.” KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 420 (2007). Here,
`
`enhancing the quality of an output image using a digital camera that uses multiple
`
`lenses and image sensors is a need shared by Border and Parulski, and provides at
`
`least one reason to combine the respective teachings.
`
` Second, A POSITA looking to improve upon the digital imaging system of
`
`Border would typically refer to references describing other similar devices to get a
`
`sense of how the industry approached and solved problems associated with such
`
`devices. (APPL-1004), Cossairt, ¶59. When evaluating the teachings of Border, a
`
`POSITA would naturally have considered the teachings of Parulski, which is a
`
`patent that has the same co-inventor (John N. Border) and the same assignee
`
`(Eastman Kodak Company, one of the top digital camera makers) as Border. (APPL-
`
`1004), Cossairt, ¶59.
`
`
`
`- 19 -
`
`

`

`
`
`
`IPR2018-01133 Petition
`
`Inter Partes Review of 9,538,152
`
`Third, Parulski itself provides an express motivation to use Parulski’s
`
`teachings in the system of Border. (APPL-1004), Cossairt, ¶60. Specifically, Parulski
`
`explicitly provides that its image augmentation process of using a secondary image to
`
`modify a primary image “can also be applied in connection with image pairs having
`
`different resolutions.” (APPL-1007), Parulski, 29:51-52; (APPL-1004), Cossairt, ¶60.
`
`Parulski refers to Border’s system as an example for such application, and explicitly
`
`incorporates Border by reference. (APPL-1007), Parulski, 29:52-67; (APPL-1004),
`
`Cossairt, ¶60.
`
`Thus, a POSITA would have been motivated to apply Parulski’s teachings
`
`for modifying a primary image using a secondary image to generate an enhanced
`
`primary image in Border’s multi-lens digital camera because the combination
`
`would provide the benefits of enhanced image quality (e.g., “a broadened depth of
`
`field,” “a broadened dynamic range,” “relatively low noise and good sharpness”) in
`
`such a digital camera. (APPL-1007), Parulski, 28:52-53, 29:4-7, 30:17-20; (APPL-
`
`1004), Cossairt, ¶61.
`
`4. Claim 1
`
`[1.0] A multi-aperture imaging system comprising:
`
`To the extent that this preamble is deemed limiting, Border teaches a multi-
`
`aperture imaging system. (APPL-1004), Cossairt, ¶¶62-65.
`
`
`
`- 20 -
`
`

`

`
`
`
`IPR2018-01133 Petition
`
`Inter Partes Review of 9,538,152
`
`Specifically, as shown in the annotated Figure 1B of Border below, Border
`
`teaches a digital camera 10B (an imaging system) including “two fixed focal length
`
`lenses 2 and 4, each providing an image to a corresponding image sensor 12 and 14.”
`
`(APPL-1006), Border, [0058]; (APPL-1004), Cossairt, ¶63. In the digital camera
`
`10B, the two fixed focus lenses 2 and 4 are selected to provide a substantial zoom
`
`range, and “a composite image is constructed from the two images captured on
`
`images sensors 12 and 14.” (APPL-1006), Border, [0058]; (APPL-1004), Cossairt,
`
`¶63. The image processor 50 may apply digital zoom to the composite image
`
`according to a zoom amount “that can be adjusted by the camera user.” (APPL-
`
`1006), Border, [0036], [0058]; (APPL-1004), Cossairt, ¶63.
`
`
`
`- 21 -
`
`

`

`
`
`
`
`
`
`IPR2018-01133 Petition
`Inter Partes Review of 9,538,152
`
`
`
`(APPL-1006), Border, Fig. 1B, annotated
`
`Border describes two lenses 2 and 4 of digital camera 10B each having its own
`
`aperture. (APPL-1004), Cossairt, ¶64.