UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.
`Patent Owner
`
`_______________
`
`IPR2018-01133
`U.S. Patent No. 9,538,152
`_______________
`
`DECLARATION OF OLIVER COSSAIRT, PH.D.
`UNDER 37 C.F.R. § 1.68 IN SUPPORT OF PETITION
`FOR INTER PARTES REVIEW
`
`
`
`
`
`
`
`
`
`
`
`APPL-1004/Page 1 of 76
`Apple Inc. v. Corephotonics
`
`

`

`
`
` Declaration of Oliver Cossairt, Ph.D.
` Inter Partes Review of U.S. Patent 9,538,152
`TABLE OF CONTENTS
`
`INTRODUCTION ........................................................................................... 1(cid:3)
`I. (cid:3)
`II. (cid:3) QUALIFICATIONS ........................................................................................ 2(cid:3)
`III.(cid:3) LEVEL OF ORDINARY SKILL IN THE ART ............................................. 6(cid:3)
`IV. (cid:3) RELEVANT LEGAL STANDARDS ............................................................. 8(cid:3)
`A. (cid:3) Anticipation ........................................................................................... 8(cid:3)
`B. (cid:3) Obviousness ........................................................................................... 8(cid:3)
`V. (cid:3) THE ’152 PATENT ....................................................................................... 10(cid:3)
`A.(cid:3)
`Summary of ’152 Patent ...................................................................... 10(cid:3)
`B.(cid:3)
`Prosecution History of the ’152 Patent ............................................... 14(cid:3)
`VI. (cid:3) CLAIM CONSTRUCTION .......................................................................... 17(cid:3)
`A. “standard color filter array (CFA)” (claims 1 and 3) .................... 18(cid:3)
`VII.(cid:3) GROUNDS .................................................................................................... 19(cid:3)
`A.(cid:3) Ground 1: Claims 1-4 are unpatentable under 35 U.S.C. § 103(a) over
`Border in view of Parulski .................................................................. 19(cid:3)
`1.(cid:3)
`Summary of Border ................................................................... 19(cid:3)
`2.(cid:3)
`Summary of Parulski ................................................................. 21(cid:3)
`3.(cid:3)
`Reasons to Combine Border and Parulski ................................. 23(cid:3)
`4.(cid:3)
`Claim 1 ...................................................................................... 25(cid:3)
`3.(cid:3)
`Claim 2 ...................................................................................... 68(cid:3)
`4.(cid:3)
`Claim 3 ...................................................................................... 71(cid:3)
`5.(cid:3)
`Claim 4 ...................................................................................... 73(cid:3)
`VIII.(cid:3) DECLARATION ........................................................................................... 74(cid:3)
`
`
`
`
`
`
`
`
`
`
`
`
`- i -
`
`APPL-1004 / Page 2 of 76
`
`

`

`
`
`I.
`
`Declaration of Oliver Cossairt, Ph.D.
`
` Inter Partes Review of U.S. Patent 9,538,152
`INTRODUCTION
`
`1.
`
`I, Oliver Cossairt, have been retained by counsel for Apple Inc.
`
`(“Apple” or “Petitioner”) as a technical expert in connection with the proceeding
`
`identified above. I submit this declaration in support of Apple’s Petition for Inter
`
`Partes Review of U.S. Patent No. 9,538,152 (“the ’152 Patent”).
`
`2.
`
`Compensation for my work in this matter is based on an hourly rate.
`
`In addition, reasonable and customary expenses associated with my work and
`
`testimony in this matter are reimbursed. This compensation is not contingent on
`
`the outcome of this matter, nor is it contingent on the specifics of my testimony. I
`
`have no personal or financial stake, nor any interest in the outcome of the present
`
`proceeding.
`
`3.
`
`(1)
`
`(2)
`
`1002;
`
`In the preparation of this declaration, I have studied:
`
`The ’152 Patent, APPL-1001;
`
`The prosecution file history of the ’152 Patent (’823 App), APPL-
`
`(3)
`
`The prosecution file history of U.S. Provisional App. No. 61/730,570
`
`(’570 App), APPL-1003;
`
`(4) U.S. Patent Application Publication No. 2008/0030592 to Border et
`
`al. (“Border”), APPL-1006;
`
`(5) U.S. Patent No. 7,859,588 to Parulski et al. (“Parulski”), APPL-1007;
`
`
`
`- 1 -
`
`APPL-1004 / Page 3 of 76
`
`

`

`
`
`(6)
`
`Declaration of Oliver Cossairt, Ph.D.
`
` Inter Partes Review of U.S. Patent 9,538,152
`Ralph E. Jacobson et al., The Manual of Photography: photographic
`
`and digital imaging, 9th Edition, 2000 (“Jacobson”), APPL-1008;
`
`(7) Michael Langford et al., Langford’s Advanced Photography, 7th
`
`Edition, 2008 (“Langford”), APPL-1009;
`
`(8)
`
`Richard Szeliski, Computer Vision: Algorithms and Applications,
`
`2011 (“Szeliski”), APPL-1010.
`
`4.
`
`(1)
`
`In forming the opinions expressed below, I have considered:
`
`The documents listed above;
`
`(2) Any additional documents discussed below; and
`
`(3) My own knowledge and experience based upon my work in the fields
`
`of imaging systems as described below.
`
`II. QUALIFICATIONS
`
`5.
`
`My qualifications and professional experience are described in my
`
`Curriculum Vitae, a copy of which can be found in exhibit APPL-1005. The
`
`following is a brief summary of my relevant qualifications and professional
`
`experience.
`
`6.
`
`For more than 15 years, I have been developing professional and
`
`academic experience in the field of imaging systems. One of the major themes of
`
`my research has been directed to computational photography, which combines
`
`expertise in optics design and image processing. One of my focused research areas
`
`
`
`- 2 -
`
`APPL-1004 / Page 4 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`is for leveraging optical designs and image processing together to enable new
`
`capabilities in conventional cameras, such as extended depth-of-field, digital
`
`refocusing, super-resolution, and measuring high dimensional appearance.
`
`7.
`
`I earned my Bachelor of Science degree in Physics from Evergreen
`
`State College, Olympia, Washington, in 2001. In 2003, I earned my Master of
`
`Science degree in the Department of Media Arts and Sciences from Massachusetts
`
`Institute of Technology, Cambridge, Massachusetts.
`
`8.
`
`From 2003 to 2006, I worked as an Optical and Software Engineer at
`
`Actuality Systems, Inc., Reading, Massachusetts. As part of this work, I managed
`
`a National Institute of Standards and Technology (NIST) Advanced Technology
`
`Program (ATP) research initiative to develop next generation 3D displays.
`
`9.
`
`I received a Ph.D. in Computer Science from Columbia University,
`
`New York City, New York in 2011. My doctoral thesis focused on computational
`
`imaging, specifically tradeoffs and limits in computational imaging. After
`
`receiving the Ph.D., I continued my research in computational imaging as a
`
`Postdoctoral Researcher at Columbia University till June 2012.
`
`10.
`
`Since 2012, I have been a professor in the Department of Electrical
`
`Engineering and Computer Science at the Northwestern University, Evanston,
`
`Illinois. I currently lead the Computational Photography Lab at Northwestern
`
`University, which develops imaging and display systems that combine a creative
`
`
`
`- 3 -
`
`APPL-1004 / Page 5 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`use of optical devices, sensor technology, and image processing algorithms to
`
`enable new functionality in cameras and displays.
`
`11. As a professor, I teach in the area of computational photography. In
`
`the course of Introduction to Computational Photography, I teach principles of
`
`computational photography through a series of hands on projects, including, for
`
`example, programming on a Nokia n900 phone to control its camera’s focus, flash,
`
`exposure, etc. In the course of Computational Photography Seminar, I teach state
`
`of the art research in computational cameras, where graduate and undergraduate
`
`students learn novel techniques and methods on computational photography,
`
`including for example, using multi-aperture camera systems to increase spatial
`
`resolution.
`
`12.
`
`I have authored and co-authored over 45 journal publications,
`
`conference proceedings, technical papers, and technical presentations in the area of
`
`imaging system technologies, including optics design, image processing, and
`
`computational photography. In 2010, I received the Best Paper Award “Spectral
`
`Focal Sweep: Extended Depth of Field from Chromatic Aberrations” at the IEEE
`
`International Conference on Computational Photography (ICCP). In 2014, I
`
`received the Best Paper Honorable Mention Award for the paper “Digital
`
`Refocusing with Incoherent Holography” at ICCP. In 2017, I received another
`
`Best Paper Honorable Mention Award for the paper “Coherent inverse scattering
`
`
`
`- 4 -
`
`APPL-1004 / Page 6 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`via transmission matrices” at ICCP. In addition, I am an inventor on 11 patents in
`
`the United States.
`
`13.
`
`In 2015, I received a National Science Foundation (NSF) Faculty
`
`Early Career Development (CAREER) on my research project “Coherent
`
`Computational Imaging: Macro Measurements in a Macro World.” The NSF
`
`CAREER award is NSF’s most prestigious award in support of early-career faculty
`
`who has the potential to serve as academic role models in research and education
`
`and to lead advances in the mission of their department or organization. The goal
`
`of this project is to build fundamentally new types of cameras that combine novel
`
`optics and algorithm design to overcome the diffraction limit for macroscopic
`
`scenes, thereby achieving high levels of precision in image, depth, and material
`
`acquisition. For example, a large array of cameras may be used together with
`
`active illumination to significantly increase resolution beyond the diffraction limit.
`
`14. My involvement in the research community extends to several
`
`organizations, journals, and conferences. Over the years, I have organized and
`
`served in the Program Committee of a variety of conferences, including
`
`International Conference on Computer Vision and Pattern Recognition (CVPR),
`
`ICCP, Optical Society of America (OSA) Computational Optics and Sensing
`
`(COSI), International Conference on Image Processing (ICIP). I have been a
`
`reviewer for many high-quality conference proceedings and journal publications
`
`
`
`- 5 -
`
`APPL-1004 / Page 7 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`including, Association for Computing Machinery (ACM) Special Interest Group
`
`on Computer Graphics and Interactive Techniques (SIGGRAPH), ACM
`
`Transactions on Graphics, Eurographics Symposium on Rendering (EGSR), IEEE
`
`Transactions on Pattern Analysis and Machine Intelligence (PAMI), International
`
`Journal of Computer Vision (IJCV), Applied Optics, and Optics Express. Since
`
`2014, I have been an Associate Editor for IEEE Transactions on Computational
`
`Imaging. During 2013-2015, I served as a National Science Foundation (NSF)
`
`panel reviewer in the area of photonic/optical systems.
`
`15. A list of my publications and patents is contained in my CV at exhibit
`
`APPL-1005.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`
`16.
`
`I understand that the level of ordinary skill may be reflected by the
`
`prior art of record, and that a Person of Ordinary Skill In The Art (“POSITA”) to
`
`which the claimed subject matter pertains would have the capability of
`
`understanding the scientific and engineering principles applicable to the pertinent
`
`art. I understand that a POSITA has ordinary creativity, and is not an automaton.
`
`17.
`
`I understand that there are multiple factors relevant to determining the
`
`level of ordinary skill in the pertinent art, including (1) the levels of education and
`
`experience of persons working in the field at the time of the invention; (2) the
`
`sophistication of the technology; (3) the types of problems encountered in the field;
`
`
`
`- 6 -
`
`APPL-1004 / Page 8 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`and (4) the prior art solutions to those problems.
`
`18.
`
`I am familiar with the imaging system art pertinent to the ’152 Patent.
`
`I am also aware of the state of the art at the time the application resulting in the
`
`’152 Patent was filed. I have been informed by counsel that the earliest claimed
`
`priority date for the ’152 Patent is November 12, 2012, although any given claim
`
`of the ’152 Patent may or may not be entitled to the earliest claimed date.
`
`19. Based on the technologies disclosed in the ’152 Patent, I believe that a
`
`POSITA would include someone who had, as of the claimed priority date of the
`
`’152 Patent, a bachelor’s or the equivalent degree in computer science or electrical
`
`and/or computer engineering or a related field and 2-3 years of experience in
`
`imaging systems including optics design and imaging processing. In addition, I
`
`recognize that someone with less formal education but more experience, or more
`
`formal education but less experience could have also met the relevant standard for
`
`a POSITA. I believe that I am a POSITA and, furthermore, I have supervised
`
`students and engineers who were also POSITAs. Accordingly, I believe that I am
`
`qualified to opine from the perspective of a POSITA regarding the ’152 Patent.
`
`20.
`
`For purposes of this Declaration, unless otherwise noted, my opinions
`
`and statements, such as those regarding the understanding of a POSITA (and
`
`specifically related to the references I consulted herein), reflect the knowledge that
`
`existed in the art before the earliest claimed priority date of the ’152 Patent.
`
`
`
`- 7 -
`
`APPL-1004 / Page 9 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`IV. RELEVANT LEGAL STANDARDS
`
`21.
`
`I have been asked to provide my opinions regarding whether claims 1-
`
`4 (the “Challenged Claims”) of the ’152 Patent would have been obvious to a
`
`POSITA at the time of the alleged invention in light of the prior art.
`
`22.
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’152 Patent, I am relying on certain legal
`
`principles explained to me by counsel.
`
`23.
`
`I understand that a claim is unpatentable if it is anticipated under 35
`
`U.S.C. § 102 or obvious under 35 U.S.C. § 103.
`
`A. Anticipation
`
`24.
`
`I have been informed by counsel that a patent claim is unpatentable as
`
`anticipated if each element of that claim is present either explicitly or inherently in
`
`a single prior art reference. I have also been informed that, to be an inherent
`
`disclosure, the prior art reference must necessarily disclose the limitation, and the
`
`fact that the reference might possibly practice or contain a claimed limitation is
`
`insufficient to establish that the reference inherently teaches the limitation.
`
`B. Obviousness
`
`25.
`
`I have been informed and I understand that a claimed invention is
`
`unpatentable under 35 U.S.C. § 103(a) if the differences between the subject matter
`
`sought to be patented and the prior art are such that the subject matter as a whole
`
`
`
`- 8 -
`
`APPL-1004 / Page 10 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`would have been obvious to a POSITA at the time the invention was made. I
`
`understand that the appropriate analysis for determining obviousness of a claimed
`
`invention takes into account factual inquiries, including the level of ordinary skill
`
`in the art, the scope and content of the prior art, and the differences between the
`
`prior art and the claimed subject matter as a whole.
`
`26.
`
`I have been informed and I understand that the United States Supreme
`
`Court has recognized several rationales for combining references or modifying a
`
`reference to show obviousness of claimed subject matter. Some of these rationales
`
`include the following: (a) combining prior art elements according to known
`
`methods to yield predictable results; (b) simple substitution of one known element
`
`for another to obtain predictable results; (c) use of a known technique to improve a
`
`similar device (method, or product) in the same way; (d) applying a known
`
`technique to a known device (method, or product) ready for improvement to yield
`
`predictable results; (e) choosing from a finite number of identified, predictable
`
`solutions, with a reasonable expectation of success; and (f) some teaching,
`
`suggestion, or motivation in the prior art that would have led a POSITA to modify
`
`the prior art reference or to combine prior art reference teachings to arrive at the
`
`claimed invention. I have also been informed and I understand that a
`
`demonstration of obviousness does not require a physical combination or bodily
`
`incorporation, but rather may be found based on consideration of what the
`
`
`
`- 9 -
`
`APPL-1004 / Page 11 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`combined teachings would have suggested to a POSITA at the time of the alleged
`
`invention.
`
`V. THE ’152 PATENT
`
`A.
`
`27.
`
`Summary of ’152 Patent
`
`The ’152 Patent is titled “High Resolution Thin Multi-Aperture
`
`Imaging Systems” and was issued on January 3, 2017. (APPL-1001), ’152 Patent,
`
`Title. The ’152 Patent is directed to a “multi-aperture imaging (‘MAI’) systems ...
`
`with high color resolution and/or optical zoom.” (APPL-1001), ’152 Patent, 1:15-
`
`18. In its background, the ’152 Patent provides, “Optical Zoom is a primary
`
`feature of many digital still cameras but one that mobile phone cameras usually
`
`lack, mainly due to camera height constraints in mobile imaging devices, cost and
`
`mechanical reliability.” (APPL-1001),’152 Patent, 1:31-34. The ’152 Patent
`
`recognizes that while mechanical zoom solutions are common in digital still
`
`cameras, they are “typically too thick for most camera phones” and may result in
`
`“resolution compromise.” (APPL-1001), ’152 Patent, 1:35-43.
`
`28.
`
`In its background, the ’152 Patent acknowledges that one of the
`
`known approaches is using a multi-aperture imaging (“MAI”) system, for example,
`
`a dual-aperture imaging system (“DAI”) including “two optical apertures which
`
`may be formed by one or two optical modules, and one or two image sensors” to
`
`implement “zoom as well as increasing the output resolution.” (APPL-1001),’152
`
`
`
`- 10 -
`
`APPL-1004 / Page 12 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`Patent, 1:52-59. However, the ’152 Patent alleges that those known multi-aperture
`
`imaging systems “often trade-off functionalities and properties, for example zoom
`
`and color resolution, or image resolution and quality for camera module height.”
`
`(APPL-1001),’152 Patent, 1:63-66. The ’152 Patent alleges that there was a need
`
`to have thin multi-aperture imaging systems that “produce an image with high
`
`resolution (and specifically high color resolution) together with zoom
`
`functionality.” (APPL-1001), ’152 Patent, 1:67-2:3.
`
`29. As an alleged solution to this problem, the’152 Patent describes a
`
`dual-aperture imaging system including a Wide sensor and a Tele sensor capturing
`
`a Wide image and a Tele image from two apertures, where color filter arrays may
`
`be used in the Wide sensor and Tele sensor. (APPL-1001), ’152 Patent, 2:34-65.
`
`The ’152 Patent also describes that “a different magnification image of the same
`
`scene is grabbed” by each sensor, “resulting in field of view (FOV) overlap.”
`
`(APPL-1001), ’152 Patent, 3:11-14. The Wide image and Tele image may be
`
`fused to “output one fused (combined) output zoom image processed according to
`
`a user [zoom factor] ZF input request.” (APPL-1001), ’152 Patent, 3:17-20.
`
`30.
`
`Figure 1A of the ’152 Patent below illustrates a dual-aperture zoom
`
`imaging system 100 including a Wide subset 104 and a Tele subset 106 each
`
`having a respective sensor, and Figure 1B of the ’152 Patent illustrates the Wide
`
`image and Tele image obtained by the Wide subset 104 and a Tele subset 106:
`
`
`
`- 11 -
`
`APPL-1004 / Page 13 of 76
`
`

`

`
`
`
`Declaration of Oliver Cossairt, Ph.D.
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`
`
`
`
`(APPL-1001), ’152 Patent, Figs. 1A and 1B
`
`31.
`
`In describing Figure 1A, the ’152 Patent explains that a processor 108
`
`“fuses a Wide image obtained with the Wide subset and a Tele image obtained
`
`
`
`- 12 -
`
`APPL-1004 / Page 14 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`with the Tele subset, into a single fused output image according to a user-defined
`
`‘applied’ ZF input or request.” (APPL-1001), ’152 Patent, 5:60-6:2. In describing
`
`Figure 1B, the ’152 Patent explains that an overlap area 110 of the Wide image and
`
`Tele image is illustrated on the Wide image in the figure. (APPL-1001), ’152
`
`Patent, 4:62-64, 6:2-9. Figure 1B illustrates by way of exemplary images, a larger
`
`FOV for the Wide image and a smaller FOV for the corresponding Tele image.
`
`32.
`
`To obtain the output image, the ’152 Patent teaches a registration
`
`process, which “chooses either the Wide image or the Tele image to be a primary
`
`image” “based on the ZF chosen for the output image.” (APPL-1001), ’152 Patent,
`
`9:20-21, 31-33. The registration process “considers the primary image as the
`
`baseline image and registers the overlap area in an auxiliary image to it,” and the
`
`“output image point of view is determined according to the primary image point of
`
`view (camera angle).” (APPL-1001), ’152 Patent, 9:20-28. In an example, “[i]f
`
`the chosen ZF is smaller than or equal to the ratio between the focal-lengths of the
`
`Tele and Wide cameras, the Wide image is set to be the primary image and the
`
`Tele image is set to be the auxiliary image.” (APPL-1001), ’152 Patent, 9:37-40.
`
`33. Representative independent claim 1 of the ’152 Patent is reproduced
`
`below:
`
`1. A multi-aperture imaging system comprising:
`a) a first camera that provides a first image, the first camera having a first
`field of view (FOV1) and a first sensor with a first plurality of sensor pixels
`covered at least in part with a standard color filter array (CFA);
`- 13 -
`
`
`
`APPL-1004 / Page 15 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`
`
`
`b) a second camera that provides a second image, the second camera having
`a second field of view (FOV2) such that FOV2<FOV1 and a second sensor with a
`second plurality of sensor pixels, the second plurality of sensor pixels being either
`Clear or covered with a standard CFA, the second image having an overlap area
`with the first image; and
`
`c) a processor configured to provide an output image from a point of view of
`the first camera based on a zoom factor (ZF) input that defines a respective field of
`view (FOVZF), the first image being a primary image and the second image being a
`non-primary image, wherein if FOV2<FOVZF<FOV1 then the point of view of the
`output image is that of the first camera, the processor further configured to register
`the overlap area of the second image as non-primary image to the first image as
`primary image to obtain the output image.
`
`(APPL-1001), ’152 Patent, 12:60-13:13.
`
`34. As I discuss below in more detail, the system and method presented in
`
`the ’152 Patent, namely, a multi-aperture imaging system producing “an image
`
`with high resolution ... together with zoom functionality” ((APPL-1001), ’152
`
`Patent, 2:1-3) using 1) two cameras having respective field of views (FOVs) and
`
`image sensors with color filter arrays to provide two images and 2) a processor
`
`providing an output image using the two images based on the relationship between
`
`a zoom factor input and the FOVs of the two cameras, was well known to persons
`
`of ordinary skill in the art before the earliest priority date of the ’152 Patent.
`
`B.
`
`Prosecution History of the ’152 Patent
`
`35. U.S. Patent Application No. 14/386,823 (“’823 App”), which
`
`ultimately issued as the ’152 Patent, was filed on September 22, 2014, and
`
`contained 32 claims. (APPL-1002), ’823 App, 204, 241-245. The ’823 App is a
`- 14 -
`
`
`
`APPL-1004 / Page 16 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`National Phase application from PCT patent application PCT/IB2013/060356 filed
`
`November 23, 2013, which claims priority from US Provisional Application No.
`
`61,730,570, filed November 28, 2012. (APPL-1002), ’823 App, 210; APPL-1003.
`
`36. On September 22, 2014, the Applicant filed a Preliminary
`
`Amendment concurrently with the National Phase application, which canceled
`
`claims 2, 3, 5-9, 11, 12 and 14-31 of the PCT patent application
`
`PCT/IB2013/060356, and adds new claims 33-43. (APPL-1002), ’823 App, 196-
`
`201.
`
`37. On September 16, 2015, the Applicant filed another Preliminary
`
`Amendment, which canceled claims 1, 4, 10, 13, and 32-43 and added new claims
`
`44-61. (APPL-1002), ’823 App, 73-80.
`
`38. A non-final Office Action was issued on May 17, 2016, rejecting
`
`claims 1, 4, 10, 13, and 44-61 “under pre-AIA 35 U.S.C. 103(a) as being
`
`unpatentable over Dagher, US 2011/0064327 in view of Koskinen, US 8179457.”
`
`(APPL-1002), ’823 App, 48. Regarding claim 44 which ultimately issued as claim
`
`1, the Examiner stated that Dagher discloses elements (a), (b), and (c) of claim 44.
`
`(APPL-1002), ’823 App, 52-53.
`
`39. On August 10, 2016, the Applicant filed an Amendment to cancel
`
`claims 52 and 61. (APPL-1002), ’823 App, 36. Regarding claim 44, the Applicant
`
`argued that Dagher “does not teach an output image being related to a zoom factor
`
`
`
`- 15 -
`
`APPL-1004 / Page 17 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`and a point of view” as recited in element (c) because Dagher’s output image “is
`
`always based on the Wide camera point of view.” (APPL-1002), ’823 App, 39
`
`(emphasis original).
`
`40. On November 10, 2016, the Applicant had a telephonic interview with
`
`the Examiner. (APPL-1002), ’823 App, 17. At the telephonic interview, the
`
`Applicant and Examiner discussed references Border, US 2008/0218612 and
`
`Williams, US 2013/0141525. (APPL-1002), ’823 App, 17. The Applicant “agreed
`
`to make amendments to claims 1, 44 & 53 to include the relationship of the zoom
`
`factor to the first and second field of view that dictates at least the switching of the
`
`point of view, via Examiner’s Amendment in order to issue the case.” (APPL-
`
`1002), ’823 App, 17.
`
`41. On November 25, 2016, a notice of allowance issued. (APPL-1002),
`
`’823 App, 7. In the Examiner’s Amendment, claims 1, 4, 10, 13, 45-51, and 54-60
`
`were deleted. (APPL-1002), ’823 App, 12. Claims 44 and 53, which were
`
`ultimately issued as claims 1 and 3, were amended. (APPL-1002), ’823 App, 12-
`
`14. Specifically, regarding claim 44, elements (a) and (b) were amended to change
`
`“first camera subset” and “second element subset” to “first camera” and “second
`
`camera,” element (b) was amended to recite “the second image having an overlap
`
`area with the first image,” and element (c) was amended to recite “the first image
`
`being a primary image and the second image being a non-primary image, wherein
`
`
`
`- 16 -
`
`APPL-1004 / Page 18 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`if FOV2 < FOVZF <FOV1 then the point of view of the output image is that of the
`
`first camera, the processor further configured to register the overlap area of the
`
`second image as non-primary image to the first image as primary image to obtain
`
`the output image.” (APPL-1002), ’823 App, 12. New claims 62 and 63, which
`
`ultimately issued as claims 2 and 4 respectively, were added. (APPL-1002), ’823
`
`App, 14.
`
`42.
`
`The Examiner’s statement of reasons for allowance provided, “The
`
`prior art teaches switching the point of view from different cameras or sensors
`
`based on an adjusted zoom level and also fusing images. It does not explicitly
`
`teach or suggest a relationship of the zoom factor to a first and second FOV that
`
`dictates which corresponding image is used as the primary image when the images
`
`are fused, in conjunction with other elements.” (APPL-1002), ’823 App, 15.
`
`43.
`
`The ’512 Patent issued on January 3, 2017. Claims 44, 53, 62, and 63
`
`were issued as claims 1, 3, 2, and 4 respectively.
`
`VI. CLAIM CONSTRUCTION
`
`44.
`
`It is my understanding that in order to properly evaluate the ’152
`
`Patent, the terms of the claims must first be interpreted. It is my understanding that
`
`for the purposes of this inter partes review, the claim terms are given their ordinary
`
`and accustomed meaning as would be understood by one of ordinary skill in the
`
`art, unless the inventor has set forth a special meaning for a term. In order to
`
`
`
`- 17 -
`
`APPL-1004 / Page 19 of 76
`
`

`

`Declaration of Oliver Cossairt, Ph.D.
`
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`construe the following claim terms, I have reviewed the entirety of the ’152 Patent,
`
`as well as its prosecution history.
`
`A. “standard color filter array (CFA)” (claims 1 and 3)
`
`45.
`
`It is my opinion that, in the context of the ’512 Patent, a POSITA would
`
`have understood that the ’152 Patent defines “standard color filter array (CFA)” to
`
`mean “a color filter array including a RGB (Bayer) pattern, RGBE, CYYM, CYGM,
`
`RGBW#l, RGBW#2, or RGBW#3.”
`
`46.
`
`The ’152 Patent provides, “A ‘standard CFA’ may include a RGB
`
`(Bayer) pattern or a non-Bayer pattern such as RGBE, CYYM, CYGM, RGBW#l,
`
`RGBW#2 or RGBW#3. Thus, reference may be made to ‘standard Bayer or
`
`‘standard non-Bayer’ patterns or filters.” (APPL-1001), ’152 Patent, 2:43-47. As
`
`contrasted with “standard CFA,” the ’153 Patent provides, “[a]s used herein, ‘non-
`
`standard CFA’ refers to a CFA that is different in its pattern that [sic] CFAs listed
`
`above as ‘standard’.” (APPL-1001), ’152 Patent, 2:48-49. As such, the ’152 Patent
`
`expressly defines the term “standard CFA” to mean a color filter array including “a
`
`RGB (Bayer) pattern or a non-Bayer pattern such as RGBE, CYYM, CYGM,
`
`RGBW#l, RGBW#2 or RGBW#3.”
`
`47.
`
`It is therefore my opinion that a POSITA would have understood that
`
`“standard color filter array (CFA)” means “a color filter array including a RGB
`
`(Bayer) pattern, RGBE, CYYM, CYGM, RGBW#l, RGBW#2, or RGBW#3.”
`
`
`
`- 18 -
`
`APPL-1004 / Page 20 of 76
`
`

`

`
`
`VII. GROUNDS
`
`Declaration of Oliver Cossairt, Ph.D.
`
` Inter Partes Review of U.S. Patent 9,538,152
`
`A. Ground 1: Claims 1-4 are unpatentable under 35 U.S.C. § 103(a)
`over Border in view of Parulski
`
`1.
`
`Summary of Border
`
`48. U.S. Patent Application Publication No. 2008/0030592 to Border et al.
`
`(“Border”) was filed on August 1, 2006. The face of Border lists John N. Border of
`
`Walworth, NY as an inventor. Further, the face of Border lists Eastman Kodak
`
`Company of Rochester NY as the assignee. Border is titled “Producing Digital Image
`
`with Different Resolution Portions,” and discloses “a digital camera that uses multiple
`
`lenses and image sensors to provide an extended zoom range and the method used to
`
`produce a digital image that combines the multiple images produced by the digital
`
`camera.” (APPL-1006), Border, Title, Abstract, [0002].
`
`49.
`
`Border recognizes that “[s]mall camera size and a large zoom range are
`
`two very important features of digital cameras.” (APPL-1006), Border, [0004].
`
`Border explains that the zoom range may include “optical zoom which is provided by
`
`variable focal length lenses and digital zoom which is provided by a magnification of
`
`the digital image after c