`
`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`APPLE INC.,
`
`Petitioner
`
`v.
`
`COREPHOTONICS, LTD.
`
`Patent Owner
`
`_______________
`
`IPR2020-00905
`U.S. Patent No. 10,225,479
`_______________
`
`DECLARATION OF FREDO DURAND, PH.D.
`UNDER 37 C.F.R. § 1.68 IN SUPPORT OF PETITION
`FOR INTER PARTES REVIEW
`
`
`
`
`
`
`
`
`
`
`
`Apple v. Corephotonics
`
`- 1 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`TABLE OF CONTENTS
`
`V.
`
`INTRODUCTION ........................................................................................... 4
`I.
`QUALIFICATIONS ........................................................................................ 6
`II.
`III. LEVEL OF ORDINARY SKILL IN THE ART ............................................. 9
`IV. RELEVANT LEGAL STANDARDS ........................................................... 11
`A. Anticipation .........................................................................................11
`B.
`Obviousness .........................................................................................11
`THE ’479 PATENT ....................................................................................... 13
`A.
`Summary of the ’479 Patent ................................................................13
`B.
`Prosecution History of the ’479 Patent ...............................................17
`VI. CLAIM CONSTRUCTION .......................................................................... 18
`A.
`“fused image with a point of view (POV) of the Wide camera” (claims
`1 and 23). .............................................................................................18
`VII. GROUNDS .................................................................................................... 20
`A. Ground 1: Claims 1, 10-14, 16, 18, 23, 32-36, 38, and 40 are
`unpatentable over Parulski in view of Konno. ....................................20
`1.
`Summary of Parulski .................................................................20
`2.
`Summary of Konno ...................................................................27
`3.
`Reasons to Combine Parulski and Konno.................................32
`4.
`Detailed Analysis ......................................................................35
`Ground 2: Claims 2-4 and 24-26 are obvious over Parulski and
`Konno, further in view of Szeliski. .....................................................66
`1.
`Summary of Szeliski .................................................................66
`2.
`Reasons to Combine Parulski, Konno, and Szeliski .................66
`3.
`Detailed Analysis ......................................................................68
`Ground 3: Claims 5-9 and 27-31 are obvious over Parulski, Konno,
`and Szeliski, further in view of Segall. ...............................................77
`1.
`Summary of Segall ....................................................................77
`
`B.
`
`C.
`
`Apple v. Corephotonics
`
`- 2 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`Reasons to Combine Parulski, Konno, Szeliski, and Segall. ....78
`2.
`Detailed Analysis ......................................................................80
`3.
`D. Ground 4: Claims 15 and 37 are obvious over Parulski and Konno,
`further in view of Stein. .......................................................................88
`1.
`Summary of Stein......................................................................88
`2.
`Reasons to Combine Parulski, Konno, and Stein. ....................91
`3.
`Detailed Analysis ......................................................................94
`VIII. DECLARATION ........................................................................................... 98
`
`
`
`
`
`
`Apple v. Corephotonics
`
`- 3 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`I.
`
`INTRODUCTION
`1.
`I, Fredo Durand, have been retained by counsel for Apple Inc.
`
`(“Apple” or “Petitioner”) as a technical expert in connection with the proceeding
`
`identified above. I submit this declaration in support of Apple’s Petition for Inter
`
`Partes Review of U.S. Patent No. 10,225,479 (“the ’479 Patent”).
`
`2.
`
`Compensation for my work in this matter is based on an hourly rate.
`
`In addition, reasonable and customary expenses associated with my work and
`
`testimony in this matter are reimbursed. This compensation is not contingent on the
`
`outcome of this matter, nor is it contingent on the specifics of my testimony. I have
`
`no personal or financial stake, nor any interest in the outcome of the present
`
`proceeding.
`
`3.
`
`In the preparation of this declaration, I have studied:
`
`(1) APPL-1001: The ’479 Patent;
`
`(2) APPL-1002: The prosecution history of the ’479 Patent (’242 App);
`
`(3) APPL-1005: U.S. Patent No. 7,859,588 to Parulski et al. (“Parulski”);
`
`(4) APPL-1013: Richard Szeliski, Computer Vision – Algorithms and
`
`Applications (2011) (“Szeliski”);
`
`(5) APPL-1015: JP Pub. No. 2013-106289 to Konno et al. (“Konno”),
`
`Certified English translation and Original;
`
`Apple v. Corephotonics
`
`- 4 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`(6) APPL-1018: U.S. Patent No. 7,206,136 to Labaziewicz et al.
`
`(“Labaziewicz”);
`
`(7) APPL-1020: Warren J. Smith, MODERN LENS DESIGN (1992)
`
`(“Smith”);
`
`(8) APPL-1021: Declaration of Dr. Jose Sasián, Ph.D.;
`
`(9) APPL-1023: U.S. Patent No. 8,908,041 to Stein et al. (“Stein”);
`
`(10) APPL-1024: U.S. Patent No. 8,406,569 to Segall et al. (“Segall”);
`
`(11) APPL-1025: U.S. Patent No. 8,824,833 to Dagher et al. (“Dagher”);
`
`(12) APPL-1027: File History for Provisional No. 61/752,515 to Stein
`
`(“Stein provisional”);
`
`(13) APPL-1031: Product announcement for Sony ICX612 12 MP image
`
`sensor;
`
`(14) APPL-1032: Product announcement for Sony ICX652 13.5 MP image
`
`sensor;
`
`(15) APPL-1034: U.S. Patent No. 7,112,774 to Baer.
`
`4.
`
`In forming the opinions expressed below, I have considered:
`
`(1) The documents listed above;
`
`(2) Any additional documents discussed below; and
`
`Apple v. Corephotonics
`
`- 5 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`(3) My own knowledge and experience based upon my work in the fields
`
`of imaging systems as described below.
`
`II. QUALIFICATIONS
`1. My qualifications and professional experience are described in my
`
`Curriculum Vitae, a copy of which can be found in exhibit APPL-1004. The
`
`following is a brief summary of my relevant qualifications and professional
`
`experience.
`
`2.
`
`I earned my Bachelor’s degree in Math and Computer Science from
`
`École Normale Superieure of Paris, France in 1993, Master of Science degree in
`
`Computer Science from Grenoble Institute of Technology, Grenoble, France in
`
`1994, and Ph.D. degree in Computer Science from Joseph Fourier University,
`
`Grenoble, France in 1999. My doctoral thesis focused on 3D visibility and lighting
`
`simulation.
`
`3.
`
`For more than 25 years, I have been developing professional and
`
`academic experience in the field of imaging systems, including integration of
`
`optics, sensors, and digital processing in imaging systems. My research interests
`
`span most aspects of picture generation and creation, and one of the major themes
`
`of my research has been directed to computational photography, which combines
`
`expertise in optics design and image processing.
`
`Apple v. Corephotonics
`
`- 6 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`I am a tenured full Professor in the Electrical Engineering and
`
`
`
`4.
`
`Computer Science Department of the Massachusetts Institute of Technology, and a
`
`member of the Computer Science and Artificial Intelligence Laboratory.
`
`5.
`
`As a professor, I teach in the area of computational photography. In
`
`the courses of Computational Photography, I teach principles of computational
`
`photography through a series of hands on projects, including applications of
`
`computational photography in high-dynamic range photography, photomontage,
`
`panoramas, image resampling, foreground extraction, Bayer sensor demosaicing,
`
`optical aberration correction, background defocusing, and morphing.
`
`6.
`
`I have authored and co-authored over two hundred journal
`
`publications, conference proceedings, technical papers, and technical presentations
`
`in the area of imaging system technologies, including optics design, image
`
`processing, and computational photography.
`
`7.
`
`In 2004, I received an inaugural Eurographics Young Researcher
`
`Award. I received a National Science Foundation (NSF) Faculty Early Career
`
`Development (CAREER) award in 2005. The NSF CAREER award is to support
`
`my research project “Transient Signal Processing for Realistic Imagery,” which is
`
`NSF’s most prestigious award in support of early-career faculty who has the
`
`potential to serve as academic role models in research and education and to lead
`
`Apple v. Corephotonics
`
`- 7 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`advances in the mission of their department or organization. The goal of this
`
`project is to characterize light transport from a signal-processing perspective with
`
`applications to image synthesis and material-appearance acquisition. I received an
`
`inaugural Microsoft Research New Faculty Fellowship in 2005, a Sloan fellowship
`
`in 2006, a Spira award for distinguished teaching in 2007. I received Association
`
`for Computing Machinery's (ACM’s) Special Interest Group on Computer
`
`Graphics and Interactive Techniques (SIGGRAPH) Computer Graphics
`
`Achievement Award in 2016, which is given by the organization each year to
`
`recognize an individual for an outstanding achievement in computer graphics and
`
`interactive techniques. I became an ACM fellow in 2016, which is ACM’s most
`
`prestigious member grade that recognizes the top 1% of ACM members for their
`
`outstanding accomplishments in computing and information technology and/or
`
`outstanding service to ACM and the larger computing community.
`
`8. My involvement in the research community extends to several
`
`organizations, journals, and conferences. Over the years, I have organized and
`
`served in the Program Committee of a variety of conferences, including IEEE
`
`International Conference on Computational Photography, Symposium on
`
`Computational Photography and Video, ACM SIGGRAPH, Eurographics
`
`Symposium on Rendering (EGSR), Graphics Interface, Eurographics, Non-
`
`Apple v. Corephotonics
`
`- 8 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`Photorealistic Animation and Rendering (NPAR), Symposium on Point-Based
`
`Rendering, ACM Transactions on Graphics, Foundations and Trends in Computer
`
`Graphics and Computer Vision. I was a Member of the advisory board of Image
`
`and Meaning 2, an interdisciplinary conference on scientific illustration and
`
`education.
`
`9.
`
`A list of my publications and patents is contained in my CV at exhibit
`
`APPL-1004.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`10.
`I understand that the level of ordinary skill may be reflected by the
`
`prior art of record, and that a Person of Ordinary Skill in The Art (“POSITA”) to
`
`which the claimed subject matter pertains would have had the capability of
`
`understanding the scientific and engineering principles applicable to the pertinent
`
`art. I understand that a POSITA has ordinary creativity and is not an automaton.
`
`11.
`
`I understand that there are multiple factors relevant to determining the
`
`level of ordinary skill in the pertinent art, including (1) the levels of education and
`
`experience of persons working in the field at the time of the invention; (2) the
`
`sophistication of the technology; (3) the types of problems encountered in the field;
`
`and (4) the prior art solutions to those problems.
`
`12.
`
`I am familiar with the imaging system art pertinent to the ’479 Patent.
`
`Apple v. Corephotonics
`
`- 9 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`I am also aware of the state of the art at the time the application resulting in the
`
`’479 Patent was filed. I have been informed by counsel that the earliest claimed
`
`priority date for the ’479 Patent is June 13, 2013, although any given claim of the
`
`’479 Patent may or may not be entitled to the earliest claimed date.
`
`13. Based on the technologies disclosed in the ’479 Patent, I believe that a
`
`POSITA would include someone who had, as of the claimed priority date of the
`
`’479 Patent, a bachelor’s or the equivalent degree in electrical and/or computer
`
`engineering or a related field and 2-3 years of experience in imaging systems
`
`including image processing and lens design. In addition, I recognize that someone
`
`with less formal education but more experience, or more formal education but less
`
`experience could have also met the relevant standard for a POSITA. For the
`
`knowledge of a POSITA regarding lens design aspects of the ’479 Patent, I rely on
`
`Dr. Sasián’s opinions set forth in his declaration at APPL-1021. Accordingly,
`
`based on Dr. Sasián’s opinion and my own education and experience, I believe that
`
`I am qualified to opine from the perspective of a POSITA regarding the ’479
`
`Patent and the claims discussed below.
`
`14. For purposes of this Declaration, unless otherwise noted, my opinions
`
`and statements, such as those regarding the understanding of a POSITA (and
`
`specifically related to the references I consulted herein), reflect the knowledge that
`
`Apple v. Corephotonics
`
`- 10 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`existed in the art before the earliest claimed priority date of the ’479 Patent.
`
`IV. RELEVANT LEGAL STANDARDS
`15.
`I have been asked to provide my opinions regarding whether claims 1-
`
`16, 18, 23-38, and 40 (the “Challenged Claims”) of the ’479 Patent would have
`
`been obvious to a POSITA at the time of the alleged invention in light of the prior
`
`art.
`
`16.
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’479 Patent, I am relying on certain legal
`
`principles explained to me by counsel.
`
`17.
`
`I understand that a claim is unpatentable if it is anticipated under 35
`
`U.S.C. § 102 or obvious under 35 U.S.C. § 103.
`
`A. Anticipation
`18.
`
`I have been informed by counsel that a patent claim is unpatentable as
`
`anticipated if each element of that claim is present either explicitly or inherently in
`
`a single prior art reference. I have also been informed that, to be an inherent
`
`disclosure, the prior art reference must necessarily disclose the limitation, and the
`
`fact that the reference might possibly practice or contain a claimed limitation is
`
`insufficient to establish that the reference inherently teaches the limitation.
`
`B. Obviousness
`19.
`
`I have been informed and I understand that a claimed invention is
`
`Apple v. Corephotonics
`
`- 11 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`unpatentable under 35 U.S.C. § 103(a) if the differences between the subject matter
`
`sought to be patented and the prior art are such that the subject matter as a whole
`
`would have been obvious to a POSITA at the time the invention was made. I
`
`understand that the appropriate analysis for determining obviousness of a claimed
`
`invention takes into account factual inquiries, including the level of ordinary skill
`
`in the art, the scope and content of the prior art, and the differences between the
`
`prior art and the claimed subject matter as a whole.
`
`20.
`
`I have been informed and I understand that the United States Supreme
`
`Court has recognized several rationales for combining references or modifying a
`
`reference to show obviousness of claimed subject matter. Some of these rationales
`
`include the following: (a) combining prior art elements according to known
`
`methods to yield predictable results; (b) simple substitution of one known element
`
`for another to obtain predictable results; (c) use of a known technique to improve a
`
`similar device (method, or product) in the same way; (d) applying a known
`
`technique to a known device (method, or product) ready for improvement to yield
`
`predictable results; (e) choosing from a finite number of identified, predictable
`
`solutions, with a reasonable expectation of success; and (f) some teaching,
`
`suggestion, or motivation in the prior art that would have led a POSITA to modify
`
`the prior art reference or to combine prior art reference teachings to arrive at the
`
`Apple v. Corephotonics
`
`- 12 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`claimed invention. I have also been informed and I understand that a demonstration
`
`of obviousness does not require a physical combination or bodily incorporation,
`
`but rather may be found based on consideration of what the combined teachings
`
`would have suggested to a POSITA at the time of the alleged invention.
`
`V. THE ’479 PATENT
`Summary of the ’479 Patent
`A.
`21. The ’479 Patent is titled “Dual Aperture Zoom Digital Camera” and
`
`directed to a “dual-aperture zoom digital camera operable in both still and video
`
`modes.” (APPL-1001), ’479 Patent, Abstract. FIG. 1A of the ’479 Patent below
`
`illustrates a dual-aperture Zoom imaging system 100 including a first Wide
`
`imaging section and a second Tele imaging section, each having a respective lens
`
`and image sensor. See APPL-1001, 6:21-29.
`
`Apple v. Corephotonics
`
`- 13 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`
`
`APPL-1001, Fig. 1A.
`
`22. FIG. 2 of the ’479 Patent below illustrates Wide and Tele sensors and
`
`their respective FOVs. A larger FOV for the Wide image is provided by the Wide
`
`sensor 202 and a smaller FOV for the corresponding Tele image is provided by the
`
`Tele sensor 204. See APPL-1001, 6:1-2.
`
`Apple v. Corephotonics
`
`- 14 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`
`
`APPL-1001, Fig. 2.
`
`23. With Wide and Tele images captured from the respective cameras, the
`
`‘479 Patent describes several processing methods that can be achieved. In the
`
`method that forms the subject of the challenged claims, the image processing first
`
`rectifies the Wide and Tele images to be aligned on an epipolar line. See APPL-
`
`1001, 9:46-47. Next, the process performs “mapping between the Wide and the
`
`Tele aligned images” to “produce a registration map.” APPL-1001, 9:48-49. The
`
`Tele image is then “resampled according to the registration map” or in other
`
`words, resized to correspond to the field of view (“FOV”) of the Wide image. See
`
`APPL-1001, 9:50-60. The process finally then fuses or combines portions of the
`
`Apple v. Corephotonics
`
`- 15 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`resampled Tele image with corresponding portions of the Wide image to produce
`
`an output image. See APPL-1001, 9:52-67. As part of this fusion step, any errors
`
`between the images are detected and if an error exists, “Wide pixel values are
`
`chosen to be used in the output image.” APPL-1001, 9:54-60.
`
`24. Representative independent claim 1 of the ’479 Patent is below:
`
`1. A dual-aperture digital camera for imaging an
`object or scene, comprising:
`
`a) a Wide camera comprising a Wide lens and a
`Wide image sensor, the Wide camera having a respective
`field of view FOVW and being operative to provide a Wide
`image of the object or scene;
`
`b) a Tele camera comprising a Tele lens and a Tele
`image sensor, the Tele camera having a respective field of
`view FOVT narrower than FOVW and being operative to
`provide a Tele image of the object or scene, wherein the
`Tele lens has a respective effective focal length EFLT and
`total track length TTLT fulfilling the condition EFLT /
`TTLT > 1;
`
`c) a first autofocus (AF) mechanism coupled
`mechanically to, and used to perform an AF action on the
`Wide lens;
`
`d) a second AF mechanism coupled mechanically
`to, and used to perform an AF action on the Tele lens; and
`
`Apple v. Corephotonics
`
`- 16 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`e) a camera controller operatively coupled to the
`first and second AF mechanisms and to the Wide and Tele
`image sensors and configured
`to control
`the AF
`mechanisms and to process the Wide and Tele images to
`create a fused image, wherein areas in the Tele image that
`are not focused are not combined with the Wide image to
`create the fused image and wherein the camera controller
`is further operative to output the fused image with a point
`of view (POV) of the Wide camera by mapping Tele
`image pixels to matching pixels within the Wide image.
`
`APPL-1001, 13:21-50.
`
`25. As I further discuss below, the system and method presented in the
`
`’479 Patent, namely, a dual-aperture camera system having 1) Wide and Tele lens
`
`systems with overlapping fields of view, and 2) a camera controller that processes
`
`Wide and Tele images to fuse focused portions of the Tele image with the Wide
`
`image, were well known in the prior art prior to the ’479 Patent.
`
`Prosecution History of the ’479 Patent
`
`B.
`26. On July 28, 2018, the Applicant filed U.S. Patent Application No.
`
`16/048,242 (“the ’242 App”) including claims 1-40, which ultimately issued as the
`
`’479 Patent. APPL-1002, 334-66. The ’242 application claimed priority, through a
`
`string of continuations, to Provisional App. 61/834,486 filed on June 13, 2013.
`
`APPL-1001, 1:5-20.
`
`Apple v. Corephotonics
`
`- 17 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`27. On January 9, 2019, a Notice of Allowance is issued. In the
`
`Allowance, the Examiner set forth the claim limitations that were found to be
`
`patentable over the prior art including “the Tele lens has a respective effective
`
`focal length EFLT and total track length TTLT fulfilling the condition EFLT / TTLT
`
`> 1” and several other limitations set forth in claims 1, 19, and 23. The ’479 Patent
`
`issued on March 5, 2019.
`
`VI. CLAIM CONSTRUCTION
`28.
`It is my understanding that in order to properly evaluate the ’479
`
`Patent, the terms of the claims must first be interpreted. It is my understanding that
`
`for the purposes of this inter partes review, the claim terms are given their ordinary
`
`and accustomed meaning as would be understood by one of ordinary skill in the
`
`art, unless the inventor has set forth a special meaning for a term. In order to
`
`construe the following claim terms, I have reviewed the entirety of the ’479 Patent,
`
`as well as its prosecution history.
`
`A.
`
`“fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23).
`
`29. This term is used in claims 1 and 23 which state: “to output the fused
`
`image with a point of view (POV) of the Wide camera by mapping Tele image
`
`pixels to matching pixels within the Wide image.”
`
`30. The Summary section of the ’479 Patent explains the concept of
`
`Apple v. Corephotonics
`
`- 18 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`“POV of the Wide camera” in this way: “In a dual-aperture camera image plane, as
`
`seen by each sub-camera (and respective image sensor), a given object will be
`
`shifted and have different perspective (shape). This is referred to as point-of-
`
`view (POV).” APPL-1001, 5:10-12.
`
`31. Due to Wide and Tele cameras having different perspectives, the ’479
`
`Patent indicates that the “output image can have the shape and position of either
`
`sub-camera image or the shape or position of a combination thereof.” APPL-
`
`1001, 5:12-15. “If the output image retains the Wide image shape then it has the
`
`Wide perspective POV. If it retains the Wide camera position, then it has the
`
`Wide position POV.” APPL-1001, 5:15-19. The same applies to the images from
`
`the Tele camera. APPL-1001, 5:19-20.
`
`32. Based on this description, it is my opinion that a POSITA would
`
`interpret “a point of view of the Wide camera” in the ’479 Patent to mean one of
`
`two things—either “Wide perspective POV” (i.e., wide camera FOV) or “Wide
`
`position POV” (i.e., wide camera FOV and position). When discussing the fusion
`
`step, the ’479 Patent does not dictate one type of POV or the other: “it is possible
`
`to register Tele image pixels to a matching pixel set within the Wide image pixels,
`
`in which case the output image will retain the Wide POV (“Wide fusion”).” APPL-
`
`1001, 5:23-26. Because the specification describes Wide POV in two ways and
`
`Apple v. Corephotonics
`
`- 19 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`does not specify which type is used by the image fusion step, a POSITA would
`
`have understood that a fused image that maintains a Wide POV either fuses images
`
`to maintain just the Wide field of view or fuses images to maintain both the Wide
`
`field of view and the Wide camera’s position.
`
`33.
`
`It is therefore my opinion that a POSITA would have understood a
`
`“fused image with a point of view (POV) of the Wide camera” to mean “a fused
`
`image that maintains the Wide camera’s field of view or both the Wide camera’s
`
`field of view and position.”
`
`VII. GROUNDS
`A. Ground 1: Claims 1, 10-14, 16, 18, 23, 32-36, 38, and 40 are
`unpatentable over Parulski in view of Konno.
`
`Summary of Parulski
`1.
`34. U.S. Patent No. 7,859,588 to Parulski, et al. (“Parulski”) was filed on
`
`March 9, 2007 and issued on December 28, 2010. See APPL-1005. The face of
`
`Parulski lists Eastman Kodak Company of Rochester NY as the assignee. Id. Parulski
`
`is titled “Method and Apparatus for Operating a Dual Lens Camera to Augment an
`
`Image,” and discloses “a digital camera that uses multiple lenses and image sensors to
`
`provide an improved imaging capability.” Id., 1:8-10. In Parulski, “digital zooming
`
`between the wide angle and the telephoto focal lengths” is used to provide an
`
`extended zoom range. Id., 23:54-58.
`
`Apple v. Corephotonics
`
`- 20 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`35. Parulski teaches that its dual lens image capture assembly may operate in
`
`still and video modes to produce “still images and motion video images.” Id., 12:36-
`
`41; see also Id., 14:5-9 (“The digital data ... is ... processed by the image processor 50
`
`to produce a processed digital image file, which may contain a still digital image or a
`
`video image.”); Id., 29:8-11 (“the images captured by the primary and secondary
`
`capture units could be a still image or a video image, and in the case of a video image
`
`could be a series of images.”).
`
`36. Parulski describes that its “digital camera with dual capture systems ...
`
`utiliz[es] both images to provide an improved output image.” Id., 7:21-24. Parulski
`
`describes an image augmentation process that “utilizes one of the images from a dual-
`
`lens camera as a secondary image that can be used to modify the other, primary image
`
`and thereby generate an enhanced primary image.” Id., 7:32-35. As shown relative to
`
`annotated FIG. 16B below, Parulski goes on to describe its technique using images
`
`from fixed focal length wide-angle and telephoto lenses and image sensors. Id., 23:28-
`
`43. A composite image may be formed “by combining a portion of the first wide
`
`angle digital image and a portion of the telephoto digital image, to produce a digital
`
`image with improved resolution during digital zooming.” Id., 29:51-64. Parulski
`
`describes that its image augmentation process may be applied to “a still image or a
`
`video image.” Id., 29:8-20.
`
`Apple v. Corephotonics
`
`- 21 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`Parulski, FIG. 16B, annotated
`
`
`
`37. Parulski teaches generating an enhanced primary image including
`
`capturing “a first (i.e., primary) still image [captured] at a first (i.e., primary) focus
`
`distance” and “simultaneously capture a second (i.e., secondary) still image at a
`
`second (i.e., secondary) focus distance,” using the “secondary output image for
`
`modifying the primary output image, thereby generating an enhanced primary image
`
`signal.” Id., 12:6-13. Parulski describes that “the secondary still image is used to
`
`provide an enhancement signal that may, e.g., sharpen portions of the primary still
`
`image that are positioned near the secondary focus distance.” Id., 12:16-20.
`
`38. Parulski describes that its image augmentation process includes digital
`
`zoom based on a requested zoom position from a user and determines the primary
`
`Apple v. Corephotonics
`
`- 22 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`image and secondary image from two capture units of the digital camera based on the
`
`user requested zoom position. Id., FIG. 23, 27:8-24, 29:51-64. If the requested zoom
`
`position is less than a zoom switch value X, the first image station with a wide lens is
`
`set as primary capture unit for providing a primary image, and the second image
`
`station with a tele lens is set as a secondary capture unit for providing a secondary
`
`image. Id., Fig. 23, Fig. 14, 22:18-21 (“In block 502, the zoom position setting is
`
`compared to a value X at which the image capture function switches” between
`
`first/second image capture units); Fig. 3, 15:54-61 (“In block 102, the zoom position
`
`setting is compared to a value X at which the image capture function switches from
`
`the first image capture stage to the second image capture stage”). In block 104, if the
`
`zoom position setting is less than X (a negative response to block 102), then the first
`
`image capture stage 1 is used [as the primary image capture stage]”); Id., 27:8-15 (if
`
`the requested zoom position is not within the zoom range of the current primary
`
`capture unit for providing a primary image, “the functions of the capture units are
`
`reversed,” where the current scene analysis unit for providing a secondary image and
`
`the current primary capture unit for providing the primary image are “reset to be the
`
`primary capture unit and scene analysis capture unit, respectively.”).
`
`39. Parulski also teaches several image enhancement processes that can be
`
`performed on the primary and secondary images. First, as shown in Fig 11, Parulski
`
`Apple v. Corephotonics
`
`- 23 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`
`
`teaches that a range map (i.e., depth map) can be created using the two images by
`
`correlating the secondary image with a “cropped 10 and upsampled [primary] image
`
`to determine the pixel offset between the images for different portions of the
`
`images.” Determining the pixel offsets would be understood by a POSITA to create a
`
`disparity or registration map. See id., 19:55-58 (“a method for producing a rangemap
`
`or depth map from a disparity map produced from the pixel offset information for a
`
`set of images captured by multiple cameras with similar fields of view”). The pixel
`
`offsets are then “converted in block 482 to distances from the image capture
`
`device us