`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`APPLE INC.,
`
`Petitioner
`
`v.
`
`COREPHOTONICS, LTD.
`
`Patent Owner
`
`_______________
`
`IPR2020-00906
`U.S. Patent No. 10,225,479
`_______________
`
`DECLARATION OF FREDO DURAND, PH.D.
`UNDER 37 C.F.R. § 1.68 IN SUPPORT OF PETITION
`FOR INTER PARTES REVIEW
`
`
`
`
`
`
`
`
`
`Apple v. Corephotonics
`
`- 1 -
`
`APPL-1003
`
`

`

`
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`TABLE OF CONTENTS
`
`V.
`
`I.
`INTRODUCTION ........................................................................................... 4
`QUALIFICATIONS ........................................................................................ 6
`II.
`III. LEVEL OF ORDINARY SKILL IN THE ART ............................................. 9
`IV. RELEVANT LEGAL STANDARDS ........................................................... 11
`A. Anticipation .........................................................................................12
`B.
`Obviousness .........................................................................................12
`THE ’479 PATENT ....................................................................................... 13
`A.
`Summary of the ’479 Patent ................................................................13
`B.
`Prosecution History of the ’479 Patent ...............................................18
`VI. CLAIM CONSTRUCTION .......................................................................... 19
`A.
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited for
`portrait photos” (claim 19) ..................................................................19
`VII. GROUNDS .................................................................................................... 22
`A.
`Claims 19 and 20 are obvious over Parulski in view of Ogata, further
`in view of Kawamura, further in view of Soga. ..................................22
`1.
`Summary of Parulski .................................................................22
`2.
`Summary of Kawamura ............................................................26
`3.
`Reasons to combine Parulski and Kawamura ...........................31
`4.
`Summary of Ogata ....................................................................34
`5.
`Reasons to combine Parulski and Ogata ...................................38
`6.
`Summary of Soga ......................................................................40
`7.
`Reasons to combine Parulski and Soga ....................................43
`8.
`Detailed Analysis ......................................................................48
`Claims 21 and 22 are obvious over the combination of Parulski,
`Ogata, Kawamura, Soga, and Morgan-Mar. .......................................75
`1.
`Summary of Morgan-Mar .........................................................75
`2.
`Reasons to Combine Parulski, Soga, and Morgan-Mar ............78
`
`B.
`
`Apple v. Corephotonics
`
`- 2 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`VIII. DECLARATION ........................................................................................... 85
`
`
`
`
`
`
`Apple v. Corephotonics
`
`- 3 -
`
`APPL-1003
`
`

`

`I.
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`
`INTRODUCTION
`1.
`I, Fredo Durand, have been retained by counsel for Apple Inc.
`
`(“Apple” or “Petitioner”) as a technical expert in connection with the proceeding
`
`identified above. I submit this declaration in support of Apple’s Petition for Inter
`
`Partes Review of U.S. Patent No. 10,225,479 (“the ’479 Patent”).
`
`2.
`
`Compensation for my work in this matter is based on an hourly rate.
`
`In addition, reasonable and customary expenses associated with my work and
`
`testimony in this matter are reimbursed. This compensation is not contingent on the
`
`outcome of this matter, nor is it contingent on the specifics of my testimony. I have
`
`no personal or financial stake, nor any interest in the outcome of the present
`
`proceeding.
`
`3.
`
`In the preparation of this declaration, I have studied:
`
`(1) APPL-1001: The ’479 Patent;
`
`(2) APPL-1002: The prosecution history of the ’479 Patent (’242 App.);
`
`(3) APPL-1005: U.S. Patent No. 7,859,588 to Parulski et al. (“Parulski”);
`
`(4) APPL-1006: JP Patent App. Pub. No. 2007-259108 to Soga and
`
`certified English translation (“Soga”);
`
`(5) APPL-1007: Jacobs et al., “Focal Stack Compositing for Depth of
`
`Field Control,” Stanford Computer Graphics Laboratory Technical
`
`Report 2012-1 (“Jacobs”);
`
`Apple v. Corephotonics
`
`- 4 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`(6) APPL-1009: U.S. Patent No. 8,989,517 to Morgan-Mar et al.
`
`(“Morgan-Mar”);
`
`(7) APPL-1010: PCT Publication No. WO2013140359 to Shalon et al.
`
`(“Shalon”);
`
`(8) APPL-1011: U.S. Patent Application Publication No. 2008/0030592
`
`to Border et al. (“Border”);
`
`(9) APPL-1012: JPS5862609A to Kawamura, (“Kawamura”);
`
`(10) APPL-1014: U.S. Patent No. 6,259,863 to Maruyama, (“Maruyama”);
`
`(11) APPL-1016: Ralph E. Jacobson et al., The Manual of Photography –
`
`Photographic and Digital Imaging (9th ed. 2000);
`
`(12) APPL-1017: U.S. Patent App. Pub. No. 2010/0321511 to Koskinen et
`
`al. (“Koskinen”);
`
`(13) APPL-1018: U.S. Patent No. 7,206,136 to Labaziewicz et al.
`
`(“Labaziewicz”);
`
`(14) APPL-1019: Milton Katz, INTRODUCTION TO GEOMETRICAL
`
`OPTICS (2002) (“Katz”);
`
`(15) APPL-1020: Warren J. Smith, MODERN LENS DESIGN (1992)
`
`(“Smith”);
`
`(16) APPL-1021: Declaration of Dr. Jose Sasián, Ph.D.;
`
`Apple v. Corephotonics
`
`- 5 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`(17) APPL-1022: ZEMAX Development Corporation, ZEMAX Optical
`
`Design Program User’s Manual, February 14, 2011 (“ZEMAX User’s
`
`Manual”);
`
`(18) APPL-1026: U.S. Patent No. 5,546,236 to Ogata et al. (“Ogata”);
`
`(19) APPL-1028: Bae et al, “Defocus Magnification,” EUROGRAPHICS
`
`2007, (“Bae”);
`
`(20) APPL-1029: Specification sheet for Sony ICX629 Image Sensor,
`
`(“ICX629”);
`
`(21) APPL-1030: Specification sheet for Sony ICX624 Image Sensor,
`
`(“ICX624”);
`
`(22) APPL-1033: Product manual for Kodak Easyshare V610;
`
`4.
`
`In forming the opinions expressed below, I have considered:
`
`(1) The documents listed above;
`
`(2) Any additional documents discussed below; and
`
`(3) My own knowledge and experience based upon my work in the fields
`
`of imaging systems as described below.
`
`II. QUALIFICATIONS
`1. My qualifications and professional experience are described in my
`
`Curriculum Vitae, a copy of which can be found in exhibit APPL-1004. The
`
`following is a brief summary of my relevant qualifications and professional
`
`Apple v. Corephotonics
`
`- 6 -
`
`APPL-1003
`
`

`

`
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`experience.
`
`2.
`
`I earned my Bachelor’s degree in Math and Computer Science from
`
`École Normale Superieure of Paris, France in 1993, Master of Science degree in
`
`Computer Science from Grenoble Institute of Technology, Grenoble, France in
`
`1994, and Ph.D. degree in Computer Science from Joseph Fourier University,
`
`Grenoble, France in 1999. My doctoral thesis focused on 3D visibility and lighting
`
`simulation.
`
`3.
`
`For more than 25 years, I have been developing professional and
`
`academic experience in the field of imaging systems, including integration of
`
`optics, sensors, and digital processing in imaging systems. My research interests
`
`span most aspects of picture generation and creation, and one of the major themes
`
`of my research has been directed to computational photography, which combines
`
`expertise in optics design and image processing.
`
`4.
`
`I am a tenured full Professor in the Electrical Engineering and
`
`Computer Science Department of the Massachusetts Institute of Technology, and a
`
`member of the Computer Science and Artificial Intelligence Laboratory.
`
`5.
`
`As a professor, I teach in the area of computational photography. In
`
`the courses of Computational Photography, I teach principles of computational
`
`photography through a series of hands on projects, including applications of
`
`computational photography in high-dynamic range photography, photomontage,
`
`Apple v. Corephotonics
`
`- 7 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`panoramas, image resampling, foreground extraction, Bayer sensor demosaicing,
`
`
`
`optical aberration correction, background defocusing, and morphing.
`
`6.
`
`I have authored and co-authored over two hundred journal
`
`publications, conference proceedings, technical papers, and technical presentations
`
`in the area of imaging system technologies, including optics design, image
`
`processing, and computational photography.
`
`7.
`
`In 2004, I received an inaugural Eurographics Young Researcher
`
`Award. I received a National Science Foundation (NSF) Faculty Early Career
`
`Development (CAREER) award in 2005. The NSF CAREER award is to support
`
`my research project “Transient Signal Processing for Realistic Imagery,” which is
`
`NSF’s most prestigious award in support of early-career faculty who has the
`
`potential to serve as academic role models in research and education and to lead
`
`advances in the mission of their department or organization. The goal of this
`
`project is to characterize light transport from a signal-processing perspective with
`
`applications to image synthesis and material-appearance acquisition. I received an
`
`inaugural Microsoft Research New Faculty Fellowship in 2005, a Sloan fellowship
`
`in 2006, a Spira award for distinguished teaching in 2007. I received Association
`
`for Computing Machinery's (ACM’s) Special Interest Group on Computer
`
`Graphics and Interactive Techniques (SIGGRAPH) Computer Graphics
`
`Achievement Award in 2016, which is given by the organization each year to
`
`Apple v. Corephotonics
`
`- 8 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`recognize an individual for an outstanding achievement in computer graphics and
`
`
`
`interactive techniques. I became an ACM fellow in 2016, which is ACM’s most
`
`prestigious member grade that recognizes the top 1% of ACM members for their
`
`outstanding accomplishments in computing and information technology and/or
`
`outstanding service to ACM and the larger computing community.
`
`8. My involvement in the research community extends to several
`
`organizations, journals, and conferences. Over the years, I have organized and
`
`served in the Program Committee of a variety of conferences, including IEEE
`
`International Conference on Computational Photography, Symposium on
`
`Computational Photography and Video, ACM SIGGRAPH, Eurographics
`
`Symposium on Rendering (EGSR), Graphics Interface, Eurographics, Non-
`
`Photorealistic Animation and Rendering (NPAR), Symposium on Point-Based
`
`Rendering, ACM Transactions on Graphics, Foundations and Trends in Computer
`
`Graphics and Computer Vision. I was a Member of the advisory board of Image
`
`and Meaning 2, an interdisciplinary conference on scientific illustration and
`
`education.
`
`9.
`
`A list of my publications and patents is contained in my CV at exhibit
`
`APPL-1004.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`10.
`I understand that the level of ordinary skill may be reflected by the
`
`Apple v. Corephotonics
`
`- 9 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`prior art of record, and that a Person of Ordinary Skill in The Art (“POSITA”) to
`
`
`
`which the claimed subject matter pertains would have had the capability of
`
`understanding the scientific and engineering principles applicable to the pertinent
`
`art. I understand that a POSITA has ordinary creativity and is not an automaton.
`
`11.
`
`I understand that there are multiple factors relevant to determining the
`
`level of ordinary skill in the pertinent art, including (1) the levels of education and
`
`experience of persons working in the field at the time of the invention; (2) the
`
`sophistication of the technology; (3) the types of problems encountered in the field;
`
`and (4) the prior art solutions to those problems.
`
`12.
`
`I am familiar with the imaging system art pertinent to the ’479 Patent.
`
`I am also aware of the state of the art at the time the application resulting in the
`
`’479 Patent was filed. I have been informed by counsel that the earliest claimed
`
`priority date for the ’479 Patent is June 13, 2013, although any given claim of the
`
`’479 Patent may or may not be entitled to the earliest claimed date.
`
`13. Based on the technologies disclosed in the ’479 Patent, I believe that a
`
`POSITA would include someone who had, as of the claimed priority date of the
`
`’479 Patent, a bachelor’s or the equivalent degree in electrical and/or computer
`
`engineering or a related field and 2-3 years of experience in imaging systems
`
`including image processing and lens design. In addition, I recognize that someone
`
`with less formal education but more experience, or more formal education but less
`
`Apple v. Corephotonics
`
`- 10 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`experience could have also met the relevant standard for a POSITA. For the
`
`
`
`knowledge of a POSITA regarding lens design aspects of the ’479 Patent, I rely on
`
`Dr. Sasián’s opinions set forth in his declaration at APPL-1021. Accordingly,
`
`based on Dr. Sasián’s opinion and my own education and experience, I believe that
`
`I am qualified to opine from the perspective of a POSITA regarding the ’479
`
`Patent and the claims discussed below.
`
`14. For purposes of this Declaration, unless otherwise noted, my opinions
`
`and statements, such as those regarding the understanding of a POSITA (and
`
`specifically related to the references I consulted herein), reflect the knowledge that
`
`existed in the art before the earliest claimed priority date of the ’479 Patent.
`
`IV. RELEVANT LEGAL STANDARDS
`15.
`I have been asked to provide my opinions regarding whether 19-22
`
`(the “Challenged Claims”) of the ’479 Patent would have been obvious to a
`
`POSITA at the time of the alleged invention in light of the prior art.
`
`16.
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’479 Patent, I am relying on certain legal
`
`principles explained to me by counsel.
`
`17.
`
`I understand that a claim is unpatentable if it is anticipated under 35
`
`U.S.C. § 102 or obvious under 35 U.S.C. § 103.
`
`Apple v. Corephotonics
`
`- 11 -
`
`APPL-1003
`
`

`

`
`
`
`A. Anticipation
`18.
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`I have been informed by counsel that a patent claim is unpatentable as
`
`anticipated if each element of that claim is present either explicitly or inherently in
`
`a single prior art reference. I have also been informed that, to be an inherent
`
`disclosure, the prior art reference must necessarily disclose the limitation, and the
`
`fact that the reference might possibly practice or contain a claimed limitation is
`
`insufficient to establish that the reference inherently teaches the limitation.
`
`B. Obviousness
`19.
`
`I have been informed and I understand that a claimed invention is
`
`unpatentable under 35 U.S.C. § 103(a) if the differences between the subject matter
`
`sought to be patented and the prior art are such that the subject matter as a whole
`
`would have been obvious to a POSITA at the time the invention was made. I
`
`understand that the appropriate analysis for determining obviousness of a claimed
`
`invention takes into account factual inquiries, including the level of ordinary skill
`
`in the art, the scope and content of the prior art, and the differences between the
`
`prior art and the claimed subject matter as a whole.
`
`20.
`
`I have been informed and I understand that the United States Supreme
`
`Court has recognized several rationales for combining references or modifying a
`
`reference to show obviousness of claimed subject matter. Some of these rationales
`
`include the following: (a) combining prior art elements according to known
`
`Apple v. Corephotonics
`
`- 12 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`methods to yield predictable results; (b) simple substitution of one known element
`
`
`
`for another to obtain predictable results; (c) use of a known technique to improve a
`
`similar device (method, or product) in the same way; (d) applying a known
`
`technique to a known device (method, or product) ready for improvement to yield
`
`predictable results; (e) choosing from a finite number of identified, predictable
`
`solutions, with a reasonable expectation of success; and (f) some teaching,
`
`suggestion, or motivation in the prior art that would have led a POSITA to modify
`
`the prior art reference or to combine prior art reference teachings to arrive at the
`
`claimed invention. I have also been informed and I understand that a demonstration
`
`of obviousness does not require a physical combination or bodily incorporation,
`
`but rather may be found based on consideration of what the combined teachings
`
`would have suggested to a POSITA at the time of the alleged invention.
`
`V. THE ’479 PATENT
`Summary of the ’479 Patent
`A.
`21. The ’479 Patent is titled “Dual Aperture Zoom Digital Camera” and
`
`directed to a “dual-aperture zoom digital camera operable in both still and video
`
`modes.” APPL-1001, ’479 Patent, Abstract. FIG. 1A of the ’479 Patent below
`
`illustrates a dual-aperture Zoom imaging system 100 including a first Wide
`
`imaging section and a second Tele imaging section, each having a respective lens
`
`and image sensor. See id., 6:21-29.
`
`Apple v. Corephotonics
`
`- 13 -
`
`APPL-1003
`
`

`

`
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`Id., Fig. 1A.
`
`22. FIG. 2 of the ’479 Patent below illustrates Wide and Tele sensors and
`
`their respective FOVs. A larger FOV for the Wide image is provided by the Wide
`
`sensor 202 and a smaller FOV for the corresponding Tele image is provided by the
`
`Tele sensor 204. See Id., 6:1-2.
`
`Apple v. Corephotonics
`
`- 14 -
`
`APPL-1003
`
`

`

`
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`
`
`Id., Fig. 2.
`
`23. With Wide and Tele images captured from the respective cameras, the
`
`‘479 Patent describes performing several image processing methods. In the method
`
`that forms the subject of the challenged claims, the image processing performs two
`
`separate functions— “find[ing] translations between matching points in the images
`
`to calculate depth information” and “creat[ing] a fused image suited for portrait
`
`photos” with a depth of field shallower than the depth of field of the Tele image.
`
`Id., 14:66-15:32.
`
`24. Regarding the process for calculating depth information, the ’479 Patent
`
`only describes this for the purposes of fast focusing of an autofocus mechanism.
`
`Id., 12:15 (“The result is fast focusing.”). This process first performs registration
`
`Apple v. Corephotonics
`
`- 15 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`“between the Wide and Tele images to output a transformation coefficient…. The
`
`
`
`transformation coefficient includes the translation between matching points in the
`
`two images” that is “measured in a number of pixels. Id., 12:7-11. The different
`
`translations between the images “result in a different number of pixel movements
`
`between matching points in the images.” Id., 12:11-12. The pixel movements are
`
`“translated into depth” and then “translated into an AF position.” Id., 12:12-15.
`
`25. Regarding the process for creating a fused image with a shallower
`
`depth of field, the ’479 Patent describes that:
`
`In some embodiments, a dual-aperture zoom system
`disclosed herein can be used to capture a shallow DOF
`photo (shallow compared with a DOF of a Wide camera
`alone), by taking advantage of the longer focal length of
`the Tele lens. The reduced DOF effect provided by the
`longer Tele focal length can be further enhanced in the
`final image by fusing data from an image captured
`simultaneously with the Wide lens. Depending on the
`distance to the object, with the Tele lens focused on a
`subject of the photo, the Wide lens can be focused to a
`closer distance than the subject so that objects behind the
`subject appear very blurry. Once the two images are
`captured, information from the out-of-focus blurred
`background in the Wide image is fused with the original
`Tele image background information, providing a blurrier
`background and even shallower DOF.
`
`Apple v. Corephotonics
`
`- 16 -
`
`APPL-1003
`
`

`

`
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`Id., 4:23-38.
`
`26. Representative independent claim 19 of the ’479 Patent is below:
`
`19. A dual-aperture digital camera for imaging an
`object or scene, comprising:
`
`a) a Wide camera comprising a Wide lens and a
`Wide image sensor, the Wide camera having a respective
`field of view FOVW and being operative to provide a Wide
`image of the object or scene;
`
`b) a Tele camera comprising a Tele lens and a Tele
`image sensor, the Tele camera having a respective field of
`view FOVT narrower than FOVW and being operative to
`provide a Tele image of the object or scene, wherein the
`Tele lens has a respective effective focal length EFLT and
`total track length TTLT fulfilling the condition EFLT /
`TTLT > 1;
`
`c) a first autofocus (AF) mechanism coupled
`mechanically to, and used to perform an AF action on the
`Wide lens;
`
`d) a second AF mechanism coupled mechanically
`to, and used to perform an AF action on the Tele lens,
`wherein the Wide and Tele lenses have different F
`numbers F#Wide and F#Tele, wherein the Wide and Tele
`image sensors have pixels with respective pixel sizes Pixel
`sizeWide and Pixel sizeTele wherein Pixel sizeWide is not equal
`to Pixel sizeTele, and wherein the Tele camera has a Tele
`
`Apple v. Corephotonics
`
`- 17 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`camera depth of field (DOFT) shallower than a DOF of the
`Wide camera (DOFW); and
`
`e) a camera controller operatively coupled to the
`first and second AF mechanisms and to the Wide and Tele
`image sensors and configured
`to control
`the AF
`mechanisms, to process the Wide and Tele images to find
`translations between matching points in the images to
`calculate depth information and to create a fused image
`suited for portrait photos, the fused image having a DOF
`shallower than DOFT and having a blurred background.
`
`Id., 14:66-15:32.
`
`27. As I further discuss below, the system and method presented in the
`
`’479 Patent, namely, a dual-aperture camera system having 1) Wide and Tele lens
`
`systems with overlapping fields of view, and 2) a camera controller that performs
`
`image processing to calculate depth information and to fuse Wide and Tele images
`
`to emphasize the foreground and blur the background, were well known in the
`
`prior art prior to the ’479 Patent.
`
`Prosecution History of the ’479 Patent
`
`B.
`28. On July 28, 2018, the Applicant filed U.S. Patent Application No.
`
`16/048,242 (“the ’242 App”) including claims 1-40, which ultimately issued as the
`
`’479 Patent. APPL-1002, 334-66. The ’242 application claimed priority, through a
`
`string of continuations, to Provisional App. 61/834,486 filed on June 13, 2013.
`
`Apple v. Corephotonics
`
`- 18 -
`
`APPL-1003
`
`

`

`
`APPL-1001, 1:5-20.
`
`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`29. On January 9, 2019, a Notice of Allowance is issued. In the
`
`Allowance, the Examiner set forth the claim limitations that were found to be
`
`patentable over the prior art including “the Tele lens has a respective effective
`
`focal length EFLT and total track length TTLT fulfilling the condition EFLT / TTLT
`
`> 1” and several other limitations set forth in claims 1, 19, and 23. The ’479 Patent
`
`issued on March 5, 2019.
`
`VI. CLAIM CONSTRUCTION
`30.
`It is my understanding that in order to properly evaluate the ’479
`
`Patent, the terms of the claims must first be interpreted. It is my understanding that
`
`for the purposes of this inter partes review, the claim terms are given their ordinary
`
`and accustomed meaning as would be understood by one of ordinary skill in the
`
`art, unless the inventor has set forth a special meaning for a term. In order to
`
`construe the following claim terms, I have reviewed the entirety of the ’479 Patent,
`
`as well as its prosecution history.
`
`A.
`
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited for
`portrait photos” (claim 19)
`
`31. This term is recited in claim 19, where it appears as follows:
`
`e) a camera controller operatively coupled to the first and
`second AF mechanisms and to the Wide and Tele image
`sensors and configured to control the AF mechanisms, to
`
`Apple v. Corephotonics
`
`- 19 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`process the Wide and Tele images to find translations
`between matching points in the images to calculate depth
`information and to create a fused image suited for portrait
`photos, the fused image having a DOF shallower than
`DOFT and having a blurred background.
`
`APPL-1001, 15:25-32.
`
`32. This claim term should be construed as requiring the “camera
`
`controller” to perform two separate functions: (1) “to find translations between
`
`matching point in the images to calculate depth information” and (2) “to create a
`
`fused image suited for portrait photos.”
`
`33. Regarding the operation “to find translations between matching point
`
`in the images to calculate depth information,” the specification references Fig. 6
`
`which is “a method disclosed herein for acquiring a zoom image in video/preview
`
`mode for 3 different zoom factor (ZF) ranges.” Id., 11:30-32. As discussed above,
`
`the specification describes obtaining two images where, for a given ROI (“Region
`
`of Interest”) depth information is obtained for faster focusing:
`
`[R]egistration is performed between the Wide and Tele
`images to output a transformation coefficient. The
`transformation coefficient is used to set an AF position.
`The transformation coefficient includes the translation
`between matching points in the two images. This
`translation can be measured in a number of pixels.
`Different translations will result in a different number of
`
`Apple v. Corephotonics
`
`- 20 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`pixel movements between matching points in the images.
`This movement can be translated into depth and the depth
`can be translated into an AF position. This enables to set
`the AF position by only analyzing two images (Wide &
`Tele). The result is fast focusing.
`
`Id., 12:7-15. Nowhere does the specification describe using the “translations
`
`between matching point” to “create a fused image suited for portrait photos.”
`
`34. Rather, the specification describes image fusion in relation to a
`
`different “Still Mode Operation” (shown in Fig. 2). See id., 7:44. In this separate
`
`embodiment, “the obtained image is fused from information obtained by both sub-
`
`cameras at all zoom levels (see FIG. 2), which shows a Wide sensor 202 and a Tele
`
`sensor 204 and their respective FOVs.” Id., 7:45-48. Fusing the Wide and Tele
`
`images is stated to “achieve optical zoom, improves SNR and provides wide
`
`dynamic range.” Id., 7:57-59. Nowhere does the specification describe fusing two
`
`images using “translations between matching points.”
`
`35. Based portions of the specification cited above, it is my opinion that a
`
`POSITA would have understood “to find translations between matching points in
`
`the images to calculate depth information and to create a fused image suited for
`
`portrait photos” as requiring the claimed camera controller to (1) “find translations
`
`between matching points in the images to calculate depth information” and (2)
`
`“creating a fused image suited for portrait photos.”
`
`Apple v. Corephotonics
`
`- 21 -
`
`APPL-1003
`
`

`

`
`
`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`
`VII. GROUNDS
`A. Claims 19 and 20 are obvious over Parulski in view of Ogata,
`further in view of Kawamura, further in view of Soga.
`
`Summary of Parulski
`1.
`36. U.S. Patent No. 7,859,588 to Parulski, et al. (“Parulski”) was filed on
`
`March 9, 2007 and issued on December 28, 2010. See APPL-1005. Parulski is
`
`titled “Method and Apparatus for Operating a Dual Lens Camera to Augment an
`
`Image,” and discloses “a digital camera that uses multiple lenses and image sensors
`
`to provide an improved imaging capability.” Id., 1:8-10.
`
`37. Parulski describes several embodiments that can utilize its dual-
`
`capture enhancement methods. Id., 12:55-13:20. An example of the camera
`
`embodiment is below:
`
`Id., Figs. 2A-2B. Parulski describes this embodiment as an “image capture
`
`assembly l0A” that includes a “first zoom lens 3” and a “second zoom lens 4.” Id.,
`
`12:55-59. The camera also includes “a color LCD image display 70 and a number
`
`
`
`Apple v. Corephotonics
`
`- 22 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`of user controls 42, including a shutter button 42a for enabling an image capture
`
`
`
`sequence ….” Id., 12:60-62.
`
`38.
`
`In this camera embodiment, “[t]he optical axes of the zoom lenses 3
`
`and 4 and the sensors 12 and 14 are generally aligned with respect to each other so
`
`as to be viewing substantially the same scene, albeit typically with different fields
`
`of view.” Id., 13:6-9. An extended zoom range is provided by “digital zooming
`
`between the wide angle and the telephoto focal lengths.” Id., 23:54-58. As an
`
`alternative, Parulski states that “one (or both) of the zoom lenses 3 and 4 could
`
`be replaced with a fixed focal length lens.” Id., 13:4-6. Reference herein to
`
`“Parulski’s camera” is to the embodiment that includes fixed-focal-length wide and
`
`telephoto lenses.
`
`39. Parulski teaches that its dual lens image capture assembly, including the
`
`embodiments from Labaziewicz, may operate in still and video modes to produce
`
`“still images and motion video images.” Id., 12:36-41; see also id., 14:5-9 (“The
`
`digital data ... is ... processed by the image processor 50 to produce a processed digital
`
`image file, which may contain a still digital image or a video image.”); Id., 29:8-11
`
`(“the images captured by the primary and secondary capture units could be a still
`
`image or a video image, and in the case of a video image could be a series of
`
`images.”).
`
`40. Parulski also describes several digital zoom features for its camera based
`
`Apple v. Corephotonics
`
`- 23 -
`
`APPL-1003
`
`

`

`Declaration of Fredo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,225,479
`
`on a requested zoom position from a user that determines the primary image and
`
`
`
`secondary image from two capture units of the digital camera based on the user
`
`requested zoom position. Id., FIG. 23, 27:8-24, 29:51-64. If the requested zoom
`
`position is less than a zoom switch value X, the first image station with a wide lens is
`
`set as primary capture unit for providing a primary image, and the second image
`
`station with a tele lens is set as a secondary capture unit for providing a secondary
`
`image. Id., Fig. 23, Fig. 14, 22:18-21 (“In block 502, the zoom position setting is
`
`compared to a value X at which the image capture function switches” between
`
`first/second image capture units); Fig. 3, 15:54-61 (“In block 102, the zoom position
`
`setting is compared to a value X at which the image capture function switches from
`
`the first image capture stage to the second image capture stage”). In block 104, if the
`
`zoom position setting is less than X (a negative response to block 102), then the first
`
`image capture stage