`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2020-00905
`Case No. IPR2020-00906
`U.S. Patent No. 10,225,479
`____________
`
`DECLARATION OF JOHN C. HART, Ph.D.
`PURSUANT TO 37 C.F.R. § 1.68
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 1
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`TABLE OF CONTENTS
`
`“fused image with a point of view (POV) of the Wide camera”
`
`BACKGROUND .................................................................... 1
`I.
`SUMMARY OF OPINIONS ................................................... 2
`II.
`III. EDUCATIONAL AND EMPLOYMENT BACKGROUND ...... 5
`IV. LEVEL OF ORDINARY SKILL IN THE ART (POSITA) ....... 9
`V. RELEVANT LEGAL STANDARDS FOR OBVIOUSNESS ... 10
`VI. OVERVIEW OF THE ’479 PATENT .................................... 14
`VII. CLAIM CONSTRUCTION ................................................... 18
`A.
`(claims 1 and 23) ........................................................................... 18
`B.
`portrait photos” (claim 19) ............................................................ 23
`VIII. PRIOR ART REFERENCES ................................................. 25
`A.
`Parulski .......................................................................................... 25
`IX. OBVIOUSNESS—CLAIMS 1 AND 23 (AND DEPENDENTS)
` ............................................................................................ 38
`A. A POSITA Would Not Have Found Obvious the Combination of
`Parulski and Konno (Ground 1, -00905 IPR) ................................ 38
`B.
`Parulski, Konno and Szeliski (Ground 2, -00905 IPR) ................. 44
`C.
`Parulski, Konno, Szeliski and Segall (Ground 3, -00905 IPR) ..... 45
`
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited for
`
`A POSITA Would Not Have Found Obvious the Combination of
`
`A POSITA Would Not Have Found Obvious the Combination of
`
`i
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 2
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`Claims 19 and 20 Are Not Obvious Over the Combination of
`
`D. A POSITA Would Not Have Found Obvious the Combination of
`Parulski, Konno and Stein (Ground 4, -00905 IPR) ..................... 46
`X. OBVIOUSNESS—CLAIM 19 (AND DEPENDENTS) ........... 48
`A.
`Parulski, Ogata, Kawamura, and Soga (Ground 1, -00906 IPR) .. 48
`1.
`and Kawamura Lens Designs with Parulski ............................ 48
`2.
`photos” Is Satisfied Under Its Proper Construction ................. 52
`B.
`00906 IPR) .................................................................................... 58
`XI. SECONDARY CONSIDERATIONS / OBJECTIVE INDICIA
`OF NON-OBVIOUSNESS .................................................... 60
`Industry Praise / Licensing ............................................................ 64
`A.
`B.
`Commercial Success ..................................................................... 71
`Failure of Others / Copying ........................................................... 73
`C.
`XII. DECLARATION .................................................................. 75
`
`A POSITA Would Not Have Been Motivated to Use the Ogata
`
`Apple Has Not Shown that the Limitation “to find translations
`between matching points in the images to calculate depth
`information and to create a fused image suited for portrait
`
`Claims 21 and 22 Are Not Obvious Over the Combination of
`Parulski, Ogata, Kawamura, Soga, and Morgan-Mar (Ground 2, -
`
`
`
`ii
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 3
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`I.
`
`BACKGROUND
`
`1.
`
`I have been retained as a technical expert by Patent Owner Core-
`
`photonics Ltd. (“Patent Owner” or “Corephotonics”) in this proceeding. Core-
`
`photonics has asked me to provide my expert opinions concerning certain
`
`technical aspects of imaging system design as they relate to the Petitioner Ap-
`
`ple Inc.’s petition for inter partes review of U.S. Patent 10,225,479 (“’479
`
`patent”) in Case Nos. IPR2020-00905 (“-00905 IPR”) IPR2020-00906 (“-
`
`00906 IPR”) and the accompanying Declarations of Fredo Durand. In partic-
`
`ular, I have been asked to respond to Dr. Durand’s opinions set forth in his
`
`declarations, Ex. 1003 in each IPR.
`
`2.
`
`The statements in this declaration summarize my opinions on
`
`these matters based on my over 30 years of study and research of imaging
`
`systems, my education, knowledge, skills, and my review and analysis of the
`
`materials referenced herein.
`
`3.
`
`I am being compensated for my work in this matter at the rate of
`
`$575 per hour. I am also being reimbursed for reasonable and customary ex-
`
`penses associated with my work and testimony in this investigation. My com-
`
`pensation is not contingent on the outcome of this matter or the substance of
`
`my testimony
`
`1
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 4
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`II. SUMMARY OF OPINIONS
`
`4.
`
`In the preparation of this declaration, I have reviewed:
`
`• The ’479 patent (Ex. 1001)1
`
`• Prosecution history of the ’479 patent (Ex. 1002)
`
`• The declarations of Dr. Fredo Durand (Ex. 1003 in each IPR)
`
`• The curriculum vitae of Dr. Fredo Durand (Ex. 1004)
`
`• U.S. Patent No. 7,859,588 (“Parulski”) (Ex. 1005)
`
`• English translation of Japanese Patent Application Publication No. 2007-
`259108 (“Soga”) (Ex. 1006)
`
`• Jacobs et al., “Focal Stack Compositing for Depth of Field Control,”
`Stanford Computer Graphics Laboratory Technical Report 2012-1 (Ex.
`1007)
`
`• Prosecution history of the Morgan-Mar patent (Ex. 1008)
`
`• U.S. Patent No. 8,989,517 (“Morgan-Mar) (Ex. 1009, Ex. 2037)
`
`• PCT Publication No. WO2013140359 (“Shalon”) (Ex. 1010)
`
`• U.S. Patent Application Publication No. 2008/0030592 (“Border”) (Ex.
`1011)
`
`
`1 Where a given Apple exhibit appears with the same exhibit number in both
`IPRs on the ’479 patent, or a given exhibit number is used in only one of
`the IPRs, I refer to the exhibit by that number. Where the same exhibit
`number is used for different exhibits in the two IPRs, e.g., for Dr. Durand’s
`declarations, I will attempt to always clarify which IPR’s exhibit I am re-
`ferring to.
`
`2
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 5
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`• English translation of Japanese Patent Application Publication
`JPS5862609 (“Kawamura”) (Ex. 1012)
`
`• Richard Szeliski, Computer Vision—Algorithms and Applications (2011)
`(“Szeliski”) (Ex. 1013)
`
`• U.S. Patent No. 6,259,863 (“Maruyama”) (Ex. 1014)
`
`• English translation of JP Pub. No. 2013-106289 (“Konno”) (Ex. 1015)
`
`• Ralph E. Jacobson et al., The Manual of Photography: photographic and
`digital imaging, 9th Edition, 2000 (“Jacobson”) (Ex. 1016)
`
`• U.S. Patent App. Pub. No. 2010/0321511 (“Koskinen”) (Ex. 1017)
`
`• U.S. Patent No. 7,206,136 (“Labaziewicz”) (Ex. 1018)
`
`• Milton Katz, Introduction to Geometrical Optics (2002) (“Katz”) (Ex.
`1019)
`
`• Warren J. Smith, Modern Lens Design (1992) (“Smith”) (Ex. 1020)
`
`• The declaration of Dr. José Sasián (Ex. 1021)
`
`• U.S. Patent No. 8,908,041 (“Stein”) (Ex. 1023)
`
`• U.S. Patent No. 8,406,569 (“Segall”) (Ex. 1024)
`
`• U.S. Patent No. 8,824,833 (“Dagher”) (Ex. 1025)
`
`• U.S. Patent No. 5,546,236 (“Ogata”) (Ex. 1026)
`
`• File History for Provisional No. 61/752,515 to Stein (“Stein provisional”)
`(Ex. 1027)
`
`• Bae et al., “Defocus Magnification,” Eurographics 2007 (“Bae”) (Ex.
`1028)
`
`3
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 6
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`• Specification sheet for Sony ICX629 image sensor (“ICX629”) (Ex.
`1029)
`
`• Specification sheet for Sony ICX624 image sensor (“ICX624”) (Ex.
`1030)
`
`• Product manual for Kodak Easyshare V610 (Ex. 1033)
`
`• U.S. Patent No. 7,112,774 (“Baer”) (Ex. 1034)
`
`• Robert E. Fischer et al., Optical System Design (2008) (Ex. 1035)
`
`• The declaration of Dr. Duncan Moore (Ex. 2015)
`
`• The transcript of the 26 Jan. 2021 deposition of Dr. Fredo Durand. (Ex.
`2036)
`
`• Forsyth and Ponce, “Computer Vision: A Modern Approach” (1st ed.)
`(2003) (Ex. 2038)
`
`• The declaration of Eli Saber filed for IPR2020-0860.
`
`• District court filings, emails, and agreements concerning Apple’s evalua-
`tion of Corephotonics’ technology relevant to secondary considerations
`(Exs. 2004–2012, 2018–2023)
`
`• The declaration of Eran Kali (Ex. 2013)
`
`5.
`
`In forming the opinions set forth herein, I have considered:
`
`a. The documents listed above;
`
`b. My education, knowledge, skills, and experience in the design and devel-
`
`opment of imaging systems; and
`
`4
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 7
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`c. The level of skill of a person having ordinary skill in the art (POSITA) at
`
`the time of the effective filing dates of the ’479 patent.
`
`6.
`
`As I explain in further detail below, it is my professional and
`
`expert opinion that Apple and Dr. Durand have failed to demonstrate that any
`
`of the challenged claims of the ’479 patent were obvious, under any of the
`
`grounds or combinations of references that Apple has raised in these two IPRs.
`
`III. EDUCATIONAL AND EMPLOYMENT BACKGROUND
`
`7.
`
`As indicated in my Curriculum Vitae, attached as Exhibit 2003,
`
`I am a tenured full Professor of Computer Science in the Department of Com-
`
`puter Science at the University of Illinois at Urbana-Champaign. As an edu-
`
`cator for the past three decades, I have taught courses in computer graphics
`
`and related areas to thousands of students. I also strive to provide opportuni-
`
`ties for the general public to learn more about computing. For example, in
`
`1999 I oversaw the production of the documentary “The Story of Computer
`
`Graphics.” I also teach an open course on data visualization on Coursera that
`
`has reached over 360,000 learners worldwide since 2016.
`
`8.
`
`I serve as the Director of Online Programs for the Department of
`
`Computer Science at the University of Illinois, and oversee its Master of Com-
`
`5
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 8
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`puter Science (“MCS”) degree program. In 2016, I redesigned the online of-
`
`fering of the MCS degree program to make it more flexible and affordable for
`
`students that could not afford to leave their job to pursue a degree fulltime.
`
`Under my leadership, this degree program quickly grew to the second largest
`
`graduate program offered by the University of Illinois at Urbana-Champaign,
`
`and contributed significantly to the campus-wide proportion of underrepre-
`
`sented minorities enrolled in the institution. The tech company C3.ai found
`
`this online degree so desirable, it pays its employee’s tuition and upon com-
`
`pletion, gives them a bonus, a raise and stock options.
`
`9.
`
`I am also the Executive Associate Dean of the Graduate College
`
`of the University of Illinois at Urbana-Champaign, where I oversee the edu-
`
`cation of over 17,800 graduate students in hundreds of graduate degree pro-
`
`grams across the entire university. I recently developed a new post-
`
`baccalaureate certificate credential at Illinois to provide the recently unem-
`
`ployed with a rapid educational opportunity to transition their skills to areas
`
`of greater prosperity.
`
`10.
`
`I have been researching computer graphics since 1987, with over
`
`a hundred papers, videos, patents and other contributions to computer
`
`graphics including photographic imaging systems. My work in computer
`
`6
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 9
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`graphics has been funded by Adobe, Intel, Microsoft, Nokia and Nvidia as
`
`well as the National Science Foundation (NSF) and the Defense Advanced
`
`Research Projects Agency (DARPA). One of my most recent contribution is
`
`on the topic of displaying text on the video screen of a VR headset, in collab-
`
`oration with Oculus. This paper, “Real-Time Analytic Antialiased Text for 3-
`
`D Environments,” was selected as one of the best papers at the 2019 High-
`
`Performance Graphics Conference in Strasbourg France in July.
`
`11.
`
`I am an internationally recognized leader in the field of computer
`
`graphics. From 2002-08 I was the Editor-in-Chief of the top journal in com-
`
`puter graphics, the Association for Computing Machinery (ACM) Transac-
`
`tions on Computer Graphics. From 1994-1999 I served on the executive
`
`committee of the main organization of computer graphics practitioners, the
`
`ACM Special Interest Group on Computer Graphics and Interactive Tech-
`
`niques (SIGGRAPH). I continue to oversee the peer review of major papers
`
`in the field through service as chair and member of various paper review com-
`
`mittees. I am also a founding member of the editorial board of ACM Books,
`
`and the area editor for computer graphics.
`
`7
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 10
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`12. This report is on the subject of photographic imaging systems. I
`
`have worked on a variety of methods and systems for the fusion of photo-
`
`graphs. For example, in 2008 I was granted patent #7,365,744 “Method and
`
`Systems for Image Modification” on techniques for learning a surface appear-
`
`ance from one photograph and realistically applying it to a different surface
`
`in another photograph.
`
`13. At the priority date of the ’479 pa-
`
`tent, I was funded by the National Science
`
`Foundation’s Advanced Digitization of Biodi-
`
`versity Collections to design and deliver an im-
`
`aging infrastructure to scan the nation’s
`
`entomological collections of insect drawers.
`
`This project, available at invertnet.org, re-
`
`quired the fusion of 51,791 photographic im-
`
`ages of small portions of insect drawers, vials
`
`and slides to make the collections available via the Internet as high-resolution
`
`zoomable composite images. This effort included the design and deployment
`
`of a custom robotic photographic imaging system, designed specifically to
`
`capture and fuse numerous photographs of each specimen drawer.
`
`8
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 11
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`IV. LEVEL OF ORDINARY SKILL IN THE ART (POSITA)
`
`14.
`
`I understand that in evaluating the validity of the ’479 patent
`
`claims, the content of a patent or printed publication prior art should be inter-
`
`preted the way a person of ordinary skill in the art would have interpreted the
`
`prior art as of the effective filing date of the challenged patent.
`
`15.
`
`I understand that factors that may be considered in determining
`
`the level of ordinary skill in the art at the time of the effective filing date of
`
`the challenged patents include: (1) the educational level of the inventor; (2)
`
`type of problems encountered in the art; (3) prior art solutions to those prob-
`
`lems; (4) rapidity with which innovations are made; (5) sophistication of the
`
`technology; and (6) educational level of active workers in the field.
`
`16. Dr. Durand at ¶13 in each declaration believes “that a POSITA
`
`would include someone who had, as of the claimed priority date of the ’479
`
`Patent, a bachelor’s or the equivalent degree in electrical and/or computer en-
`
`gineering or a related field and 2-3 years of experience in imaging systems
`
`including image processing and lens design.” He further recognizes “that
`
`someone with less formal education but more experience, or more formal ed-
`
`ucation but less experience could have also met the relevant standard for a
`
`9
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 12
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`POSITA.” My opinions in reply to Dr. Durand use this definition of a
`
`POSITA.
`
`17.
`
`I understand that the ’479 patent shares a specification with and
`
`claims priority to U.S. App. No. 14/365,711 filed June 16, 2014 and issued as
`
`U.S. Patent No. 9,185,291. (Ex. 1001, ’479 patent at 1:7–29.) I understand
`
`that U.S. App. No. 14/365,711 was a § 371 application from international pa-
`
`tent application PCT/IB2014/062180 filed June 12, 2014 and is related to and
`
`claims priority from U.S. Provision Patent Application No. 61/834,486 filed
`
`June 13, 2013. (Ex. 1001, ’479 patent at 1:7–29.) I therefore understand that
`
`the effective filing date of the ’479 patent is June 13, 2013.
`
`18.
`
`I would have met the requirements of a POSITA on June 13,
`
`2013. I have used the perspective of a POSITA at that time to form my opin-
`
`ions in reply to Dr. Durand’s opinons.
`
`V. RELEVANT LEGAL STANDARDS FOR OBVIOUSNESS
`
`19.
`
`I have been informed of the legal standards for establishing pa-
`
`tent invalidity in inter partes review proceedings before the Patent Trial and
`
`Appeal Board.
`
`10
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 13
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`20.
`
`I understand that the petitioner must prove invalidity of a patent
`
`claim by a preponderance of the evidence, that is, the evidence must be suffi-
`
`cient to show that a fact or legal conclusion is more likely than not.
`
`21.
`
`I understand that a claim may be anticipated if (1) the claimed
`
`invention was patented, described in a printed publication, or in public use, on
`
`sale, or otherwise available to the public before the effective filing date of the
`
`claimed invention; or (2) the claimed invention was described in a patent or
`
`published application, in which the patent or application names another in-
`
`ventor and was effectively filed before the effective filing date of the claimed
`
`invention.
`
`22.
`
`I understand that, once the claims of a patent have been properly
`
`construed, the next step in determining anticipation of a patent claim requires
`
`a comparison of the properly construed claim language to the prior art on a
`
`limitation-by-limitation basis.
`
`23.
`
`I understand that even if a patent claim is not anticipated, it may
`
`still be invalid if the differences between the claimed subject matter and the
`
`prior art are such that the subject matter as a whole would have been obvious
`
`at the time the invention was made to a person of ordinary skill in the pertinent
`
`art.
`
`11
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 14
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`24.
`
`I also understand that a patent may be rendered obvious based on
`
`an alleged prior art reference or a combination of such references plus what a
`
`person of ordinary skill in the art would understand based on his or her
`
`knowledge and the references. It is also my understanding that in assessing
`
`the obviousness of claimed subject matter one should evaluate obviousness
`
`over the prior art from the perspective of one of ordinary skill in the art at the
`
`time the invention was made (and not from the perspective of either a layman
`
`or a genius in that art).
`
`25.
`
`I understand that a patent claim composed of several elements is
`
`not proved obvious merely by demonstrating that each of its elements was
`
`known in the prior art. There must be a reason for combining the elements in
`
`the manner claimed. That is, there must be a showing that a person of ordinary
`
`skill in the art at the time of the invention would have thought of either com-
`
`bining two or more references or modifying a reference to achieve the claimed
`
`invention.
`
`26.
`
`I understand that an obviousness determination includes the con-
`
`sideration of the following factors: (1) the scope and content of the prior art,
`
`(2) the differences between the prior art and the claims at issue, (3) the level
`
`of ordinary skill in the art, and (4) objective evidence of nonobviousness.
`
`12
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 15
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`27.
`
`I understand that, when available, so-called objective indicia of
`
`non-obviousness (also known as “secondary considerations” and or the real
`
`world factors) like the following are also to be considered when assessing
`
`obviousness: (1) widespread acclaim; (2) commercial success; (3) long-felt
`
`but unresolved needs; (4) copying of the invention by others in the field; (5)
`
`initial expressions of disbelief by experts in the field; (6) failure of others to
`
`solve the problem that the inventor solved; and (7) unexpected results, among
`
`others. I also understand that evidence of objective indicia of non-obviousness
`
`must be commensurate in scope with the claimed subject matter. I understand
`
`this is commonly referred to as a “nexus.”
`
`28.
`
`I understand that the burden is on the petitioner to explain how
`
`specific references could be combined, which combinations of elements in
`
`specific references would yield a predictable result, and how any specific
`
`combination would operate or read on the claims. I further understand that the
`
`petitioner cannot rely on conclusory statements but must instead provide a
`
`reasoned explanation supported by evidence. I also understand that obvious-
`
`ness does not exist where the prior art discourages or teaches away from the
`
`claimed invention. I also understand that even if a reference does not teach
`
`away, its statements regarding preferences are relevant to ta finding whether
`
`13
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 16
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`a person skilled in the art would be motivated to combine that reference with
`
`another reference.
`
`29.
`
`I understand that it is impermissible to use hindsight to arrive at
`
`the claimed invention. My understanding is that the inventor’s own path never
`
`leads to a conclusion of obviousness. I also understand that, when assessing
`
`whether there was a motivation to combine references to teach a claim ele-
`
`ment, defining the problem in terms of its solution reveals improper hindsight.
`
`30.
`
`I understand that, in this proceeding, prior art to the ’479 patent
`
`includes patents and printed publications in the relevant art that predate the
`
`effective filing date of the ‘479 patent’s challenged claims, which I understand
`
`to be June 13, 2013. (Ex. 1001, ’479 patent at 1:7–20.)
`
`VI. OVERVIEW OF THE ’479 PATENT
`
`31. The ’479 patent describes and claims techniques for making
`
`“thin digital cameras with optical zoom operating in both video and still
`
`mode.” (Ex. 1001, ’479 patent at 3:27–28.) As the patent explains, zoom in
`
`“commonly understood as a capability to provide different magnifications of
`
`the same scene and/or object by changing the focal length of an optical sys-
`
`tem.” (Ex. 1001, ’479 patent at 1:44–49.) Traditionally, this was accomplished
`
`by mechanically moving lens elements relative to one another. (Ex.
`
`14
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 17
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`1001, ’479 patent at 1:49–51.) Another approach is “digital zooming,” where
`
`the focal length of the lens is kept unchanged, but the image is cropped and
`
`digitally manipulated to produce an image that is magnified but has a lower
`
`resolution. (Ex. 1001, ’479 patent at 1:55–38.)
`
`32. The ’479 patent describes an approach to approximating the ef-
`
`fect of a zoom lens (which varies its focal length) by using two lens systems
`
`(a “wide” and a “tele” lens system) with different fixed focal lengths. (Ex.
`
`1001, ’479 patent at 3:34–54.) Various computational means are used to take
`
`the images from these two lenses to produce an output that approximate a
`
`system with mechanical zoom. This approach can produce a device that is
`
`smaller, lower cost, and more reliable than devices that use mechanical zoom.
`
`(Ex. 1001, ’479 patent at 1:51–53.)
`
`(Ex. 1001, ’479 patent, Fig. 1B)
`
`
`
`15
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 18
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`33. Relevant to the claims of the ’479 patent, the specification de-
`
`scribes combining still images using the technique of “fusion.” (Ex.
`
`1001, ’479 patent at 3:48–54.) A “fused” image includes information from
`
`both the wide and tele images. (Id.) One approach to performing fusion is
`
`shown in Figure 5:
`
`(Ex. 1001, ’479 patent, Fig. 5.)
`
`
`
`16
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 19
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`34. Making a compact, high-quality dual-aperture zoom system re-
`
`quires lenses with particular characteristics. The ’479 patent teaches lens de-
`
`signs for the tele lens which provide a small “total track length” relative to
`
`their focal length, which means that they have a compact size in light of the
`
`degree of magnification that they provide. (Ex. 1001, ’479 patent at 12:38–
`
`53.) One of the lens designs taught by the ’479 patent and covered by several
`
`of the challenged claims is shown in Figure 9:
`
`
`
`(Ex. 1001, ’479 patent, Fig. 9.)
`
`35. The lens aspects of the ’479 patent are described further in Dr.
`
`Moore’s declaration. (E.g., Ex. 2015, Moore Decl., ¶¶ 31–34.)
`
`17
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 20
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`VII. CLAIM CONSTRUCTION
`
`A.
`
`“fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23)
`
`36. Dr. Durand’s opinion is that this term should be construed as “a
`
`fused image that maintains the Wide camera’s field of view or both the Wide
`
`camera’s field of view and position.” (Ex. 1003 from -00905 IPR, ¶¶ 29–33.)
`
`I do not agree that is the understanding of this term to a POSITA, in view of
`
`the ’479 patent.
`
`37. Under this construction there are two ways to meet the “point of
`
`view” requirement. Either, the fused image can maintain the Wide camera’s
`
`(a) field of view or (b) field of view and position. However, the second of
`
`these two options is superfluous, as if the image has both the field of view and
`
`position of the Wide camera, then it also necessarily has the field of view of
`
`the Wide camera. So, Dr. Durand’s construction is logically equivalent to the
`
`construction “a fused image that maintains the Wide camera’s field of view.”
`
`38. Even the superfluous “Wide camera’s . . . position” portion of
`
`the construction does not line up with the term “position” as it is used in
`
`the ’479 patent’s discussion of “POV.” During his deposition, Dr. Durand
`
`confirmed that he understood the “Wide camera’s . . . position” to refer to the
`
`“3D XYZ location of the camera.” (Ex. 2036, Durand Depo. at 21:3–7.) But
`
`18
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 21
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`when the specification refers to “position POV” in its discussion of “combi-
`
`nation” POVs, it is referring to the “position of either sub-camera image.” (Ex.
`
`1001, ’479 patent at 5:14–16.) That is, “position POV” is based on the posi-
`
`tions of images, not the positions of cameras. An image position may differ
`
`because the camera is located in a different position, but it also may differ
`
`because camera, located in the same position, has been pointed in a different
`
`direction. For this reason as well, Dr. Durand’s proposed construction is in-
`
`consistent with how the patent specification uses the relevant terms.
`
`39. The effect of Dr. Durand’s construction is to replace the term
`
`“point of view” in the claims with the term “field of view” in his construction.
`
`This is not consistent with how a POSITA would understand these phrases or
`
`with how they are used in the ’479 patent. For example, claim 1 refers to both
`
`“a field of view FOVW” of the wide camera and “a point of view (POV)” of
`
`the wide camera, with no suggestion they are the same thing or that one term
`
`is the antecedent basis for the other. (Ex. 1001, ’479 patent at 13:25–26,
`
`13:48.)
`
`40.
`
`In the specification, the ’479 patent clearly defines “FOV” as a
`
`planar angle, representable in degrees: “As used herein, the FOV is measured
`
`from the center axis to the corner of the sensor (i.e. half the angle of the normal
`
`19
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 22
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`definition).” (Ex. 1001 at 7:11–13.) Examples of FOV values are given in de-
`
`grees (id. at 7:20–22), and FOV is used as a parameter to the tangent function,
`
`further confirming that it is a simple angle (id. at 7:7–8).
`
`41. Dr. Durand agreed during his deposition that his construction of
`
`“POV” matches what the patent calls field of view:
`
`Q. So when you’re using the term “field of view” in this con-
`struction, you’re reviewing -- you're referring to how much of
`the scene is captured by the camera; is that right?
`
`A. This is a vague version of the definition, I would say one
`definition of the field of view. For example, the horizontal field
`of view is to look at the angle between the two edges of the --
`of the image.
`
`(Ex. 2036 at 22:4–12.)
`
`42. This is definition matches what the ’479 patent specification calls
`
`the “normal definition” of FOV (the ’479 patent uses half that “normal” value
`
`in its formulas). (Ex. 1001, ’479 patent at 7:11–13.) As Dr. Durand testified,
`
`this FOV is an inherent property of the camera and lens, and independent of
`
`where they are pointed or what they see:
`
`Q. Would you agree that a camera’s field of view is a property
`of the camera that’s independent of what direction the camera is
`pointing?
`
`A. So one definition or understanding of field of view would be
`-- would indeed be just an angle that’s a property of the combi-
`nation of a camera and the lens.
`
`20
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 23
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`(Ex. 2036 at 22:25–23:6.)
`
`43. POV is defined in the specification quite differently. It refers to
`
`how objects are “seen by each sub-camera,” i.e., how objects “with be shifted
`
`and have different perspective (shape)” for the two cameras. (Ex. 1001 at
`
`5:10–14.) This POV depends on the position and orientation of the camera
`
`and cannot be expressed fully by a single numerical angle. Rather, as the ’479
`
`patent explains, using a camera with a different POV can both shift an object
`
`(change its position in the image) and change the perspective of an object
`
`(changes its apparent shape in the image). (Ex. 1001 at 5:10–16.)
`
`44. Examples of changing POV can be seen in image pairs (a)-(b)
`
`and (d)-(e) from Szeliski Figure 1.1:
`
`21
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 24
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`
`
`(Ex. 1013, Szeliski at 468.)
`
`45. The ’479 patent refers to “combination” possibilities where an
`
`output image reflects only some aspects of a given POV, such as “Wide per-
`
`spective POV” or “Wide position POV.” (Ex. 1001, ’479 patent at 5:15–19.)
`
`But, when it refers to “Wide POV,” without qualification, it is referring to the
`
`complete Wide POV, both perspective and position. (Ex. 1001, ’479 patent at
`
`5:10–14; 5:23–26.)
`
`46.
`
`In summary, a POSITA would not agree that the term POV in
`
`the phrase “fused image with a point of view (POV) of the Wide camera” can
`
`22
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 25
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`be replaced with the distinct term FOV. Further, a POSITA would understand
`
`that POV of the Wide camera in this phrase refers to the full Wide camera
`
`POV and not to “combination” outputs that have a Wide “perspective POV”
`
`and Tele “position POV” or vice versa. (Ex. 1001, ’479 patent at 5:13–23.) In
`
`my opinion a POSITA would understand this term to mean “fused image in
`
`which the positions and shapes of objects reflect the POV of the Wide cam-
`
`era.”
`
`B.
`
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited
`for portrait photos” (claim 19)
`
`47. Dr. Durand’s opinion