UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2020-00905
`Case No. IPR2020-00906
`U.S. Patent No. 10,225,479
`____________
`
`DECLARATION OF JOHN C. HART, Ph.D.
`PURSUANT TO 37 C.F.R. § 1.68
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 1
`
`
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2020-00905
`Case No. IPR2020-00906
`U.S. Patent No. 10,225,479
`____________
`
`DECLARATION OF JOHN C. HART, Ph.D.
`PURSUANT TO 37 C.F.R. § 1.68
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 1
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`TABLE OF CONTENTS
`
`“fused image with a point of view (POV) of the Wide camera”
`
`BACKGROUND .................................................................... 1
`I.
`SUMMARY OF OPINIONS ................................................... 2
`II.
`III. EDUCATIONAL AND EMPLOYMENT BACKGROUND ...... 5
`IV. LEVEL OF ORDINARY SKILL IN THE ART (POSITA) ....... 9
`V. RELEVANT LEGAL STANDARDS FOR OBVIOUSNESS ... 10
`VI. OVERVIEW OF THE ’479 PATENT .................................... 14
`VII. CLAIM CONSTRUCTION ................................................... 18
`A.
`(claims 1 and 23) ........................................................................... 18
`B.
`portrait photos” (claim 19) ............................................................ 23
`VIII. PRIOR ART REFERENCES ................................................. 25
`A.
`Parulski .......................................................................................... 25
`IX. OBVIOUSNESS—CLAIMS 1 AND 23 (AND DEPENDENTS)
` ............................................................................................ 38
`A. A POSITA Would Not Have Found Obvious the Combination of
`Parulski and Konno (Ground 1, -00905 IPR) ................................ 38
`B.
`Parulski, Konno and Szeliski (Ground 2, -00905 IPR) ................. 44
`C.
`Parulski, Konno, Szeliski and Segall (Ground 3, -00905 IPR) ..... 45
`
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited for
`
`A POSITA Would Not Have Found Obvious the Combination of
`
`A POSITA Would Not Have Found Obvious the Combination of
`
`i
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 2
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`Claims 19 and 20 Are Not Obvious Over the Combination of
`
`D. A POSITA Would Not Have Found Obvious the Combination of
`Parulski, Konno and Stein (Ground 4, -00905 IPR) ..................... 46
`X. OBVIOUSNESS—CLAIM 19 (AND DEPENDENTS) ........... 48
`A.
`Parulski, Ogata, Kawamura, and Soga (Ground 1, -00906 IPR) .. 48
`1.
`and Kawamura Lens Designs with Parulski ............................ 48
`2.
`photos” Is Satisfied Under Its Proper Construction ................. 52
`B.
`00906 IPR) .................................................................................... 58
`XI. SECONDARY CONSIDERATIONS / OBJECTIVE INDICIA
`OF NON-OBVIOUSNESS .................................................... 60
`Industry Praise / Licensing ............................................................ 64
`A.
`B.
`Commercial Success ..................................................................... 71
`Failure of Others / Copying ........................................................... 73
`C.
`XII. DECLARATION .................................................................. 75
`
`A POSITA Would Not Have Been Motivated to Use the Ogata
`
`Apple Has Not Shown that the Limitation “to find translations
`between matching points in the images to calculate depth
`information and to create a fused image suited for portrait
`
`Claims 21 and 22 Are Not Obvious Over the Combination of
`Parulski, Ogata, Kawamura, Soga, and Morgan-Mar (Ground 2, -
`
`
`
`ii
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 3
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`I.
`
`BACKGROUND
`
`1.
`
`I have been retained as a technical expert by Patent Owner Core-
`
`photonics Ltd. (“Patent Owner” or “Corephotonics”) in this proceeding. Core-
`
`photonics has asked me to provide my expert opinions concerning certain
`
`technical aspects of imaging system design as they relate to the Petitioner Ap-
`
`ple Inc.’s petition for inter partes review of U.S. Patent 10,225,479 (“’479
`
`patent”) in Case Nos. IPR2020-00905 (“-00905 IPR”) IPR2020-00906 (“-
`
`00906 IPR”) and the accompanying Declarations of Fredo Durand. In partic-
`
`ular, I have been asked to respond to Dr. Durand’s opinions set forth in his
`
`declarations, Ex. 1003 in each IPR.
`
`2.
`
`The statements in this declaration summarize my opinions on
`
`these matters based on my over 30 years of study and research of imaging
`
`systems, my education, knowledge, skills, and my review and analysis of the
`
`materials referenced herein.
`
`3.
`
`I am being compensated for my work in this matter at the rate of
`
`$575 per hour. I am also being reimbursed for reasonable and customary ex-
`
`penses associated with my work and testimony in this investigation. My com-
`
`pensation is not contingent on the outcome of this matter or the substance of
`
`my testimony
`
`1
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 4
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`II. SUMMARY OF OPINIONS
`
`4.
`
`In the preparation of this declaration, I have reviewed:
`
`• The ’479 patent (Ex. 1001)1
`
`• Prosecution history of the ’479 patent (Ex. 1002)
`
`• The declarations of Dr. Fredo Durand (Ex. 1003 in each IPR)
`
`• The curriculum vitae of Dr. Fredo Durand (Ex. 1004)
`
`• U.S. Patent No. 7,859,588 (“Parulski”) (Ex. 1005)
`
`• English translation of Japanese Patent Application Publication No. 2007-
`259108 (“Soga”) (Ex. 1006)
`
`• Jacobs et al., “Focal Stack Compositing for Depth of Field Control,”
`Stanford Computer Graphics Laboratory Technical Report 2012-1 (Ex.
`1007)
`
`• Prosecution history of the Morgan-Mar patent (Ex. 1008)
`
`• U.S. Patent No. 8,989,517 (“Morgan-Mar) (Ex. 1009, Ex. 2037)
`
`• PCT Publication No. WO2013140359 (“Shalon”) (Ex. 1010)
`
`• U.S. Patent Application Publication No. 2008/0030592 (“Border”) (Ex.
`1011)
`
`
`1 Where a given Apple exhibit appears with the same exhibit number in both
`IPRs on the ’479 patent, or a given exhibit number is used in only one of
`the IPRs, I refer to the exhibit by that number. Where the same exhibit
`number is used for different exhibits in the two IPRs, e.g., for Dr. Durand’s
`declarations, I will attempt to always clarify which IPR’s exhibit I am re-
`ferring to.
`
`2
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 5
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`• English translation of Japanese Patent Application Publication
`JPS5862609 (“Kawamura”) (Ex. 1012)
`
`• Richard Szeliski, Computer Vision—Algorithms and Applications (2011)
`(“Szeliski”) (Ex. 1013)
`
`• U.S. Patent No. 6,259,863 (“Maruyama”) (Ex. 1014)
`
`• English translation of JP Pub. No. 2013-106289 (“Konno”) (Ex. 1015)
`
`• Ralph E. Jacobson et al., The Manual of Photography: photographic and
`digital imaging, 9th Edition, 2000 (“Jacobson”) (Ex. 1016)
`
`• U.S. Patent App. Pub. No. 2010/0321511 (“Koskinen”) (Ex. 1017)
`
`• U.S. Patent No. 7,206,136 (“Labaziewicz”) (Ex. 1018)
`
`• Milton Katz, Introduction to Geometrical Optics (2002) (“Katz”) (Ex.
`1019)
`
`• Warren J. Smith, Modern Lens Design (1992) (“Smith”) (Ex. 1020)
`
`• The declaration of Dr. José Sasián (Ex. 1021)
`
`• U.S. Patent No. 8,908,041 (“Stein”) (Ex. 1023)
`
`• U.S. Patent No. 8,406,569 (“Segall”) (Ex. 1024)
`
`• U.S. Patent No. 8,824,833 (“Dagher”) (Ex. 1025)
`
`• U.S. Patent No. 5,546,236 (“Ogata”) (Ex. 1026)
`
`• File History for Provisional No. 61/752,515 to Stein (“Stein provisional”)
`(Ex. 1027)
`
`• Bae et al., “Defocus Magnification,” Eurographics 2007 (“Bae”) (Ex.
`1028)
`
`3
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 6
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`• Specification sheet for Sony ICX629 image sensor (“ICX629”) (Ex.
`1029)
`
`• Specification sheet for Sony ICX624 image sensor (“ICX624”) (Ex.
`1030)
`
`• Product manual for Kodak Easyshare V610 (Ex. 1033)
`
`• U.S. Patent No. 7,112,774 (“Baer”) (Ex. 1034)
`
`• Robert E. Fischer et al., Optical System Design (2008) (Ex. 1035)
`
`• The declaration of Dr. Duncan Moore (Ex. 2015)
`
`• The transcript of the 26 Jan. 2021 deposition of Dr. Fredo Durand. (Ex.
`2036)
`
`• Forsyth and Ponce, “Computer Vision: A Modern Approach” (1st ed.)
`(2003) (Ex. 2038)
`
`• The declaration of Eli Saber filed for IPR2020-0860.
`
`• District court filings, emails, and agreements concerning Apple’s evalua-
`tion of Corephotonics’ technology relevant to secondary considerations
`(Exs. 2004–2012, 2018–2023)
`
`• The declaration of Eran Kali (Ex. 2013)
`
`5.
`
`In forming the opinions set forth herein, I have considered:
`
`a. The documents listed above;
`
`b. My education, knowledge, skills, and experience in the design and devel-
`
`opment of imaging systems; and
`
`4
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 7
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`c. The level of skill of a person having ordinary skill in the art (POSITA) at
`
`the time of the effective filing dates of the ’479 patent.
`
`6.
`
`As I explain in further detail below, it is my professional and
`
`expert opinion that Apple and Dr. Durand have failed to demonstrate that any
`
`of the challenged claims of the ’479 patent were obvious, under any of the
`
`grounds or combinations of references that Apple has raised in these two IPRs.
`
`III. EDUCATIONAL AND EMPLOYMENT BACKGROUND
`
`7.
`
`As indicated in my Curriculum Vitae, attached as Exhibit 2003,
`
`I am a tenured full Professor of Computer Science in the Department of Com-
`
`puter Science at the University of Illinois at Urbana-Champaign. As an edu-
`
`cator for the past three decades, I have taught courses in computer graphics
`
`and related areas to thousands of students. I also strive to provide opportuni-
`
`ties for the general public to learn more about computing. For example, in
`
`1999 I oversaw the production of the documentary “The Story of Computer
`
`Graphics.” I also teach an open course on data visualization on Coursera that
`
`has reached over 360,000 learners worldwide since 2016.
`
`8.
`
`I serve as the Director of Online Programs for the Department of
`
`Computer Science at the University of Illinois, and oversee its Master of Com-
`
`5
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 8
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`puter Science (“MCS”) degree program. In 2016, I redesigned the online of-
`
`fering of the MCS degree program to make it more flexible and affordable for
`
`students that could not afford to leave their job to pursue a degree fulltime.
`
`Under my leadership, this degree program quickly grew to the second largest
`
`graduate program offered by the University of Illinois at Urbana-Champaign,
`
`and contributed significantly to the campus-wide proportion of underrepre-
`
`sented minorities enrolled in the institution. The tech company C3.ai found
`
`this online degree so desirable, it pays its employee’s tuition and upon com-
`
`pletion, gives them a bonus, a raise and stock options.
`
`9.
`
`I am also the Executive Associate Dean of the Graduate College
`
`of the University of Illinois at Urbana-Champaign, where I oversee the edu-
`
`cation of over 17,800 graduate students in hundreds of graduate degree pro-
`
`grams across the entire university. I recently developed a new post-
`
`baccalaureate certificate credential at Illinois to provide the recently unem-
`
`ployed with a rapid educational opportunity to transition their skills to areas
`
`of greater prosperity.
`
`10.
`
`I have been researching computer graphics since 1987, with over
`
`a hundred papers, videos, patents and other contributions to computer
`
`graphics including photographic imaging systems. My work in computer
`
`6
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 9
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`graphics has been funded by Adobe, Intel, Microsoft, Nokia and Nvidia as
`
`well as the National Science Foundation (NSF) and the Defense Advanced
`
`Research Projects Agency (DARPA). One of my most recent contribution is
`
`on the topic of displaying text on the video screen of a VR headset, in collab-
`
`oration with Oculus. This paper, “Real-Time Analytic Antialiased Text for 3-
`
`D Environments,” was selected as one of the best papers at the 2019 High-
`
`Performance Graphics Conference in Strasbourg France in July.
`
`11.
`
`I am an internationally recognized leader in the field of computer
`
`graphics. From 2002-08 I was the Editor-in-Chief of the top journal in com-
`
`puter graphics, the Association for Computing Machinery (ACM) Transac-
`
`tions on Computer Graphics. From 1994-1999 I served on the executive
`
`committee of the main organization of computer graphics practitioners, the
`
`ACM Special Interest Group on Computer Graphics and Interactive Tech-
`
`niques (SIGGRAPH). I continue to oversee the peer review of major papers
`
`in the field through service as chair and member of various paper review com-
`
`mittees. I am also a founding member of the editorial board of ACM Books,
`
`and the area editor for computer graphics.
`
`7
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 10
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`12. This report is on the subject of photographic imaging systems. I
`
`have worked on a variety of methods and systems for the fusion of photo-
`
`graphs. For example, in 2008 I was granted patent #7,365,744 “Method and
`
`Systems for Image Modification” on techniques for learning a surface appear-
`
`ance from one photograph and realistically applying it to a different surface
`
`in another photograph.
`
`13. At the priority date of the ’479 pa-
`
`tent, I was funded by the National Science
`
`Foundation’s Advanced Digitization of Biodi-
`
`versity Collections to design and deliver an im-
`
`aging infrastructure to scan the nation’s
`
`entomological collections of insect drawers.
`
`This project, available at invertnet.org, re-
`
`quired the fusion of 51,791 photographic im-
`
`ages of small portions of insect drawers, vials
`
`and slides to make the collections available via the Internet as high-resolution
`
`zoomable composite images. This effort included the design and deployment
`
`of a custom robotic photographic imaging system, designed specifically to
`
`capture and fuse numerous photographs of each specimen drawer.
`
`8
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 11
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`IV. LEVEL OF ORDINARY SKILL IN THE ART (POSITA)
`
`14.
`
`I understand that in evaluating the validity of the ’479 patent
`
`claims, the content of a patent or printed publication prior art should be inter-
`
`preted the way a person of ordinary skill in the art would have interpreted the
`
`prior art as of the effective filing date of the challenged patent.
`
`15.
`
`I understand that factors that may be considered in determining
`
`the level of ordinary skill in the art at the time of the effective filing date of
`
`the challenged patents include: (1) the educational level of the inventor; (2)
`
`type of problems encountered in the art; (3) prior art solutions to those prob-
`
`lems; (4) rapidity with which innovations are made; (5) sophistication of the
`
`technology; and (6) educational level of active workers in the field.
`
`16. Dr. Durand at ¶13 in each declaration believes “that a POSITA
`
`would include someone who had, as of the claimed priority date of the ’479
`
`Patent, a bachelor’s or the equivalent degree in electrical and/or computer en-
`
`gineering or a related field and 2-3 years of experience in imaging systems
`
`including image processing and lens design.” He further recognizes “that
`
`someone with less formal education but more experience, or more formal ed-
`
`ucation but less experience could have also met the relevant standard for a
`
`9
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 12
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`POSITA.” My opinions in reply to Dr. Durand use this definition of a
`
`POSITA.
`
`17.
`
`I understand that the ’479 patent shares a specification with and
`
`claims priority to U.S. App. No. 14/365,711 filed June 16, 2014 and issued as
`
`U.S. Patent No. 9,185,291. (Ex. 1001, ’479 patent at 1:7–29.) I understand
`
`that U.S. App. No. 14/365,711 was a § 371 application from international pa-
`
`tent application PCT/IB2014/062180 filed June 12, 2014 and is related to and
`
`claims priority from U.S. Provision Patent Application No. 61/834,486 filed
`
`June 13, 2013. (Ex. 1001, ’479 patent at 1:7–29.) I therefore understand that
`
`the effective filing date of the ’479 patent is June 13, 2013.
`
`18.
`
`I would have met the requirements of a POSITA on June 13,
`
`2013. I have used the perspective of a POSITA at that time to form my opin-
`
`ions in reply to Dr. Durand’s opinons.
`
`V. RELEVANT LEGAL STANDARDS FOR OBVIOUSNESS
`
`19.
`
`I have been informed of the legal standards for establishing pa-
`
`tent invalidity in inter partes review proceedings before the Patent Trial and
`
`Appeal Board.
`
`10
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 13
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`20.
`
`I understand that the petitioner must prove invalidity of a patent
`
`claim by a preponderance of the evidence, that is, the evidence must be suffi-
`
`cient to show that a fact or legal conclusion is more likely than not.
`
`21.
`
`I understand that a claim may be anticipated if (1) the claimed
`
`invention was patented, described in a printed publication, or in public use, on
`
`sale, or otherwise available to the public before the effective filing date of the
`
`claimed invention; or (2) the claimed invention was described in a patent or
`
`published application, in which the patent or application names another in-
`
`ventor and was effectively filed before the effective filing date of the claimed
`
`invention.
`
`22.
`
`I understand that, once the claims of a patent have been properly
`
`construed, the next step in determining anticipation of a patent claim requires
`
`a comparison of the properly construed claim language to the prior art on a
`
`limitation-by-limitation basis.
`
`23.
`
`I understand that even if a patent claim is not anticipated, it may
`
`still be invalid if the differences between the claimed subject matter and the
`
`prior art are such that the subject matter as a whole would have been obvious
`
`at the time the invention was made to a person of ordinary skill in the pertinent
`
`art.
`
`11
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 14
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`24.
`
`I also understand that a patent may be rendered obvious based on
`
`an alleged prior art reference or a combination of such references plus what a
`
`person of ordinary skill in the art would understand based on his or her
`
`knowledge and the references. It is also my understanding that in assessing
`
`the obviousness of claimed subject matter one should evaluate obviousness
`
`over the prior art from the perspective of one of ordinary skill in the art at the
`
`time the invention was made (and not from the perspective of either a layman
`
`or a genius in that art).
`
`25.
`
`I understand that a patent claim composed of several elements is
`
`not proved obvious merely by demonstrating that each of its elements was
`
`known in the prior art. There must be a reason for combining the elements in
`
`the manner claimed. That is, there must be a showing that a person of ordinary
`
`skill in the art at the time of the invention would have thought of either com-
`
`bining two or more references or modifying a reference to achieve the claimed
`
`invention.
`
`26.
`
`I understand that an obviousness determination includes the con-
`
`sideration of the following factors: (1) the scope and content of the prior art,
`
`(2) the differences between the prior art and the claims at issue, (3) the level
`
`of ordinary skill in the art, and (4) objective evidence of nonobviousness.
`
`12
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 15
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`27.
`
`I understand that, when available, so-called objective indicia of
`
`non-obviousness (also known as “secondary considerations” and or the real
`
`world factors) like the following are also to be considered when assessing
`
`obviousness: (1) widespread acclaim; (2) commercial success; (3) long-felt
`
`but unresolved needs; (4) copying of the invention by others in the field; (5)
`
`initial expressions of disbelief by experts in the field; (6) failure of others to
`
`solve the problem that the inventor solved; and (7) unexpected results, among
`
`others. I also understand that evidence of objective indicia of non-obviousness
`
`must be commensurate in scope with the claimed subject matter. I understand
`
`this is commonly referred to as a “nexus.”
`
`28.
`
`I understand that the burden is on the petitioner to explain how
`
`specific references could be combined, which combinations of elements in
`
`specific references would yield a predictable result, and how any specific
`
`combination would operate or read on the claims. I further understand that the
`
`petitioner cannot rely on conclusory statements but must instead provide a
`
`reasoned explanation supported by evidence. I also understand that obvious-
`
`ness does not exist where the prior art discourages or teaches away from the
`
`claimed invention. I also understand that even if a reference does not teach
`
`away, its statements regarding preferences are relevant to ta finding whether
`
`13
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 16
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`a person skilled in the art would be motivated to combine that reference with
`
`another reference.
`
`29.
`
`I understand that it is impermissible to use hindsight to arrive at
`
`the claimed invention. My understanding is that the inventor’s own path never
`
`leads to a conclusion of obviousness. I also understand that, when assessing
`
`whether there was a motivation to combine references to teach a claim ele-
`
`ment, defining the problem in terms of its solution reveals improper hindsight.
`
`30.
`
`I understand that, in this proceeding, prior art to the ’479 patent
`
`includes patents and printed publications in the relevant art that predate the
`
`effective filing date of the ‘479 patent’s challenged claims, which I understand
`
`to be June 13, 2013. (Ex. 1001, ’479 patent at 1:7–20.)
`
`VI. OVERVIEW OF THE ’479 PATENT
`
`31. The ’479 patent describes and claims techniques for making
`
`“thin digital cameras with optical zoom operating in both video and still
`
`mode.” (Ex. 1001, ’479 patent at 3:27–28.) As the patent explains, zoom in
`
`“commonly understood as a capability to provide different magnifications of
`
`the same scene and/or object by changing the focal length of an optical sys-
`
`tem.” (Ex. 1001, ’479 patent at 1:44–49.) Traditionally, this was accomplished
`
`by mechanically moving lens elements relative to one another. (Ex.
`
`14
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 17
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`1001, ’479 patent at 1:49–51.) Another approach is “digital zooming,” where
`
`the focal length of the lens is kept unchanged, but the image is cropped and
`
`digitally manipulated to produce an image that is magnified but has a lower
`
`resolution. (Ex. 1001, ’479 patent at 1:55–38.)
`
`32. The ’479 patent describes an approach to approximating the ef-
`
`fect of a zoom lens (which varies its focal length) by using two lens systems
`
`(a “wide” and a “tele” lens system) with different fixed focal lengths. (Ex.
`
`1001, ’479 patent at 3:34–54.) Various computational means are used to take
`
`the images from these two lenses to produce an output that approximate a
`
`system with mechanical zoom. This approach can produce a device that is
`
`smaller, lower cost, and more reliable than devices that use mechanical zoom.
`
`(Ex. 1001, ’479 patent at 1:51–53.)
`
`(Ex. 1001, ’479 patent, Fig. 1B)
`
`
`
`15
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 18
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`33. Relevant to the claims of the ’479 patent, the specification de-
`
`scribes combining still images using the technique of “fusion.” (Ex.
`
`1001, ’479 patent at 3:48–54.) A “fused” image includes information from
`
`both the wide and tele images. (Id.) One approach to performing fusion is
`
`shown in Figure 5:
`
`(Ex. 1001, ’479 patent, Fig. 5.)
`
`
`
`16
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 19
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`34. Making a compact, high-quality dual-aperture zoom system re-
`
`quires lenses with particular characteristics. The ’479 patent teaches lens de-
`
`signs for the tele lens which provide a small “total track length” relative to
`
`their focal length, which means that they have a compact size in light of the
`
`degree of magnification that they provide. (Ex. 1001, ’479 patent at 12:38–
`
`53.) One of the lens designs taught by the ’479 patent and covered by several
`
`of the challenged claims is shown in Figure 9:
`
`
`
`(Ex. 1001, ’479 patent, Fig. 9.)
`
`35. The lens aspects of the ’479 patent are described further in Dr.
`
`Moore’s declaration. (E.g., Ex. 2015, Moore Decl., ¶¶ 31–34.)
`
`17
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 20
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`VII. CLAIM CONSTRUCTION
`
`A.
`
`“fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23)
`
`36. Dr. Durand’s opinion is that this term should be construed as “a
`
`fused image that maintains the Wide camera’s field of view or both the Wide
`
`camera’s field of view and position.” (Ex. 1003 from -00905 IPR, ¶¶ 29–33.)
`
`I do not agree that is the understanding of this term to a POSITA, in view of
`
`the ’479 patent.
`
`37. Under this construction there are two ways to meet the “point of
`
`view” requirement. Either, the fused image can maintain the Wide camera’s
`
`(a) field of view or (b) field of view and position. However, the second of
`
`these two options is superfluous, as if the image has both the field of view and
`
`position of the Wide camera, then it also necessarily has the field of view of
`
`the Wide camera. So, Dr. Durand’s construction is logically equivalent to the
`
`construction “a fused image that maintains the Wide camera’s field of view.”
`
`38. Even the superfluous “Wide camera’s . . . position” portion of
`
`the construction does not line up with the term “position” as it is used in
`
`the ’479 patent’s discussion of “POV.” During his deposition, Dr. Durand
`
`confirmed that he understood the “Wide camera’s . . . position” to refer to the
`
`“3D XYZ location of the camera.” (Ex. 2036, Durand Depo. at 21:3–7.) But
`
`18
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 21
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`when the specification refers to “position POV” in its discussion of “combi-
`
`nation” POVs, it is referring to the “position of either sub-camera image.” (Ex.
`
`1001, ’479 patent at 5:14–16.) That is, “position POV” is based on the posi-
`
`tions of images, not the positions of cameras. An image position may differ
`
`because the camera is located in a different position, but it also may differ
`
`because camera, located in the same position, has been pointed in a different
`
`direction. For this reason as well, Dr. Durand’s proposed construction is in-
`
`consistent with how the patent specification uses the relevant terms.
`
`39. The effect of Dr. Durand’s construction is to replace the term
`
`“point of view” in the claims with the term “field of view” in his construction.
`
`This is not consistent with how a POSITA would understand these phrases or
`
`with how they are used in the ’479 patent. For example, claim 1 refers to both
`
`“a field of view FOVW” of the wide camera and “a point of view (POV)” of
`
`the wide camera, with no suggestion they are the same thing or that one term
`
`is the antecedent basis for the other. (Ex. 1001, ’479 patent at 13:25–26,
`
`13:48.)
`
`40.
`
`In the specification, the ’479 patent clearly defines “FOV” as a
`
`planar angle, representable in degrees: “As used herein, the FOV is measured
`
`from the center axis to the corner of the sensor (i.e. half the angle of the normal
`
`19
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 22
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`definition).” (Ex. 1001 at 7:11–13.) Examples of FOV values are given in de-
`
`grees (id. at 7:20–22), and FOV is used as a parameter to the tangent function,
`
`further confirming that it is a simple angle (id. at 7:7–8).
`
`41. Dr. Durand agreed during his deposition that his construction of
`
`“POV” matches what the patent calls field of view:
`
`Q. So when you’re using the term “field of view” in this con-
`struction, you’re reviewing -- you're referring to how much of
`the scene is captured by the camera; is that right?
`
`A. This is a vague version of the definition, I would say one
`definition of the field of view. For example, the horizontal field
`of view is to look at the angle between the two edges of the --
`of the image.
`
`(Ex. 2036 at 22:4–12.)
`
`42. This is definition matches what the ’479 patent specification calls
`
`the “normal definition” of FOV (the ’479 patent uses half that “normal” value
`
`in its formulas). (Ex. 1001, ’479 patent at 7:11–13.) As Dr. Durand testified,
`
`this FOV is an inherent property of the camera and lens, and independent of
`
`where they are pointed or what they see:
`
`Q. Would you agree that a camera’s field of view is a property
`of the camera that’s independent of what direction the camera is
`pointing?
`
`A. So one definition or understanding of field of view would be
`-- would indeed be just an angle that’s a property of the combi-
`nation of a camera and the lens.
`
`20
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 23
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`(Ex. 2036 at 22:25–23:6.)
`
`43. POV is defined in the specification quite differently. It refers to
`
`how objects are “seen by each sub-camera,” i.e., how objects “with be shifted
`
`and have different perspective (shape)” for the two cameras. (Ex. 1001 at
`
`5:10–14.) This POV depends on the position and orientation of the camera
`
`and cannot be expressed fully by a single numerical angle. Rather, as the ’479
`
`patent explains, using a camera with a different POV can both shift an object
`
`(change its position in the image) and change the perspective of an object
`
`(changes its apparent shape in the image). (Ex. 1001 at 5:10–16.)
`
`44. Examples of changing POV can be seen in image pairs (a)-(b)
`
`and (d)-(e) from Szeliski Figure 1.1:
`
`21
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 24
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`
`
`(Ex. 1013, Szeliski at 468.)
`
`45. The ’479 patent refers to “combination” possibilities where an
`
`output image reflects only some aspects of a given POV, such as “Wide per-
`
`spective POV” or “Wide position POV.” (Ex. 1001, ’479 patent at 5:15–19.)
`
`But, when it refers to “Wide POV,” without qualification, it is referring to the
`
`complete Wide POV, both perspective and position. (Ex. 1001, ’479 patent at
`
`5:10–14; 5:23–26.)
`
`46.
`
`In summary, a POSITA would not agree that the term POV in
`
`the phrase “fused image with a point of view (POV) of the Wide camera” can
`
`22
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2001
`Page 25
`
`
`
`Case Nos. IPR2020-00905, IPR2020-00906
`U.S. Patent No. 10,225,479
`
`be replaced with the distinct term FOV. Further, a POSITA would understand
`
`that POV of the Wide camera in this phrase refers to the full Wide camera
`
`POV and not to “combination” outputs that have a Wide “perspective POV”
`
`and Tele “position POV” or vice versa. (Ex. 1001, ’479 patent at 5:13–23.) In
`
`my opinion a POSITA would understand this term to mean “fused image in
`
`which the positions and shapes of objects reflect the POV of the Wide cam-
`
`era.”
`
`B.
`
`“to find translations between matching points in the images to
`calculate depth information and to create a fused image suited
`for portrait photos” (claim 19)
`
`47. Dr. Durand’s opinion is that this term appearing in
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
