`
`Paper No. 32
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`____________
`
`
`PATENT OWNER’S SUR-REPLY
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`TABLE OF CONTENTS
`
`“fused image with a point of view (POV) of the Wide camera”
`
`Petitioner’s Does Not Demonstrate Unpatentability Under the
`
`Claims 1, 10–14, 16, 18, 23, 32–36, 38, and 40 Are Not Obvious
`
`INTRODUCTION .................................................................. 1
`I.
`CLAIM CONSTRUCTION ..................................................... 1
`II.
`A.
`(claims 1 and 23) ............................................................................. 1
`III. PATENTABILITY OF CHALLENGED CLAIMS .................... 5
`A.
`Correct Claim Construction ............................................................ 5
`B.
`Over the Combination of Parulski and Konno (Ground 1) ............. 6
`C.
`Grounds 2, 3 and 4 Also Do Not Establish Unpatentability ........... 8
`D.
` ......................................................................................................... 9
`CONCLUSION .................................................................... 10
`
`Secondary Considerations / Objective Indicia of Non-Obviousness
`
`II.
`
`
`
`
`
`
`i
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Exhibit No.
`2001
`2002
`2003
`2004
`
`2005
`
`2006
`
`2007
`2008
`2009
`2010
`2011
`2012
`2013
`2014
`
`2015
`2016
`2017
`2018
`2019
`2020
`2021
`2022
`2023
`2024
`2025
`2026
`2027
`
`Patent Owner’s Exhibit List
`
`
`Description
`Declaration of John C. Hart, Ph.D.
`Fredo Durand, Presentation Titled “Photography 101”
`Curriculum Vitae of John C. Hart, Ph.D.
`Complaint for Patent Infringement, Dkt. No. 1, Case No.
`19-cv-4809 (United States District Court, Northern Dis-
`trict of California)
`Answer to Complaint for Patent Infringement, Dkt. No.
`17, Case No. 19-cv-4809 (United States District Court,
`Northern District of California)
`Corephotonics Proposal: “Dual Aperture Image Fusion
`Technology, Proposed Engagement Framework” (June 22,
`2014)
`Email chain with emails dating from July and August 2014
`Email chain with emails dating from March 2015
`Email dated December 21, 2015
`Email chain with emails dating from August 2016
`Email dated May 23, 2013
`Email dated May 23, 2013
`Declaration of Eran Kali
`Transcript of January 21, 2021 Video-Recorded Deposi-
`tion of Fredo Durand, Ph.D.
`Declaration of Duncan Moore, Ph.D.
`Rudolf Kingslake, “Optics in Photography” (1992)
`Curriculum Vitae of Duncan Moore, Ph.D.
`Email chain with emails dating from June and July 2013
`Email chain with emails dating from June and July 2013
`Email chain with emails dating from October 2013
`Technology Evaluation Agreement dated August 8, 2013
`Email chain with emails dating from September 18, 2013
`Email dated May 21, 2014
`Reserved
`Reserved
`Deposition transcript of José Sasián, November 9, 2020
`José Sasián, Introduction to Lens Design (2019), hardcopy
`
`ii
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`2028
`2029
`
`2030
`
`2031
`
`2032
`
`2033
`
`2034
`
`2035
`2036
`
`2037
`2038
`
`2039
`
`2040
`
`2041
`
`Tigran V. Galstian, Smart Mini-Cameras (2014)
`Dmitry Reshidko and Jose Sasián, “Optical analysis of
`min- iature lenses with curved imaging surfaces,” Applied
`Optics, Vol. 54, No. 28, E216-E223 (October 1, 2015)
`José Sasián, Introduction to Aberrations in Optical Imag-
`ing Systems (2013), hardcopy
`Yufeng Yan and Jose Sasián, “Miniature Camera Lens De-
`sign with a Freeform Surface,” Design and Fabrication
`Congress (2017)
`Peter Clark, “Mobile platform optical design,” Proc. SPIE
`9293, International Optical Design Conference 2017,
`92931M (17 December 2014)
`Jane Bareau and Peter P. Clark, “The Optics of Miniature
`Digital Camera Modules,” SPIE Vol. 6352, International
`Op- tical Design Conference 2006, 63421F.
`Yufeng Yan, “Selected Topics in Novel Optical Design,”
`Ph.D. Dissertation (2019)
`Declaration of Jose Sasián, Ph.D. from IPR2020-00489
`Transcript of January 26, 2021 Video-Recorded Deposi-
`tion of Fredo Durand, Ph.D.
`U.S. Patent No. 8,989,517 (“Morgan-Mar”)
`Forsyth and Ponce, “Computer Vision: A Modern Ap-
`proach” (1st ed.) (2003)
`Declaration of Marc A. Fenster in Support of Motion to
`Appear Pro Hac Vice on Behalf of Patent Owner Corepho-
`tonics, Ltd.
`Declaration of James S. Tsuei in Support of Motion to Ap-
`pear Pro Hac Vice on Behalf of Patent Owner Corephoton-
`ics, Ltd.
`Transcript of June 8, 2021 Video-Recorded Deposition of
`Frédo Durand, Ph.D.
`
`
`
`
`
`iii
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`I.
`
`INTRODUCTION
`
`On Reply, Petitioner fails to rebut the core arguments made in Corepho-
`
`tonics’ response. Petitioner even modifies it construction of a critical claim
`
`limitation. Even with this improper revision, it continues to rely on an unten-
`
`able construction for a critical claim phrase to improperly broaden the scope
`
`of the claims to better match its asserted prior art. Petitioner also fails address
`
`the motivation of a person of ordinary skill in the art (“POSITA”) to combine
`
`the various embodiments of Parulski and Konno.
`
`Accordingly, the Board should deny the Petition and find the challenged
`
`claims not unpatentable.
`
`II. CLAIM CONSTRUCTION
`
`A. “fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23)
`
`Corephotonics’ Construction
`
`Apple’ Construction
`
`“fused image in which the positions
`and shapes of objects reflect the
`POV of the Wide camera.”
`
`“a fused image that maintains the
`Wide camera’s field of view or both
`the Wide camera’s field of view and
`position.”
`
`
`
`In its Reply, Petitioner, and its expert Dr. Durand, appear to concede that
`
`its original construction in the Petition is non-sensical. Ex. 1003 at ¶¶ 6-8. In
`
`1
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`an effort to cure this, Petitioner is now offering up a new construction not
`
`previously provided or analyzed. Petitioner now argues that the a “fused im-
`
`age with a point of view (POV) of the Wide camera” should be construed to
`
`mean a “fused image in which the positions or shapes of objects reflect those
`
`of the Wide camera.” Reply at 6; Ex. 1038 at ¶8. Neither Petitioner nor Dr.
`
`Durand proffered this construction in the Petition and supporting declaration.
`
`As such, this new analysis is improper and should not be permitted.
`
`Even if it is permitted, Petitioner’s new construction is still wrong. Peti-
`
`tioner continues to misinterpret the ’479 patent.1 The ’479 patent defines point
`
`of view (“POV”) as requiring matching both shape and position. Ex. 1001,
`
`5:11-13. The ’479 patent then describes “position POV” and “perspective
`
`POV.” Id. at 5:13-20. While the concepts of “position POV” and “perspective
`
`POV” are related, nothing in the ’479 patent discloses that one of these by
`
`itself is actually POV, as Petitioner’s construction would require. In fact, the
`
`’479 patent explicitly states that the “system output image can have the shape
`
`and position of either sub-camera image or the shape or position of a
`
`
`1 In the claim construction arguments, both the Petitioner and Dr. Durand’s
`Declaration erroneously cite to APPL-1005, which is the Parulski reference.
`Based on the quoted language, it appears those citations should be to APPL-
`1001, which is the ‘479 patent.
`
`2
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`combination thereof.” Id. at 5:14-16. That is, both shape (perspective) and
`
`position are necessary. This is consistent with Patent Owner’s construction.2
`
`With its misunderstanding of the teachings of the ’479 patent, Petitioner
`
`further argues that the ’479 patent’s description of position POV is actually
`
`the POV. Reply at 3-4. This is incorrect, as it ignores that the registering pixels
`
`to matching pixels will necessarily address both position (shift) and perspec-
`
`tive (shape).
`
`To support this argument, Petitioner relies heavily on the teaching in the
`
`’479 patent that registration can be performed “after either sub-camera image
`
`is shifted, in which case the output image will retain the respective Wide or
`
`Tele perspective POV.” Id., 5:30-33 (emphasis added). Petitioner asserts that
`
`because “perspective POV” is explicitly recited here, the previous recitation
`
`of registration does not address perspective POV. Reply at 3-4. But this inter-
`
`pretation ignores the logical construction of this sentence. The ’479 patent, in
`
`the sentence identified by Petitioner above, certainly describes shifting and
`
`
`2 Petitioner’s argument that this construction is improper because it excludes
`a preferred embodiment is wrong. Patent Owner’s construction is consistent
`with the specification and the embodiments described therein. Further, Peti-
`tioner has not demonstrated that any particular embodiment is a “preferred
`embodiment.” Rather, the ‘479 patent contemplates numerous embodiments,
`of which some or all may be covered by a particular claim.
`
`3
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`then registering as addressing perspective. Ex. 1001, 5:30-33. But that cannot
`
`mean, as Petitioner argues, that registering without shifting only addresses
`
`position and not perspective. At best, Petitioner’s argument logically would
`
`mean that shifting, without registering, would not address perspective. But
`
`Petitioner fails to identify any portion of the ’479 patent that even describes
`
`such a scenario. And this conclusion, even if accurate, does not support Peti-
`
`tioner’s claim construction.
`
`Petitioner’s arguments ignore the correct way to interpret these portions
`
`of the ’479 patent – that the examples in the ’479 patent at col. 5:26-33 are
`
`just a parallel to the examples at col. 5:20-23. More specifically, col. 5:26-30
`
`describes a scenario where the the unmodified position of either the wide or
`
`the tele is used, thereby matching both position and perspective POV of that
`
`camera. This is no different than the description in the ‘479 patent that “the
`
`perspective POV may be of the Wide or Tele sub-cameras” as described at col.
`
`5:21-22.
`
`Further to that, col. 5:30-33 describes the scenario where images are
`
`shifted to create a position POV that is intermediate between the two cameras.
`
`Again, this is the same as the teaching of the ’479 patent that “the position
`
`POV may shift continuously between the Wide and Tele sub-cameras” as
`
`4
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`described at col. 5:22-23. While this may be very desirable, the resulting po-
`
`sition does not match the position POV of a sub-camera. However, the result-
`
`ing image does have the perspective POV of one of the cameras.
`
`In the context of the ’479 patent specification – and thus the claims -
`
`“POV” without qualification means matching both position and shape. Peti-
`
`tioner’s construction is wrong and based on a fundamental misreading of the
`
`specification of the ‘479 patent. Therefore, the “fused image with a point of
`
`view (POV) of the Wide camera” limitation of claims 1 and 23 of the ’479
`
`patent should be construed as a “fused image in which the positions and
`
`shapes of objects reflect the POV of the Wide camera.”
`
`III. PATENTABILITY OF CHALLENGED CLAIMS
`
`A.
`
`Petitioner’s Does Not Demonstrate Unpatentability Under the
`Correct Claim Construction
`
`As an initial matter, Dr. Durand admits that he and Petitioner have offered
`
`no argument that the prior art would satisfy the claims of the ’479 patent under
`
`Patent Owner’s claim construction. Ex. 2041, 52:25-54:20 (“I have not pro-
`
`vided an opinion about whether Parulski satisfies Dr. Hart’s claim construc-
`
`tion.”) Without such an argument, and when using the proper construction of
`
`the claims as set forth by the Patent Owner, Petitioner’s arguments regarding
`
`the unpatentability of the ’479 patent must necessarily be rejected.
`
`5
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`In addition, as noted above, Petitioner is now offering up a new construc-
`
`tion, arguing that a “fused image with a point of view (POV) of the Wide
`
`camera” should be construed to mean a “fused image in which the positions
`
`or shapes of objects reflect those of the Wide camera.” Reply at 6; Ex. 1038
`
`at ¶8. Petitioner and Dr. Durand did not offer this construction in the Petition
`
`and supporting declaration. Petitioner and Dr. Durand did not previously ap-
`
`ply this construction to the claims. Therefore, as previously noted, this new
`
`construction and analysis is improper and should not be permitted in its Reply.
`
`B. Claims 1, 10–14, 16, 18, 23, 32–36, 38, and 40 Are Not Obvious
`Over the Combination of Parulski and Konno (Ground 1)
`
`Petitioner further admits that its Petitioner relies on hindsight and to com-
`
`bine cherry-picked portions of embodiments of Parulski to create an embodi-
`
`ment that Parulski neither disclosed nor preferred. Reply at 7-8. Instead,
`
`Petitioner doubles-down by arguing that a POSITA would have had the “com-
`
`mon sense, common wisdom and common knowledge” to yield the claim in-
`
`vention. Id. at 8. However, this argument ignores that a POSITA would need
`
`motivation to do so. This motivation is missing from the Petition, and Peti-
`
`tioner’s improper attempts to provide that motivation now should be ignored.
`
`Petitioner argues that a range map “is the only method taught by Parul-
`
`ski” for identifying portions of an image positioned near a focal distance.
`
`6
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Reply at 8-9. However, that fact does not mean a POSITA would necessarily
`
`use that method in different embodiments. Rather, a POSITA would need to
`
`be motived to use that specific methodology depending on the characteristics
`
`of a particular embodiment. Petitioner ignores this.
`
`The Petition argued that an image that has both the dog and the moun-
`
`tains in focus, with the intermediate parts of the scene blurred, (Ex. 1005 at
`
`21:40–44) would provide such a motivation to use the range map in other em-
`
`bodiments of Parulski. However, as noted in Patent Owner’s Response, this
`
`could be achieved using image data from the wide image, without any need
`
`for importing image data from the tele image. The dog in Parulski is described
`
`as being “5 feet away.” Id. at 21:12–13. Parulski further teaches that a single
`
`wide angle lens, set to its hyperfocal distance, will have objects from 4 feet to
`
`infinity in focus. Id. at 21:59–61; see also Ex. 2001, Hart Decl., ¶ 76.
`
`Petitioner asserts that this argument fails because “it does not consider
`
`the combination with Konno’s wide lens (which Patent Owner does not dis-
`
`pute) which, when similarly focused at 8 feet, has a smaller depth of field of
`
`about 6 feet to infinity.” Reply at 11. But this would mean a POSITA would
`
`alter the teachings in Parulski to create an embodiment with a smaller depth
`
`of field, and more nearby objects would be out of focus. Thus, Petitioner’s
`
`7
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`modification would create a system with lesser performance results than it
`
`originally had because fewer objects would be in focus up close.
`
`Further, because the dog in the example would be out of focus, additional
`
`steps of sharpening using the range-map and the tele image to bring the dog
`
`into focus would now be required. Reply at 11. Even in its Reply, Petitioner
`
`does not explain why a POSITA would do this. That is because there is no
`
`reason why a POSITA would alter Parulski to change its functions such that it
`
`has a smaller depth of field, thereby degrading performance, and then requir-
`
`ing additional steps to achieve what it does as taught. A POSITA having “com-
`
`mon sense, common wisdom and common knowledge” would not do this, and
`
`Petitioner knows it.
`
`For these reasons, Petitioner’s arguments should be rejected and claims
`
`1, 10–14, 16, 18, 23, 32–26, 38, and 40 should not be found unpatentable.
`
`C. Grounds 2, 3 and 4 Also Do Not Establish Unpatentability
`
`Ground 2 (claims 2-4 and 24-26), Ground 3 (5-9 and 27-31), and Ground
`
`4 (claims 15 and 37) are all based on the combination of Parulski, Konno and
`
`one or more additional prior art references added to address limitations in
`
`these dependent claims. None of these additional prior art references cure the
`
`8
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`deficiencies in Parulski and Konno. Therefore, for the same reasons, these
`
`grounds should be denied and the claims found not unpatentable.
`
`D.
`
`Secondary Considerations / Objective Indicia of Non-Obvious-
`ness
`
`The Reply’s response to Patent Owner’s extensive evidence of secondary
`
`considerations fails to rebut Patent Owner’s demonstration of non-obvious-
`
`ness.
`
`First, Petitioner asserts that Patent Owner is not entitled to a presumption
`
`of nexus because it first needs to show that its objective evidence is “coexten-
`
`sive with … any specific feature of the challenged claims.” Reply, 19. Peti-
`
`tioner argues that there is no evidence of “specific features of the claimed
`
`dual-aperture camera implementing the claimed fusion methods.” But Peti-
`
`tioner does not deny it vigorously pursued Patent Owner’s “image fusion al-
`
`gorithms” and “wide and tele camera image data” technology and the ability
`
`to license all of its intellectual property. These technology features are specific
`
`to the challenged claims.
`
`Second, Petitioner argues that Patent Owner’s use of secondary consid-
`
`erations with respect to multiple patents undermines Patent Owner’s reliance
`
`on the secondary considerations of non-obviousness regarding these patents.
`
`This ignores that more than one patent in a portfolio may have value. Under
`
`9
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Petitioner’s theory, only one patent could ever be the basis of a secondary
`
`consideration of nonobvious. This ignores that Petitioner specifically asked
`
`Patent Owner for the ability to license all of Corephotonics IP, and asked for
`
`and received samples of Corephotonics’ image fusion algorithm. See Exs.
`
`2007, 2011, 2012, 2018, 2019, 2020, 2021, 2022; Ex. 2013 Kali Decl., at ¶¶
`
`17-30. Thus, the technical features claimed in the ’479 patent are specifically
`
`what Petitioner requested to evaluate and license.
`
`Petitioner also refers to much of the industry praise as “self-serving,”
`
`dismissing it coming from investors, business partners and Patent Owner’s
`
`press releases. But this hand waives the fact that other companies entered into
`
`a business relationship with Patent Owner because of the patented technology.
`
`Petitioner’s position would require one to ignore the very actions of compa-
`
`nies who “put their money where their mouth is” regarding the claimed tech-
`
`nology of the ’479 patent.
`
`II. CONCLUSION
`
`For the reasons set forth above, Corephotonics respectfully requests that
`
`the Board affirm the validity of claims 1–16, 18, 23–36–38, and 40 of the ’479
`
`patent.
`
`10
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`
`
`Dated: June 22, 2021
`
`Respectfully submitted,
`
`
`
` /Neil A. Rubin/
`Neil A. Rubin (Reg. No. 67,030)
`RUSS AUGUST & KABAT
`12424 Wilshire Boulevard, 12th Floor
`Los Angeles, CA 90025
`Telephone: 310-826-7474
`
`Attorney for Patent Owner,
`COREPHOTONICS, LTD.
`
`11
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`CERTIFICATE OF WORD COUNT
`
`I certify that there are 2156 words in this paper, excluding the portions ex-
`
`empted under 37 C.F.R. § 42.24(a)(1), according the word count tool in Mi-
`
`crosoft Word.
`
`
`
`
` /Neil A. Rubin/
`Neil A. Rubin (Reg. No. 67,030)
`
`
`12
`
`
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`CERTIFICATE OF SERVICE
`
`I hereby certify that “Patent Owner’s Sur-Reply,” and accompanying exhibits
`
`was served on June 22, 2021 by email sent to:
`
`Michael S. Parsons
`Andrew S. Ehmke
`Jordan Maucotel
`Bethany Love
`Stephanie N. Sivinski
`HAYNES AND BOONE, LLP
`2323 Victory Ave. Suite 700
`Dallas, TX 75219
`Telephone: 214-651-5000
`Email: michael.parsons.ipr@haynesboone.com
`Email: andy.ehmke.ipr@haynesboone.com
`Email: jordan.maucotel.ipr@haynesboone.com
`Email: andy.ehmke.ipr@haynesboone.com
`Email: andy.ehmke.ipr@haynesboone.com
`
`David W. O’Brien
`Hong Shi
`HAYNES AND BOONE, LLP
`600 Congress Ave. Suite 1300
`Austin, TX 78701
`Telephone: 512-867-8400
`Email: david.obrien.ipr@haynesboone.com
`Email: hong.shi.ipr@haynesboone.com
`
`
`
` /Neil A. Rubin/
`
`
`
`
`13
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
