`
`
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`_______________
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`_____________
`
`APPLE INC.,
`Petitioner
`v.
`COREPHOTONICS, LTD.,
`Patent Owner
`———————
`
`IPR2020-00905
`U.S. Patent 10,225,479
`_______________
`
`
`PETITIONER’S REPLY
`
`

`

`
`
`
`Petitioner’s Reply
`
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`Table of Contents
`
`I.
`II.
`
`2.
`
`Introduction ...................................................................................................... 1
`Claim Construction .......................................................................................... 1
`A.
`“fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23) ................................................................................... 1
`III. Obviousness ..................................................................................................... 6
`A.
`Patent Owner’s expert, Dr. Hart, applies the wrong standard for a
`POSITA in his obviousness analysis. .................................................... 6
`Claims 1, 10-14, 16, 18, 23, 32-36, 38, and 40 are obvious over the
`combination of Parulski and Konno. ..................................................... 7
`1.
`A POSITA would have implemented Parulski’s Fig. 14
`method to output a combined image with a broadened depth
`of field by using Parulski’s range mapping method in Fig. 11
`to identify and extract objects. .................................................... 7
`Parulski teaches outputting a “fused image with a point of
`view (POV) of the Wide camera” when this term is properly
`construed to include Wide position POV. ................................11
`Claims 2-4 and 24-26 are obvious over the combination of Parulski,
`Konno, and Szeliski. ............................................................................14
`Claims 5-9 and 27-31 are obvious over the combination of Parulski,
`Konno, Szeliski, and Segall .................................................................15
`Claims 15 and 37 are obvious over the combination of Parulski,
`Konno, and Stein. ................................................................................16
`IV. Secondary Considerations .............................................................................18
`A. No nexus. .............................................................................................18
`1.
`Patent Owner is not entitled to a presumption of nexus. ..........19
`2.
`Patent Owner fails to prove nexus. ...........................................20
`Praise/licensing lacks nexus and is self-serving. ................................23
`B.
`Patent Owner did not show commercial success. ...............................25
`C.
`D. No failure of others..............................................................................25
`E.
`No evidence of copying. ......................................................................26
`- i -
`
`B.
`
`C.
`
`D.
`
`E.
`
`
`
`

`

`
`
`
`Petitioner’s Reply
`
` IPR2020-00905 (Patent No. 10,225,479)
`
`V.
`Conclusion .....................................................................................................27
`VI. Certificate of Word Count .............................................................................28
`CERTIFICATE OF SERVICE ............................................................................29
`
`
`
`
`
`
`
`
`- ii -
`
`

`

`
`
`
`Petitioner’s Reply
`
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`PETITIONER’S EXHIBIT LIST
`Updated: May 7, 2021
`
`APPL-1001 U.S. Patent No. 10,225,479 to Shabtay et al. (the “’479 Patent”)
`
`APPL-1002 Prosecution File History of the ’479 Patent (the “’242 App”)
`
`APPL-1003 Declaration of Dr. Fredo Durand Ph.D.
`
`APPL-1004 CV of Dr. Fredo Durand
`
`APPL-1005 U.S. Patent No. 7,859,588 to Parulski et al. (“Parulski”)
`
`APPL-1006 Used in co-filed Petition
`
`APPL-1007 Used in co-filed Petition
`
`APPL-1008 Used in co-filed Petition
`
`APPL-1009 Used in co-filed Petition
`
`APPL-1010 Used in co-filed Petition
`
`APPL-1011 Used in co-filed Petition
`
`APPL-1012 Used in co-filed Petition
`APPL-1013 Richard Szeliski, COMPUTER VISION – ALGORITHMS AND
`APPLICATIONS (2011) (“Szeliski”)
`APPL-1014 Used in co-filed Petition
`JP Pub. No. 2013-106289 to Konno et al. (“Konno”), Certified
`English translation and Original
`APPL-1016 Ralph E. Jacobson et al., The Manual of Photography:
`photographic and digital imaging, 9th Edition, 2000 (“Jacobson”)
`APPL-1017 U.S. Patent App. Pub. No. 2010/0321511 to Koskinen et al.
`(“Koskinen”)
`
`APPL-1015
`
`
`
`- iii -
`
`

`

`
`
`
`Petitioner’s Reply
`
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`APPL-1018 U.S. Patent No. 7,206,136 to Labaziewicz et al. (“Labaziewicz”)
`APPL-1019 Used in co-filed Petition
`APPL-1020 Warren J. Smith, MODERN LENS DESIGN (1992) (“Smith”)
`
`APPL-1021 Declaration of Dr. Jose Sasián, Ph.D.
`
`APPL-1022
`
`ZEMAX Development Corporation, ZEMAX Optical Design
`Program User’s Manual, February 14, 2011 (“ZEMAX User’s
`Manual”)
`
`APPL-1023 U.S. Patent No. 8,908,041 to Stein et al. (“Stein”)
`
`APPL-1024 U.S. Patent No. 8,406,569 to Segall et al. (“Segall”)
`
`APPL-1025 U.S. Patent No. 8,824,833 to Dagher et al. (“Dagher”)
`
`APPL-1026 Used in co-filed Petition
`APPL-1027 File History for Provisional No. 61/752,515 to Stein (“Stein
`provisional”)
`
`APPL-1028 Used in co-filed Petition
`
`APPL-1029 Used in co-filed Petition
`
`APPL-1030 Used in co-filed Petition
`
`APPL-1031 Product announcement for Sony ICX612 12 MP image sensor
`
`APPL-1032 Product announcement for Sony ICX652 13.5 MP image sensor
`
`APPL-1033 Used in co-filed Petition
`
`APPL-1034 U.S. Patent No. 7,112,774 to Baer
`
`APPL-1035 Robert E. Fischer et al., OPTICAL SYSTEM DESIGN (2008)
`APPL-1036 Email from Patent Owner’s counsel authorizing electronic
`service
`
`
`
`- iv -
`
`

`

`
`
`
`Petitioner’s Reply
`
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`APPL-1037 Deposition transcript of John Hart, Ph.D., April 29, 2021
`APPL-1038 Declaration of Dr. Fredo Durand Ph.D. in support of Petitioner’s
`Reply
`
`APPL-1039 Used in co-filed Petition
`
`APPL-1040 Used in co-filed Petition
`
`APPL-1041 Used in co-filed Petition
`
`
`
`- v -
`
`

`

`
`
`
`
`I.
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`Introduction
`For the reasons in the Petition and below, the challenged claims of the ’479
`
`Patent are unpatentable. Patent Owner relies on an incorrect claim construction that
`
`improperly imports a limitation from the specification and a POSITA with an
`
`automaton-like level of knowledge, skill, and creativity to defend against the
`
`detailed reasons to combine the prior art. Petitioner therefore requests that the
`
`Board find the challenged claims unpatentable.
`
`II. Claim Construction
`“fused image with a point of view (POV) of the Wide camera”
`A.
`(claims 1 and 23)
`
`Patent Owner incorrectly construes “fused image with a point of view (POV)
`
`of the Wide camera” as “fused image in which the positions and shapes of objects
`
`reflect the POV of the Wide camera.” Response at 13. As an initial matter, Patent
`
`Owner’s construction redundantly replaces “with a point of view (POV)” with “in
`
`which the positions and shapes of objects reflect the POV.” This proposal
`
`unhelpfully fails to provide any meaning to the construed term “point of view
`
`(POV),” as the construction repeats the term within the construction. Patent Owner
`
`asserts that “POV” itself refers to a particular position of objects in the image and
`
`perspective/shape of objects in an image. Response at 12. Thus, a more appropriate
`
`
`
`- 1 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`construction under Patent Owner’s theory would be “fused image in which the
`
`positions and shapes of objects reflect those of the Wide camera.” APPL-1038, ¶2.
`
`But Patent Owner’s construction, even when clarified to remove the
`
`redundancy, is still incorrect because it uses the word “and” instead of the word
`
`“or.” Id., ¶3. Specifically, Patent Owner’s construction erroneously requires a
`
`fused image in which both object positions and shapes reflect those of the Wide
`
`camera, which is not required by the claim language and conflicts with the
`
`specification. Id. The ’479 patent refers to two different types of Wide POV:
`
`“Wide position POV” describes objects maintaining the same position as in the
`
`wide image, and “Wide perspective POV” describes objects maintaining the same
`
`shape. Id.; APPL-1005, 5:16-19 (“If the output image retains the Wide image
`
`shape then it has the Wide perspective POV. If it retains the Wide camera position,
`
`then it has the Wide position POV.”). Critically, maintaining either one of those
`
`types of Wide POV can satisfy the claim language, which requires only “a” point
`
`of view (POV) of the Wide camera. APPL-1038, ¶3. Patent Owner admits that the
`
`‘479 patent contemplates embodiments in which the output image reflects only the
`
`“Wide position POV” or “Wide perspective POV”. Response at 13.
`
`Patent Owner incorrectly argues, however, that when the ’479 patent “refers
`
`to ‘Wide POV,’ without qualification, it is referring to the complete Wide POV,
`
`
`
`- 2 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`both perspective and position.” Response at 13 (emphasis added). Patent Owner is
`
`wrong. The ‘479 Patent discloses two alternative embodiments of fusion
`
`maintaining Wide POV: one that maintains both Wide perspective POV and Wide
`
`position POV, and one that maintains only the Wide position POV. APPL-1038,
`
`¶4. Patent Owner’s construction erroneously excludes the latter embodiment. Id.
`
`Specifically, the ’479 Patent discusses “Wide POV” in the context of fusion as the
`
`fused image obtained by registering Tele image pixels to a matching pixel set with
`
`in the Wide image pixels, maintaining only the Wide position POV: “it is possible
`
`to register Telephoto image pixels to a matching pixel set within the Wide image
`
`pixels, in which case the output image will retain the Wide POV (‘Wide fusion’).”
`
`APPL-1005, 5:23-26. While this sentence refers to “Wide POV,” the specification
`
`continues on to confirm that this embodiment maintains only Wide position POV.
`
`Only when an additional function is performed (the Telephoto image is “shifted”
`
`prior to registration mapping) is the Wide perspective POV also retained: “It is also
`
`possible to perform the registration after either sub-camera image is shifted, in
`
`which case the output image will retain the respective Wide or Tele perspective
`
`POV.” Id., 5:30-33 (emphasis added), 5:14-16 (“[t]he system output image can
`
`have the shape and position of either sub-camera image or the shape or position of
`
`a combination thereof.”). In other words, an additional “possible” step is required
`
`
`
`- 3 -
`
`

`

`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`in order to retain the perspective POV (i.e., retain the shapes), and the embodiment
`
`
`
`
`
`that does not perform that optional step maintains only the position POV, not the
`
`perspective POV. APPL-1038, ¶4.
`
`Consequently, Patent Owner’s construction, that requires both the position
`
`(i.e., Wide position POV) and shape (i.e., Wide perspective POV) of objects be
`
`maintained from the wide image, improperly excludes the embodiment that
`
`maintains only the Wide position POV. Kaneka Corp. v. Xiamen Kingdomway
`
`Group Co., 790 F.3d 1298, 1304 (Fed. Cir. 2015) (“A claim construction that
`
`excludes a preferred embodiment is rarely, if ever, correct.”) (citation omitted).
`
`Patent Owner’s construction erroneously imports an unclaimed limitation, ignoring
`
`the specification’s teaching that Wide POV can be only the Wide position POV.
`
`APPL-1038, ¶5; see Liebel-Flarsheim Co. v. Medrad, Inc., 358 F.3d 898, 904 (Fed.
`
`Cir. 2004) (“[I]t is improper to read a limitation from the specification into the
`
`claims.”).
`
`Third, Patent Owner raises a red herring by contrasting the use of “FOV” in
`
`Petitioner’s construction with “POV.” Response at 8-12. This is an immaterial
`
`distinction for present purposes. The Petition proposed the construction: “fused
`
`image that maintains [1] the Wide camera’s field of view or [2] the Wide camera’s
`
`position.” Petition, pp.7-8. Dr. Durand phrased this construction in this way to
`
`
`
`- 4 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`reflect that the Wide camera POV is based on either [1] the scene seen through the
`
`wide camera field of view (i.e., the position of objects seen in the camera); or [2]
`
`how the wide camera was positioned in 3D space when the image was captured
`
`(i.e., determining the perspective or shape of the objects). APPL-1038, ¶6; Ex.
`
`2036, Durand Depo. at 21:3–7. Thus, maintaining the Wide camera FOV in
`
`Petitioner’s construction would ensure that the position of objects in the image is
`
`that of the Wide camera, as required by Patent Owner’s construction. APPL-1038,
`
`¶6. Similarly, maintaining the Wide camera 3D position in Petitioner’s
`
`construction would ensure that the perspective/shape of objects in the image is that
`
`of the Wide camera, as required by Patent Owner’s construction. Id.
`
`Patent Owner asserts that FOV can be described using a single number, and
`
`thus cannot be relevant to the POV. However, Petitioner and Dr. Durand’s original
`
`construction referred to the “fused image” maintaining the Wide camera FOV.
`
`APPL-1038, ¶7. While the Wide camera FOV is a single number, by saying the
`
`fused image maintains that FOV, Petitioner’s construction indicates that the image
`
`shows the same objects/scene as seen from the Wide camera FOV. Id. While FOV
`
`can be described as a single number, Patent Owner gives no explanation as to why
`
`the image maintaining the FOV must somehow be described by a single number.
`
`Id.
`
`
`
`- 5 -
`
`

`

`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`As previously explained, the parties’ constructions reflect the same concepts
`
`
`
`
`
`in capturing a scene but merely use different terminology. APPL-1038, ¶2. The
`
`only significant difference between the parties’ constructions is whether the claim
`
`language encompasses maintaining Wide position POV or Wide perspective POV
`
`(as Petitioner contends) or requires maintaining both Wide position POV and Wide
`
`perspective POV (as Patent Owner incorrectly contends). Id., ¶8. To narrow the
`
`disputes for resolution by the Board, Petitioner’s construction can be rephrased
`
`using Patent Owner’s terminology, consistent with the specification, as “fused
`
`image in which the positions or shapes of objects reflect those of the Wide
`
`camera.” Id. As shown in the Petition and explained further below, Parulski
`
`satisfies the meaning required by the claim language, including under this
`
`alternative construction.
`
`III. Obviousness
`Patent Owner’s expert, Dr. Hart, applies the wrong standard for
`A.
`a POSITA in his obviousness analysis.
`
`Patent Owner’s flawed arguments against combining the prior art relies on
`
`Dr. Hart’s unsupported conclusions minimizing a POSITA’s knowledge, skill, and
`
`understanding. See APPL-1037, 61:22-10, 62:14-63:17, 100:25-101:13, 111:17-
`
`112:5, 114:3-13, 114:14-116:15, 129:7-14, 138:24-139:12, 140:23-141:9.
`
`
`
`- 6 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`This is improper because a POSITA is not an automaton but “is also a
`
`person of ordinary creativity ….” KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 421
`
`(2007). This means that “courts must ‘consider common sense, common wisdom,
`
`and common knowledge in analyzing obviousness.’” B/E Aerospace, Inc. v. C&D
`
`Zodiac, Inc., 962 F. 3d 1373, 1380 (Fed. Cir. 2020) (quoting Arendi S.A.R.L. v.
`
`Apple Inc., 832 F.3d 1355, 1361 (Fed. Cir. 2016)).
`
`Because m Dr. Hart treats the POSITA as an automaton incapable of
`
`applying knowledge outside of what each reference explicitly teaches, his
`
`declaration should carry little weight.
`
`B. Claims 1, 10-14, 16, 18, 23, 32-36, 38, and 40 are obvious over the
`combination of Parulski and Konno.
`
`1.
`
`A POSITA would have implemented Parulski’s Fig. 14 method
`to output a combined image with a broadened depth of field by
`using Parulski’s range mapping method in Fig. 11 to identify
`and extract objects.
`Patent Owner does not dispute that Parulski’s Fig. 11 teaches a method for
`
`creating a range map that “map[s] tele image pixels to matching pixels within the
`
`wide image.” See Response at 26-28. Rather, Patent Owner argues that a POSITA
`
`would not have combined the range mapping method with the image combination
`
`method in Fig. 14 because “[n]owhere … does Parulski describe using the Figure
`
`11 method together with image fusion.” Id. at 27.
`
`
`
`- 7 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`But to do so, Patent Owner and Dr. Hart impermissibly treat the POSITA as
`
`an automaton incapable of applying two of Parulski’s compatible teachings
`
`because Parulski doesn’t explicitly say to do so. Id. at 26-28; APPL-1038, ¶16;
`
`APPL-1037, 61:22-10, 62:14-63:17, 100:25-101:13, 111:17-112:5, 114:3-13. But
`
`as discussed above in III.A, a POSITA “is also a person of ordinary creativity, not
`
`an automaton,” and can use common sense, common wisdom, and common
`
`knowledge, and doing so here would have yielded the proposed combinations. KSR
`
`v. Teleflex, 550 U.S. at 421; B/E Aerospace, Inc. v. C&D Zodiac, Inc., 962 F. 3d
`
`1373, 1380 (Fed. Cir. 2020). Patent Owner’s argument thus fail.
`
`Patent Owner recognizes that “[i]dentifying objects within an image,”
`
`among others are “operations that could utilize a range map ….” Response at 27.
`
`But Patent Owner fails to mention the operation to “enable object extraction from
`
`an image by identifying the continuous boundaries of the object so it can be
`
`segmented within the image.” APPL-1005, 20:56-59. While Fig. 14 may not
`
`explicitly state that it uses a range map, a POSITA, not being an automaton, would
`
`have recognized the applicability of using object identification and extraction to
`
`the last step in Fig. 14 (block 514) which teaches that the second image (e.g., tele
`
`image) “is used to enhance the depth of field of the primary image” (e.g., wide
`
`image), “for instance, where the secondary still image is used to … sharpen
`
`
`
`- 8 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`portions of the primary still image that are positioned near the secondary focus
`
`distance.” APPL-1005, 22:37-42; APPL-1038, ¶17. A range map is the only
`
`method taught by Parulski that a POSITA could have used for identifying and
`
`extracting the portions of the tele image “positioned near the secondary focus
`
`distance” to “sharpen portions” of the wide image, thereby broadening the depth of
`
`field of the wide image. Id.
`
`Rather than use what Parulski already teaches to implement Fig. 14 (i.e.,
`
`identifying and extracting portions of the tele image to combine with the wide),
`
`Patent Owner argues that a POSITA would have looked to another method outside
`
`of Parulski that does not “incorporat[e] data from the tele image” like “sharpening”
`
`by “enhance[ing] edges based solely on the data from a single image.” Response at
`
`28-29. If Parulski was only concerned with sharpening edges of an object, it would
`
`said so. Instead, Parulski describes using portions of the tele image “captured
`
`near” the tele camera’s “focus distance” (i.e., tele camera’s depth of field) to
`
`“sharpen portions” of the wide image, or in other words, combining two images
`
`“into a modified image with a broadened depth of field.” Id.; APPL-1005, 22:37-
`
`43, 28:48-53. This is evident from Parulski’s example with the mountains, the
`
`flowers, and the dog. APPL-1038, ¶11; see APPL-1005, 21:7-44.
`
`
`
`- 9 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`Patent Owner argues that Parulski’s example does not provide motivation
`
`for using a range map to combine image data from the tele image with the wide
`
`image to broaden the depth of field. Response at 27-29. But Parulski’s example is
`
`included to instruct a POSITA about the “use of a range map for purposes such as
`
`noted above,” which include object identification and extraction. APPL-1038, ¶19;
`
`see APPL-1005, 20:50-21:8. Parulski then uses the example to explain that “the
`
`depth of field can be adjusted so that, e.g., the dog is in focus, the mountains are in
`
`focus and so are those great flowers.” APPL-1005, 21:25-27. This as well as other
`
`depth of field enhancements is achieved in part by sharpening the dog using the in-
`
`focus portions of the tele image, thereby broadening the depth of field to include
`
`objects closer to the wide camera that may be outside of the wide camera’s depth
`
`of field. APPL-1038, ¶19; see APPL-1005, 21:27-31.
`
`Patent Owner asserts that a POSITA would not have read Parulski’s example
`
`this way because Parulski’s states that an ultra-wide lens can capture an image
`
`focused at 8 feet “so that objects from 4 feet to infinity are in focus.” See Response
`
`at 28; APPL-1005, 21:57-65. This would presumably include the dog in the
`
`example at 5 feet, which Patent Owner interprets as evidence that a POSITA would
`
`not use the focused portions of the dog from the tele image to broaden the depth of
`
`field of the wide image. See Response at 28; APPL-1005, 21:12-13.
`
`
`
`- 10 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`This argument fails because it does not consider the combination with
`
`Konno’s wide lens (which Patent Owner does not dispute) which, when similarly
`
`focused at 8 feet, has a smaller depth of field of about 6 feet to infinity. APPL-
`
`1038, ¶21. In this case, the dog being at 5 feet would be slightly out of focus in the
`
`wide image (id.) which, based on Parulski’s example, could be sharpened by using
`
`a range map to identify the in-focus portions of the dog from the tele image,
`
`extracting these in-focus portions, and outputting an image that combines these
`
`portions with the wide image to sharpen the slightly-out-of-focus dog. Id., ¶¶22-26.
`
`Thus, Patent Owner fails to show that a POSITA would not have used
`
`Parulski’s range mapping methods for object identification and extraction to
`
`broaden the depth of field of the wide image via combination with corresponding
`
`portions of the tele image captured at the tele camera’s focus distance. Id., ¶27.
`
`2.
`
`Parulski teaches outputting a “fused image with a point of
`view (POV) of the Wide camera” when this term is properly
`construed to include Wide position POV.
`Patent Owner argues that Petitioner has not shown how the combination
`
`satisfies “fused image with a point of view (POV) of the Wide camera” under its
`
`construction. Response at 30-31. As discussed above, Patent Owner’s construction
`
`improperly imports a limitation requiring that Wide perspective POV be
`
`maintained, when in fact the claim also encompasses maintaining only Wide
`
`
`
`- 11 -
`
`

`

`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`position POV. See Response at 8-13, 30-31. Patent Owner does not directly dispute
`
`
`
`
`
`that Parulski teaches maintaining the Wide position POV when combining portions
`
`of the tele image with the wide image. See id. at 29-31; APPL-1038, ¶28; APPL-
`
`1005, Fig. 14.
`
`As discussed previously, the ‘479 specification describes that Wide position
`
`POV is maintained by “register[ing] Tele image pixels to a matching pixel set
`
`within the Wide image pixels, in which case the output image will retain the Wide
`
`POV (‘Wide fusion’).” APPL-1001, 5:23-25. The Wide position POV is therefore
`
`maintained when the FOV of the Wide camera is maintained. APPL-1038, ¶29.
`
`Only when the pixels from the Tele image are “shifted” is the Wide
`
`perspective POV for the objects also maintained. APPL-1001, 5:30-33. Claims 1
`
`and 23 only require that the POV of the wide camera be maintained “by mapping
`
`Tele image pixels to matching pixels within the Wide image” (e.g., using a
`
`registration map as in Parulski’s Fig. 11). APPL-1001, 13:48-50; APPL-1038, ¶30.
`
`Nothing in the claims also requires shifting the pixels in the Tele image to also
`
`maintain the Wide perspective POV. APPL-1038, ¶30. While the Wide perspective
`
`POV could also be maintained by also performing shifting, the claims do not
`
`require it.
`
`
`
`- 12 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`Parulski teaches in Fig. 14 that when the first capture stage is the wide
`
`camera and the second capture state is the tele camera, that portions of the tele
`
`image are combined with the wide image, thereby producing an output image that
`
`maintains the Wide position POV or the field of view of the Wide camera when the
`
`image was captured:
`
`
`
`Petition at 27; APPL-1003, p.48; APPL-1038, ¶31; APPL-1005, 28:48-53.
`
`Dr. Durand explains the same thing—that Parulski maintains the Wide
`
`position POV in terms of the field of view of the wide camera when the image was
`- 13 -
`
`
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`captured by outputting an image that maps Tele image pixels to matching pixels
`
`within the Wide image. APPL-1038, ¶32; see Petition at 29-30; APPL-1003, p.51-
`
`52.
`
`Because Patent Owner’s construction is overly narrow and improperly
`
`imports a limitation into the claims, it should be rejected and the Board’s finding in
`
`the Institution Decision should be maintained. See Institution Decision at 23-25.
`
`C. Claims 2-4 and 24-26 are obvious over the combination of
`Parulski, Konno, and Szeliski.
`
`Patent Owner admits that “image rectification was already well known to a
`
`POSITA” at the time Parulski was filed (see Response at 31), but argues that “[i]f
`
`rectification was an obvious improvement to Puralski [sic] to a POSITA on June
`
`13, 2013, then it would have been an obvious improvement to Puralski [sic] to a
`
`POSITA (including Puralski [sic]) on Mar. 9, 2007.” Id. at 31-32; Ex. 2001, ¶81.
`
`Yet again, Patent Owner and Dr. Hart treat a POSITA as an automaton incapable
`
`of applying a well-known rectification technique to Parulski’s range mapping
`
`method despite the evidence of record (i.e., Szeliski) showing that rectification
`
`makes range mapping more efficient. See APPL-1038, ¶26; APPL-1013, p.472.
`
`But the creativity of a normal POSITA would have informed the use of
`
`Parulski’s range mapping to yield a more efficient algorithm given its application
`
`in mobile devices with limited processing, memory, and power than traditional
`
`
`
`- 14 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`computer systems. APPL-1038, ¶34. Common sense also dictates that more
`
`efficient range mapping would have produced faster image processing results,
`
`thereby improving the user’s experience. Id.
`
`Patent Owner also argues that Dr. Durand “provides no reason why a
`
`POSITA would use rectification over other alternatives” like a plane sweep
`
`algorithm. Response at 32. This is not the standard, but even if it were, Szeliski
`
`teaches that rectification “makes most sense if the cameras are next to each other”
`
`(APPL-1013, p.472, n.2), which is the exact situation in Parulski (see APPL-1005,
`
`Fig. 16); APPL-1038, ¶35. Plain sweep, though, is better suited for “stereo
`
`matching with arbitrary camera configurations” where rectification cannot be
`
`performed. APPL-1013, pp.471, 472 n.2.
`
`Thus, Patent Owner’s arguments either apply the wrong standard or fail
`
`based on the evidence of record and do not overcome the Petition’s showing of
`
`unpatentability.
`
`D. Claims 5-9 and 27-31 are obvious over the combination of
`Parulski, Konno, Szeliski, and Segall
`
`Patent Owner does not dispute that Segall teaches certain limitations of
`
`claims 5 and 6, but instead argues that Parulski and Segall are not combinable.
`
`Response at 32-33; Ex. 2001, ¶83.
`
`
`
`- 15 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`The Petition and Dr. Durand provided extensive reasons why a POSITA
`
`would have combined Segall’s teachings about its fusion process with Parulski’s
`
`image combination process. Petition at 52-54; APPL-1003, ¶¶ 67-70. One reason
`
`specifically given was that Segall teaches how to deal with fusing pixels where
`
`errors exist in the registration map (see Petition at 52), which Patent Owner admits
`
`is still an issue with Parulski’s registration process. Response at 31; APPL-1038,
`
`¶37. Thus, Patent Owner’s argument does not overcome the evidence relied on in
`
`the Petition by simply relying on Parulski’s method generating “fewer errors.”
`
`Patent Owner also argues that implementing Segall’s entire fusion method
`
`“including portions on temporal consistency and other issues of motion
`
`compensation registration … would have added significant wasted effort when
`
`applied to a simpler and less error-prone stereo registration.” Response at 32-33.
`
`But yet again, Patent Owner and Dr. Hart ignore the common sense, wisdom, and
`
`knowledge of a POSITA who would know which portions of Segall’s fusion
`
`method are applicable to Parulski’s registration and combination methods—like
`
`how to fuse pixels when errors exist in the registration map. APPL-1038, ¶38.
`
`E. Claims 15 and 37 are obvious over the combination of Parulski,
`Konno, and Stein.
`
`Patent Owner does not dispute that Stein teaches synchronizing rolling
`
`shutters in claims 15 and 17. Response at 33-34. Instead, Patent Owner first argues
`
`
`
`- 16 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`that because the ’479 patent developed shutter synchronization to allegedly
`
`“minimize the required bandwidth from both sensors for the ISPs,” Parulski’s
`
`example (with mountains, flowers, and dog) with no movement “provides little
`
`motivation for the need of increased bandwidth.” Response at 34. Patent Owner
`
`also argues that “a POSITA would not have looked to the automotive industry”
`
`because Stein’s problem and the problem in the ’479 patent “were solved by a
`
`similar synchronization approach [which] is coincidental,” but not obvious.
`
`Response at 34-35.
`
`Patent Owner applies flawed analysis that starts with the problem the ’479
`
`patent was allegedly trying to solve (see Response at 33-34) and concludes that a
`
`POSITA would not have been motivated to use shutter synchronization. APPL-
`
`1038, ¶40. Patent Owner also fail to address the extensive reasons in the Petition to
`
`combine Parulski and Stein including Stein’s teaching that shutter synchronization
`
`reduces registration errors that lead to incorrect depth information, an issue directly
`
`relevant to improving accuracy in Parulski’s range mapping process. Petition at 66-
`
`67; APPL-1003, ¶¶ 77-79; APPL-1038, ¶40. This teaching also contradicts Patent
`
`Owner’s argument that “driver assist systems … are less concerned with rolling
`
`shutter image artifacts.” See Response at 34-35; Petition at 66-67; APPL-1003,
`
`¶79; APPL-1038, ¶40.
`
`
`
`- 17 -
`
`

`

`
`
`
`
`
`
`
`
`
`Petitioner’s Reply
` IPR2020-00905 (Patent No. 10,225,479)
`
`
`Patent Owner also argues that Parulski’s design “include[s] an interface to
`
`an external PC that can be used in the computation of a range map,” but neither the
`
`Response nor Dr. Hart’s declaration cite to where Parulski supposedly teaches this
`
`and Dr. Durand could not find anything in Parulski besides image processor 50 in
`
`the camera device being used to process images. APPL-1038, ¶41.
`
`IV. Secondary Considerations
`Patent Owner’s evidence of secondary considerations is not entitled to any
`
`weight. There is no nexus between Patent Owner’s evidence and the challenged
`
`claims. Even if there were, the evidence is insufficient to overcome Petitioner’s
`
`strong showing of obviousness. Asyst Techs., Inc. v. Emtrak, Inc., 544 F.3d 1310,
`
`1316 (Fed. Cir. 2008). The combinations presented in the Petition represent the
`
`predictable use of prior art elements, and as such, evidence of secondary
`
`considerations is inadequate to establish non-obviousness. Ohio Willow Wood Co.
`
`v. Alps S., LLC, 735 F.3d 1333, 1344 (Fed. Cir. 2013).
`
`
`
`Patent Owner’s various theories: industry praise/licensing, commercial
`
`success, and failure of others/copying have no merit. See Response, 35-46.
`
`A. No nexus.
`
`For secondary considerations evidence to be accorded substantial weight, the
`
`“patentee bears the burden of showing that a nexus exists” to the claimed
`
`
`