UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________
`
`
`APPLE INC.
`Petitioner
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner
`____________
`
`Case IPR2018-01133
`U.S. Patent No. 9,538,152
`____________
`
`
`PATENT OWNER’S SUR-REPLY
`TO PETITION FOR INTER PARTES REVIEW
`
`
`
`
`
`
`
`
`
`

`

`Table of Contents
`
`INTRODUCTION .......................................................................................... 1
`
`CLAIN CONSTRUCTION ............................................................................ 2
`
`I.
`
`II.
`
`
`A. The PTAB Should Construe “Point of View” as “Camera Angle” ................ 2
`
`III. APPLICABLE LEGAL STANDARDS ......................................................... 3
`
`IV. ARGUMENT ................................................................................................. 4
`
`
`A. Apple’s Petition Fails to Show that Border Discloses An Output Image From
`the Point of View of One Camera .................................................................. 5
`
`1. Apple misapplies the meaning of “camera angle” in its Reply ................... 5
`
`2. Apple’s reliance on Border’s “coordinate transformation” is misplaced; it
`does not meet the “point of view” limitation when properly construed as
`“camera angle” ........................................................................................... 8
`
`B. The Board Should Disregard Apple’s New Arguments on Reply ............... 11
`
`
`1. Apple’s new obviousness arguments based on “various registration
`models” in Border are improper ............................................................... 12
`
`2. Apple’s new obviousness theory for modifying Border with any “suitable
`registration method known in the art” is improper ................................... 18
`
`3. Apple’s new obviousness combination of Border and Parulski against
`elements [1.8] and [1.10] is improper ....................................................... 19
`
`C. Apple’s Attacks on Dr. Kosmach’s Qualifications and Expertise Are
`Without Merit ............................................................................................... 21
`
`D. Apple’s Expert Conceded that the Petition’s Reliance on Parulski Was
`Illusory .......................................................................................................... 22
`
`CONCLUSION ............................................................................................ 23
`
`
`V.
`
`
`
`i
`
`

`

`Table of Authorities
`
`Cases
`
`ActiveVideo Networks, Inc. v. Verizon Commc’ns, Inc.,
`694 F.3d 1312 (Fed. Cir. 2012) ............................................................................. 3
`
`Argentum Pharm. LLC v. Research Corp. Techs., Inc.,
`IPR2016-00204, Pap. 85 (Mar. 22, 2017) ........................................................... 12
`
`Intelligent Bio-Sys., Inc. v. Illumina Cambridge Ltd.,
`821 F.3d 1359 (Fed. Cir. 2016) ....................................................................... 4, 19
`
`Pfizer Inc. v. Chugai Pharmaceutical,
`IPR2017-01357, Paper 56 (Nov. 28, 2018) ........................................................... 3
`
`Wasica v. Con’t Auto Sys.,
`853 F.3d 1272 (Fed. Cir. 2017) ............................................................. 3, 4, 11, 20
`Statutes
`
`35 U.S.C. § 312(a)(3) ........................................................................................... 1, 3
`Rules
`
`37 C.F.R. § 42.108 .................................................................................................... 3
`
`37 C.F.R. § 42.23(b) ............................................................................................... 19
`
`
`
`
`
`i
`
`

`

`Patent Owner’s Updated Exhibit List for IPR2018-01133
`
`Pursuant to 37 C.F.R. § 42.63(e), Patent Owner Corephotonics Ltd., hereby
`
`submits its updated exhibit list associated with the above-captioned inter partes
`
`review of U.S. Patent No. 9,538,152.
`
`2004
`
`2005
`
`2006
`
`2007
`
`Exhibit No. Description
`2003
`Joint Claim Construction Statement filed in Corephotonics, Ltd.
`v. Apple Inc., Case No. 5:17-cv-06457-LHK in the U.S. District
`Court for the Northern District of California.
`
`Transcript of Deposition of Oliver Cossairt (February 5, 2019)
`
`Declaration of James Kosmach, Ph.D.
`
`Curriculum Vitae of James Kosmach
`
`Corephotonics’ Opening Claim Construction Brief
`
`Apple’s Responsive Claim Construction Brief
`
`U.S. Patent No. 9,185,291, “Dual Aperture Zoom Digital
`Camera”
`
`Transcript of Deposition of Oliver Cossairt (July 12, 2019)
`
`
`2008
`
`2009
`
`2010 (New)
`
`
`
`
`
`
`
`ii
`
`

`

`I.
`
`Introduction
`
`Apple’s Reply makes, for the first time, multiple arguments that were either
`
`absent from or never substantively discussed in its initial Petition. Not only did
`
`Apple submit a new 33-page declaration from its technical expert replete with new
`
`opinions, Apple uses a newly-filed, 211-page exhibit of excerpts from the Szeliski
`
`textbook1 (other portions of which were discussed only as background prior art in
`
`the Petition) to make new arguments for obviousness and motivation to combine.
`
`The scope of the trial, however, is limited to the grounds in the Petition, in which
`
`Apple was required to identify “with particularity . . . the grounds on which the
`
`challenge to each claim is based, and the evidence that supports the grounds for the
`
`challenge to each claim.” 35 U.S.C. § 312(a)(3). Each of Apple’s new arguments
`
`on Reply are improper and should be disregarded.
`
`Even if the new Reply arguments are credited, Apple still failed to show that
`
`either of its asserted references, Border or Parulski, disclose an “output image,”
`
`generated from multiple cameras, that has the “point of view” of only one of the
`
`multiple cameras. As explained below, the correct construction of “point of view”
`
`is “camera angle,” and none of Apple’s arguments based on Border or Parulski show
`
`
`1 Richard Szeliski, COMPUTER VISION: ALGORITHMS AND APPLICATIONS (2011).
`Apple’s Petition included APPL-1010 (“Szeliski I”), which comprised 16 pages of
`the Szeliski textbook. On Reply, Apple filed 211 pages of new material from the
`Szeliski textbook as APPL-1012 (“Szeliski II”).
`
`
`
`1
`
`

`

`teaching of challenged claim element 1(c), enumerated in the Petition as [1.8], [1.9],
`
`[1.10], and [1.11], or that Border should even be combined with Parulski for the
`
`purpose disclosed in the Petition.
`
`II. Claim Construction
`
`A. The PTAB Should Construe “Point of View” as “Camera
`Angle”
`
`The Reply makes clear that the claim term “point of view” requires
`
`construction. Though neither party initially proposed that “point of view” be
`
`construed, the Petition appeared to apply an understanding of that term inconsistent
`
`with its plain meaning. See Patent Owner Response (Paper 15, “POR”), at 15-16.
`
`Accordingly, Corephotonics stated that a POSITA would have understood “point of
`
`view” to mean “camera angle.” Id. On Reply, Apple asserts that construing “point
`
`of view” “as camera angle” would “not materially change Petitioner’s analysis.” As
`
`shown below, however, Apple’s Petition fails to meet the “point of view” elements
`
`so construed.
`
`As previously explained, construing “point of view” to mean “camera angle”
`
`is consistent with the intrinsic evidence and the inventors’ usage of the that term.
`
`Apple’s construction, “viewpoint,” lacks any support in the specification of the ‘152
`
`patent (or the ’291 patent) and also suffers from inconsistent and indeterminate scope
`
`when used by Apple and its expert. See POR, at 13-16.
`
`
`
`2
`
`

`

`Because one of skill would understand the plain meaning of “point of view”
`
`to be camera angle in light of the intrinsic and extrinsic record, the PTAB should
`
`construe “point of view” as “camera angle.”
`
`III. Applicable Legal Standards
`
`A petition challenging a claim on grounds of obviousness must sufficiently
`
`explain (1) “how specific references could be combined,” (2) “which combination(s)
`
`of elements in specific references would yield a predictable result,” and (3) “how
`
`any specific combination would operate or read on” the claims. ActiveVideo
`
`Networks, Inc. v. Verizon Commc’ns, Inc., 694 F.3d 1312, 1327-28 (Fed. Cir. 2012).
`
`The scope of the trial is controlled, and limited, by the Petition. In its Petition,
`
`Apple was required to identify “with particularity . . . the grounds on which the
`
`challenge to each claim is based, and the evidence that supports the grounds for the
`
`challenge to each claim. 35 U.S.C. § 312(a)(3). Though “introduction of new
`
`evidence in the course of the trial is to be expected,” the “shifting of arguments is
`
`not.” Pfizer Inc. v. Chugai Pharmaceutical, IPR2017-01357, Paper 56, at 19 (Nov.
`
`28, 2018) (declining to consider petitioner’s “subsequent arguments” on Reply
`
`which “amount[ed] to an entirely new theory of [inherent anticipation] absent from
`
`the petition” (quoting Wasica v. Con’t Auto Sys., 853 F.3d 1272, 1286 (Fed. Cir.
`
`2017))). This is because, “unlike district court litigation—where parties have greater
`
`freedom to revise and develop their arguments over time and in response to newly
`
`
`
`3
`
`

`

`discovered material—the expedited nature of IPRs bring with it an obligation for
`
`petitioners to make their case in their petition to institute.” Intelligent Bio-Sys., Inc.
`
`v. Illumina Cambridge Ltd., 821 F.3d 1359, 1369 (Fed. Cir. 2016).
`
`Importantly, the Petitioner may not remedy fatal deficiencies in its Petition
`
`with new arguments in its Reply. See Wasica, 853 F.3d at 1286 (“Rather than
`
`explaining how its original petition was correct, Continental’s subsequent arguments
`
`amount to an entirely new theory of prima facie obviousness absent from the
`
`petition. Shifting arguments in this fashion is foreclosed by statute, our precedent,
`
`and Board guidelines.”) (internal citations omitted).
`
`IV. Argument
`
`Apple failed to show that the prior art teaches several critical claim elements
`
`of the ‘152 patent. Whether the claim term “point of view” is given its plain and
`
`ordinary meaning or an appropriate construction of “camera angle,” Apple did not
`
`show that Border disclosed “a processor configured to provide an output image from
`
`the point of view of the first camera” and when “FOV2<FOVZF<FOV1 then the point
`
`of view of the output image is that of the first camera,” as required in elements [1.8]
`
`and [1.10]. Neither did Apple show a POSITA would have been motivated to
`
`combine Border with Parulski to render obvious elements [1.9] “the first image
`
`being a primary image and the second image being a non-primary image” or [1.11],
`
`
`
`4
`
`

`

`“register[ing] the overlap area of the second image as non-primary image to the first
`
`image as primary image to obtain the output image.”
`
`A. Apple’s Petition Fails to Show that Border Discloses An Output
`Image From the Point of View of One Camera
`
`As explained in the POR, Apple fails to show that Border or Parulski disclose
`
`an “output image,” generated from multiple cameras, that has the camera angle (i.e.,
`
`point of view), of only one of the multiple cameras. The inventors of the ‘152 patent
`
`explained that a composite output image that combines data from two cameras could
`
`have the point of view of either camera, or a combination of both:
`
`In a dual-aperture camera image plane, as seen by each sub-camera (and
`respective image sensor), a given object will be shifted and have
`different perspective (shape). This is referred to as point-of-view
`(POV). The system output image can have the shape and position of
`either sub-camera image or the shape or position of a combination
`thereof.
`
`Ex. 2009, at 4:60-4:65. While the ’152 patent’s inventors understood that a
`
`composite output image could have the point of view of either camera, or a
`
`combination of both, the challenged claims require that the output image have a
`
`single point of view of one sub-camera. Apple fails to show that Border meets this
`
`limitation.
`
`1.
`
`Apple misapplies the meaning of “camera angle” in its
`Reply
`
`Though Corephotonics has already explained the concept of “camera angle”
`
`and cited to the unambiguous discussion in Corephotonics’ ’291 patent regarding
`
`
`
`5
`
`

`

`the same (POR, at 14), Apple now contends that a composite image generated from
`
`two cameras will have the “point of view” of a single camera even if the composite
`
`image has occlusions that result from the two different camera angles of the input
`
`image data. See Reply, at 18 (“Border’s stitched composite image, with occlusion
`
`or not, is from the point of view of the wide image.” (emphasis by Apple)).2
`
`The presence of occlusions (or “discontinuities”) in a composite image is a
`
`function of the different camera angles of the input cameras. In other words, the
`
`presence of an occlusion means necessarily that the composite image includes image
`
`data from at least two different camera angles or points of view.
`
`Apple’s technical expert, Dr. Cossairt, admitted in deposition that the
`
`composite image disclosed in Border will be a combination of both points of view.
`
`He explained that if there is an object occlusion in the camera angle of the telephoto
`
`camera (i.e., the “point of view” of the telephoto camera), the portion of the
`
`composite image comprising image data stitched from the telephoto camera will
`
`contain that occlusion even if no such occlusion appears from the camera angle of
`
`the wide camera:
`
`Q. So those occlusions that are present in the telephoto image would
`also be present in the region of the output image inside the dashed lines
`even if that same region of the wide image doesn't contain that
`occlusion, correct?
`
`
`
`2 All emphases are added unless otherwise stated.
`
`
`
`6
`
`

`

`image
`the
`that
`interpretation
`the
`[DR. COSSAIRT]. Under
`transformation that is applied to the tele images is just a scaling and
`translation, I believe what you said is correct.
`
`Ex. 2004, at 98:3-12 (objection omitted). In other words, Dr. Cossairt admits that
`
`the combined output image will have a combination of both points of view. Dr.
`
`Cossairt’s testimony confirms that Border does not meet the single point of view
`
`limitation.
`
`Further underscoring Apple’s misapplication of the terms “camera angle” and
`
`“point of view” is its argument that, for “planar scenes,” Border “teaches providing
`
`an output image without such artifacts” such as “occlusion, parallax.” Reply, at 18
`
`(emphasis in original). Apple argues that when the “scene” being photographed is
`
`“planar,” such as the landscape vista depicted in Fig. 1B of the ‘152 patent, the
`
`composite image contains no “artifacts” resulting from the different camera angles
`
`(or “points of view”) represented in the input image data. See Reply id. at 18-19.
`
`The claims require a processor configured to “provide an output image from
`
`a point of view of the first camera.” As Dr. Cossairt admits, the output image from
`
`Border will have image data from both the telephoto and wide angle cameras. While
`
`there may not be any noticeable occlusions if the scene is planar, that does not mean
`
`that the output image has the point of view of one camera. It just means that one
`
`cannot, upon visible inspection, see that the output image has image data from both
`
`points of view. But Apple cannot dispute that when Border is used to photograph a
`
`
`
`7
`
`

`

`scene with depth, in which the different camera angles will result in different image
`
`data for each sub-camera, the output image will have discontinuities. See Ex. 1005,
`
`¶ 48. That there will be occlusions in a composite image of a scene with depth, on
`
`the other hand, proves that the output image contains multiple “points of view.” The
`
`Petition therefore fails to show that Border meets the requirement of a “processor
`
`configured to” generate an “output image” with only the “point of view” of one input
`
`camera as the ’152 patent requires. Ex. 1001, cl 1.
`
`2.
`
`“coordinate
`Border’s
`on
`reliance
`Apple’s
`transformation” is misplaced; it does not meet the
`“point of view” limitation when properly construed as
`“camera angle”
`
`Where Apple discusses Border’s translation and scale homography (see
`
`Reply, at 16-17), Apple contends that the coordinate transformation used for the
`
`translation and scale—i.e., “transform[ing] the coordinates of the telephoto image
`
`206 to the wide image 204”—means that the coordinates of the telephoto image have
`
`been “transformed to the point of view of the wide image.” Id.; see also id. at 24
`
`(“Border teaches obtaining an output image including ”). That “coordinate
`
`transformation,” however, “is simply a translation and scale.” Ex. 1006, ¶ 38.
`
`Border’s use of the verb “transform” does not mean, as Apple implies, that the
`
`telephoto image is modified to have the “camera angle” or the “point of view” of the
`
`wide image. Rather, the “coordinate transformation” cited by Apple refers to the
`
`
`
`8
`
`

`

`function of correlating pixels between the two input images, the product of which is
`
`then stored in a homography matrix:
`
`
`
`Id.; see id. ¶ 38 (“The coordinate translation is simply a translation and scale because
`
`the image sensors that capture the wide image 204 and the telephoto image 206 are
`
`coplanar.”). The homography array (“HTW”) contains the information reflecting that
`
`image-to-image pixel correlation:
`
`
`
`9
`
`
`
`

`

`Border acknowledges that the homography disclosed in ¶¶ 38-41 shows only
`
`a “pure translate and scale transformation,” which is not sophisticated enough to
`
`“correct for a difference in tilt between the two imaging systems.” Id. ¶ 41. In other
`
`words, the “pure translate and scale transformation” of Border does nothing to
`
`ensure that the resulting composite image has either the “point of view” or “camera
`
`angle” of the telephoto camera or the wide camera as opposed to the combination of
`
`both.
`
`By the admission of Apple’s expert, the “pure translation and scale”
`
`registration technique applied between two input images cannot mitigate or correct
`
`for occlusions in a composite image generated from that input:
`
`[DR. COSSAIRT:] [I]f the image is captured from a point of view such
`that some objects are occluded, there is no way using the image
`information from the image captured from that point of view only to
`remove that occlusion via a simple image transformation, generally.
`There are very complicated forms of image transformation that say
`that’s generally accepted to be true.
`
`Ex. 2004, at 95:15-23. As Dr. Cossairt concedes, correcting for occlusions in an
`
`output image, which is necessary to produce a composite image that has just one
`
`“point of view,” is “more complicated than just a scaling and translation.” Id. at
`
`96:10-13. And because the pure translation and scale cannot mitigate occlusions or
`
`other artifacts caused by parallax errors, the “coordinate transformation” disclosure
`
`is incapable of ensuring that an “output image” have a single “point of view” of one
`
`input camera or another.
`
`
`
`10
`
`

`

`
`
`B.
`
` The Board Should Disregard Apple’s New Arguments on Reply
`
`On Reply, rather than “explaining how its original petition was correct”
`
`(Wasica, 853 F.3d at 1286), Apple asserts multiple new arguments to correct the
`
`deficiencies apparent in its Petition. The Reply points to portions of the prior art
`
`Apple had previously overlooked and offers new expert testimony on concepts not
`
`discussed in the Petition. Apple even asserts new obviousness combinations and
`
`obviousness theories against specific claim elements, with the transparent
`
`justification that elements [1.8] through [1.11] (which comprise claim 1(c)) are all
`
`“interrelated” and that therefore Apple’s “analysis of interrelated limitations . . .
`
`based on Border and Parulski [allegedly] applies to element (c) in its entirety.”
`
`Reply, at 21. Apple even submits, cites, and discusses a newly-submitted 211-page
`
`(!) excerpt of the Szeliski textbook (Ex. 1012) to support its new obviousness
`
`arguments and theories of motivations to combine or modify the prior art, even
`
`though for its Petition Apple had filed only 16 page pages from the same textbook
`
`(Ex. 1010) only for use as background material.
`
`It is “improper to present in reply new evidence (including new expert
`
`testimony) that could have been presented in a prior filing.” PTAB Rev’d Tr.
`
`
`
`11
`
`

`

`Practice Guide, at 15 (August 2018 Update).3 A “reply or sur-reply that raises a new
`
`issue or belatedly presents evidence may not be considered.” Id. Each of Apple’s
`
`new arguments, enumerated and addressed below, reflect a “new direction with a
`
`new approach as compared to positions taken in a prior filing” and should therefore
`
`be disregarded. Id.4
`
`1.
`
`Apple’s new obviousness arguments based on “various
`registration models” in Border are improper
`
`Apple argues, for the first time, that Border discloses elements of the
`
`challenged claims because it purportedly teaches “various registration models”
`
`beyond pure translation and scaling homography, such as “feature-based
`
`registration, registration with true depth map, and registration correcting tilt.”
`
`Reply, at 6-7; see 14, 19, and 24. Apple’s arguments based on these three
`
`“registration models” are both improper and untimely.
`
`First, the Petition failed to discuss any disclosure in Border of “feature-based
`
`registration” or even quote from Border’s discussion of “feature-based registration”
`
`
`3 Available at
`https://www.uspto.gov/sites/default/files/documents/2018_Revised_Trial_Practice
`_Guide.pdf (accessed July 20, 2019).
`4 See also Argentum Pharm. LLC v. Research Corp. Techs., Inc., IPR2016-00204,
`Pap. 85, 5-6 (Mar. 22, 2017) (disallowing Petitioner “to advance [in Reply]
`additional evidence to supplement its original contentions of unpatentability”); 77
`Fed. Reg. 48,756, 48,767 (“Examples of indications that a new issue has been raised
`in a reply include new evidence necessary to make out a prima facie case for the
`patentability or unpatentability of an original or proposed substitute claim, and new
`evidence that could have been presented in a prior filing.”).
`
`
`
`12
`
`

`

`(which appears at Ex. 1006 ¶ 42). The Petition cites Ex. 1006 ¶ 42 only on page 46
`
`of the Petition—but as a “see also” citation lacking any parenthetical or contextual
`
`explanation, and in a section of the Petition exclusively discussing the pure
`
`translation and scale homography discussed above. See Pet., at 46. And, notably,
`
`the portions of the Petition discussing the registration-related element of claim 1
`
`(limitation [1.11]) contain no argument about, or even a reference to, “feature based
`
`registration.” See Pet, at 59.
`
`In support of its new argument, Apple cites the new supplemental Cossairt
`
`Declaration (Ex. 1013) and the new 211-page excerpt from the Szeliski textbook
`
`(Ex. 1012) to explain what “feature-base registration” actually means and how it
`
`teaches elements in the challenged claims. See, e.g., Reply, at 6 (citing Ex.1012, at
`
`181-235 and 273-301 as “describing feature-based registration); 19 (citing Szeliski
`
`II to argue “Border’s feature-based registration was well-known to address artifacts
`
`(e.g., occlusion)”), 20 (excerpting a figure from Szeliski II to show “feature-based
`
`registration”). All of this new argument is untimely and improper.
`
`Second, the concept of “registration correcting tilt” appears nowhere in the
`
`Petition or the expert declaration Apple submitted with the Petition. Apple’s
`
`opening papers do not even include the word “tilt.” To shore up this gap, Apple
`
`relies on its supplemental expert declaration filed with its Reply, which opines that
`
`Border “actually teaches various registration models including . . . registration
`
`
`
`13
`
`

`

`correcting tilt.” Ex. 1013, ¶ 10. Both Apple and Dr. Cossairt, however, fail to
`
`actually describe how Border discloses a “registration correcting tilt” or how it
`
`would be devised or implemented. See Ex. 1013, ¶¶ 10-14.
`
`The sole reference in Border to the problem of “tilt” is in the context of
`
`pointing out a shortcoming of a pure translation and scale model: “It should be
`
`further noted that while the example shows a pure translate and scale transformation,
`
`it may be necessary to correct for a difference in tilt between the two imaging
`
`systems.” Ex. 1006, ¶ 41. Not only is Apple’s argument that Border teaches
`
`“registration correcting tilt” untimely, it is unsupported by the evidence because
`
`Border only observes the problem of tilt but does not disclose how to correct tilt,
`
`much less an entirely separate form of “registration” to correct for tilt. Id.
`
`Third, the Petition contains only a single, oblique reference to Border’s
`
`mention of a “true depth map” for removing discontinuities in a composite image.
`
`See Pet., at 61. No substantive discussion of a “true depth map” or a registration
`
`based thereon appears anywhere in the Petition. On Reply, Apple attempts to
`
`remedy this deficiency by having Dr. Cossairt opine on and explain “true depth
`
`maps” and cites Szeliski II describing the same. See Ex. 1013, ¶ 12; Reply, at 6-7
`
`(citing Ex. 1012, at 50-51 as “explaining using true depth map including depth
`
`coordinates for mapping”).
`
`
`
`14
`
`

`

`None of the registration techniques disclosed in Border, other than pure
`
`translation and scale homography, were cited or discussed in a substantive manner
`
`in the Petition. It is this simple homography that Corephotonics responded to in its
`
`Patent Owner Response, and it is a simple homography which, as Apple does not
`
`deny, fails to teach elements of the challenged claims.
`
`Even if Apple’s newly-raised “various registration models” are considered at
`
`face value, its argument fails because the Border does not actually disclose
`
`“registration models” but instead only image processing techniques for populating
`
`values into a homography matrix, which Dr. Cossairt confirmed in deposition. See
`
`Ex. 2010, at 28:19-10 (“[Paragraph] 42 does not describe the nature of the
`
`registration, or the mapping that the registration might, or even examples of
`
`registration.”). A homography, however, is not “registration” or a “registration
`
`model.” Id. at 78:16-19. More importantly, the homographies at issue are incapable
`
`of modifying one image to have the point of view of another:
`
`Q. If you have an image of a concave scene taken from one point of
`view, will applying a homography to that image produce an image of
`that concave scene taken from a different point of view?
`
`A. No, no, it will not.
`
`See Ex. 2004, 174:10-15.
`
`Besides being untimely, Apple’s new arguments are deficient also because
`
`Dr. Cossairt never explains how a POSITA would have used “feature-based
`
`
`
`15
`
`

`

`registration,” depth mapping, or “registration correcting tilt,” either from Border
`
`specifically or as known in the art generally (e.g., Szeliski II), to “address artifacts
`
`(e.g., occlusion).” See Ex. 1013, ¶ 30. At paragraph 30 of his supplemental
`
`declaration, Dr. Cossairt claims that “occlusion and parallax” were “well known
`
`issues” and that a “POSITA would have known how to address those issues.” Id.
`
`But the portions of Szeliski II cited by Dr. Cossairt which pertain to fixing occlusions
`
`and parallax errors are specifically in the context of generating rotational panoramas.
`
`See Ex. 1012, at 380-97.5 A rotational panorama image depicts multiple points of
`
`view stitched together from multiple points of view, as illustrated in Szeliski II’s
`
`Figure 9.11:
`
`
`
`
`5 The other portions of Szeliski cited by Cossairt in paragraph 30 (i.e., Ex. 1012, at
`219, 485-86) do not pertain to image stitching or combining images at all.
`
`
`
`16
`
`

`

`
`
`
`Ex. 1012, at 394. The Reply fails to show that rotational panoramas, which are
`
`composite images with multiple points of view, are relevant to the challenged claims,
`
`which require a composite image with the point of view of a single camera.
`
`Apple’s untimely citation to Szeliski II’s discussion of feature-based
`
`registration is deficient for similar reasons. To illustrate: the Reply at page 20
`
`reproduces Szeliski II’s Figure 4.13 (shown below) to contend that feature-based
`
`
`
`17
`
`

`

`registration is usable to “remove [] artifacts” from a composite image. Reply, at 19-
`
`20.
`
`
`
`Figure 4.13 of Szeliski II depicts two images that have different occlusions—e.g.,
`
`the left wall and the roof gable visible in the left image are occluded in the right
`
`frame. But the figure shows only the matching of features that are visible in both
`
`images, whereas it is silent about how features that are visible from only one point
`
`of view or how any other feature-based registration disclosures in Szeliski II would
`
`be used to create a composite image with only one point of view.
`
`2.
`
`Apple’s new obviousness theory for modifying Border
`with any “suitable registration method known in the
`art” is improper
`
`The Reply asserts, for the first time, that a POSITA would have understood
`
`the system in Border to be modifiable to use any number of image registration
`
`techniques known in the art. See Reply, at 14. In Apple’s view, notwithstanding the
`
`Petition’s narrow focus on Border’s pure translation and scale homography-based
`
`registration, “a POSITA would have understood how to modify the image
`
`registration algorithm implemented in the image processor [of Border] with any
`
`
`
`18
`
`

`

`other suitable registration method known in the art.” Id. Apple aims this new
`
`obviousness theory specifically at claim 1’s “registration” limitation. See Reply, at
`
`23-24: “the registration information may be represented by . . . any other registration
`
`models (e.g., registration to correct tilt, feature-based registration, and registration
`
`using a true depth map) taught by Border or known in the art.”
`
`In addition to being untimely, this new obviousness theory for modifying
`
`Border with “other suitable registration method[s] known in the art” is wholly
`
`unsupported by any specific motivation to combine. The word “motivation” does
`
`not even appear in the Reply (and if Apple did attempt to substantiate such a
`
`motivation in its Reply, it also would be improper as a previously-undisclosed
`
`obviousness argument). See Intelligent Bio-Sys., 821 F.3d at 1370 (“the Board did
`
`not err in refusing the reply brief as improper under 37 C.F.R. § 42.23(b) because
`
`IBS relied on an entirely new rationale to explain why one of skill in the art would
`
`have been motivated to combine Tsien or Ju with a modification of Zavgorodny.”).
`
`3.
`
`Apple’s new obviousness combination of Border and
`Parulski against elements [1.8] and [1.10] is improper
`
`As an initial matter, Apple tacitly concedes that its Petition did not discuss
`
`any disclosure from Parulski except as against elements [1.9] and [1.11]. But, on
`
`Reply, Apple tries to shoehorn previously-undiscussed portions of Parulski into its
`
`grounds against [1.8] and [1.10]. See Reply, at 21. Apple’s justification is that
`
`
`
`19
`
`

`

`elements [1.8] through [1.11] are all “interrelated” and, therefore, Apple’s arguments
`
`on Parulski for [1.9] and [1.11] apply also to [1.8] and [1.10]. Id.
`
`Apple’s excuse, however, conflicts with permitted practice. “It is of the
`
`utmost importance that petitioners in the IPR proceedings adhere to the requirement
`
`that the initial petition identify ‘with particularity’ the ‘evidence that supports the
`
`grounds for the challenge to each claim.’” Intelligent Bio-Sys., 821 F.3d at 1369.
`
`Here, Apple elected in its Petition to divide claim 1(c) into four separate elements,
`
`[1.8] through [1.11], and asserted a combination of Border and Parulski against some
`
`of those elements but not others. Now, facing critical deficiencies in its Petition,
`
`Apple seeks to expand the scope of its institution grounds so that a combination of
`
`Border and Parulski is applied to all four elements of claim 1(c). See, e.g., Reply
`
`at 26 (“Petitioner’s Border and Parulski-based analysis of [1.8]-[1.11] applies to
`
`section (c) of claim 1 in its entirety.”). Practically speaking, the Reply seeks to inject
`
`into the proceedings an “entirely new theory of prima facie obviousness absent from
`
`the petition.” Wasica, 8