`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`Paper 19
`
`
`APPLE INC.,
`Petitioner
`v.
`COREPHOTONICS, LTD.
`Patent Owner
`
`_______________
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`_______________
`PETITIONER’S REPLY
`
` 
`
`
`
`
`
`
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`Table of Contents
`
`I. 
`II. 
`
`Introduction ...................................................................................................... 1 
`Claim Construction .......................................................................................... 1 
`A.  No construction is necessary for the term “point of view.” ................... 1 
`III.  The evidence of record supports the Border/Parulski combination. ............... 2 
`A. 
`Patent Owner mischaracterizes Border’s image stitching and image
`registration. ............................................................................................ 2 
`1. 
`Patent Owner oversimplifies Border’s image stitching. ............. 3 
`2. 
`Patent Owner incorrectly limits Border’s image registration
`to a “simple homography registration.” ...................................... 5 
`Patent Owner misrepresents Petition’s use of Parulski in the
`combination. .......................................................................................... 7 
`1. 
`Petitioner relies on Border for establishing a primary/non-
`primary image relationship. .......................................................... 8 
`Petition relies on Parulski’s image combination teachings for
`obtaining an output image using primary/non-primary images.
` ..................................................................................................... 9 
`A POSITA would have understood why and how to combine
`Border with Parulski’s generating an enhanced primary image
`using a secondary image. .................................................................... 11 
`1. 
`Patent Owner’s “addition” of Parulski’s “primary/non-primary
`image designation step” to Border is unnecessary because
`Border itself establishes the primary/non-primary relationship. .. 11 
`2.  Modifications to Border to accommodate Parulski’s
`teachings would have been within the skill of a POSITA. ....... 12 
`IV.  Border and Parulski render claims 1 and 3 obvious. ..................................... 14 
`A. 
`The Border/Parulski combination renders obvious [1.10]: “wherein
`if FOV2<FOVZF<FOV1 then the point of view of the output image is
`that of the first camera.” ..................................................................... 15 
`1. 
`Patent Owner’s importation of extraneous requirements into
`the claim language “the point of view of the output image is
`that of the first camera” should be rejected. ............................. 15 
`
`B. 
`
`C. 
`
`2. 
`
`
`
`- i -
`
`

`

`
`
`
`V. 
`
`B. 
`
`2. 
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`Border teaches this limitation. .................................................. 16 
`2. 
`The Border/Parulski combination teaches this limitation. ........ 20 
`3. 
`The Border/Parulski combination renders obvious [1.11]: “the
`processor further configured to register the overlap area of the
`second image as non-primary image to the first image as primary
`image to obtain the output image.” ..................................................... 22 
`1. 
`Patent Owner’s importation of extraneous requirements into
`the claim language “register the overlap area…to obtain the
`output image” should be rejected. ............................................. 22 
`Border teaches the partial claim limitation “configured to
`register the overlap area…to obtain the output image.” .......... 23 
`The Border/Parulski combination teaches this limitation. ........ 25 
`3. 
`Border and Parulski render claims 2 and 4 obvious. ..................................... 27 
`A. 
`
`Border teaches that if FOV2≧FOVZF, then the processor is further
`
`configured to provide an output image from a point of view of the
`second camera. .................................................................................... 27 
`The Border/Parulski combination renders claims 2 and 4 obvious. ... 28 
`B. 
`VI.  Conclusion ..................................................................................................... 30 
`VII.  Certificate of Word Count ............................................................................. 31 
`CERTIFICATE OF SERVICE ............................................................................ 32 
`
`
`
`
`
`
`
`
`- ii -
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`PETITIONER’S EXHIBIT LIST
`Updated: June 3, 2019
`
`APPL-1001
`APPL-1002
`APPL-1003
`
`APPL-1004
`APPL-1005
`APPL-1006
`
`APPL-1007
`APPL-1008
`
`U.S. Patent No. 9,538,152 to Shabtay et al. (the “’152 Patent”)
`Prosecution File History of the ’152 Patent (the “’823 App”)
`Prosecution File History of U.S. Provisional App. No.
`61/730,570 (the “’570 App”)
`Declaration of Dr. Oliver Cossairt (“Cossairt”)
`Curriculum Vitae of Dr. Oliver Cossairt
`U.S. Patent Application Publication No. 2008/0030592 to
`Border et. al (“Border”)
`U.S. Patent No. 7,859,588 (“Parulski”)
`Ralph E. Jacobson et al., The Manual of Photography:
`photographic and digital imaging, 9th Edition, 2000
`(“Jacobson”)
`APPL-1009 Michael Langford et al., Langford’s Advanced Photography,
`7th Edition, 2008 (“Langford”)
`Richard Szeliski, Computer Vision: Algorithms and
`Applications, 2011 (“Szeliski”)
`Transcript of Deposition of Dr. James Kosmach
`
`APPL-1010
`
`APPL-1011
`(New)
`APPL-1012
`(New)
`APPL-1013
`(New)
`
`
`Additional Excerpt from Richard Szeliski, Computer Vision:
`Algorithms and Applications, 2011 (“Szeliski II”)
`Supporting Declaration of Dr. Oliver Cossairt, under 37 C.F.R.
`§ 1.68 (“Cossairt Supp.”)
`
`
`
`- iii -
`
`

`

`
`
`I.
`
`
`
`
`Petitioner’s Reply
`IPR2018-01133 (Patent No. 9,538,152)
`
`Introduction
`The Petition and record as a whole provide detailed reasons why the cited art
`
`renders the challenged claims of the ’152 patent obvious. None of the arguments
`
`in the Response adequately refute the evidence of record.
`
`Patent Owner’s arguments fail because they import unclaimed limitations
`
`into the claims, mischaracterize Border’s teachings regarding image stitching and
`
`image registration, and disregard Parulski’s teachings for modifying a primary
`
`image using a secondary image to generate an enhanced primary image. In fact,
`
`Patent Owner attempts to recast Petitioner’s Border and Parulski obviousness ground
`
`as Border-only anticipation (Response, 26n2 and 27) for the claim limitations in
`
`dispute. As discussed below, Patent Owner’s arguments are incorrect,
`
`unsupported, and should be rejected.
`
`II. Claim Construction
`A. No construction is necessary for the term “point of view.”
`
`Patent Owner “believes that the term ‘point of view’ requires no construction,”
`
`but proposes that “[s]hould the Board conclude that it is necessary to construe
`
`‘point of view,’ that term should be construed as ‘camera angle.’” Response, 13.
`
`Because Patent Owner fails to explain why construing the term “point of view” is
`
`necessary and states that it believes that the term ‘point of view’ requires no
`
`construction, no construction is necessary for the term “point of view.” APPL-
`
`1013, ¶¶2-3.
`
`
`
`- 1 -
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`To the extent the Board adopts Patent Owner’s proposed construction of “point
`
`of view” as “camera angle,” such a construction does not materially change
`
`Petitioner’s analysis. APPL-1013, ¶3; see also APPL-1011, 40-41 (Patent Owner’s
`
`expert defining point of view as a “viewpoint,” the construction used by Petitioner).
`
`III. The evidence of record supports the Border/Parulski combination.
`Patent Owner alleges the Petition “fails to explain why or how a POSITA
`
`would combine Border with Parulski’s teaching of modifying a primary image
`
`with a non-primary image.” Response, 31-33. As explained below, the allegation
`
`seeks to manufacture incompatibility based on mischaracterizations of the Border
`
`and Parulski teachings and of Petition’s reliance on Parulski for the combination.
`
`APPL-1013, ¶¶4-27.
`
`A.
`
`Patent Owner mischaracterizes Border’s image stitching and
`image registration.
`
`Patent Owner asserts that Border cannot be combined with Parulski because
`
`“the simple homography registration technique at the heart of Border’s image
`
`stitching system has no use for Parulski’s teaching of ‘determining the primary
`
`image and secondary image from two capture units.’” Response, 32 (emphasis
`
`added). This argument relies on overly simplistic mischaracterizations of Border’s
`
`image stitching and image registration disclosures, which permeate all arguments
`
`in Patent Owner’s Response. Thus, in this section, Petitioner first corrects these
`
`mischaracterizations. APPL-1013, ¶¶5-14.
`
`
`
`- 2 -
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`Patent Owner oversimplifies Border’s image stitching.
`1.
`Patent Owner mischaracterizes Border’s image stitching as “simple image
`
`stitching” that does not provide any geometric transformation for combining
`
`images with different point of views. See, e.g., Response, 2, 9, 20-22, 29-30, and
`
`32. Rather than affording “stitching” the meaning appropriate to its image
`
`processing context as would be understood by a POSITA, the Response seeks to
`
`cabin “stitching” with overly-simplistic meaning as that in a quilt-block-style
`
`stitching context. The mischaracterization is inconsistent with Border’s teachings,
`
`contrary to a POSITA’s understanding of image stitching technology (APPL-1013,
`
`¶¶6-8), and supported only by the declaration of Dr. Kosmach, who admitted in
`
`deposition his lack of knowledge and expertise in image stitching (APPL-1011,
`
`164:9-13).
`
`First, Border’s image stitching uses registration information that
`
`“transforms the coordinates of the telephoto image 206 to the wide image 204” to
`
`transform the telephoto image, such that the transformed telephoto image is from a
`
`point of view of the wide image. Pet.46-47. As such, the composite image includes
`
`information from the wide image from the point of view of the wide image, and
`
`information from the transformed telephoto image with a transformed point of view,
`
`which is the point of view of the wide image. APPL-1013, ¶7. Accordingly, the
`
`
`
`- 3 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`composite image has the same point of view as the wide image. Pet.43-48 (citing
`
`APPL-1010, Fig. 2.12 (below)).
`
`
`
`APPL-1010, Fig. 2.12, annotated
`
`Second, as confirmed by Dr. Cossairt, a POSITA would have understood that
`
`stitching in the context of image processing, like stitching in Border, has a long and
`
`rich history and is different from Patent Owner’s oversimplified quilt-block-style
`
`stitching. APPL-1013, ¶8. Specifically, a POSITA would have understood that
`
`image stitching includes various image registration, compositing, and other image
`
`processing techniques for forming a seamless composite image using images from
`
`different point of views. Id.; see e.g., APPL-1012, 375-408 (explaining that
`
`algorithms “for aligning images and stitching them into seamless photo-mosaics
`
`
`
`- 4 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`are among the oldest and most widely used in computer vision” and that image
`
`stitching includes various registration, e.g., direct pixel-based and feature-based
`
`registration, and compositing techniques).
`
`Third, Patent Owner’s mischaracterization of Border’s image stitching is
`
`supported only by the declaration of its expert Dr. Kosmach, who admitted his lack
`
`of knowledge and expertise in image stitching:
`
`[Q.] So as you sit here today, you don’t know what image
`
`stitching includes and does not include because you’re not an expert
`
`in that area, correct?
`
`A. That – that’s fair to say, yes.
`
`APPL-1011, 164:9-13. Further, Dr. Kosmach’s declaration does not provide any
`
`citations to evidence supporting the simplistic understanding of image stitching.
`
`2.
`
`Patent Owner incorrectly limits Border’s image registration
`to a “simple homography registration.”
`Patent Owner’s assertion that Border cannot be combined with Parulski is also
`
`based on its premise that Border’s image registration teachings are limited to a
`
`“simple homography registration.” Response, 32-33. Again, the
`
`mischaracterization is inconsistent with Border’s teachings, is overly simplistic
`
`even as to homography, and is contrary to a POSITA’s understanding of image
`
`registration. APPL-1013, ¶¶9-14.
`
`
`
`- 5 -
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`First, contrary to Patent Owner’s assertion that Border only uses a simple
`
`homography registration, Border actually teaches various registration models
`
`including homography, feature-based registration, registration with true depth map,
`
`and registration correcting tilt. APPL-1013, ¶10. Border explains one simple 3x3
`
`homography, HTW, with particular coefficients (APPL-1006, [0040]) as an
`
`“example [that] shows a pure translate and scale transformation,” while teaching
`
`that to obtain a composite image, homography or various alternative registration
`
`models may be used. Pet.61.
`
`For example, Border describes an alternative to homography, feature-based
`
`registration:
`
`Alternatively, the registration between images can be determined
`using the image information contained in the wide image 204 and
`telephoto image 20[6]. This is well known in the art of image
`processing…and generally includes the steps of finding interest points
`in each image, making guesses at corresponding points (i.e. a scene
`feature that appears in both images), determining an initial guess at
`the registration, using that initial guess to refine the correspondence
`point guess, and so on based on comparing pixel values or contrast in
`the two images.
`
`
`APPL-1006, [0042]; see also APPL-1012, 181-234 and 273-301 (describing
`
`feature-based registration); APPL-1013, ¶11.
`
`Border describes a “registration model [that] is no longer a simple scale
`
`translation model,” where a “true depth map can also be created and used by the
`
`image resampler 214 to sample the appropriate locations within the telephoto
`
`
`
`- 6 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`image 206 and the wide image 204,” and a registration model used to “correct for
`
`a difference in tilt between the two imaging systems.” APPL-1006, [0041],
`
`[0048]; APPL-1012, 50-51 (explaining using true depth map including depth
`
`coordinates for mapping “two images of 3D scene from different camera positions
`
`or orientations”); APPL-1013, ¶12.
`
`
`
`Second, as discussed above at III.A.1, a POSITA would have understood
`
`that image stitching as described in Border may use various image registration
`
`methods.
`
`Accordingly, because Border itself teaches various registration models (e.g.,
`
`homography, feature-based registration, registration with true depth map, and
`
`registration correcting tilt), and because it was well known in the art that image
`
`stitching may use various registration models (e.g., direct pixel-based or feature-
`
`based registration), Patent Owner’s characterization of Border’s image registration
`
`as limited to “simple homography registration” is incorrect. APPL-1013, ¶¶13-14.
`
`Patent Owner’s incorrect premise is supported only by Dr. Kosmach, who admitted
`
`limitations of his expertise in “registration procedures in DAI [(dual-aperture
`
`imaging)] systems.” See APPL-1011, 31:19-33:2.
`
`B.
`
`Patent Owner misrepresents Petition’s use of Parulski in the
`combination.
`
`Patent Owner alleges, “The Petition uses Parulski in only one way: as a
`
`combination reference to shore up Border’s silence in discussing how a ‘primary
`- 7 -
`
`
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`image’ is designated to be modified by a ‘non-primary image.’” Response, 32. By
`
`misrepresenting Petition’s use of Parulski, Patent Owner attempts to attack
`
`Petitioner’s Border and Parulski obviousness ground as a Border-only anticipation
`
`ground for the claim limitations in dispute. APPL-1013, ¶¶15-20.
`
`1.
`
`Petitioner relies on Border for establishing a primary/non-
`primary image relationship.
`As explained in the Petition, Border teaches establishing the primary/non-
`
`primary image relationship as claimed. Pet.48-49. Specifically, when the zoom
`
`amount, Z, is between 1 and M, in registering telephoto image 206 to wide image
`
`204, Border establishes a primary/non-primary relationship between first image
`
`(wide image 204 as primary image) and second image (telephoto image 206 as
`
`non-primary image), though without express usage of the “primary image” and
`
`“non-primary image” labels. Id.
`
`Patent Owner thus misrepresents the record when it suggests that Border is
`
`silent in discussing a primary/non-primary image relationship, and that the Petition
`
`modifies Border to add Parulski’s “primary/non-primary image designation step.”
`
`See Response, 32 (emphasis added). Rather, the Petition uses Parulski’s express
`
`“primary image” and “secondary image” labels to supplement Border’s teachings of
`
`establishing the primary/non-primary image relationship. Pet.49; APPL-1013, ¶16.
`
`
`
`- 8 -
`
`

`

`
`
`
`2.
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`Petition relies on Parulski’s image combination teachings
`for obtaining an output image using primary/non-primary
`images.
`As explained with Fig. 26 of Parulski below, by modifying a primary image
`
`using a secondary image, Parulski teaches generating an enhanced primary image,
`
`which is an output image from a point of view of the first camera. Pet.51 (citing
`
`APPL-1007, Fig. 26); APPL-1013, ¶17. A POSITA would have been motivated to
`
`apply Parulski’s image combination teachings using primary/non-primary images in
`
`Border’s multi-lens digital camera because the combination would provide the
`
`benefits of enhanced image quality (e.g., “a broadened depth of field,” “a
`
`broadened dynamic range,” “relatively low noise and good sharpness”) in such a
`
`digital camera. Pet.20, 53-54.
`
`
`
`- 9 -
`
`

`

`
`
`
`
`
`
`Petitioner’s Reply
`IPR2018-01133 (Patent No. 9,538,152)
`
`APPL-1007, Fig. 26, annotated
`
`
`
`Specifically, Parulski teaches various image combination techniques for
`
`enhancing the primary image. Pet.51-52. For example, Parulski teaches that the
`
`image combination may include “replac[ing] portions of the primary image [] with
`
`corresponding portions of the secondary image [] to obtain a modified image with
`
`relatively low noise and good sharpness,” APPL-1007, 7:54-8:5, or include
`
`“creat[ing] the modified image in which the pixel values are determined by
`
`considering the pixel values of both the primary and secondary images” to produce an
`- 10 -
`
`
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`extended dynamic range digital image. APPL-1007, 29:36-50; Pet.17; APPL-1013,
`
`¶19.
`
`Petitioner explains that in the combination of Border and Parulski, when the
`
`zoom amount, Z, is between 1 and M, the wide/first image is primary image as Z is
`
`within the zoom range (e.g., from 1 to M) of the wide camera, and the
`
`telephoto/second image is non-primary image, and the telephoto image is used to
`
`modify the wide image to generate a composite image that is an enhanced primary
`
`image. Pet.54. A POSITA would have understood that such an enhanced primary
`
`image is from a point of view of the primary image, i.e., the wide/first image. Id.
`
`A POSITA would have understood that the combination of Border and Parulski may
`
`use various image combination techniques (e.g., image stitching, replacing portions of
`
`the primary image, or determining the pixel values of the output image by considering
`
`pixel values of both primary and secondary images) as taught by Border and Parulski.
`
`APPL-1007, 7:54-8:5, 29:36-50; APPL-1013, ¶20.
`
`C. A POSITA would have understood why and how to combine
`Border with Parulski’s generating an enhanced primary image
`using a secondary image.
`
`1.
`
`Patent Owner’s “addition” of Parulski’s “primary/non-
`primary image designation step” to Border is unnecessary
`because Border itself establishes the primary/non-primary
`relationship.
`Patent Owner argues “Border’s system is unsuitable for modification in the
`
`manner suggested by the Petition, since the addition of a computationally complex
`
`
`
`- 11 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`primary/non-primary image designation step produces no benefit without a further
`
`modification of Border’s image registration technique to something other than a
`
`homography that scales and translates the image.” Response, 32-33.
`
`Because Border itself establishes the image relationship, there is no need for
`
`“addition” of Parulski’s primary/non-primary image “designation step” to Border as
`
`alleged by Patent Owner. APPL-1013, ¶¶21-23. Patent Owner’s incorrect arguments
`
`that Border’s image registration is limited to “a homography that scales and translates
`
`the image,” and that Parulski’s primary/non-primary image determination is
`
`“computationally complex,” are irrelevant. Id. Parulski’s determination is a simple
`
`comparison, and Border’s registration, even in the form of simple homography,
`
`establishes the primary/non-primary relationship. Id.; see also APPL-1007, Fig. 23,
`
`27:8-24, Fig. 14, 22:18-21.
`
`2. Modifications to Border to accommodate Parulski’s
`teachings would have been within the skill of a POSITA.
`A POSITA would have possessed the technical skills needed to perform the
`
`proposed combination—specifically, modifying Border’s image processor to
`
`accommodate Parulski’s teachings. APPL-1013, ¶¶24-27.
`
`As shown with Fig. 5 of Border below, the Petition explains that processor 50
`
`of Border’s digital camera includes image compositor 202 to form a composite image
`
`208 using wide image 204 and telephoto image 206 of the same scene. Pet.14. The
`
`Petition also explains that image registration determiner 212 determines the
`
`
`
`- 12 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`registration between the wide and telephoto images, and image resampler 214
`
`produces the composite image based on a zoom amount Z specifying the desired
`
`relative zoom amount. Pet.14-15.
`
`
`
`APPL-1006, Fig. 5, annotated
`
`As summarized below, a POSITA would have understood how to modify
`
`Border’s image processor to accommodate Parulski’s teachings of generating an
`
`enhanced primary image by modifying a primary image with a secondary image.
`
`First, regarding implementing Parulski’s teachings of image combination methods in
`
`the combination of Border and Parulski, a POSITA would have understood how to
`
`modify Border’s image combination algorithm (e.g., image stitching) implemented in
`
`
`
`- 13 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`image processor 50 (e.g., using image resampler 214 of image compositor 202), with
`
`the image combination algorithms taught by Parulski (e.g., replacing portions of the
`
`primary image with corresponding portions of the secondary image, determining the
`
`pixel values of the output image by considering the pixel values of both the primary
`
`and secondary images). APPL-1013, ¶26.
`
`Second, regarding implementing image registration methods in the
`
`combination of Border and Parulski, a POSITA would have understood the various
`
`registration methods taught by Border (e.g., homography, feature-based registration,
`
`registration with true depth map, and registration correcting tilt) may be used in the
`
`combination. APPL-1013, ¶27. Further, a POSITA would have understood how to
`
`modify the image registration algorithm implemented in the image processor (e.g.,
`
`using image registration determiner 212 of image compositor 202) with any other
`
`suitable registration method known in the art. APPL-1013, ¶27.
`
`IV. Border and Parulski render claims 1 and 3 obvious.
`Element (c) (reproduced below with in-line labels [1.8]-[1.11]) is the only
`
`disputed language of claim 1:
`
`c) [1.8] a processor configured to provide an output image from a
`point of view of the first camera based on a zoom factor (ZF) input
`that defines a respective field of view (FOVZF), [1.9] the first image
`being a primary image and the second image being a non-primary
`image, [1.10] wherein if FOV2<FOVZF<FOV1 then the point of view
`of the output image is that of the first camera, [1.11] the processor
`further configured to register the overlap area of the second image
`
`
`
`- 14 -
`
`

`

`
`
`
`Petitioner’s Reply
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`as non-primary image to the first image as primary image to obtain
`the output image.
`
`
`Patent Owner only disputes limitation [1.10] and a partial limitation of [1.11]
`
`(“configured to register the overlap area…to obtain the output image”), based only on
`
`a Border-only reformulation of Petitioner’s actual ground. Response, 19 (“Border
`
`fails to disclose [1.10]….”), 27 (“Border fails to disclose [1.11]….”). Patent Owner
`
`does not raise any dispute regarding Parulski. Instead, Patent Owner attempts to
`
`recast Petitioner’s Border and Parulski obviousness ground as Border-only
`
`anticipation, completely disregarding the relied upon teachings from Parulski. Even
`
`Patent Owner’s Border-only reformulation imports extraneous requirements into the
`
`claim and is based on previously described mischaracterizations of Border’s image
`
`stitching and image registration, and on mischaracterizations of Dr. Cossairt’s
`
`testimony. As such, Patent Owner’s arguments should be rejected.
`
`A.
`
`The Border/Parulski combination renders obvious [1.10]:
`“wherein if FOV2<FOVZF<FOV1 then the point of view of the
`output image is that of the first camera.”
`
`1.
`
`Patent Owner’s importation of extraneous requirements into
`the claim language “the point of view of the output image is
`that of the first camera” should be rejected.
`Patent Owner argues that “Border fails to discuss any concept of creating an
`
`output image from the images of multiple cameras that is from the point of view of
`
`any specific camera.” Response, 19-20. However, all the claim language requires
`
`is that “point of view of the output image is that of the first camera.” As distraction
`
`
`
`- 15 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`from this actual requirement, Patent Owner seeks to import an extraneous
`
`requirement that the output image is without occlusion or parallax artifacts. See
`
`Response, 22-26. The maneuver seeks to avoid express claim construction.
`
`However, such extraneous requirements are not supported by the ’152 Patent.
`
`The ’152 Patent does not mention occlusion or parallax, much less any solution to
`
`such “requirements” that Patent Owner now formulates years after patent grant.
`
`APPL-1013, ¶29. The Response and Dr. Kosmach’s declaration do not provide any
`
`credible basis for a stealth construction of any term to require an output image free of
`
`any occlusion or parallax artifact. In fact, Dr. Kosmach admitted that a dual-aperture
`
`camera system that cannot resolve occlusion issues still falls within the scope of claim
`
`1. APPL-1011, 147.
`
`Finally, occlusion and parallax were well-known issues in image combination
`
`applications (e.g., image stitching), and a POSITA would have known how to address
`
`those issues when providing a composite image using images with different point of
`
`views. APPL-1013, ¶30; see e.g., APPL-1012, 378-403.
`
`Border teaches this limitation.
`2.
`Patent Owner argues that, in Border’s output image, the overlap area “has
`
`the point of view of the second image” and the portion outside of the overlap
`
`image “has the point of view of the first image” (Response, 20), and that the
`
`
`
`- 16 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`Petition does not explain how Border’s output image is from a particular point of
`
`view (Response, 21).
`
`Patent Owner’s arguments (Response, 20-22) fail because they are based on the
`
`mischaracterizations of Border’s image stitching as oversimplified quilt-block-style
`
`stitching explained above in III.A. APPL-1013, ¶¶31-33. As explained in the
`
`Petition, Border teaches that when zoom amount, Z, is between 1 and M
`
`(FOV2<FOVZF<FOV1), the composite image (i) is generated by modifying wide
`
`image 204 using telephoto image 206 with registration information (e.g.,
`
`represented by homography HTW) that “transforms the coordinates of the
`
`telephoto image 206 to the wide image 204” and (ii) has the point of view of wide
`
`image 204. Pet.44-47. As such, the overlap area of Border’s composite image has
`
`the point of view of the wide image (first image), because the overlap area includes
`
`the telephoto image transformed to the point of view of the wide image using
`
`registration information. APPL-1013, ¶32. Border does not simply include the
`
`original telephoto image as suggested by Patent Owner’s quilt-block-style
`
`“stitching.” Id. In fact, Patent Owner does not directly dispute that Border uses
`
`registration information to transform the telephoto image to the wide image, such
`
`that the transformed telephoto image has the point of view of the wide image.
`
`Further, the Response mischaracterizes Dr. Cossairt’s testimony when it states,
`
`“Dr. Cossairt confirmed that a stitched composite image would have the points of
`
`
`
`- 17 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`view from both the Wide and Tele cameras.” Response, 24. However, the cited
`
`testimony is simply directed to the possibility of occlusion in an output image in the
`
`case of a homography with just scaling and translation. See Response, 24; APPL-
`
`1013, ¶34. Instead, as Dr. Cossairt explains, Border’s stitched composite image, with
`
`occlusion or not, is from the point of view of the wide image. APPL-1013, ¶34.
`
`Specifically, the composite image includes: (1) the overlap region which includes the
`
`transformed telephoto image having the point of view of the wide image, by using
`
`registration information that transforms the coordinates of the telephoto image to
`
`the wide image, and (2) the portion outside of the overlap region, which includes
`
`the corresponding wide image portion, and necessarily has the point of view of the
`
`wide image. Id.
`
`Even though the ’152 Patent does not so require (see IV.A.1), Border
`
`teaches techniques to provide an output image without artifacts (e.g., occlusion,
`
`parallax) that are possible for some images from different point of views. APPL-
`
`1013, ¶¶35-37. In fact, there is no dispute that for planar scenes, Border’s
`
`homography registration teaches providing an output image without such artifacts.
`
`Response, 29; APPL-1011, 117-120 (Dr. Kosmach admitting that planar scenes do
`
`not have occlusion or parallax issues between two images). Indeed, as shown in
`
`the exemplary Fig. 1B from the ’152 Patent itself, a POSITA would have
`
`
`
`- 18 -
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`understood that when a three-dimensional scene is sufficiently far away from a
`
`multi-aperture imaging system, it becomes effectively planar. APPL-1013, ¶35.
`
`APPL-1001, FIG. 1B
`
`
`
`Even for three-dimensional scenes at closer range, Border teaches image
`
`registration methods (e.g., feature-based registration, registration using a true depth
`
`map) for providing an output image with no occlusion from the point of view of
`
`the first camera. APPL-1013, ¶36. For example, Border’s feature-based
`
`registration was well-known to address artifacts (e.g., occlusion) when combining
`
`images from different point of views. Id.; see e.g., APPL-1012, 183 (explaining
`
`“features [] can also be good indicator of [] occlusion events”), 194, Fig. 4.13
`
`(below) (illustrating feature-based registration used to match two images taken
`
`from dramatically different viewpoints). Further, a POSITA would have also
`
`understood other well-known image processing techniques may be used in Border
`- 19 -
`
`
`
`

`

`Petitioner’s Reply
`
`
`IPR2018-01133 (Patent No. 9,538,152)
`
`
`to remove those artifacts. APPL-1013, ¶37; see e.g., APPL-1012, 377, 391, 393-
`
`397, 485-48