UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`APPLE INC.,
`Petitioner,
`v.
`COREPHOTONICS, LTD.
`Patent Owner
`
`_______________
`IPR2018-01133
`U.S. Patent No. 9,538,152
`_______________
`DECLARATION OF OLIVER COSSAIRT, PH.D.
`UNDER 37 C.F.R. § 1.68
`IN SUPPORT OF PETITIONER’S REPLY
`
`Apple v. Corephotonics
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`Table of Contents
`
`2.
`
`2.
`
`I.
`II.
`
`III.
`
`B.
`
`C.
`
`INTRODUCTION ........................................................................................... 1
`CLAIM CONSTRUCTION ............................................................................ 1
`A.
`No construction is necessary for the term “point of view.” ................... 1
`THE COMBINATION OF BORDER WITH PARULSKI ............................ 2
`A.
`Patent Owner mischaracterizes Border’s Image Stitching and Image
`Registration ........................................................................................... 2
`1.
`Patent Owner oversimplifies Border’s image stitching. ............. 3
`2.
`Patent Owner incorrectly limits Border’s image registration to a
`“simple homography registration.” ............................................. 5
`Patent Owner misrepresents Petition’s use of Parulski in the
`combination, and disregards Parulski’s augmentation/combination
`teachings for generating an enhanced primary image. .......................... 8
`1.
`Border is not silent on establishing primary/non-primary image
`relationship. ................................................................................. 8
`Petition relies on Parulski’s image augmentation/combination
`teachings for obtaining an output image using primary/non-
`primary images. ........................................................................... 9
`A POSITA would have understood why and how to combine Border
`with Parulski’s generating an enhanced primary image by modifying a
`primary image using a secondary image. ............................................12
`1.
`The “addition” of Parulski’s “primary/non-primary image
`designation step” to Border as alleged by Patent Owner is
`unnecessary because Border establishes the primary/non-
`primary relationship. .................................................................12
`Any modifications to Border to accommodate Parulski’s
`teachings would have been within the skill of a POSITA. .......13
`IV. BORDER AND PARULSKI RENDER CLAIMS 1 AND 3 OBVIOUS. .... 14
`A.
`The combination of Border and Parulski renders obvious [1.10]:
`“wherein if FOV2<FOVZF<FOV1 then the point of view of the output
`image is that of the first camera.” .......................................................15
`
`Apple v. Corephotonics
`
`- i -
`
`IPR2018-01133 / APPL-1013
`
`

`

`1.
`
`2.
`3.
`
`2.
`
`3.
`
` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`Patent Owner’s importation of extraneous requirements into the
`claim language “the point of view of the output image is that of
`the first camera” should be rejected. ........................................15
`Border teaches this limitation. ..................................................16
`The combination of Border and Parulski teaches this limitation.
` ...................................................................................................20
`The combination of Border and Parulski renders obvious [1.11]: “the
`processor further configured to register the overlap area of the
`second image as non-primary image to the first image as primary
`image to obtain the output image.” .....................................................22
`1.
`Patent Owner’s importation of extraneous requirements into the
`claim language “register the overlap area … to obtain the
`output image” should be rejected. .............................................22
`Border teaches the partial claim limitation “configured to
`register the overlap area … to obtain the output image.” ........22
`The combination of Border and Parulski teaches this limitation.
` ...................................................................................................24
`IV. BORDER AND PARULSKI RENDER CLAIMS 2 AND 4 OBVIOUS. .... 26
`A.
`Border teaches that if FOV2≧FOVZF, then the processor is further
`configured to provide an output image from a point of view of the
`second camera. ....................................................................................26
`To the extent that the condition “FOV2≧FOVZF” requires that both
`FOV2= FOVZF and FOV2> FOVZF, the combination of Border and
`Parulski renders each of claims 2 and 4 obvious. ...............................27
`VI. DECLARATION ........................................................................................... 30
`
`B.
`
`B.
`
`Apple v. Corephotonics
`
`- ii -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`
`Introduction
`
`I am Oliver Cossairt who previously submitted a declaration as APPL-
`
`I.
`
`1.
`
`1004 in this proceeding. The terms of my engagement, my background,
`
`qualifications and prior testimony, and the legal standards and claim constructions
`
`I am applying are set forth in my previous CV and declaration. See APPL-1004;
`
`APPL-1005. I offer this declaration in reply to the Response the Patent Owner filed
`
`in this proceeding. In forming my opinion, I have considered the materials noted in
`
`my previous declaration, as well as the following additional materials:
`
`(1) Additional Excerpt from Richard Szeliski, Computer Vision:
`
`Algorithms and Applications, 2011 (“Szeliski II”), APPL-1012; and
`
`(2) Dr. Kosmach’s declaration, Ex. 2005.
`
`(3)
`
`II.
`
`Paper No. 15 – Patent Owner’s Corrected Response to Petition.
`
`Claim Construction
`
`A.
`
`No construction is necessary for the term “point of view.”
`
`2.
`
`Patent Owner “believes that the term ‘point of view’ requires no
`
`construction,” but proposes that “[s]hould the Board conclude that it is necessary to
`
`construe ‘point of view,’ that term should be construed as ‘camera angle.’”
`
`Response, 13. Because Patent Owner fails to explain why construing the term “point
`
`of view” is necessary and believes that the term ‘point of view’ requires no
`
`construction, no construction is necessary for the term “point of view.”
`
`Apple v. Corephotonics
`
`- 1 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`As explained in my previous declaration, a POSITA would have
`
`3.
`
`understood that prior art’s description of “viewpoint” is consistent with the plain and
`
`ordinary meaning of “point of view.” APPL-1004, ¶¶114, 148. Patent Owner does
`
`not explain any difference between “camera angle” and “viewpoint,” or how such
`
`difference affects the claim analysis. Further, Patent Owner and its expert Dr.
`
`Kosmach use “point of view” and “viewpoint” interchangeably in the Response
`
`and the expert’s declaration. See, e.g., Response, 23-24 (citing Kosmach Decl.,
`
`¶36) (explaining “[a]n example of the differences that can occur with a change in
`
`camera point of view” using Fig. 4.24 of Jacobson with images labeled as “Distant
`
`central viewpoint” and “Closer oblique viewpoint” respectively). Accordingly, to
`
`the extent “point of view” is construed to mean “camera angle,” my analysis in my
`
`previous declaration and this declaration remains unchanged.
`
`III. The combination of Border with Parulski
`
`4.
`
`Patent Owner alleges that the Petition “fails to explain why or how a
`
`POSITA would combine Border with Parulski’s teaching of modifying a primary
`
`image with a non-primary image.” Response, 31-33. The allegation seeks to
`
`manufacture incompatibility based on mischaracterizations of the Border and
`
`Parulski teachings of Petition’s reliance on Parulski for the combination.
`
`A.
`
`Patent Owner mischaracterizes Border’s image stitching and
`image registration
`
`5.
`Patent Owner asserts that Border cannot be combined with Parulski
`Apple v. Corephotonics
`- 2 -
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`because “the simple homography registration technique at the heart of Border’s
`
`image stitching system has no use for the Parulski’s teaching of ‘determining the
`
`primary image and secondary image from two capture units.’” Response, 32.
`
`Both the premise (the simple homography registration of Border has no use for
`
`Parulski’s teachings) and conclusion (cannot combine) are based on simplistic
`
`mischaracterizations of Border’s image stitching and image registration disclosure
`
`Patent Owner oversimplifies Border’s image stitching.
`1.
`Patent Owner mischaracterizes Border’s image stitching as “a simple
`
`6.
`
`image stitching” that does not provide any geometric transformation for combining
`
`images with different point of views. See, e.g., Response, 2, 9, 20-22, 29-30, and
`
`32. Rather than affording Border’s “stitching” the meaning appropriate to its
`
`image processing context would have been understood by a POSITA, the Response
`
`seeks to cabin “stitching” with a simplistic meaning as that in a quilt block
`
`stitching context. The mischaracterization is inconsistent with Border’s teachings,
`
`contrary to a POSITA’s understanding of image stitching, and supported only by
`
`the declaration of Dr. Kosmach, who admitted in deposition his lack of knowledge
`
`and expertise in image stitching.
`
`7.
`
`First, Patent Owner’s mischaracterization of Border’s image stitching
`
`is inconsistent with Border’s teachings. Border’s image stitching uses registration
`
`information that “transforms the coordinates of the telephoto image 206 to the
`
`Apple v. Corephotonics
`
`- 3 -
`
`IPR2018-01133 / APPL-1013
`
`

`

`Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`wide image 204”to transform the telephoto image 206, such that the transformed
`
`telephoto image 206 is from a point of view of the wide image 204. APPL-1004,
`
`111, 114. As such, Border teaches generating the composite image includes
`
`information from the wide image from the point of view of the wide image, and
`
`information from the transformed telephoto image with a transformed point of
`
`view, whichis the point of view of the wide image. Accordingly, the composite
`
`image has the samepoint of view as the wide image. APPL-1004, 4114.
`
`a point of view
`a point of view
`
`from a second fromafirstregistration scene
`camera
`
`
`
`
`
`camera
`
`Figure 2.12 A point is projected into two images: (a) relationship between the 3D point coordinate (X, Y, Z, 1)
`and the 2D projected point (a, y, 1, d); (b) planar homography induced by points all lying on a common plane
`No: pt+c=0.
`
`(APPL-1010), Szeliski, Fig. 2.12, annotated
`
`8.|A POSITA would have understoodthatstitching in the context of image
`
`processing,like stitching in Border, has a long andrich history andis different from
`
`Patent Owner’s oversimplified quilt block-style stitching. Specifically, a POSITA
`
`Apple v. Corephotonics
`
`-4-
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`would have understood that image stitching includes various image
`
`registration/alignment e.g. direct pixel based registration and feature based
`
`registration, compositing, and other image processing techniques for forming a
`
`seamless composite image using images from different point of views. See e.g.,
`
`APPL-1012, 375-408 (Chapter 9 titled “Image stitching” describing that image
`
`stitching includes various registration including e.g., direct pixel based registration
`
`and feature based registration, compositing, and other techniques to form a
`
`seamless composite image from images from different point of views), 377
`
`(“Algorithms for aligning images and stitching them into seamless photo-mosaics
`
`are among the oldest and most widely used in computer vision”), 391 (describing
`
`parallax removal), 393-396 (comparing direct pixel based registration and feature
`
`based registration), 396-397 (describing performing coordinate transformations for
`
`providing a composite image with a single point of view).
`
`2.
`
`Patent Owner incorrectly limits Border’s image registration
`to a “simple homography registration.”
`Patent Owner’s assertion that Border cannot be combined with
`
`9.
`
`Parulski is also based on its premise that Border’s image registration teachings are
`
`limited to “a simple homography registration.” Response, 32-33. Again, the
`
`mischaracterization is inconsistent with Border’s teachings, is overly simplistic
`
`even as to homography, and is contrary to a POSITA’s understanding of image
`
`registration.
`
`Apple v. Corephotonics
`
`- 5 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
` Petition’s Reply in IPR2018-01133
`First, Patent Owner’s mischaracterization of Border’s image
`
`10.
`
`registration is inconsistent with Border’s actual teachings. Contrary to Patent
`
`Owner’s assertion that Border only uses a simple homography registration, Border
`
`actually teaches various registration models including homography, feature-based
`
`registration, registration with true depth map, and registration correcting tilt.
`
`11.
`
`For example, Border describes an alternative feature-based
`
`registration to homography that was “well known in the art of image processing:”
`
`Alternatively, the registration between images can be determined
`using the image information contained in the wide image 204 and
`telephoto image 20[6]. This is well known in the art of image
`processing (for example, image registration is described in U.S. Pat.
`No. 6,078,701) and generally includes the steps of finding interest
`points in each image, making guesses at corresponding points (i.e. a
`scene feature that appears in both images), determining an initial
`guess at the registration, using that initial guess to refine the
`correspondence point guess, and so on based on comparing pixel
`values or contrast in the two images.”
`
`APPL-1006, [0042]. A POSITA would have understood that such a registration
`
`determined using the image information contained in the wide image and telephoto
`
`image is also referred to as a feature-based registration. See e.g., APPL-1012, 181-
`
`234 and 273-301 (describing feature detection and feature-based registration and
`
`explaining that localized features often called keypoint features or interest points
`
`“can be matched based on their orientation and local appearance (edge profiles)
`
`and can also be good indicator of object boundaries and occlusion events”).
`
`12.
`In addition to the homography registration and feature-based
`Apple v. Corephotonics
`- 6 -
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`registration, Border describes a “registration model [that] is no longer a simple
`
`scale translation model,” where a “true depth map can also be created and used
`
`by the image resampler 214 to sample the appropriate locations within the
`
`telephoto image 206 and the wide image 204.” APPL-1006, [0048]; APPL-
`
`1012, 50-51 (explaining using true depth map including depth coordinates for
`
`mapping “two images of 3D scene from different camera positions or
`
`orientations”). Border also teaches that yet another example of a registration
`
`model other than “the example show[ing] a pure translation and scale
`
`transformation” is a registration that is used to “correct for a difference in tilt
`
`between the two imaging systems.” APPL-1006, [0041].
`
`13. As discussed above at ¶61, a POSITA would have understood that
`
`image stitching as described in Border may use various image registration
`
`methods. See e.g., APPL-1012, 393-396 (discussing pros and cons of direct pixel-
`
`based registration/alignment and feature-based registration/alignment in image
`
`stitching).
`
`14. Accordingly, because Border’s itself teaches various registration
`
`models (e.g., homography, feature-based registration, registration with true depth
`
`map, and registration correcting tilt), and because it was well known in the art that
`
`image stitching may use various registration models (e.g., direct pixel-based or
`
`feature-based registration), Patent Owner’s characterization of Border’s image
`
`Apple v. Corephotonics
`
`- 7 -
`
`IPR2018-01133 / Ex. 1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`registration as limited to “simple homography registration” is incorrect.
`
`B.
`
`Patent Owner misrepresents Petition’s use of Parulski in the
`combination, and disregards Parulski’s
`augmentation/combination teachings for generating an enhanced
`primary image.
`
`15.
`
`Patent Owner alleges, “The Petition uses Parulski in only one way: as
`
`a combination reference to shore up Border’s silence in discussing how a ‘primary
`
`image’ is designated to be modified by a ‘non-primary image.’” Response, 32.
`
`However, Patent Owner misrepresents Parulski’s teachings applied in the combination
`
`of Border and Parulski.
`
`1.
`
`Border is not silent on establishing primary/non-primary
`image relationship.
`16. As explained in my previous declaration, Border teaches establishing
`
`the primary/non-primary relationship between first and second images as claimed,
`
`while Parulski uses the labels “primary image” and “secondary image” to describe
`
`the roles of respective images used in forming a composite image. APPL-1004,
`
`¶¶117-118. As such, it is not necessary to use Parulski’s “primary/non-primary
`
`image designation step” in Border as incorrectly asserted by the Response. See
`
`Response, 32 (emphasis added). Instead, in the combination, Parulski’s teachings of
`
`primary/secondary image relationship using express “primary image” and
`
`“secondary image” labels supplement Border’s teachings of establishing the
`
`primary/non-primary image relationship, where Border does not use express
`
`Apple v. Corephotonics
`
`- 8 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`“primary image” and “non-primary image” labels.
`
`2.
`
`Petition relies on Parulski’s image
`augmentation/combination teachings for obtaining an
`output image using primary/non-primary images.
`17. As discussed in my previous declaration and explained with Fig. 26 of
`
`Parulski below, by modifying a primary image using a secondary image, Parulski
`
`teaches generating an enhanced primary image, which is an output image from a
`
`point of view of the first camera. A POSITA would have been motivated to apply
`
`Parulski’s teachings for modifying a primary image using a secondary image to
`
`generate an enhanced primary image in Border’s multi-lens digital camera because
`
`the combination would provide the benefits of enhanced image quality (e.g., “a
`
`broadened depth of field,” “a broadened dynamic range,” “relatively low noise and
`
`good sharpness”) in such a digital camera. APPL-1004, ¶61.
`
`18. As shown in Fig. 26 of Parulski annotated in my previous declaration
`
`below, by modifying a primary image using a secondary image, Parulski teaches
`
`generating an output image from a point of view of the first camera.
`
`Apple v. Corephotonics
`
`- 9 -
`
`IPR2018-01133 / APPL-1013
`
`

`

`Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`
`Sanson NO
`
`
`PREVIEW)TIERBUT CRETORNTOPREVI)
`(_RETURNTO
`YES
`
`1802
`
`1804
`
`1806
`
`1808
`
`1810
`
`1812~L
`
`.
`.
`First image being a
`primary image
`
`Second image being
`.
`anon-primary ~
`
`1816
`
`meet
`
`i
`
`,
`Output image from
`a point of view o
`the first camera
`
`CAPTUREPREVIEWIMAGE
`FROM PRIMARY CAPTURE UNIT
`
`CAPTURE IMAGE FROM
`SCENE ANALYSIS CAPTURE UNIT
`
`ANALYZE SCENE UTILIZING CAPTURED
`PREVIEW AND SCENE ANALYSIS IMAGES
`
`SET PRIMARY CAPTURE UNIT PARAMETERS
`UTILIZING RESULTS OF THE SCENE ANALYSIS
`
`SET SCENE ANALYSIS CAPTURE UNIT
`AS A SECONDARY CAPTURE UNIT
`
`SET SECONDARY CAPTURE UNIT PARAMETERS
`UTILIZING RESULTS OF THE SCENE ANALYSIS
`1814
`
`iS $2
`SHUTTER BUTTON
`PRESSED
`
`No
`
`YES
`
`CAPTURE PRIMARY IMAGE
`
`FROM SCENE ANALYSIS CAPTURE UNIT
`
`THE PRIMARY AND AUGMENTATION IMAGES
`
`(APPL-1007), Parulski, Fig. 26, annotated
`
`19.
`
`_Parulski teaches various image augmentation/combination techniques
`
`for enhancing the primary image using the secondary imageto obtain the enhanced
`
`primary image(the output image) from a point of view ofthe first camera.
`
`Parulski teaches that the image combination may includereplacing portions of the
`
`primary image with correspondingportions of the secondary image, or by determining
`
`the pixel values of the output image “by considering the pixel values ofboth the
`
`primary and secondary images.” APPL-1007, Fig. 26, 7:54-8:5 (“replac[ing] portions
`
`Apple v. Corephotonics
`
`-10-
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`of the primary image (areas of lower noise but with some motion blur) with
`
`corresponding portions of the secondary image (areas of higher noise but little or no
`
`motion blur) to obtain a modified image with relatively low noise and good
`
`sharpness”), 29:36-50 (“a sliding scale is used to create the modified image in which
`
`the pixel values are determined by considering the pixel values of both the primary
`
`and secondary images, as described in commonly-assigned, copending U.S. patent
`
`application Ser. No. 11/460,364 (which was filed Jul. 27, 2006 in the names of John
`
`Border and Efrain Morales, and entitled “Producing an Extended Dynamic Range
`
`Digital Image”), which is incorporated herein by reference.”).
`
`20. A POSITA would have understood that in the combination of Border
`
`and Parulski, various image combination techniques may be used to generate the
`
`enhanced primary image, e.g., image stitching as taught by Border, replacing portions
`
`of the primary image with corresponding portions of the secondary image as taught by
`
`Parulski, or determining the pixel values of the output image by considering the pixel
`
`values of both the primary and secondary images as taught by Parulski. APPL-1007,
`
`7:54-8:5, 29:36-50.
`
`Apple v. Corephotonics
`
`- 11 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`A POSITA would have understood why and how to combine
`Border with Parulski’s generating an enhanced primary image by
`modifying a primary image using a secondary image.
`
`1.
`
`The “addition” of Parulski’s “primary/non-primary image
`designation step” to Border as alleged by Patent Owner is
`unnecessary because Border establishes the primary/non-
`primary relationship.
`Patent Owner argues “Border’s system is unsuitable for modification in
`
`C.
`
`21.
`
`the manner suggested by the Petition, since the addition of a computationally complex
`
`primary/non-primary image designation step produces no benefit without a further
`
`modification of Border’s image registration technique to something other than a
`
`homography that scales and translates the image.” Response, 32-33.
`
`22. Because Border itself establishes the image relationship, there is no need
`
`for “the addition” of Parulski’s primary/non-primary image designation step to
`
`Border’s digital imaging system as alleged by Patent Owner. Patent Owner’s
`
`argument Border’s image registration is limited to only “a homography that scales and
`
`translates the image,” and that Parulski’s primary/non-primary image determination as
`
`“computationally complex,” which themselves are incorrect, are not relevant here.
`
`Furthermore, a POSITA would have understood that Parulski’s primary/non-primary
`
`image determination is not “computationally complex.” See APPL-1007, Fig. 23,
`
`27:8-24, Fig. 14, 22:18-21 (zoom checking blocks 502 and 1502 determines primary
`
`and secondary images by a computationally simple comparison).
`
`23. Because Border establishes the primary/non-primary image relationship,
`
`Apple v. Corephotonics
`
`- 12 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`there is no need for “the addition” of Parulski’s primary/non-primary image
`
`designation step to Border’s digital imaging system as alleged by Patent Owner.
`
`Instead, the combination applies Parulski’s image augmentation/combination
`
`teachings for generating an enhanced primary image by modifying the primary image
`
`with the secondary image in Border’s system.
`
`2. Modifications to Border to accommodate Parulski’s
`teachings would have been within the skill of a POSITA.
`24. A skilled artisan before the ’512 Patent would have possessed the
`
`technical skills needed to perform the proposed combination—specifically,
`
`modifying Border’s image processor 50 (including image compositor 202, image
`
`registration determiner 212, and image resampler 214) to accommodate Parulski’s
`
`teachings.
`
`25. As explained in my previous declaration and illustrated in Fig. 5 of
`
`Border below, image processor 50 of Border’s digital camera includes an image
`
`compositor 202 to form a composite image 208 using the two images, wide image 204
`
`and telephoto image 206 of the same scene. APPL-1004, ¶¶51-52.
`
`26. As summarized below, a POSITA would have understood how to
`
`modify the image processor 50 of Border to accommodate Parulski’s teachings of
`
`generating an enhanced primary image by modifying a primary image with a
`
`secondary image. First, regarding implementing Parulski’s teachings of image
`
`combination methods in the combination of Border and Parulski, a POSITA would
`
`Apple v. Corephotonics
`
`- 13 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`have understood how to modify the image combination algorithm (e.g., image
`
`stitching) as taught by Border implemented in the in the image processor 50 (e.g.,
`
`using image resampler 214 of image compositor 202) with image combination
`
`algorithms (e.g., replacing portions of the primary image with corresponding portions
`
`of the secondary image, determining the pixel values of the output image by
`
`considering the pixel values of both the primary and secondary images) as taught by
`
`Parulski.
`
`27.
`
`Second, regarding implementing image registration methods in the
`
`combination of Border and Parulski, a POSITA would have understood any
`
`registration methods taught by Border (e.g., homography, feature-based registration,
`
`registration with true depth map, and registration correcting tilt) may also be used
`
`in the combination of Border and Parulski. Further, a POSITA would have
`
`understood how to modify the image registration algorithm implemented in the image
`
`processor 50 (e.g., using image registration determiner 212 of image compositor 202)
`
`with any other suitable registration method known in the art.
`
`IV. Border and Parulski render claims 1 and 3 obvious.
`
`Element (c) (reproduced below with in-line labels [1.8]-[1.11]) is the only
`
`disputed language of claim 1:
`
`c) [1.8] a processor configured to provide an output image from a
`point of view of the first camera based on a zoom factor (ZF) input
`that defines a respective field of view (FOVZF), [1.9] the first image
`being a primary image and the second image being a non-primary
`Apple v. Corephotonics
`- 14 -
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`image, [1.10] wherein if FOV2<FOVZF<FOV1 then the point of view
`of the output image is that of the first camera, [1.11] the processor
`further configured to register the overlap area of the second image
`as non-primary image to the first image as primary image to obtain
`the output image.
`
`A.
`
`The combination of Border and Parulski renders obvious [1.10]:
`“wherein if FOV2<FOVZF<FOV1 then the point of view of the
`output image is that of the first camera.”
`
`1.
`
`Patent Owner’s importation of extraneous requirements into
`the claim language “the point of view of the output image is
`that of the first camera” should be rejected.
`Patent Owner argues that “Border fails to discuss any concept of
`
`28.
`
`creating an output image from the images of multiple cameras that is from the
`
`point of view of any specific camera.” Response, 19-20. However, all the claim
`
`language requires is that “point of view of the output image is that of the first
`
`camera.” As distraction from this actual requirement, Patent Owner seeks to
`
`import extraneous requirements (e.g., an output image without occlusion or
`
`parallax artifact caused by differences in the first and second images) into the term
`
`“the point of view of the output image is that of the first camera” without express
`
`claim construction. See Response, 22-26.
`
`29.
`
`Such importation of extraneous requirements into the term “the point
`
`of view of the output image is that of the first camera” is not supported by the ’152
`
`Patent. The ’152 Patent does not mention occlusion or parallax, much less detail a
`
`solution to such requirements.
`
`Apple v. Corephotonics
`
`- 15 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`In fact, occlusion and parallax were well known issues in image
`
`30.
`
`combination applications (e.g., image stitching), and a POSITA would have known
`
`how to address those issues caused by differences in images from different point of
`
`views when providing a composite image using those images with different point of
`
`views. See e.g., APPL-1012, 377 (“Algorithms for aligning images and stitching
`
`them into seamless photo-mosaics are among the oldest and most widely used in
`
`computer vision”), 391 (describing parallax removal), 393-396 (comparing direct
`
`pixel based registration and feature based registration), 396-397 (describing
`
`performing coordinate transformations for providing a composite image with a
`
`single point of view), 485-486 (“Partial occlusion is handled explicitly by
`
`assigning a group of pixels in one image to a single pixel in the other image.”), 219
`
`(“More local descriptors of curve shape such as shape contexts [] can also be used
`
`for recognition and are potentially more robust to missing parts due to
`
`occlusions.”).
`
`Border teaches this limitation.
`2.
`First, Patent Owner argues that, in the overlap area, Border’s output
`
`31.
`
`image “has the point of view of the second image” and that the portion outside of
`
`the overlap image “has the point of view of the first image” (Response, 20) and
`
`that the Petition does not explain how Border’s output image is from a particular
`
`point of view (Response, 21). However, the Petition explains that Border teaches
`
`Apple v. Corephotonics
`
`- 16 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver Cossairt, Ph.D. in Support of
`Petitioner’s Reply in IPR2018-01133
`limitation [1.10] because it teaches that when zoom amount, Z, is between 1 and M
`
`( (FOV2<FOVZF<FOV1), the composite image (i) is generated by modifying wide
`
`image 204 using telephoto image 206 with registration information (e.g.,
`
`represented by homography HTW) that “transforms the coordinates of the telephoto
`
`image 206 to the wide image 204” and (ii) has the point of view of wide image
`
`204. APPL-1004, ¶¶111-114.
`
`32. As such, the overlap area of Border’s composite image has the point of
`
`view of the wide image (first image) because it does not include the original telephoto
`
`image as suggested by Patent Owner’s quilt-block style “stitching.” Instead, the
`
`overlap area includes the telephoto image transformed to the point of view of the wide
`
`image using registration information. In fact, Patent Owner does not directly
`
`dispute that Border uses registration information to transform the telephoto image
`
`to the wide image, such that the transformed telephoto image has the point of view
`
`of the wide image.
`
`33.
`
`Second, Patent Owner’s arguments fail because they are based on
`
`mischaracterizations of (i) Border’s image stitching (Response, 20-22) as an
`
`oversimplified stitching and (ii) Border’s image registration as limited to a “scale and
`
`translation” only homography (Response, 22, 24), which are the same
`
`mischaracterizations Patent Owner relied upon to allege that Border and Parulski do
`
`not combine as discussed in Section III.A.
`
`Apple v. Corephotonics
`
`- 17 -
`
`IPR2018-01133 / APPL-1013
`
`

`

` Declaration of Oliver