Paper No. 36
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`____________
`
`
`PATENT OWNER’S REQUEST
`FOR REHEARING BY THE DIRECTOR
`
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`____________
`
`
`PATENT OWNER’S REQUEST
`FOR REHEARING BY THE DIRECTOR
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`
`TABLE OF CONTENTS
`
`BACKGROUND ............................................................................ 1
`I.
`THE MEANING OF “POINT OF VIEW” IS DISPOSITIVE ............ 5
`II.
`III. THE BOARD ERRONEOUSLY FAILED TO CONSTRUE “POINT
`OF VIEW” ..................................................................................... 8
`IV. THE DIRECTOR SHOULD REVIEW AND REVERSE THE
`BOARD’S FINAL WRITTEN DECISION ...................................... 10
`V.
`CONCLUSION ............................................................................. 12
`
`
`
`i
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`TABLE OF AUTHORITIES
`
`Cases
`
`Corephotonics, Ltd. v. Apple Inc.,
`No. 2020-1425, 2021 WL 2012601 (Fed. Cir. May 20, 2021) ............................ 5
`
`In re Sang Su Lee,
`277 F.3d 1338 (Fed. Cir. 2002) .......................................................................... 10
`
`Phillips v. AWH Corp.,
`415 F.3d 1303 (Fed. Cir. 2005) .......................................................................... 10
`
`Trivascular, Inc. v. Samuels,
`812 F.3d 1056 (Fed. Cir. 2016) .......................................................................... 10
`
`United States v. Arthrex, Inc.,
`141 S.Ct. 1970 (2021) ................................................................................... 1, 11
`
`Vivid Techs., Inc. v. Am. Sci. & Eng’g, Inc.,
`200 F.3d 795 (Fed. Cir. 1999) ........................................................................ 9, 10
`
`Statutes
`
`
`
`5 U.S.C. §§ 3345, et seq. ........................................................................................ 11
`
`
`
`ii
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`Patent Owner Corephotonics Ltd. (“Corephotonics”) respectfully requests re-
`
`view by the Director of the Final Written Decision issued by the Board in this matter.
`
`Pursuant to the Supreme Court’s recent decision in United States v. Arthrex, Inc.,
`
`141 S.Ct. 1970 (2021), such review must be conducted by a principal officer
`
`properly appointed by the President and confirmed through advice and consent of
`
`the Senate. This matter has been remanded to the Patent and Trademark Office for
`
`purposes of requesting such review. See Order at 2, Corephotonics, Ltd. v. Apple Inc.,
`
`No. 20-1425, ECF No. 68 (Fed. Cir., July 29, 2021).
`
`Corephotonics submits that the Board’s Final Written Decision in this matter
`
`must be reviewed and rejected because the Board failed to apply a proper construc-
`
`tion for “point of view” in the patent. Indeed, the Board refused to construe that term,
`
`even though Corephotonics proposed construction for it would be dispositive of non-
`
`obviousness. That was clear legal and procedural error that requires the Board’s Fi-
`
`nal Written Decision of unpatentability be reversed and is the type of straightforward
`
`and important error that warrants Director review.
`
`I.
`
`BACKGROUND
`
`The ’152 patent at issue in this proceeding involves an innovative a dual-aper-
`
`ture imaging (“DAI”) system that captures and combines image data from two sep-
`
`arate cameras (a wide-angle camera and a tele-zoom camera), to output a single high-
`
`quality zoomed image. A “different magnification image of the same scene is
`
`1
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`grabbed by each [camera], resulting in field of view (FOV) overlap between the two
`
`[images],” as shown below in annotated Fig. 1B from the patent. ’152 patent (3:11–
`
`14, 6:3–5); Fig. 1b (label 110 indicting the “overlap area” and 112 the “non-overlap
`
`area” between the two images). Here, the red annotation indicates a desired output
`
`image field of view that is intermediate between the fields of view of the two images:
`
`
`
`Since each camera is at a different spatial position, the images taken from the
`
`wide-angle and tele-zoom cameras also each have, even if only slightly, a different
`
`point of view (“POV”), which the patent expressly defines as the “camera angle”
`
`from which an image is captured. ’152 patent (9:26–28). As illustrated in the anno-
`
`tated images reproduced below from a textbook cited in Apple’s petition (Ex. 1008
`
`at 29), the same objects in images taken at different camera angles (i.e., with differ-
`
`ent points of view) will appear to have (1) different relative positions (i.e., appear
`
`2
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`“shifted” in each relative frame) and (2) different “shapes” (i.e., look different be-
`
`cause of the “different perspectives” of each camera). POPR at 14; see also Exhibit
`
`2005, Kosmach Decl., ¶¶ 25-29.
`
`
`
`Apple’s Petition challenged claims 1–4 of ’152 patent. Among other elements,
`
`each of those claims (directly or through dependence) require the final output image
`
`to be from the “point of view of the first camera” (e.g., the wide-angle camera) if the
`
`desired field of view is between that of the two images generated by the cameras
`
`(e.g., as shown by the red box in annotated Fig. 1b above). But because each image
`
`has a different point of view (i.e., taken from different camera angle), the image
`
`generated by the tele-zoom camera cannot simply be “stitched” (i.e., pasted) onto
`
`the appropriate overlapping area of the image from the wide-angle camera, otherwise
`
`objects in the pasted portions would have inconsistent positions and shapes or per-
`
`spective versus the rest of the image. The clearest example of that inconsistency that
`
`3
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`would arise from such simple “stitching” is shown in the image of the buildings
`
`reproduced above, where part of the building is entirely occluded in one image, yet
`
`is visible in the other. Solving that problem was one of the innovative aspects of the
`
`claimed invention—the prior art cited by Petitioner did not account for different
`
`shapes and perspectives when combining two images generated at different camera
`
`angles.
`
`The Board’s Final Written Decision held that claims 1 through 4 are obvious
`
`over the prior art Border and Parulski references. The Board found, inter alia, that
`
`Border disclosed the required claim element of outputting an image from a “point of
`
`view” of the first camera when the field of view is between that of the two cameras.
`
`Border, though, only disclosed image “stitching” using basic mathematical tech-
`
`niques (called “homography”) to paste one image with a narrower field of view into
`
`a small part of another image with a wider field of view, without altering the shape
`
`or perspective of objects in that narrow-field image to account for camera angle, as
`
`the claimed invention does. See infra at II. To reach that finding, the Board expressly
`
`refused to construe “point of view,” which Corephotonics had asked to be construed
`
`as “camera angle” (as it was defined in the patent) which encompasses both position
`
`and shape or perspective. FWD at 7-8.
`
`Corephotonics appealed the Final Written Decision to the Federal Circuit. The
`
`panel that heard the appeal affirmed the Board’s decision in a short nonprecedential
`
`4
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`opinion, but neglected to address the construction of “point of view.” Corephotonics,
`
`Ltd. v. Apple Inc., No. 2020-1425, 2021 WL 2012601, at *3 (Fed. Cir. May 20, 2021).
`
`Instead, after addressing arguments related to Arthrex, the Court curtly affirmed by
`
`referencing the Board’s conclusion that Border disclosed how to mathematically
`
`transform “coordinates from the telephoto to the wide-angle image” (i.e., changing
`
`relative position) and the testimony of Apple’s expert that such alterations satisfied
`
`Apple’s definition for “point of view,” which required position or shape/perspective
`
`to be addressed. Id. Before Corephotonics could file a petition for rehearing to re-
`
`mind the panel about the unadjudicated dispute over the construction of “point of
`
`view” and permit the panel to address it, the Court remanded Corephotonics’s appeal
`
`(along with dozens of others) to allow for Director review in light of the Supreme
`
`Court’s decision in Arthrex.
`
`Corephotonics now seeks Director review to correct the significant error com-
`
`mitted by the Board related to the construction of “point of view.”
`
`II. THE MEANING OF “POINT OF VIEW” IS DISPOSITIVE
`
`Border fails to disclose providing an output image from a “point of view” of
`
`the first camera, as required by the claims. The patent defines “point of view” as
`
`“camera angle.” ’152 patent (9:26-28) (“The output image point of view is deter-
`
`mined according to the primary image point of view (camera angle).”). And, as Core-
`
`photonics explained to the Board, “camera angle” is understood by a POSITA as
`
`5
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`reflecting both the position of objects in the field of view and the shape or perspec-
`
`tive of those objects. POPR at 14; see also PO Sur-Reply at 2-8. In other words, the
`
`same objects in images taken from different camera angles are “shifted and have
`
`different perspectives” (i.e., “shapes”) because of the different points of view. Id.;
`
`see also Exhibit 2005, Kosmach Decl., ¶¶ 25-29.1
`
`Thus, “an output image from a point of view of the first camera” must reflect
`
`the position and shape/perspective of objects relative to the camera angle for the first
`
`camera. And that point of view must be from the first camera alone. See PO Sur-
`
`Reply at 5. Indeed, the ’152 inventors recognized that an output image could have a
`
`“combination” of the “shape or position” of objects from both cameras, but the
`
`claims here require the output be from the point of view of just one camera at a time.
`
`See id.; Ex. 2009 (4:60-65).
`
`Under a correct construction of “point of view,” there can be no dispute that
`
`Border fails to teach outputting an image from the “point of view” of the first camera
`
`when combining images from two cameras. As depicted below in Fig. 6 from Border
`
`(replicated from Apple’s Petition at 38), Border transforms two images (204 and 206)
`
`with different fields of view into one composite image (208) with an intermediate
`
`field of view by “stitching” the image from the camera with a narrower field of view
`
`(e.g., a tele-zoom camera producing image 206) into the appropriate position for the
`
`
`1 All emphasis added throughout unless otherwise noted.
`
`6
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`same objects depicted in the image from a camera with a wider field of view (e.g., a
`
`wide-angle camera producing image 204).
`
`
`
`As Border explains, that image-stitching technique creates a “composite image
`
`208 us[ing] pixel data from the telephoto image 206 for those portions … within the
`
`dashed line 220 that are in the view of the telephoto image 206 and us[ing] pixel
`
`data from the wide image 204 otherwise.” Exhibit 1006 ¶ 47. And as Border teaches,
`
`that simple pasting of the tele-zoom camera’s image into the wide-angle camera’s
`
`image is accomplished by just mathematically “transform[ing] the coordinates of the
`
`telephoto image 206 to the wide image 204” with a simple, well-known “homogra-
`
`phy” function that places the pixels corresponding to certain objects in 206 in their
`
`7
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`correct relative position within the x/y coordinate frame of image 204 while retaining
`
`their original pixel data and resolution so that “the telephoto image 206 covers a
`
`smaller portion of the scene, but with greater resolution.” Exhibit 1006 ¶ 47; see id.
`
`¶¶ 38-40 (showing simple math). Cut and paste/squeeze image 206 into image 204—
`
`that is all that Border does.
`
`Accordingly, under a correct construction of the claims, there can be no dispute
`
`that Border’s output image will not be entirely from the required “point of view of
`
`the first camera” (e.g., the point of view of the wide-camera 204). Though placed in
`
`the relatively correct position in the final scene in the output image, objects inside
`
`the dashed line 220 will be reproduced with the shape and perspective for the point
`
`of view of the telephoto camera 206 (the second camera).
`
`III. THE BOARD ERRONEOUSLY FAILED TO CONSTRUE
`“POINT OF VIEW”
`
`Corephotonics repeatedly explained to the Board how Border does not disclose
`
`outputting an image from the “point of view” of a first camera as required by a proper
`
`construction of that term, which addresses both position and shape/perspective of
`
`objects. See, e.g., POPR at 14; PO Resp. 22-23; PO Sur-Reply at 2-8. And the Board
`
`recognized that in its opinion. See FWD 20-22 (discussing how Corephotonics ar-
`
`gued that Border failed to produce an output image that accounted for “shape and
`
`perspective” of objects in the first image, including the obvious potential “occlusion”
`
`of those objects based on the camera angle of the second camera, as shown in the
`
`8
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`picture of buildings above). But even though there was significant dispute over
`
`whether the term “point of view” required accounting for position and shape/per-
`
`spective of objects (as Corephotonics argued), the Board refused to construe the term
`
`because it concluded that Corephotonics’s proffered construction “does not change
`
`the analysis in this case.” FWD at 8.
`
`The Board therefore did not conduct the required full claim construction inquiry
`
`for “point of view,” including consideration of all of the evidence put forth by the
`
`parties. See, e.g., Vivid Techs., Inc. v. Am. Sci. & Eng’g, Inc., 200 F.3d 795, 803 (Fed.
`
`Cir. 1999) (terms in dispute must be construed if relevant to inquiry); FWD at 8
`
`(acknowledging same precedent). Compounding that error even further, the Board
`
`then purported to reject Corephotonics’s “approach to construing the ‘point of view’
`
`limitation.” FWD at 22. But again, the Board did not actually conduct a claim con-
`
`struction inquiry that addressed the construction that Corephotonics advanced for
`
`“point of view” (“camera angle,” which would have required replicating both posi-
`
`tion and shape/perspective for objects in the first camera image). See, e.g., POPR at
`
`14; PO Resp. 22-23; PO Sur-Reply at 2-8. Instead, the Board “decline[d] to import
`
`a limitation to resolving ‘occlusions’ into the claims” because the claimed invention
`
`would supposedly not be “enabled” if “occlusions” were included—a point that nei-
`
`ther party raised, since enablement is irrelevant to the claim construction inquiry
`
`here. Id. at 22-23; see, e.g., Phillips v. AWH Corp., 415 F.3d 1303, 1327 (Fed. Cir.
`
`9
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`2005) (en banc) (claims may be construed to preserve validity only if still ambiguous
`
`“after applying all the available tools of claim construction”—a rule that does not
`
`apply to “point of view,” which is amenable to construction).
`
`The meaning of “point of view” was dispositive of whether Border’s disclosure
`
`met the claim limitations here, as discussed above. The Board, therefore, should have
`
`conducted a full claim construction analysis of that term—but it did not. See, e.g.,
`
`Vivid Techs., 200 F.3d at 803; FWD at 8 (acknowledging same precedent). That was
`
`legal and procedural error under controlling precedent and the Administrative Pro-
`
`cedure Act. See, e.g., Trivascular, Inc. v. Samuels, 812 F.3d 1056, 1062 (Fed. Cir.
`
`2016) (the Board does not have “unfettered license to interpret the words in a claim”
`
`under the broadest reasonable interpretation standard—it has to follow appropriate
`
`procedure and precedent); In re Sang Su Lee, 277 F.3d 1338, 1344 (Fed. Cir. 2002)
`
`(“Omission of a relevant factor required by precedent is both legal error and arbitrary
`
`agency action” under the “Administrative Procedure Act”).
`
`IV. THE DIRECTOR SHOULD REVIEW AND REVERSE THE
`BOARD’S FINAL WRITTEN DECISION
`
`The meaning of “point of view” is dispositive here and the Board’s decision
`
`was erroneously grounded in a failure to construe that limitation. The Director
`
`should remedy the Board’s error by adopting the correct definition of “point of view”
`
`and, accordingly, reverse the finding of invalidity for the reasons discussed above.
`
`Legal and APA errors like those committed by the Board here should not be left to
`
`10
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`fester while awaiting correction by the Federal Circuit after further action on appeal,
`
`particularly when there are unadjudicated issues that the appellate court cannot ad-
`
`dress in the first instance. The clear and concise yet critical errors by the Board here
`
`are exactly the type that warrant Director review.2
`
`Corephotonics respectfully submits that, consistent with the Supreme Court’s
`
`holding, the review requested herein must be conducted by a principle constitutional
`
`officer properly appointed by the President with the advice and consent of the Senate.
`
`Because Andrew Hirshfeld is not an appointed principle officer, he may not conduct
`
`the requested review without creating the same Constitutional infirmity held to exist
`
`in United States v. Arthrex, 141 S.Ct. 1970 (2021). See, e.g., id. at 1985 (“Only an
`
`officer properly appointed to a principal office may issue a final decision binding
`
`the Executive Branch.”). Also, even if an “Acting Director” (Arthrex, 141 S.Ct. at
`
`1987) could conduct such review (he may not), there is no Acting Director to do so
`
`at present within the meaning of the Federal Vacancies Reform Act of 1998. 5 U.S.C.
`
`§§ 3345, et seq.
`
`
`2 Alternatively, the Board’s decision should be vacated and the matter remanded for
`
`adjudication under a proper construction of “point of view.”
`
`11
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`V. CONCLUSION
`
`For the foregoing reasons, the Director should reverse the decision of the Board
`
`or, at a minimum, vacate that decision and remand for adjudication under a proper
`
`construction of “point of view.”
`
`
`
`Dated: August 30, 2021
`
`
`
` /Neil A. Rubin/
`Neil A. Rubin (Reg. No. 67,030)
`RUSS AUGUST & KABAT
`12424 Wilshire Boulevard, 12th Floor
`Los Angeles, CA 90025
`Telephone: 310-826-7474
`
`Attorney for Patent Owner,
`COREPHOTONICS, LTD.
`
`12
`
`
`
`Case No. IPR2018-01133
`U.S. Patent No. 9,538,152
`CERTIFICATE OF SERVICE
`
`I hereby certify that “Patent Owner’s Request for Rehearing by the Director” was
`
`served on August 30, 2021 by email sent to:
`
`David W. O’Brien
`Hong Shi
`HAYNES AND BOONE, LLP
`600 Congress Ave. Suite 1300
`Austin, TX 78701
`Telephone: 512-867-8400
`Email: david.obrien.ipr@haynesboone.com
`Email: hong.shi.ipr@haynesboone.com
`
`Andrew S. Ehmke
`Michael S. Parsons
`Jamie H. McDole
`HAYNES AND BOONE, LLP
`2323 Victory Ave. Suite 700
`Dallas, TX 75219
`Telephone: 214-651-5000
`Email: andy.ehmke.ipr@haynesboone.com
`Email: michael.parsons.ipr@haynesboone.com
`Email: jamie.mcdole@haynesboone.com
`
` /Neil A. Rubin/
`
`
`
`
`13
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
