ELECTRONICALLY FILED IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`CAMERA-BASED TRANSPARENT DISPLAY
`
`Docket
`
`Art Unit
`
`Conf. No.
`
`:
`
`:
`
`:
`
`P33303USl (119-1289USl)
`
`2699
`
`6497
`
`Title
`
`Inventors
`
`Serial No.
`
`:
`
`:
`
`I
`
`Brett D. Miller, et al.
`
`16/125,363
`
`Examiner
`
`: Nicholas G. Giles
`
`Customer
`
`:
`
`61947
`
`Filed via EF SWEB
`
`Commissioner for Patents
`
`PO. Box 1450
`
`Alexandria, VA 22313-1450
`
`PRE-APPEAL BRIEF REQUEST FOR REVIEW
`
`RESPONSIVE TO OFFICE ACTION MAILED AUGUST 23, 2019
`
`This paper is intended to be a complete response to the above-identified Action. Assignee
`
`thanks the Examiner for carefully considering the application.
`
`Reiection of Claims 1'5: 7'10: 13: 14: 16x 18 and 19 Under Section 102
`
`Claims 1-5, 7-10,
`
`l3,
`
`l4, 16, 18 and 19 stand rejected under 35 U.S.C. 102(a)(1) as
`
`allegedly being anticipated by Robinson (US Pub. No. 2013/0207896, hereinafter Robinson).
`
`Office Action at page 3. Assignee respectfully traverses the rejection.
`
`Claim 1 is independent and recites, in part:
`
`capturing an image of a scene by a camera of an electronic device, wherein the
`electronic device comprises the camera and a display, and wherein the camera and the
`display have a first spatial relationship,
`determining a second spatial relationship between a viewpoint and the display of
`the electronic device,
`warping the image to obtain an image of a first portion of the scene based on the
`first spatial relationship and the second spatial relationship...
`wherein, from the viewpoint, the image of the first portion of the scene is
`substantially contiguous with a second portion of the scene visible outside an edge of the
`electronic device.
`
`In rejecting the claims, the Examiner relies generally on paragraphs 20-53 and paragraphs
`
`68-76, along with Figs. 1, 2A, 2BN, 2C, 3, 4A, 4B, 5A, 5B, and 8. Initially Applicant respectfully
`
`lof5
`
`

`

`Reply to Action Mailed August 23, 2019
`Filed Electronically on November 25, 2019
`
`Serial No: 16/125,363
`Docket No: P33303USl (119-1289USl)
`
`reminds the Examiner that “the pertinence of each reference, if not apparent, must be clearly
`
`explained and each rejected claim specified.” See 37 CFR 1.104. Further, a review of the reference
`
`reveals that the cited art is actually directed to determining a viewpoint of a user positioned in front
`
`of a display screen and an object tracking component to track the user manipulation of an object
`
`positioned behind the display screen. See Robinson, Abstract. At best, an image is displayed
`
`based on the viewpoint of the user. More specifically, as described with respect to FIG. 4A, an
`
`image is captured behind a display device. A user’s gaze is determined. The object behind the
`
`display is presented on the display at a location based on the user’s gaze. As such, Robinson does
`
`not present “the image of the first portion of the scene is substantially continuous with a second
`
`portion of the scene visible outside an edge of the electronic device” because Robinson is
`
`unconcerned with the portion of the scene that is n_ot obstructed by the display device.
`
`Said another way, in Robinson, the object and the arm are obfuscated by the display device.
`
`If the display is transparent, then the arm and object are visible. If the display is opaque, then they
`
`are presented as captured by a camera on the underside of the device. Then, digital content is
`
`presented and displayed based on the user viewpoint and the object location. Thus, the image is
`
`not wamed based on a second portion of the scene visible outside an edge of the electronic device
`
`because in Robinson, the digital content is merely presented in a particular location based on a
`
`viewpoint of a user and a location of the object behind the screen.
`
`In the Response to Arguments section, the Examiner takes the position that “as seen in the
`
`Figs. cited, the part of the arm displayed on ten display and the part of the same arm in the space
`
`not blocked by the display is seen as the same continuous arm and is a result of Robinson
`
`generating pixel locations on the display screen that correspond to the content for display based
`
`on the user’s line of sight between two points.” See Final Rejection, p. 3. However, Robinson at
`
`paragraph [0021]
`
`indicates that “lit
`
`the emhetfiment Shawn in FIGS. 1-3 and 5-45,
`
`the
`
`display 1'10 includes a display screen l 12 that is eemprised of a, transparent screen material.”
`
`(ea/rphtm‘s added). That is, in direct contrast to the Examiner’s centehtions, the display in the
`
`fighting is intended to be transparent, and thug, the user’s arm shown is the actual arm, and not a
`
`rendering of the arm,
`
`2of5
`
`

`

`Reply to Action Mailed August 23, 2019
`Filed Electronically on November 25, 2019
`
`Serial No: 16/125,363
`Docket No: P33303USl (119-1289USl)
`
`Only at paragraph [0024} {lees Rehinsen inentinn that “in an alternative enihndiment, the
`
`display screen is spades but transparency is sinuilated by displaying a View at" objects behind the
`
`display screen captured by a video camera that captures yirlee behind the display screen.” in an
`
`embodiment in which the screen is npaaue, Rehinsen merely indicates that the view if objects
`
`behind the display screen is captured by the videe camera and displayed on the sereen. Robinson
`
`inentiens that “transparency is simulated,” but is siient regarding how that happens.
`
`in fact,
`
`nothing in the text indicates that a warping function of Robinson causes the displayed portion to
`
`appear substantially contiguous with a second portion of the scene visible outside an edge of the
`
`electronic device.
`
`For at least the reasons described above, Robinson fails to disclose each and every feature
`
`of the independent claims. As such, claims 1, 7, and 16 are allowable over Robinson. The
`
`remaining claims depend, directly or indirectly from Robinson and, thus, are also allowable at least
`
`by virtue of their dependence from allowable independent claims. Withdrawal of the rejection is
`
`respectfully requested.
`
`Reiection of Claims 2: 8x and 16
`
`Claims 2, 8, and 16 are allowable over Robinson for additional reasons. Namely, claim 2
`
`recites:
`
`back-projecting the image based on intrisics of the camera and a depth in the scene; and
`re-projecting the image to the display based on the second spatial relationship.
`
`In rejecting the claims, the Examiner takes the position that the “back-projecting” feature
`
`is disclosed by Robinson because “when using depth cameras the 3D information is converted into
`
`a 2D image for content display.” See Office Action, p. 6. The portion of Robinson describing the
`
`conversion of 3D to 2D imagery is found at paragraph [0059] of Robinson.
`
`In fact, Robinson
`
`merely mentions that a 2D description of the overlaid image can be converted into a 3D overlaid
`
`object on a 2D display screen. Nothing in Robinson indicates that the image of the scene may be
`
`back-projected based on intrinsics of the camera.
`
`Moreover, Robinson is silent regarding “re-projecting the image to the display based on
`
`the second spatial relationship.” Nothing in the cited portion indicates any reprojection based on
`
`3of5
`
`

`

`Reply to Action Mailed August 23, 2019
`Filed Electronically on November 25, 2019
`
`Serial No: 16/ 125,363
`Docket No: P33303USl (119-1289USl)
`
`a spatial relationship between the viewpoint and the display ofthe electronic device. The Examiner
`
`merely points out a location of an object behind the screen and a viewpoint, but nothing in the
`
`Examiner’s response addresses the reprojection of the image of the scene.
`
`Rejection of Claims 4, 10, and 18
`
`Claims 4, 10, and 18 are allowable over Robinson for additional reasons. Namely, claim
`
`2 recites:
`
`and
`
`identifying four image corners, based on the intrinsics of the camera and the depth;
`
`intersecting the four image corners with a plane defined by the display to obtain a
`2D image transform.
`
`In rejecting the claims, the Examiner takes the position that “while the corners are not
`
`explicitly mentioned, the display screen has four comers as seen and the image is displayed within
`
`and bound by the four corners based on the position of the object tracking sensors 128a and 148b
`
`and the viewpoint of the user.” See Office Action, pp. 6-7. However, the object tracking sensors
`
`are only utilized to track an object in the scene. The comers of the display scene in Robinson are
`
`not related to the intrinsics of the camera or a depth.
`
`Regarding the “intersection” feature,
`
`the Examiner takes the position that “the 2D
`
`displayed image is based on using the user’s line of sight between two points.” In fact, the user’s
`
`line of sight and the object are used to determine where to display the generated content, and n_ot
`
`for how to display the image of the scene. Moreover, nothing in Robinson describes any such 2D
`
`image transformation by intersecting four image corners with a plane defined by the display.
`
`Rejection of Claims 6, 11, 12, 15, 17 and 20 Under Section 103
`
`Claims 6, 11, 12, 15, 17 and 20 stand rejected under 35 U.S.C. 103 as allegedly being
`
`unpatentable over Robinson. Office Action at page 9. Assignee respectfully traverses the rejection.
`
`Claim 6 depends from claim 1, claims 11, 12, and 15 depend from claim 7, and claims 17
`
`and 20 depend from claim 16. As such, claims 6, 11, 12, 15, 17 and 20 are allowable at least by
`
`virtue of their dependence from an allowable independent claim. Withdrawal of the rejection is
`
`respectfully requested.
`
`4of5
`
`

`

`Reply to Action Mailed August 23, 2019
`Filed Electronically on November 25, 2019
`
`Serial No: 16/125,363
`Docket No: P33303USl (119-1289USl)
`
`CONCLUSIONS
`
`In view of the foregoing amendments and remarks, it is submitted that each pending claim
`
`is allowable for at least the reasons set forth herein. Reconsideration and withdrawal of all
`
`rejections and a Notice of Allowance is therefore requested.
`
`Assignee respectfully requests the constructive assistance and suggestions of the Examiner
`
`pursuant to M.P.E.P. 706(11) and 707.07(j)(II) in order that the undersigned can place this
`
`application in allowable condition. If the Examiner believes prosecution may be accelerated by an
`
`interview, Examiner Giles is encouraged to contact the undersigned attorney at the number listed.
`
`Respectfully submitted,
`
`/Katherine A. Franco/
`
`Katherine A. Franco, Reg. No. 68,383
`
`Date: November 25 2019
`
`Blank Rome LLP
`
`717 Texas Avenue, Suite 1400
`
`Houston, TX 77002
`Telephone: (713) 632-8664
`
`50f5
`
`