UNITED STATES PATENT AND TRADEMARK OFFICE
`
`_______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`_______________
`
`APPLE INC. and GOOGLE LLC
`Petitioners
`
`v.
`
`SPACETIME3D, INC.
`Patent Owner
`
`_______________
`
`Case IPR2023-002421
`Patent No. 8,881,048
`
`_______________
`
`
`
`DECLARATION OF SCOTT SCHAEFER IN SUPPORT OF
`PATENT OWNER'S RESPONSE
`
`
`
`1
`
`
` Google LLC, which filed a petition in IPR2023-00577, has been joined as a
`petitioner in this proceeding.
`
`71584469
`
`
`
`
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 1
`
`

`

`TABLE OF CONTENTS
`
`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`Page
`
`I.
`
`II.
`
`INTRODUCTION ........................................................................................... 3
`
`BACKGROUND AND QUALIFICATIONS ................................................. 4
`
`A.
`
`Education and Experience ..................................................................... 4
`
`B. Materials Considered ............................................................................. 7
`
`C.
`
`Compensation ........................................................................................ 8
`
`III. LEGAL PRINCIPLES REGARDING VALIDITY ........................................ 9
`
`A. Validity Generally ................................................................................. 9
`
`B.
`
`C.
`
`Priority Date .......................................................................................... 9
`
`Anticipation ......................................................................................... 10
`
`D. Obviousness ......................................................................................... 10
`
`IV. PERSON OF ORDINARY SKILL IN THE ART ........................................ 13
`
`V.
`
`THE '048 PATENT ....................................................................................... 14
`
`A. Description of The Patented Invention ............................................... 14
`
`1.
`
`Chronological Presentation of Images in 3D Space ................. 17
`
`2. Mirroring 2D and 3D Space ...................................................... 19
`
`B.
`
`Prosecution History Of The ‘048 Patent ............................................. 26
`
`VI. CLAIM CONSTRUCTION .......................................................................... 36
`
`VII. PETITIONER’S CITED REFERENCES ...................................................... 38
`
`A.
`
`B.
`
`C.
`
`D.
`
`E.
`
`Robertson ............................................................................................. 38
`
`Gralla ................................................................................................... 50
`
`Gettman ............................................................................................... 51
`
`Sauve ................................................................................................... 59
`
`Tsuda ................................................................................................... 65
`
`VIII. THE CLAIMS OF THE ‘048 PATENT ARE NOT OBVIOUS .................. 70
`
`A.
`
`The Prior Art Combinations Do Not Disclose All of the
`Limitations of the ‘048 Patent ............................................................. 70
`
`
`
`71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 2
`
`

`

`TABLE OF CONTENTS
`(CONTINUED)
`
`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`Page
`
`1.
`
`2.
`
`Combining Robertson, Gralla, and Gettman Does Not
`Disclose The Claimed Limitations (Ground 1) ........................ 70
`
`Combining Sauve and Tsuda Does Not Disclose The
`Claimed Limitation (Ground 2) ................................................ 91
`
`B.
`
`C.
`
`The Prosecution History Confirms that the Prior Art Does Not
`Disclose All of the Limitations of the '048 Patent .............................. 95
`
`Petitioner Has Not Provided A Sufficient Rationale To
`Combine The Prior Art References To Yield The Claimed
`Invention ............................................................................................ 112
`
`IX. CONCLUSION ............................................................................................ 118
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 3
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`I, Scott Schaefer, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1.
`
`I have personal knowledge of the facts and opinions set forth in this
`
`declaration, and if called upon to do so, I would testify competently thereto.
`
`2.
`
`I am over the age of eighteen (18) and competent to make this
`
`Declaration.
`
`3.
`
`I have been retained by Patent Owner SpaceTime 3D, Inc. to provide
`
`assistance, analysis, and opinions regarding IPR2023-00242 and U.S. Patent No.
`
`8,881,048 (“the ‘048 patent”) as an expert in the field of computer science, and
`
`particularly in the area of 2D and 3D graphics and graphical user interfaces.
`
`Attached hereto as Appendix A is a true and correct copy of my curriculum vitae
`
`describing my background and experience.
`
`4.
`
`I have been asked to evaluate the validity of the ‘048 patent in light of
`
`the assertions found in Petitioners Apple Inc. and Google LLC’s (collectively,
`
`“Petitioner”) Petition for Inter Partes Review of U.S. Patent No. 8,881,048 Under
`
`35 U.S.C. §§ 311-319 And 37 C.F.R. § 42 (“Petition”). (See Paper 2) To this end,
`
`below are my opinions, and their underlying bases, regarding the person of ordinary
`
`skill in the art at the time of the claimed inventions (“POSITA”); the fields of
`
`graphical user interfaces (“GUI”), including two dimensional (“2D”) and three
`
`dimensional (“3D”) spaces or environment; the ‘048 patent; claim construction; the
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 4
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`prior art of record; and the validity of the ‘048 patent, including why the prior art
`
`does not render the ‘048 patent obvious and why a person of ordinary skill would
`
`not have been motivated to make the prior art combinations asserted by Petitioner.
`
`II. BACKGROUND AND QUALIFICATIONS
`
`A. Education and Experience
`
`5.
`
`I received a B.S. in Computer Science and Mathematics in 2000 from
`
`Trinity University. Afterwards, I attended Rice University and received an M.S. in
`
`Computer Science in 2003 and a Ph.D. in Computer Science in 2006. After
`
`graduating from Rice University, I began a faculty position at Texas A&M
`
`University in 2006 as an Assistant Professor. I was promoted to Associate Professor
`
`in 2012 and to Full Professor in 2016. I served as the person in charge of
`
`accreditation for our Computer Science degree from 2013-2017. I became Associate
`
`Department Head of the Department of Computer Science & Engineering in 2017
`
`and I was appointed Department Head in 2019, a role in which I have served since
`
`that time. In 2019 I was awarded the Eppright Professorship in Engineering and was
`
`named the holder of the Lynn ‘83 and Bill Crane ‘84 Department Head Chair in
`
`2020.
`
`6.
`
`I have been awarded multiple awards for my teaching and research
`
`activities. I was named a member of the 2008 DARPA Computer Science Study
`
`Panel. I was awarded the Gunter Enderle Award in 2011 for my research. In 2012,
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 5
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`I received an NSF CAREER Award for my research work. My research was
`
`subsequently recognized in 2015 when I was named the Herbert H. Richardson
`
`Faculty Fellow by the College of Engineering. I was also awarded the 2019 TEES
`
`Research Impact Award to recognize significant contributions and impact in the area
`
`of research. In addition to these awards, I have received multiple best paper awards
`
`for my research in the past.
`
`7.
`
`I have received multiple teaching awards from the Computer Science
`
`& Engineering department for teaching excellence over the years. In 2017, I was
`
`awarded the Distinguished Achievement Award in Teaching at the College level. In
`
`2019, I was awarded the Distinguished Achievement Award for Teaching at the
`
`University level, one of the highest university honors that is awarded.
`
`8.
`
`In addition to my academic experience, I have worked for a number of
`
`companies in different capacities. Before becoming involved in academia, I was
`
`employed by Southwest Research Institute to work on radio direction finding
`
`algorithms for the U.S. Navy in 1999-2000. I worked at WholeBrain Media on early
`
`mobile development from 1997-1999 including network communications. At Rare
`
`Medium, I performed what is now called full stack development and developed e-
`
`commerce platforms for various companies in 2000. I spent time at SensAble
`
`Technologies in 2003 developing implicit modeling algorithms for their haptic
`
`feedback system. In 2004, I was also employed by a startup company called Mok3
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 6
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`that was developed out of MIT to work on core computer vision algorithms for their
`
`image-based modeling software. This product constructed 3D models from sets of
`
`photographs of objects. I wrote the code that computed correspondences between
`
`images and ran a nonlinear optimization to align the images in 3D so that a virtual
`
`model of the object could be reconstructed. I was also employed by Microsoft
`
`Research as an intern in 2005 and subsequently as a visiting researcher in 2006
`
`where we worked on a number of surface modeling methods in computer graphics.
`
`My work at Microsoft Research motivated the inclusion of what is now called the
`
`tessellator unit in DirectX 11, which is an application programming interface
`
`(“API”) that governs the relationship between multimedia hardware. Every graphics
`
`card produced since 2009 has been influenced by my work there.
`
`9. My research broadly lies in the field of computer graphics, though I
`
`have worked in many areas of Computer Science as is evidenced by my background.
`
`For example, I have researched and published extensively on topics from sampling
`
`methods used for rendering to how to represent surfaces and volumes in computer-
`
`generated images. My research has been implemented by a number of companies
`
`including Microsoft, Pixar, Adobe, and Nvidia. In addition, my work on
`
`architectural optimization to create surfaces out of small numbers of discrete panels
`
`was used to design and construct a ski lift in Switzerland.
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 7
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`10.
`
`In addition, I have served as an Associate or Guest Editor for a number
`
`of journals including Graphical Models, The Visual Computer, Computer Aided
`
`Design, Computer Aided Geometric Design, Transactions on Visualization and
`
`Computer Graphics, Computer Graphics and Applications, and ACM Transactions
`
`on Graphics. In addition to this service I have served as papers chair for most major
`
`computer science conferences in my area. I also serve on the steering committee of
`
`the Geometric Modeling and Processing conference.
`
`11.
`
`In terms of instruction, I teach and have developed courses on a variety
`
`of topics in Computer Science. I have taught courses in computer graphics,
`
`geometric modeling, game development, data structures and algorithms, as well as
`
`both our lower level and upper-level undergraduate seminar courses. I also created
`
`the Computer Game Development course at Texas A&M University. Besides
`
`teaching, I previously served as the faculty advisor for the Texas A&M E-Sports
`
`Student Group.
`
`B. Materials Considered
`
`12. My analysis in this Declaration is based on my knowledge and
`
`experience. Based on my above-described qualifications in Section II.A, the Patent
`
`Trial and Appeal Board should consider me to be an expert in the field. Also, based
`
`on my experiences, I understand and know of the capabilities of persons of ordinary
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 8
`
`

`

`skill in this field at the time of the claimed inventions (“Critical Date”), i.e.,
`
`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`September 13, 2005.
`
`13. As part of my independent analysis for this Declaration, I have
`
`considered the following: the ’048 Patent (EX1001) and its prosecution history
`
`(EX1002) and that of the parent application resulting in U.S. Patent No. 7,735,018
`
`(EX2002); the background knowledge/technologies that were commonly known to
`
`persons of ordinary skill; my own knowledge and experience gained from my work
`
`in the field; my experience in teaching and advising others in this field; and my
`
`experience working with others involved in this field. In addition, I have reviewed
`
`and analyzed the Petition (Paper 2), the publications and materials listed on
`
`Petitioners' updated exhibit list (Paper 10), including the Declaration of Henry Fuchs
`
`(EX1003), the Patent Owner's Preliminary Response (Paper 6), Petitioner's reply
`
`(Paper 7), Patent Owner's sur-reply (Paper 8), the publications and materials listed
`
`in Patent Owner's preliminary response, its sur-reply and this Response, as well as
`
`the Board's Order Granting Institution (Paper 11).
`
`C. Compensation
`
`14.
`
`I am being compensated at my normal hourly rate for my time spent on
`
`this proceeding and for expenses incurred as a result of my role as an expert. My
`
`compensation is not in any way contingent on my performance, the result of this
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 9
`
`

`

`proceeding, or any of the issues involved therein. I have no financial interest in the
`
`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`outcome of this case.
`
`III. LEGAL PRINCIPLES REGARDING VALIDITY
`
`15.
`
`I am not an attorney. For purposes of this report, I have been informed
`
`by counsel of the following legal standards that apply to the issues I address in this
`
`report.
`
`A. Validity Generally
`
`16.
`
`I understand that the claims of an issued patent are presumed to be
`
`valid. The basis for the presumption of validity is the fact that the allowed claims
`
`went through a rigorous examination process at the U.S. Patent Office.
`
`17.
`
`I understand that Petitioner has the burden of proving invalidity by a
`
`preponderance of the evidence, and that the burden of proof never shifts to the
`
`plaintiff to prove the validity of the ’048 patent.
`
`B.
`
`Priority Date
`
`18.
`
`I understand that an inventor establishes priority of an invention by
`
`conceiving of the invention and reducing the invention to practice.
`
`19.
`
`I understand that a patent application may establish priority of invention
`
`when it supports all elements of the asserted claims.
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 10
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`C. Anticipation
`
`20.
`
`I understand from counsel for Patent Owner that invalidation by
`
`anticipation occurs only when a single alleged prior art reference discloses each and
`
`every limitation of the claim at issue, either expressly or inherently. In other words,
`
`every limitation of the claim must identically appear in a single prior art reference
`
`for the reference to anticipate that claim. I also understand that all elements of the
`
`claim must be disclosed in the reference as they are arranged in the claim.
`
`21.
`
`I understand that to be considered anticipatory, the prior art reference
`
`must be enabling and must describe the patentee’s claimed invention sufficiently to
`
`have placed it in possession of a person of ordinary skill in the field of the invention.
`
`22.
`
`I understand that the Petitioner did not assert, and the Board has not
`
`instituted, any grounds of unpatentability pursuant to Section 102.
`
`D. Obviousness
`
`23.
`
`It has further been explained to me by counsel for Patent Owner that a
`
`claim is obvious pursuant to 35 U.S.C. § 103 if the differences between the claimed
`
`invention and the prior art are such that the claimed invention as a whole would have
`
`been obvious before the effective filing date of the claimed invention to a person
`
`having ordinary skill in the art to which the claimed invention pertains. As such, I
`
`understand that a claim is valid unless the differences between the claimed invention
`
`and the prior art are such that the subject matter as a whole would have been obvious
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 11
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`at the time the invention was made to a person having ordinary skill in the art to
`
`which the claimed invention pertains.
`
`24.
`
`It has further been explained to me by counsel for Patent Owner that
`
`the following factors are used to make an obviousness determination: (i) the scope
`
`and content of the prior art; (ii) the level of ordinary skill in the art; (iii) the
`
`differences between the claimed invention and the prior art; and (iv) objective or
`
`secondary considerations of nonobviousness. Any relevant objective factors or
`
`secondary considerations of non-obviousness, which may include commercial
`
`success of a product using the invention, if that commercial success is due to the
`
`invention; long-felt need for the invention; evidence of copying of the claimed
`
`invention; industry acceptance; the taking of licenses under the patent by others;
`
`initial skepticism; failure of others; and praise of the invention.
`
`25.
`
`I understand that when a patentee can demonstrate commercial success,
`
`usually shown by significant sales in a relevant market, and that the successful
`
`product is the invention disclosed and claimed in the patent, it is presumed that the
`
`commercial success is due to the patented invention. I also understand that courts
`
`may find the commercial success of the infringer and other members of the industry
`
`persuasive even though the patentee was not successful with a product covered by
`
`the patent.
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 12
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`26.
`
`I understand that a patent claim composed of several elements is not
`
`proved obvious merely by demonstrating that each of its elements was independently
`
`known in the prior art. But multiple prior art references or elements may, in some
`
`circumstances, be combined to render a patent claim obvious. I understand that I
`
`should consider whether there is an “apparent reason” to combine the prior art
`
`references or elements in the way the patent claims. Requiring a reason for the prior
`
`art combination protects against the distortion caused by hindsight. Along the same
`
`lines, one cannot use the asserted patent as a blueprint to piece together the prior art
`
`in order to combine the right ones in the right way as to create the claimed inventions.
`
`To determine whether such an “apparent reason” exists to combine the prior art
`
`references or elements in the way a patent claims, it will often be necessary to look
`
`to the interrelated teachings of multiple patents, to the effects of demands known to
`
`the design community or present in the marketplace, and to the background
`
`knowledge possessed by a person having ordinary skill in the art.
`
`27.
`
`It has further been explained to me by counsel for Patent Owner that
`
`the motivation to combine must be more than mere conclusory statements and that
`
`a generic motivation to combine known elements is insufficient to establish
`
`obviousness. I understand that the motivation to combine prior art references must
`
`be identified in the Petition.
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 13
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`28.
`
`I understand that when the prior art “teaches away” from combining
`
`prior art references or certain known elements, discovery of a successful means of
`
`combining them is less likely to be obvious. A prior art reference may be said to
`
`“teach away” from a patent when a person of ordinary skill, upon reading the
`
`reference, would be discouraged from following the path set out in the patent or
`
`would be led in a direction divergent from the path that was taken by the patent.
`
`29.
`
`I understand that in developing opinions as to whether or not certain
`
`claimed subject matter would have been obvious, each claim of a given patent should
`
`be considered in its entirety and separately from any other claims. In so doing, it is
`
`my further understanding that while I should consider any differences between the
`
`claimed invention and the prior art, I should also assess the obviousness or non-
`
`obviousness of the entirety of a claim covering an alleged invention, not merely
`
`some portion of it.
`
`IV. PERSON OF ORDINARY SKILL IN THE ART
`
`30.
`
`I understand that my interpretation of the patent claims and my validity
`
`analysis must be undertaken from the perspective of a hypothetical person of
`
`ordinary skill in the art at the time of the claimed inventions or POSITA. In
`
`determining the characteristics of a hypothetical person having ordinary skill in the
`
`art of the asserted patents at the time of the claimed inventions, I considered several
`
`things, including the type of problems encountered in the art, the solutions to those
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 14
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`problems, the rapidity with which innovations are made, the sophistication of the
`
`technology, and the education level and experience of people working in the field.
`
`It has further been explained to me by counsel for Patent Owner that the relevant
`
`timeframe is in or around September 13, 2005.
`
`31. Based on my experience, it is my opinion that a person of ordinary skill
`
`in the art at the time of the invention of the Asserted Patent would have had a
`
`Bachelor’s Degree in Computer Science or a related field, and two years of
`
`experience working in the field of computer graphics or graphical user interfaces or
`
`GUIs. An individual with less technical education but more experience, or vice
`
`versa, could also qualify as a person of ordinary skill in the art.
`
`32. Given my education and professional experience described above and
`
`in my curriculum vitae, I am in a position to opine on the understanding of a person
`
`of ordinary skill in the art of the ‘048 patent.
`
`V. THE '048 PATENT
`
`A. Description of The Patented Invention
`
`33. The ‘048 Patent provides “an improved three-dimensional graphical
`
`user interface.” According to the specification, “[b]ecause the 3D GUI creates the
`
`illusion of infinite space in 3D, it can create a visual history of the user’s computing
`
`session, whereby the user can visit past visual computing events (or a snapshot in
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 15
`
`

`

`time) by simply navigating to previously recorded states or viewpoints.” EX1001 at
`
`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`5:6-11.2
`
`34. The specification describes several ways of presenting and interacting
`
`with information in 3D space. The ‘048 Patent, however, focuses on one way, where
`
`images of webpages are presented in 3D space, and individual webpages are
`
`interacted with, one at a time, in 2D space. In particular, Claim 1 of the ‘048 Patent
`
`provides, in part, a 3D GUI where objects are displayed in chronological sequence
`
`wherein (1) a user provides at least first and second web addresses, (2) corresponding
`
`first and second webpages are received and rendered, (3) images of the rendered
`
`webpages are captured and textured onto first and second objects, respectively, and
`
`(4) the first and second objects (with images of the first and second webpages) are
`
`then presented in 3D space, with the first object (i.e., with the first image of the first
`
`website that was requested) being displayed in the foreground of the 3D space and
`
`the second object (i.e., with the second image of the second website that was
`
`
`2 Note that my citations to non-patent literature throughout this Declaration refer to
`
`the page number added to the exhibit (and not the original page number of the
`
`document). For patents or patent publications, I’ve used the column/line numbers
`
`or paragraph numbers, respectively. Unless indicated otherwise, I have added all
`
`emphasis (bold/italics/underline) in any quoted text.
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 16
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`requested) being displayed in the background of the 3D space. EX1001 at 37:49–
`
`38:17. Claim 6 provides for a third object (i.e., with a third image of the third website
`
`that was requested) being displayed in a further background of the 3D space, behind
`
`the second object. Id. at 38:43-54.
`
`35.
`
`In the embodiment of Claim 1, when the user interacts with the first
`
`image, a window is presented to the user in 2D space, where the window in 2D space
`
`(a) replaces the plurality of objects (or images) in 3D space and (b) includes the
`
`rendered first webpage (i.e., the previously rendered webpage, from which the first
`
`image was captured). EX1001 at 38:6–38:17 (Claim 1). The user can then interact
`
`with (click on) links embedded within the rendered first webpage to acquire
`
`additional information, where the additional information is rendered and displayed
`
`in the window within the 2D space. Id. Claim 2 further provides that an image of
`
`the rendered additional information is captured and textured onto the first object,
`
`thereby replacing the first image on the first object in the 3D space. Id. at 38:18-27.
`
`36. A critical feature of the present invention is presenting images of
`
`webpages in 3D space and, when the user interacts with a particular image,
`
`presenting the corresponding webpage in 2D space, thereby (1) replacing the
`
`plurality of images and (2) allowing the user to interact with the webpage to acquire
`
`additional information. Not only does this allow the user to go back and forth
`
`between 3D space, where images are presented, and 2D space, where individual,
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 17
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`active webpages are presented, but it does so in a way so that “the user can visit past
`
`visual computing events (or snapshots in time) by simply navigating to previously
`
`recorded states or viewpoints.” EX1001 at 5:6-13. Those skilled in the art would
`
`have understood that this is accomplished by presenting the images chronologically
`
`in 3D space, where each image matches or “mirrors” where the user last left off in
`
`2D space.
`
`1.
`
`Chronological Presentation of Images in 3D Space
`
`37. A feature of the ‘048 Patent is to provide “a visual history of the user’s
`
`computing session.” EX1001 at 5:6-11; see also EX2001 at [56-57]. To this end,
`
`the specification provides that “depth (z) … is also known as time” and that “[t]his
`
`notion of expressing depth or time in a visual computing metaphor is important for
`
`the creation of a visual history of the end user’s computer sessions.” EX1001 at
`
`2:14-20. By plotting images of webpages in 3D space, where the first webpage is
`
`presented in the foreground, a second webpage is presented in a background, behind
`
`the first image, a third webpage is presented further in the background, behind the
`
`second image, and so forth, “the 3D GUI can function as a visual chronological
`
`history of the user’s computing session.” Id. at 5:6-13.
`
`38. This can be seen in Figure 11 of the ‘048 Patent (reproduced below),
`
`where “an end user types http://www.yahoo.com … [and] the application then draws
`
`the HTML page through the proper web browser control into the 3D virtual space as
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 18
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`depicted” (i.e., 510). EX1001 at 29:23-38. “This process can be repeated
`
`indefinitely, entering additional URLs … to have them filed in a 3D stack.” Id.
`
`“Four webpages (510, 512, 514, 516) are created in a 3D GUI virtual space having
`
`sequentially typed in http://www.yahoo.com followed by a carriage return,
`
`http://www.google.com followed by a carriage return, http://www.ebay.com
`
`followed by a carriage return and then http://www.msn.com followed by a carriage
`
`return.” Id.
`
`Figure 11 of the ‘048 Patent
`
`
`
`39. The process of presenting the webpages (or images thereof) in
`
`chronological order is described at col. 17, ll. 45-54 of the ‘048 Patent:
`
`The system as a default draws each new webpage in what we call a “3D
`
`stack” (i.e., a stack visualized in 3D, sometimes referred to as a 3D
`
`stack) as shown where each new webpage occupies an x,y,z coordinate
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 19
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`similar to the position of the existing webpage, except it is drawn
`
`further into the distance along the z axis (where it appears smaller from
`
`the given perspective) and is translated on the x or y or both x, y axis to
`
`allow the end user to see multiple webpages from any given
`
`perspective.
`
`40. Using this system, where “a position of depth (z) [is] known as time …
`
`it is possible to create a visual history of the end user’s computing session by plotting
`
`new output in a new position further along the (z) axis.” EX1001 at 20:15-32.
`
`“Because the 3D GUI creates the illusion of infinite space in 3D, it can create a
`
`visual history of the user’s computing session, whereby the user can visit past visual
`
`computing events (or a snapshot in time) by simply navigating [along the z-axis] to
`
`previously recorded states.” Id. at 25:55-59. “This way the end user can visit past
`
`visual computing moments in time.” Id. at 2:62-63; see also id. at 22:15-16 (“This
`
`way, the 3D GUI lets one travel back (visually) in computing time.”).
`
`2. Mirroring 2D and 3D Space
`
`41.
`
`In providing “a visual history of the user’s computing session” (id. at
`
`5:6-11), the images are not only plotted chronologically but updated to reflect
`
`interactions with the active webpage. See, e.g., EX1001 at 8:27-30 (“the program
`
`will redraw the scene to reflect the user-initiated input”). This “updating” takes
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 20
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`place “in cyclical fashion to create a real time experience of interacting with said
`
`operating system controls or operating system output.” Id. at 25:9-22.
`
`42. This can be seen in annotated Figure 3 of the ‘048 Patent, which is
`
`reproduced below. A user request (e.g., for a webpage) is captured and transferred
`
`to the 2D desktop (124, 126) and an image is captured (146), stored (148), and
`
`presented in 3D space (142). User-interactions are then passed to the 2D
`
`environment (152) and the results (e.g., displaying additional information) are then
`
`captured (146), stored (148), and presented in 3D space (142). This process is
`
`repeated (156) as necessary, in a cyclical fashion.
`
`Figure 3 of the ‘048 Patent
`
`
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 21
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`43.
`
`In describing this process, the specification provides that “the
`
`application will transfer and initiate user requests from the 3D virtual space to 2D
`
`desktop … by simulating/emulating or reproducing the request or device input from
`
`event handler off-screen onto the hidden 2D mirror component and capturing the
`
`response or change in output from the 2D mirror again in synchronous fashion and
`
`mapping it back to arbitrary 3D geometry.” EX1001 at 24:24-40 (emphasis added).
`
`“In doing so, the GUI system redraws the 3D virtual space in a cyclical fashion to
`
`reflect the changes of content and perspective within the 3D space based on the end
`
`user’s input.” Id. at 35:25-30. This updating is necessary to present an accurate
`
`“visual history of where the user last left off” when viewing images in 3D space. Id.
`
`at 20:33-39.
`
`44. This feature is described in Claim 1, where “displaying … the first
`
`webpage … [and] the second webpage … in 3D space compris[es]: rendering the
`
`first and second webpages; capturing first and second images of … the first
`
`webpage and … the second webpage, respectively; and texturing the first image on
`
`the first object and the second image on the second object [in 3D space], the first
`
`object being displayed in a foreground of the 3D space and the second object being
`
`displayed in a background of the 3D space.” EX1011 at 37:57-38:3 (emphasis
`
`added).
`
`
`
`
`
`71584469 71584469
`
`SPACETIME3D
`EXHIBIT 2014 - PAGE 22
`
`

`

`IPR2023-00242
`U.S. Patent No. 8,881,048
`
`45.
`
`If the user int