Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`NIANTIC, INC.,
`Petitioner
`
`v.
`
`NANT HOLDINGS IP, LLC,
`Patent Owner
`
`U.S. Patent No. 10,403,051 B2
`Issue Date: September 3, 2019
`
`Title: Interference Based Augmented Reality Hosting Platforms
`
`DECLARATION OF DR. MICHAEL ZYDA
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 001
`
`

`

`TABLE OF CONTENTS
`
`
`Page
`
`
`I.
`
`V.
`
`INTRODUCTION AND QUALIFICATIONS .............................................. 1
`A. Qualifications and Experience ............................................................. 1
`B. Materials Considered ............................................................................ 5
`PERSON OF ORDINARY SKILL IN THE ART ....................................... 10
`II.
`III. LEGAL PRINCIPLES USED IN THE ANALYSIS ................................... 12
`A.
`Prior Art .............................................................................................. 12
`B.
`Claim Construction ............................................................................ 12
`C.
`Legal Standards for Obviousness ....................................................... 14
`D.
`Reasonable Expectation of Success ................................................... 20
`IV. RELEVANT TECHNOLOGY BACKGROUND ....................................... 20
`A.
`Introduction to Augmented Reality .................................................... 20
`B. History of Augmented Reality ........................................................... 23
`C.
`Location-Based Tracking in AR Systems .......................................... 28
`D. Object Recognition in AR Systems .................................................... 34
`E.
`Augmented Reality Gaming ............................................................... 36
`F.
`Conducting Commercial Transactions in AR Systems ...................... 41
`THE ’051 PATENT ...................................................................................... 43
`A. Overview of the Specification ............................................................ 43
`B. Overview of the Challenged Claims .................................................. 52
`C. Disputed Claim Term Constructions .................................................. 53
`VI. APPLICATION OF THE PRIOR ART TO THE CHALLENGED
`CLAIMS ....................................................................................................... 54
`A.
`Brief Description and Summary of the Prior Art ............................... 55
`1.
`Yu [Ex. 1003] ........................................................................... 55
`2.
`Sanz-Pastor [Ex. 1004] ............................................................ 60
`3. Mullen [Ex. 1005] .................................................................... 63
`B. Ground 1: Obviousness of the Challenged Claims Over Yu ............. 66
`-i-
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 002
`
`

`

`
`
`1.
`
`2.
`
`TABLE OF CONTENTS
`(continued)
`
`Page
`
`b.
`
`c.
`
`d.
`
`Independent claim 1 ................................................................. 66
`a.
`“An augmented reality (AR) platform system
`comprising…” (Claim 1 preamble) ............................... 67
`“an AR object repository storing available AR
`objects in a first non-transitory computer readable
`memory; and” (Claim 1[a]) ........................................... 72
`“an AR server coupled with the AR object
`repository and, upon execution of software
`instructions stored in a second non-transitory
`computer readable memory by a processor, is
`configured to:” (Claim 1[b]) .......................................... 75
`“obtain digital data representative of an
`environment of an AR capable mobile device, the
`digital data including a device location of the AR
`capable device and a virtual element attribute;”
`(Claim 1[c]) ................................................................... 87
`“determine at least one context related to the AR
`capable device and pertinent to the environment
`based at least on the device location;” (Claim 1[d]) ..... 96
`“identify relevant AR objects from the AR object
`repository representing available AR objects
`corresponding to the at least one context;” (Claim
`1[e]) .............................................................................. 104
`“determine whether to alter presence of a relevant
`AR object based on at least the device location and
`the virtual element attribute; and” (Claim 1[f]) ........... 107
`“cause the AR capable device to render the
`relevant AR object according to its altered
`presence.” (Claim 1[g]) ............................................... 122
`Claim 5: “The system of claim 1, wherein the AR server
`is a remote server coupled with the AR capable device
`via a wireless network.” ......................................................... 126
`
`e.
`
`f.
`
`g.
`
`h.
`
`
`
`
`
`-ii-
`
`
`
`Niantic's Exhibit No. 1002
`Page 003
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`3.
`
`4.
`
`Claim 6: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on a position of
`the AR capable device relative to the environment.” ............ 128
`Claim 7: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on an
`orientation of the AR capable device relative to the
`environment.” ......................................................................... 130
`Claim 8: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered within an AR game.”..... 132
`Claim 9: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered by superimposing a
`visual image of at least one relevant AR object over an
`image of the environment.” .................................................... 134
`Claim 18: “The system of claim 1, wherein the presence
`of the relevant AR object is altered to include a non-
`visible presence.” ................................................................... 136
`Claim 22: “The system of claim 1, wherein the
`determination of whether to alter presence of the relevant
`AR object depends on a time.” .............................................. 137
`Claim 23: “The system of claim 22, wherein the presence
`alteration of the relevant AR object changes with the
`time.” ...................................................................................... 139
`10. Claim 27: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`populate the environment with at least one of the relevant
`AR objects.” ........................................................................... 141
`11. Claim 28: “The system of claim 1, wherein the
`environment comprises an AR environment.” ....................... 143
`12. Claim 29: “The system of claim 28, wherein the AR
`environment includes real-world elements and virtual
`elements.” ............................................................................... 146
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`-iii-
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 004
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`16.
`
`b.
`
`13. Claim 34: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`have an interaction with the rendered relevant AR
`object.” ................................................................................... 147
`14. Claim 36: “The system of claim 1, wherein the mobile
`device is a cell phone.” .......................................................... 149
`15. Claim 38: “The system of claim 1, wherein the mobile
`device is a tablet computer.” .................................................. 150
`Independent claim 43 ............................................................. 150
`a.
`“An Augmented Reality (AR) server coupled with
`an AR object repository and configured to:”
`(Claim 43 preamble) .................................................... 152
`“obtain digital data representative of an
`environment of an AR capable mobile device, the
`digital data including a device location of the AR
`capable device and a virtual element attribute;”
`(Claim 43[a]) ............................................................... 153
`“determine at least one context related to the AR
`capable device and pertinent to the environment
`based at least on the device location;” (Claim
`43[b]) ........................................................................... 153
`“identify relevant AR objects from the AR object
`repository representing available AR objects
`corresponding to the at least one context;” (Claim
`43[c]) ............................................................................ 154
`“determine whether to make present relevant AR
`objects based on at least the device location and
`the virtual element attribute; and” (Claim 43[d]) ....... 154
`“cause the AR capable device to render one or
`more of the relevant AR objects if they have been
`determined to be made present.” (Claim 43[e]) ......... 157
`C. Ground 2: Obviousness of the Challenged Claims Over Yu in
`View of Sanz-Pastor ......................................................................... 159
`-iv-
`
`
`c.
`
`d.
`
`e.
`
`f.
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 005
`
`

`

`
`
`1.
`
`2.
`
`TABLE OF CONTENTS
`(continued)
`
`Page
`
`b.
`
`c.
`
`d.
`
`Independent claim 1 ............................................................... 159
`a.
`“An augmented reality (AR) platform system
`comprising…” (Claim 1 preamble) ............................. 160
`“an AR object repository storing available AR
`objects in a first non-transitory computer readable
`memory; and” (Claim 1[a]) ......................................... 168
`“an AR server coupled with the AR object
`repository and, upon execution of software
`instructions stored in a second non-transitory
`computer readable memory by a processor, is
`configured to:” (Claim 1[b]) ........................................ 171
`“obtain digital data representative of an
`environment of an AR capable mobile device, the
`digital data including a device location of the AR
`capable device and a virtual element attribute;”
`(Claim 1[c]) ................................................................. 178
`“determine at least one context related to the AR
`capable device and pertinent to the environment
`based at least on the device location;” (Claim 1[d]) ... 184
`“identify relevant AR objects from the AR object
`repository representing available AR objects
`corresponding to the at least one context;” (Claim
`1[e]) .............................................................................. 186
`“determine whether to alter presence of a relevant
`AR object based on at least the device location and
`the virtual element attribute; and” (Claim 1[f]) ........... 188
`“cause the AR capable device to render the
`relevant AR object according to its altered
`presence.” (Claim 1[g]) .............................................. 195
`Claim 5: “The system of claim 1, wherein the AR server
`is a remote server coupled with the AR capable device
`via a wireless network.” ......................................................... 201
`
`e.
`
`f.
`
`g.
`
`h.
`
`
`
`
`
`-v-
`
`
`
`Niantic's Exhibit No. 1002
`Page 006
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`3.
`
`4.
`
`5.
`
`6.
`
`Claim 6: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on a position of
`the AR capable device relative to the environment.” ............ 202
`Claim 7: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on an
`orientation of the AR capable device relative to the
`environment.” ......................................................................... 204
`Claim 8: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered within an AR game.”..... 205
`Claim 9: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered by superimposing a
`visual image of at least one relevant AR object over an
`image of the environment.” .................................................... 208
`Claim 10: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered within an overlapping
`augmented reality among multiple AR capable devices.” ..... 208
`Claim 11: “The system of claim 10, wherein the
`overlapping augmented reality comprises a team-based
`augmented reality.” ................................................................ 211
`Claim 15: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered according to a haptic
`format.” .................................................................................. 212
`10. Claim 18: “The system of claim 1, wherein the presence
`of the relevant AR object is altered to include a non-
`visible presence.” ................................................................... 216
`11. Claim 22: “The system of claim 1, wherein the
`determination of whether to alter presence of the relevant
`AR object depends on a time.” .............................................. 217
`12. Claim 23: “The system of claim 22, wherein the presence
`alteration of the relevant AR object changes with the
`time.” ...................................................................................... 218
`
`7.
`
`8.
`
`9.
`
`-vi-
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 007
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`13. Claim 27: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`populate the environment with at least one of the relevant
`AR objects.” ........................................................................... 220
`14. Claim 28: “The system of claim 1, wherein the
`environment comprises an AR environment.” ....................... 221
`15. Claim 29: “The system of claim 28, wherein the AR
`environment includes real-world elements and virtual
`elements.” ............................................................................... 221
`16. Claim 34: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`have an interaction with the rendered relevant AR
`object.” ................................................................................... 222
`17. Claim 36: “The system of claim 1, wherein the mobile
`device is a cell phone.” .......................................................... 224
`18. Claim 38: “The system of claim 1, wherein the mobile
`device is a tablet computer.” .................................................. 224
`Independent claim 43 ............................................................. 225
`19.
`D. Ground 3: Obviousness of the Challenged Claims Over Yu in
`View of Sanz-Pastor and in Further View of Mullen ...................... 229
`1.
`Independent claim 1 ............................................................... 229
`a.
`“An augmented reality (AR) platform system
`comprising…” (Claim 1 preamble) ............................. 230
`“an AR object repository storing available AR
`objects in a first non-transitory computer readable
`memory; and” (Claim 1[a]) ......................................... 230
`“an AR server coupled with the AR object
`repository and, upon execution of software
`instructions stored in a second non-transitory
`computer readable memory by a processor, is
`configured to:” (Claim 1[b]) ........................................ 230
`
`b.
`
`c.
`
`-vii-
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 008
`
`

`

`e.
`
`f.
`
`g.
`
`h.
`
`TABLE OF CONTENTS
`(continued)
`
`Page
`
`d.
`
`“obtain digital data representative of an
`environment of an AR capable mobile device, the
`digital data including a device location of the AR
`capable device and a virtual element attribute;”
`(Claim 1[c]) ................................................................. 231
`“determine at least one context related to the AR
`capable device and pertinent to the environment
`based at least on the device location;” (Claim 1[d]) ... 231
`“identify relevant AR objects from the AR object
`repository representing available AR objects
`corresponding to the at least one context;” (Claim
`1[e]) .............................................................................. 231
`“determine whether to alter presence of a relevant
`AR object based on at least the device location and
`the virtual element attribute; and” (Claim 1[f]) ........... 232
`“cause the AR capable device to render the
`relevant AR object according to its altered
`presence.” (Claim 1[g]) .............................................. 232
`Claim 5: “The system of claim 1, wherein the AR server
`is a remote server coupled with the AR capable device
`via a wireless network.” ......................................................... 232
`Claim 6: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on a position of
`the AR capable device relative to the environment.” ............ 233
`Claim 7: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered based on an
`orientation of the AR capable device relative to the
`environment.” ......................................................................... 233
`Claim 8: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered within an AR game.”..... 233
`
`-viii-
`
`
`
`
`
`
`
`
`
`2.
`
`3.
`
`4.
`
`5.
`
`Niantic's Exhibit No. 1002
`Page 009
`
`

`

`7.
`
`8.
`
`9.
`
`TABLE OF CONTENTS
`(continued)
`
`Page
`
`6.
`
`Claim 9: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered by superimposing a
`visual image of at least one relevant AR object over an
`image of the environment.” .................................................... 239
`Claim 10: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered within an overlapping
`augmented reality among multiple AR capable devices.” ..... 239
`Claim 11: “The system of claim 10, wherein the
`overlapping augmented reality comprises a team-based
`augmented reality.” ................................................................ 239
`Claim 15: “The system of claim 1, wherein the relevant
`AR object is caused to be rendered according to a haptic
`format.” .................................................................................. 239
`10. Claim 18: “The system of claim 1, wherein the presence
`of the relevant AR object is altered to include a non-
`visible presence.” ................................................................... 240
`11. Claim 22: “The system of claim 1, wherein the
`determination of whether to alter presence of the relevant
`AR object depends on a time.” .............................................. 240
`12. Claim 23: “The system of claim 22, wherein the presence
`alteration of the relevant AR object changes with the
`time.” ...................................................................................... 240
`13. Claim 24: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`conduct a commercial transaction with a commerce
`engine.” .................................................................................. 240
`14. Claim 25: “The system of claim 24, wherein the
`commercial transaction includes an exchange of virtual
`currency.” ............................................................................... 243
`15. Claim 26: “The system of claim 24, wherein the
`commercial transaction includes a transfer of real-world
`funds.” .................................................................................... 244
`
`-ix-
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 010
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`b.
`
`c.
`
`16. Claim 27: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`populate the environment with at least one of the relevant
`AR objects.” ........................................................................... 245
`17. Claim 28: “The system of claim 1, wherein the
`environment comprises an AR environment.” ....................... 245
`18. Claim 29: “The system of claim 28, wherein the AR
`environment includes real-world elements and virtual
`elements.” ............................................................................... 246
`19. Claim 34: “The system of claim 1, wherein the AR server
`is further configured to enable the AR capable device to
`have an interaction with the rendered relevant AR
`object.” ................................................................................... 246
`20. Dependent claim 35 ............................................................... 246
`a.
`“The system of claim 34, wherein the interaction
`includes at least one of the following:” (Claim 35
`preamble) ..................................................................... 247
`“executing instructions of the rendered relevant
`AR object,” (Claim 35[a]) ........................................... 248
`“presenting AR content via at least one of a
`speaker or tactile interface depending on nature of
`AR content,” (Claim 35[b]) ......................................... 248
`“sharing an AR object with other devices,” (Claim
`35[c]) ............................................................................ 250
`“presenting a purchasable product,” (Claim 35[d]) ..... 251
`“incorporating a shopping context,” (Claim 35[e]) ..... 252
`“allowing access to data records,” (Claim 35[f]) ........ 253
`“allowing a user to link to object information via
`an AR object,” (Claim 35[g]) ...................................... 254
`“interacting with a remote computing device, and”
`(Claim 35[h]) ............................................................... 255
`
`d.
`
`e.
`f.
`g.
`h.
`
`i.
`
`-x-
`
`
`
`
`
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 011
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`
`
`“managing AR objects” (Claim 35[i]) ......................... 256
`j.
`21. Claim 36: “The system of claim 1, wherein the mobile
`device is a cell phone.” .......................................................... 257
`22. Claim 38: “The system of claim 1, wherein the mobile
`device is a tablet computer.” .................................................. 257
`Independent claim 43 ............................................................. 258
`23.
`VII. NO SECONDARY CONSIDERATIONS OF NON-OBVIOUSNESS .... 258
`VIII. ENABLEMENT OF THE PRIOR ART .................................................... 261
`IX. CONCLUSION ........................................................................................... 262
`
`
`
`
`
`-xi-
`
`
`
`Niantic's Exhibit No. 1002
`Page 012
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`I, Michael Zyda, declare as follows:
`I.
`INTRODUCTION AND QUALIFICATIONS
`A. Qualifications and Experience
`1.
`I began my career in Computer Graphics in 1973 as part of an
`
`undergraduate research group, the Senses Bureau, at the University of California,
`
`San Diego. I received a BA in Bioengineering from the University of California,
`
`San Diego in La Jolla in 1976, an MS in Computer and Information Science from
`
`the University of Massachusetts, Amherst in 1978 and a D.Sc. in Computer Science
`
`from Washington University, St. Louis, Missouri in 1984. I am an ACM Fellow, an
`
`IEEE Fellow, an IEEE Virtual Reality Technical Achievement Award winner, a
`
`Senior Member of the National Academy of Inventors, and a National Associate of
`
`the National Academies.
`
`2.
`
`I am currently a Professor of Engineering Practice in the USC
`
`Department of Computer Science. At USC, I founded the Computer Science Games
`
`Program and the year-long advanced game projects course that forms the core of
`
`USC Games and took that program from no program to the #1 Games program in
`
`the world.
`
`3. My research
`
`interests
`
`include computer graphics,
`
`large-scale,
`
`networked 3D virtual environments and games, agent-based simulation, modeling
`1
`
`
`
`
`Niantic's Exhibit No. 1002
`Page 013
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`human and organizational behavior,
`
`interactive computer-generated story,
`
`computer-generated
`
`characters,
`
`video
`
`production,
`
`entertainment/defense
`
`collaboration, modeling and simulation, and serious and entertainment games. I am
`
`considered a pioneer in the fields of computer graphics, virtual environments,
`
`modeling and simulation, and 3D games.
`
`4.
`
`From 1986 until 2000, I was the Director of the NPSNET Research
`
`Group at the Naval Postgraduate School (“NPS”). I began working in the networked
`
`visual simulation field in 1987. From Fall 2000 to Fall 2004, I was the Founding
`
`Director of the MOVES Institute located at the NPS and a Professor in the
`
`Department of Computer Science at NPS as well. At NPS, my NPSNET Research
`
`Group built the first networked virtual environment with fully instrumented body
`
`suits that played across the Internet. My work on the networking of virtual
`
`environments contributed to the development of the IEEE 1278.1 standard for
`
`distributed interactive simulation. While at NPS, I was an Associate Editor and then
`
`Senior Editor for the MIT Press Journal Presence: Teleoperators. During this time,
`
`I was also a member of the National Research Council Committee that put out the
`
`report “Virtual Reality - Scientific and Technological Challenges.”
`
`
`
`
`2
`
`Niantic's Exhibit No. 1002
`Page 014
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`
`5.
`
`In 1995, I chaired the SIGGRAPH Symposium on Interactive 3D
`
`Graphics. The symposium was the fourth in an ongoing series of conferences
`
`focused on the frontier of real-time interactive 3D graphics. These conferences grew
`
`out of the Workshop on Interactive 3D Graphics, held in 1986. Following that
`
`workshop I decided to found the Symposium on Interactive 3D Graphics. I served
`
`as chair of the symposium in 1990, 1995, and 2003. From 1993 to 2004, I was the
`
`Senior Editor for the MIT Press Journal Presence: Teleoperators and Virtual
`
`Environments
`
`6.
`
`I hold a lifetime appointment as a National Associate of the National
`
`Academies, an appointment made by the Council of the National Academy of
`
`Sciences in November 2003, awarded in recognition of “extraordinary service” to
`
`the National Academies. In March 2017, I was awarded the IEEE Virtual Reality
`
`Technical Achievement Award “for fundamental work in virtual reality networking,
`
`body tracking & institutionalizing the application of virtual reality.” In May 2017,
`
`I was appointed a member of the National Academy of Inventors in recognition of
`
`advanced technological development and innovation as issued by the United States
`
`Patent & Trademark Office. In November 2018, I was promoted to IEEE Fellow
`
`with the citation “for contributions to game design and networking.” In August
`
`
`
`
`3
`
`Niantic's Exhibit No. 1002
`Page 015
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`2019, I was appointed Distinguished Collaborator for Stanford’s Human
`
`Performance Laboratory, part of the Institute for Human-Centered AI. In February
`
`2020, the National Academy of Inventors elected me as a Senior Member for
`
`“success in patents, licensing, and commercialization” and for producing
`
`“technologies that have brought, or aspire to bring, real impact on the welfare of
`
`society.” In January 2021, I was promoted to ACM Fellow “for contributions to
`
`game design, game and virtual reality networking, and body tracking.”
`
`7.
`
`I have published over 150 technical books, reports, and papers relating
`
`to modeling, simulation, virtual reality, virtual environments, and computing.
`
`8.
`
`The foregoing summary of my experience in the areas of human-
`
`computer interaction, virtual reality, and augmented reality is not comprehensive,
`
`nor does it provide specific details on my numerous research efforts and
`
`publications. Additional details of my background are set forth in my curriculum
`
`vitae, attached as Exhibit A to this Declaration, which provides a more complete
`
`description of my educational background and work experience. My CV also
`
`includes a listing of all cases in which I have testified at deposition or trial.
`
`9.
`
`I am being compensated for the time I have spent on this matter at the
`
`rate of $600 per hour. My compensation does not depend in any way upon the
`
`
`
`
`4
`
`Niantic's Exhibit No. 1002
`Page 016
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`outcome of this proceeding. I hold no interest in the Petitioner (Niantic, Inc.
`
`(referred to herein as “Niantic”)) or the Patent Owner (Nant Holdings IP, LLC. I am
`
`informed that the exclusive licensee of the ’051 patent is NantWorks, LLC, and I
`
`hold no interest in NantWorks, LLC either. In this declaration, I refer to Nant
`
`Holdings IP, LLC and NantWorks, LLC collectively as “NantWorks.”
`
`B. Materials Considered
`10. The analysis that I provide in this Declaration is based on my education
`
`and experience in the areas of human-computer interaction, virtual reality, and
`
`augmented reality, as well as the documents I have considered, including U.S. Patent
`
`No. 10,403,051 (“’051 Patent” or “’051”) [Ex.1001], which states on its face that it
`
`issued from an application filed on November 9, 2018, in turn claiming priority back
`
`to an earliest provisional application filed on April 8, 2011. For purposes of this
`
`Declaration, I have assumed April 8, 2011 as the effective filing date of the ’051
`
`Patent; however, I have not performed an analysis as to whether any issued claims
`
`in the ’051 Patent are actually entitled to that date, and I reserve the right to offer
`
`analysis and opinions as to whether the ’051 Patent is entitled to the provisional
`
`application’s filing date or only to the filing date of a later utility application. My
`
`opinions and conclusions in this Declaration would not change if it were determined
`
`
`
`
`5
`
`Niantic's Exhibit No. 1002
`Page 017
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`that the ’051 Patent were entitled to the April 8, 2011 date or to the June 30, 2011
`
`filing date of the earliest utility application to which the ’051 Patent claims priority.
`
`11.
`
`I have cited to the following documents in my analysis below:
`
`Description of Document
`Ex. No.
`1001 U.S. Patent No. 10,403,051 (“’051 Patent” or “’051”)
`1003 U.S. Patent Application Publication No. 2010/0066750 A1 to Yu et
`al. (“Yu”)
`1004 U.S. Patent Application Publication No. 2007/0242131 A1 to Sanz-
`Pastor et al. (“Sanz-Pastor”)
`1005 U.S. Patent Application Publication No. 2006/0105838 A1 to Mullen
`(“Mullen”)
`
`1006
`1007
`1008
`
`1009
`
`Reserved
`
`Reserved
`
`Excerpts from Oliver Bimber and Ramesh Raskar, Chapter 1: A Brief
`Introduction to Augmented Reality, in Spatial Augmented Reality:
`Merging Real and Virtual Worlds, CRC Press (2005)
`
`Paul Milgram and Fumio Kishino, A Taxonomy of Mixed Reality
`Visual Displays, IEICE Transactions on Information Systems, E77-D
`(12):1321-29 (1994)
`
`1010
`
`Jean-Marie Normand et al., A new typology of augmented reality
`applications, AH ’12 Proceedings of the 3rd Augmented Human Int’l
`Conference (Mar. 2012)
`1011 U.S. Patent No. 3,050,870 (“Heilig”)
`
`
`
`
`6
`
`Niantic's Exhibit No. 1002
`Page 018
`
`

`

`Declaration of Dr. Michael Zyda in Support of
`Petition for Inter Partes Review of
`U.S. Patent No. 10,403,051 B2
`
`
`
`Ex. No.
`1012
`
`Description of Document
`
`Ivan E. Sutherland, A head-mounted three dimensional display,
`AFIPS ’68 (Fall, Part I): Proceedings of the December 9-11, 1968
`Fall Joint Comput. Conf., Part I, pp. 757-64 (1968)
`1013 Grégory Maubon, A little bit of history from 2006: Nokia MARA
`project, https://www.augmented-reality.fr/2009/03/un-petit-peu-
`dhistoire-de-2006-projet-mara-de-nokia/
`1014 U.S. Patent No. 6,553,310 (“Lopke”)
`1015 U.S. Patent Application Publication No. 2002/0044152 (“Abbott”)
`1016
`
`Excerpts from Lester Madden, Professional Augmented Reality
`Browsers for Smartphones, Programming for junaio, Layar, and
`Wikitude (2011)
`
`1017
`
`1018
`
`1019
`
`Jonathan Raper et al., Applications of location-based