`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`Medivis, Inc.
`Petitioner
`
`v.
`
`Novarad Corp.
`Patent Owner
`
`
`
`Case IPR2023-00042
`US Patent No. 11,004,271
`
`
`
`DECLARATION OF CRAIG ROSENBERG, Ph.D.
`
`1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 1 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`TABLE OF CONTENTS
`
`
`I. QUALIFACTIONS................................................................................................................. 7
`II. MY EXPERTISE AND THE PERSON OF ORDINARY SKILL IN THE ART ................ 15
`III.
`LEGAL STANDARDS ..................................................................................................... 17
`IV. OVERVIEW OF THE USE OF AUGMENTED REALITY (AR) AT THE TIME OF
`THE ’271 PATENT (MARCH 2017) ................................................................................ 20
`V. THE ’271 PATENT AND THE CHALLENGED CLAIMS ................................................ 22
`A. The ’271 Patent Provides a Novel Virtual Control for Navigating 3D Medical Data
`Projected in an Augmented Reality (AR) Environment. .................................................. 22
`B. Virtual Controls in an AR Environment Are Very Different From Using a Mouse,
`Keyboard, and Conventional Computer Monitor. ............................................................ 25
`VI. CONSTRUCTION OF CERTAIN CLAIM TERMS ........................................................ 27
`1. “three-dimensional (3D) data … including an outer layer of the patient and multiple inner
`layers of the patient” ......................................................................................................... 29
`“inner layer(s) of the patient” ..................................................................................... 30
`2.
`“being confined within a virtual 3D shape” ............................................................... 31
`3.
`“being having” ............................................................................................................ 35
`4.
`Summary Table of Claim Interpretation ..................................................................... 35
`5.
`VII. ANALYSIS OF THE PRIOR ART ................................................................................... 36
`A.
`Doo (Ex. 1008) ............................................................................................................... 36
`B.
`Chen (Ex. 1009) ............................................................................................................. 39
`C.
`3D Slicer (Ex. 1010)....................................................................................................... 41
`D.
`3D Visualization (Ex. 1007)........................................................................................... 44
`E.
`Amira (Ex. 1005)............................................................................................................ 45
`VIII.
`GROUNDS OF PATENTABILITY .............................................................................. 49
`A.
`Ground 1: Doo Does Not Disclose All of the Elements of Claims 1, 5, and 6. ............. 49
`1.
`Doo does not disclose all of the elements of independent claim 1. ............................ 49
`2.
`Doo does not disclose all of the elements of dependent claims 5 and 6. .................... 53
`B.
`Ground 2: Doo Alone or in View of Amira Does Not Render Claims 1-6 Obvious. .... 54
`1.
`Non-disclosure with respect to claims 1-6 ................................................................. 54
`2
`
`
`
`Page 2 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Non-disclosure with respect to independent claim 11 ................................................ 59
`2.
`Non-Disclosure with respect to claims 12-20............................................................. 62
`3.
`No motive to combine Doo and Amira ....................................................................... 63
`4.
`C. Ground 3: Chen in View of 3D Visualization and 3D Slicer Does Not Render Claim 1-6
`and 11-20 Obvious. ........................................................................................................... 65
`1. Chen’s, 3D Slicer’s, and 3D Visualization’s non-disclosure with respect to independent
`claim 1 ............................................................................................................................... 65
`2. Disclosures of Chen, 3D Slicer, and 3D Visualization with respect to dependent claims
`2-6 .................................................................................................................................... 68
`Chen’s disclosure with respect to independent claim 11 ............................................ 73
`Non-disclosure with respect to dependent claims 12-20 ............................................ 74
`No motive to combine Chen and 3D Slicer or 3D Visualization................................ 75
`
`3.
`4.
`5.
`
`
`
`
`
`
`
`
`
`
`
`3
`
`Page 3 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`I, Craig Rosenberg, Ph.D., declare as follows:
`
`1.
`
`I am over 21 years of age and otherwise competent to make this
`
`declaration. I make this declaration based upon facts and matters within my own
`
`knowledge and on information provided to me by others.
`
`2.
`
`I have been retained as an expert witness to provide testimony on
`
`behalf of Novarad Corp. (“Patent Owner”) as part of the above-captioned inter
`
`partes review proceeding (“IPR”).
`
`3. More specifically, I have been asked primarily to assist in evaluating
`
`the grounds raised in the Petition (Paper 3) and in Dr. Kazanzides’ declaration (Ex.
`
`1012) in support of that Petition.
`
`4.
`
`I reserve the right to supplement this declaration in response to
`
`additional evidence that may come to light.
`
`5.
`
`I am not currently, nor have I ever been, employed by Patent Owner.
`
`Nor have I previously served as an expert witness on behalf of Patent Owner.1
`
`6.
`
`I understand that Patent Owner owns U.S. Patent No. 11,004,271
`
`(“the ’271 Patent”), entitled AUGMENTING REAL-TIME VIEWS OF A
`
`PATIENT WITH THREE-DIMENSIONAL DATA,” the validity of which
`
`Medivis, Inc. (“Petitioner”) challenges.
`
`
`1 Nor have I performed work for, met, or ever spoken with Dr. Steven Cvetko or Dr. Wendell
`Gibby, the named inventors in the ’271 Patent.
`
`4
`
`
`
`Page 4 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`7.
`
`I have reviewed the specifications of the ’271 Patent as well as its
`
`prosecution history. I am familiar with the ’271 Patent. A copy of the ’271 Patent
`
`was provided as Exhibit 1001.
`
`8.
`
`I am familiar with the technology at issue at the time of the ’271
`
`Patent, which Petitioner has assumed to be on or before March 30, 2017. Petition
`
`at 10.
`
`9.
`
`I have also reviewed the Petition. The Petition presents several
`
`grounds for challenging the claims of the ’271 Patent, which appear deficient to me
`
`for reasons I address in more detail below. See infra at ¶¶ 121-188.
`
`10.
`
`I have also reviewed the declaration of Dr. Kazanzides (Ex. 1012),
`
`which I also address.
`
`11.
`
`I have also reviewed the following references cited by Petitioner as
`
`prior art and other documents included in the exhibit list below:
`
`Description
`
`U.S. Patent No. 11,004,271 (the ’271 Patent)
`
`Excerpts of file history of Application No. 16/574,524, now the ’271
`Patent
`
`Excerpts of file history of Application No. 15/894,595, now U.S.
`Patent No. 10,475,244.
`
`Excerpts of file history of Application No. 15/474,702, now U.S.
`Patent No. 9,892,564.
`
`
`5
`
`Exhibit
`
`
`Ex.
`1001
`Ex.
`1002
`
`Ex.
`1003
`
`Ex.
`1004
`
`
`
`Page 5 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Ex.
`1005
`Ex.
`1006
`Ex.
`1007
`
`Ex.
`1008
`
`Ex.
`1009
`
`Ex.
`1010
`
`Ex.
`1011
`
`Ex.
`1012
`Ex.
`1013
`
`12.
`
`Excerpt of Amira 5 User’s Guide (“Amira”)
`
`U.S. Publication No. US 2016/0191887 A1 to Casas (“Casas”)
`
`S. Pujols, Ph.D. et al., 3D Visualization of DICOM Images for
`Radiological Applications (“3D Visualization”)
`
`International Publication No. WO 2015/164402 A1 to Doo et al.
`(“Doo”)
`
`X. Chen et al., “Development of Surgical Navigation System Based
`On Augmented Reality Using an Optical See-Through Head-Mounted
`Display,” 55 JOURNAL OF BIOMEDICAL INFORMATICS 124-131 (2015)
`(“Chen”)
`
`Main Application GUI for 3D Slicer
`<https://www.slicer.org/wiki/Documentation/4.6/Slicer/Application/M
`ainApplicationGUI> (“3D Slicer”)
`E. Azimi et al. “Augmentation Reality Goggles with an Integrated
`Tracking System for Navigations in Neurosurgery,” IEEE VIRTUAL
`REALITY 123-124, 123 (IEEE 2012) (“AR Goggle for Neurosurgery”)
`
`Declaration of Peter Kazanzides, Ph.D.
`
`Curriculum Vitae of Peter Kazanzides Ph.D.
`
`
`I am being compensated for my time in connection with the IPR at a
`
`rate of $700 per hour. I am also being compensated for any out-of-pocket
`
`expenses for my work in this review. My compensation as an expert is in no way
`
`dependent upon the results of any investigations I undertake, the substance of any
`
`opinion I express, or the ultimate outcome of the review proceedings.
`
`
`
`
`
`6
`
`Page 6 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`I.
`
`QUALIFACTIONS
`
`13.
`
`I believe I am qualified to serve as a technical expert in this
`
`proceeding based on my educational and work experience, including my 30+ years
`
`of experience in human factors, user interface design, user interaction design,
`
`human-computer interaction, and software engineering from 1988 through today.
`
`14. This declaration sets forth my opinions, which I formed based on my
`
`study of the evidence; my understanding as an expert in the field; and my training,
`
`education, research, knowledge, and personal and professional experience. All of
`
`my opinions stated in this declaration are based on my own personal knowledge,
`
`expertise, training, education, and professional judgment. In forming my opinions,
`
`I have relied on my knowledge and experience in human factors, user interface
`
`design, user interaction design, human-computer interaction, and software
`
`engineering.
`
`15.
`
`If I am called upon to do so, I would be competent to testify to the
`
`matters set forth herein. My qualifications to testify about the technical subject
`
`matter in this case and the relevant technology are outlined in my curriculum vitae
`
`and this section of this declaration. A copy of my current curriculum vitae is
`
`provided for this proceeding as Exhibit 2003.
`
`16.
`
`I earned my Bachelor of Science degree in Industrial Engineering, my
`
`Master of Science degree in Human Factors, and Doctor of Philosophy in Human
`
`
`
`7
`
`Page 7 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Factors from the University of Washington School of Engineering. For 30 years, I
`
`have worked in human factors, user interface design, software development,
`
`software architecture, systems engineering, and modeling and simulation across
`
`various application areas, including aerospace, communications, entertainment,
`
`and healthcare.
`
`17.
`
`I graduated from the University of Washington in 1988 with a
`
`Bachelor of Science in Industrial Engineering. After graduation, I continued my
`
`studies at the University of Washington College of Engineering. In 1990, I
`
`obtained a Master of Science degree in Human Factors. Human Factors is an
`
`engineering discipline that studies the design of products, processes, equipment,
`
`and systems to work more efficiently with humans. Human factors is concerned
`
`with reducing human error, increasing productivity and efficiency, and enhancing
`
`safety and comfort with a specific focus on the interactions and interfaces between
`
`humans and the products, processes, or systems with which they interact. In 1994,
`
`I graduated with a Doctor of Philosophy in Human Factors, focusing on advanced
`
`interface design. My Bachelor of Science, Master of Science, and Doctor of
`
`Philosophy degrees were all obtained at the University of Washington College of
`
`Engineering.
`
`18. During my doctoral studies, I worked as an Associate Assistant
`
`Human Factors Professor at the University of Washington Industrial Engineering
`
`
`
`8
`
`Page 8 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Department. My duties included teaching, writing research proposals, designing
`
`and conducting funded human factors experiments for the National Science
`
`Foundation, and hiring and supervising students. While studying at the University
`
`of Washington, I also worked as a human factors researcher. I designed and
`
`performed advanced human factors experiments relating to virtual environments
`
`and interface design, stereoscopic displays, and advanced visualization research,
`
`which the National Science Foundation funded. My duties included user interface
`
`design, systems design, software development, graphics programming,
`
`experimental design, and hardware and software interfacing.
`
`19.
`
`I have published twenty-one research papers in professional journals
`
`and proceedings in user interface design, computer graphics, and the design of
`
`spatial, stereographic, and auditory displays. I also authored a book chapter on
`
`augmented reality displays in the book “Virtual Environments and Advanced
`
`Interface Design” (Oxford University Press, 1995). In addition, I created one of
`
`the first virtual spatial musical instruments called the MIDIBIRD that, utilized the
`
`MIDI protocol, two six-dimensional spatial trackers, a music synthesizer, and a
`
`computer graphics workstation to create an advanced and novel musical
`
`instrument.
`
`20. For the past 21 years, I have been a consultant for Global Technica,
`
`Sunny Day Software, Stanley Associates, Techrizon, CDI Corporation, and the
`
`
`
`9
`
`Page 9 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Barr Group. In this capacity, I have provided advanced engineering services for
`
`many companies.
`
`21.
`
`I consulted for the Boeing Company for 15 years as a senior human
`
`factors engineer, user interface designer, and software architect for various
`
`advanced commercial and military programs. Many of the projects that I was
`
`involved with at Boeing involved advanced software development, user interface
`
`design, agent-based software, and modeling and simulations in the areas of missile
`
`defense, homeland security, battle command management, computer-aided design,
`
`networking and communications, air traffic control, location-based services, and
`
`Unmanned Aerial Vehicle (“UAV”) command and control. Additionally, I was the
`
`lead system architect developing advanced air traffic controller workstations and
`
`air traffic control analysis applications, toolsets, and trade study simulations for
`
`Boeing Air Traffic Management.
`
`22.
`
`I was also the architect of the Boeing Human Agent Model. The
`
`Boeing Human Agent Model is an advanced model for the simulation of human
`
`sensory, cognitive, and motor performance as applied to the roles of air traffic
`
`controllers, pilots, and UAV operators. In another project, I was the lead human
`
`factors engineer and user interface designer for Boeing’s primary vector and raster
`
`computer-aided drafting and editing system that produces the maintenance
`
`manuals, shop floor illustrations, and service bulletins for aircraft produced by the
`
`
`
`10
`
`Page 10 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Boeing Commercial Aircraft Company. Additional responsibilities in my time as a
`
`consultant include system engineering, requirements analysis, functional
`
`specification, use case development, user stories, application prototyping,
`
`modeling and simulation, object-oriented software architecture, graphical user
`
`interface analysis, and design, as well as UML, C++, C#, and Java software
`
`development.
`
`23.
`
`In 1995 and 1996, I was hired as the lead human factors engineer and
`
`user interface designer for the first two-way pager produced by AT&T. Before this
`
`technology, individuals could receive pages but had no way to respond utilizing
`
`their pager. This new technology allowed users to use a small handheld device to
`
`receive and send canned or custom text messages, access and update an address
`
`book, and access and update a personal calendar. This high-profile project
`
`involved designing the entire feature set, user interface/user interaction design and
`
`specification, and all graphical design standards.
`
`24. From 1999 to 2001, I was the lead human factors engineer and user
`
`interface designer for a company called Eyematic Interfaces, where I was
`
`responsible for all user interface design and development activities associated with
`
`real-time mobile handheld 3D facial tracking, animation, avatar creation, and
`
`editing software for a product for Mattel. My work involved user interface design,
`
`
`
`11
`
`Page 11 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`human factors analysis, requirements gathering and analysis, and functional
`
`specifications.
`
`25.
`
`I was the lead user interface designer for a company called
`
`ObjectSpeed, which developed a portable handheld telephone for use in homes and
`
`businesses with many of the same capabilities we take for granted in mobile
`
`cellular phones. This portable multifunction device supports voice, email, chat,
`
`video conferencing, internet radio, streaming media, Microsoft Outlook
`
`integration, photo taking, sharing, etc. The ObjectSpeed device was specifically
`
`designed and developed as a portable handheld device.
`
`26.
`
`I was the lead user interface designer for Ahaza, which was building
`
`IPv6 routers. I designed the user interfaces to configure and control these
`
`advanced network hardware devices. My responsibilities included requirements
`
`analysis, functional specification, user interface design, user experience design,
`
`and human factors analysis.
`
`27.
`
`I am the founder, inventor, user interface designer, and software
`
`architect of WhereWuz. WhereWuz is a company that produces advanced mobile
`
`software running on GPS-enabled smartphones and handheld devices. WhereWuz
`
`allows users to record exactly where they have been and query this data for
`
`subsequent retrieval based on time or location. WhereWuz was specifically
`
`designed and developed to run on small handheld devices.
`
`
`
`12
`
`Page 12 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`28.
`
`I am the co-founder of a medical technology company called
`
`Healium. Healium developed advanced wearable and handheld user interface
`
`technology to allow physicians to interact more effectively with electronic medical
`
`records.
`
`29.
`
`I am the co-founder of a medical technology company called
`
`StratoScientific. StratoScientific is developing an innovative case for a smartphone
`
`that turns a standard handheld smartphone into a full-featured digital stethoscope
`
`that incorporates visualization and machine learning for telemedicine and
`
`automated diagnosis.
`
`30.
`
`I designed and developed a large software project for Disney World
`
`called xVR that allowed the operational employees of Disney World to utilize a
`
`handheld device to view the current and historical status of all of the guests of
`
`Disney World within multiple attractions and one of their restaurants. The
`
`application could run in a real-time/live mode where it would display data
`
`collected from sensors that showed the location and status of all guests within the
`
`attraction; the application could also be run in a fast-time/simulated mode. The
`
`application was developed on a laptop computer and designed to run on various
`
`devices, including laptops, personal computers, smartphones, and tablets.
`
`31.
`
`I have received several awards for my engineering work relating to
`
`interface design, computer graphics, and the design of spatial, stereographic, and
`
`
`
`13
`
`Page 13 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`auditory displays, including a $10,000 scholarship from the I/ITSEC for advancing
`
`the field of interactive computer graphics for flight simulation and a Link
`
`Foundation award for furthering the field of flight simulation and virtual interface
`
`design. I have also created computer graphic illustrations for several popular book
`
`covers and animations for a movie produced by MIRAMAR.
`
`32. More information about my educational and professional background
`
`can be found in my curriculum vitae, Exhibit 2003.
`
`33. The combination of my education, research, training, and work
`
`experience in software development, human factors, user interface design, user
`
`interaction design, and human-computer interaction enables me to provide analysis
`
`and opinions on the subject matter of this litigation.
`
`34.
`
`I am being compensated for my work on this case and my fee is not
`
`contingent on the outcome of this case or on any of my opinions or the technical
`
`positions I explain in this report. In addition, I have no financial interest in the
`
`outcome of this case, or any of the parties of involved this case. I have worked on a
`
`significant number of patent litigation cases, several of which were related to
`
`virtual and augmented reality. These include the following cases:
`
`• ARCzar v. Nintendo (Artificial Reality Patent Case)
`
`• Glasses.com v. Ditto (Artificial Reality Patent Case)
`
`• Lennon v. Sephora (Artificial Reality Patent Case)
`14
`
`
`
`Page 14 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`• RPX v. Virtual Immersion Technologies (VR Patent Case)
`
`• Princeton Digital Image Corporation v. Harmonix Music Systems (VR
`
`Patent Case)
`
`• Princeton Digital Image Corporation v. Konami Digital Entertainment
`
`(VR Patent Case).
`
`• Barbaro v. Niantic (VR Patent Case)
`
`• Worlds, Inc v. Linden Labs (VR Patent Case)
`
`• Utherverse v. Epic Games (VR Patent Case)
`
`35.
`
`I have been retained over 100 times, been deposed 46 times, and have
`
`testified seven times at trial. And I have written many expert reports and
`
`declarations.
`
`36. Based on the above credentials and experience, and as further
`
`documented in my attached CV, I believe that I have the necessary education,
`
`training, research, scholarship, and experience to analyze the ’271 Patent and opine
`
`on how a person of ordinary skill in the art (as further defined below) would have
`
`understood it and its validity in the context of the relevant prior art.
`
`II. MY EXPERTISE AND THE PERSON OF ORDINARY SKILL IN
`
`THE ART
`
`37. My opinions in this declaration are based on my experience in this
`
`technical field. More specifically, my over 30 years of experience designing and
`15
`
`
`
`Page 15 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`implementing virtual reality (VR) and augmented reality (AR) environments.
`
`38.
`
`I am certainly qualified to provide expert opinions on some aspects of
`
`the technology described in the ’271 Patent and on the references cited by
`
`Petitioner and Dr. Kazanzides.
`
`39.
`
`I understand that my opinion must be taken from the perspective of
`
`what would have been known or understood by a person of ordinary skill in the art
`
`(“POSITA”) at the time of the invention.
`
`40.
`
`I understand that Dr. Kazanzides has concluded that the relevant art is
`
`“systems and methods for using augmented reality during medical procedures.”
`
`Ex. 1012, ¶ 23.
`
`41.
`
`I further understand that Petitioner has defined POSITA at the time of
`
`the ’271 Patent as “a person with a bachelor’s degree in computer science,
`
`electrical engineering, or a related field with several years of experience in the
`
`design, development, and study of augmented reality devices and either (a) familiar
`
`with conventional medical imaging data and visualization of data for medical
`
`procedures or (b) working with a team including someone with such familiarity.”
`
`Ex. 1012, ¶ 25.
`
`42. A POSITA at the time of the ’271 Patent would include someone with
`
`a degree in human factors or human-computer interaction. Human factors
`
`professionals play a crucial role in the design and development of AR systems to
`
`
`
`16
`
`Page 16 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`ensure they are user friendly, effective, and safe. AR combines virtual elements
`
`with the real world, overlaying computer-generated information onto the user’s
`
`view of the physical environment. It involves the integration of various
`
`technologies, including hardware, software, and user interfaces. Human factors
`
`considerations become essential in optimizing the usability, interaction, and overall
`
`user experience of AR applications.
`
`43.
`
`I understand, and have been informed, that for purposes of assessing
`
`the obviousness of a claimed invention, the level of ordinary skill possessed by a
`
`POSITA is informed by, among other things, the type of problems encountered in
`
`the relevant art, the prior art solutions to those problems, the rapidity with which
`
`innovations are made in the relevant art, the sophistication of the relevant
`
`technology, and the educational level of active workers in the field.
`
`III. LEGAL STANDARDS
`
`44.
`
`I am not an attorney. My analysis and opinions are based on my
`
`expertise in this technical field, as well as the instructions I have been given by
`
`counsel for the legal standards relating to patent validity.
`
`45.
`
`I understand that a claim of an issued patent may be found to be
`
`unpatentable if the claim is not novel or would have been obvious in view of the
`
`prior art. I understand that this determination is made from the perspective of a
`
`person having ordinary skill in the art who is presumed to be aware of all prior art.
`
`
`
`17
`
`Page 17 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`46.
`
`I understand that a patent claim will not be unpatentable due to what
`
`is called anticipation unless all elements of a claim, arranged in the same way, are
`
`found in a single reference. I have also been informed that for anticipation, each
`
`and every claim limitation must be explicitly or inherently disclosed in the prior
`
`art.
`
`47.
`
`I have also been informed and understand that anticipation by inherent
`
`disclosure is appropriate only when the reference necessarily includes the unstated
`
`limitation. Thus, it has been explained to me that inherency may not be established
`
`by probabilities or possibilities, and the mere fact that a certain thing may result
`
`from a given set of circumstances is insufficient.
`
`48.
`
`I understand that a patent claim will not be unpatentable due to
`
`obviousness unless the differences between the claimed invention and the prior art
`
`are such that the subject matter as a whole would have been obvious to a POSITA
`
`at the time of the invention.
`
`49.
`
`I understand that the following factors are considered in assessing
`
`obviousness: (1) the scope and content of the prior art; (2) the differences between
`
`the prior art and the claimed invention; (3) the level of ordinary skill in the art at
`
`the time of the invention; and (4) “objective indicia of non-obviousness,” also
`
`referred to as secondary considerations of non-obviousness. Those objective
`
`indicia include considerations such as whether a product covered by the claims is
`
`
`
`18
`
`Page 18 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`commercially successful due to the merits of the claimed invention, whether there
`
`was a long felt need for the solution provided by the claimed invention, whether
`
`others failed to find the solution provided by the claimed invention, and whether
`
`there was acceptance by others of the claimed invention as shown by praise from
`
`others in the field.
`
`50.
`
`I also understand that one should analyze whether the prior art
`
`references disclose each and every element of the claim as those references would
`
`have been read by a POSITA. I understand that obviousness cannot be proven by
`
`mere conclusory statements.
`
`51.
`
`I understand that a claim is not proved obvious merely by showing
`
`that each of the individual elements existed somewhere in the prior art.
`
`52.
`
`I understand that when a combination of prior art references is relied
`
`upon to assert obviousness, the party asserting obviousness must also identify a
`
`reason or motivation that would have prompted one of ordinary skill in the art to
`
`combine the elements of the references in the same way as the claimed invention
`
`and with a reasonable expectation of success.
`
`53.
`
`I also understand that in assessing obviousness, it is improper to rely
`
`upon hindsight and that one should avoid using the claimed invention as a roadmap
`
`to piece together disclosures from the prior art.
`
`
`
`19
`
`Page 19 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`54.
`
`I understand that in this inter partes review proceeding the claims of
`
`the ’271 Patent are generally given their ordinary and customary meaning as
`
`understood by a POSITA in light of the patent’s specification and the prosecution
`
`history.
`
`55.
`
`I also understand that claim terms should be given their broadest
`
`reasonable interpretation to a POSITA, which interpretation must be consistent
`
`with the patent’s specification.
`
`56. Similarly, I understand that the prosecution history contains the
`
`complete record of all the proceedings before the Patent and Trademark Office and
`
`may contain contemporaneous exchanges between the patent applicant and the
`
`PTO about what the claims mean. Thus, I understand that that the prosecution
`
`history can inform the meaning of the claim language by demonstrating how the
`
`inventor understood the invention and whether the inventor limited the invention in
`
`the course of prosecution.
`
`57.
`
`I understand that where a patent expressly defines a claim term, that
`
`definition may also inform the understanding of a POSITA. I have applied these
`
`standards in my analysis in this declaration.
`
`IV. OVERVIEW OF THE USE OF AUGMENTED REALITY (AR) AT
`THE TIME OF THE ’271 PATENT (MARCH 2017)
`
`58. The concept of Augmented Reality (AR) has been around for several
`
`
`
`20
`
`Page 20 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`decades. The earliest forms of AR can be traced back to the 1960s and 1970s.
`
`However, the term “Augmented Reality” was not coined until 1990 by Tom
`
`Caudell, a researcher at Boeing. The 2000s and 2010s saw a significant increase in
`
`AR development, spurred by advances in technology and the increasing ubiquity of
`
`smartphones.
`
`59.
`
`In terms of AR head-mounted displays (HMDs), the field is still
`
`relatively new. Google introduced Google Glass, an early example of an AR head-
`
`mounted display, in 2013. In 2015, Microsoft introduced its HoloLens, a more
`
`advanced AR headset. A development edition of the HoloLens wasn’t available
`
`until 2016. The first generation of commercial VR/AR headsets from Oculus,
`
`HTC, and others also launched in 2016. Despite these developments in hardware,
`
`AR in March 2017 was still facing significant hurdles, particularly in surgical
`
`applications.
`
`60. While AR was showing considerable potential in areas such as
`
`surgical planning, intraoperative guidance, training, and patient education, several
`
`significant challenges were limiting widespread adoption and usage in 2017,
`
`including by way of example:
`
`61. Technological Limitations: To be effective in surgical applications,
`
`AR had to provide accurate, real-time information. This required significant
`
`
`
`21
`
`Page 21 of 77
`
`Ex. 2004
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`computing power and advanced imaging technology. Issues like latency,
`
`resolution, and field of view were major concerns.
`
`62. Registration and Tracking: Accurate alignment of virtual data (like
`
`preoperative imaging) with the actual patient anatomy is critical in AR. Imperfect
`
`registration can lead to significant errors, which can have severe consequences in a
`
`surgical context.
`
`63. Usability: Surgeons needed to feel comfortable with the technology
`
`and trust it enough to rely on it during procedures. There were also ergonomic
`
`considerations, such as how the technology would fit into the surgical workflow
`
`without disrupting or slowing it.
`
`64. While solutions to these problems have been pursued in the years
`
`following 2017, many of these challenges still exist today.
`
`V. THE ’271 PATENT AND THE CHALLENGED CLAIMS
`
`A. The ’271 Patent Provides a Novel Virtual Control for Navigating
`3D Medical Data Projected in an Augmented Reality (AR)
`Environment.
`
`65. The ’271 Patent explains that a “common problem faced by AR
`
`systems is proper placement of virtual controls for managing virtual elements.”
`
`Ex. 1001, 1