Paper No._______
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_______________
`
`MEDIVIS, INC.
`Petitioner
`
`v.
`
`NOVARAD CORP.
`Patent Owner
`
`US Patent No. 11,004,271
`
`Inter Partes Review No. IPR2023-00042
`
`_______________
`
`DECLARATION OF PETER KAZANZIDES, Ph.D.
`
`ME1 42794222v.2
`
`Medivis Exhibit 1012
`
`

`

`I.
`II.
`III. MATERIALS CONSIDERED ........................................................................ 6
`IV. OPINIONS AND ANALYSIS ........................................................................ 8
`A.
`B.
`C.
`D.
`E.
`A.
`
`
`
`V.
`
`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`TABLE OF CONTENTS
`
`
`QUALIFICATIONS ........................................................................................ 1
`LEGAL PRINCIPLES ..................................................................................... 4
`
`Page
`
`The Relevant Field of Invention and the Person of Ordinary
`Skill in the Art ....................................................................................... 9
`Challenged Claims .............................................................................. 10
`Claim Interpretation ............................................................................ 12
`Reasons for Allowance ........................................................................ 15
`Overview of the Technology and State of the Art .............................. 15
`PRIOR ART AND STATUTORY GROUNDS............................................ 28
`Ground 1: Anticipation of Claims 1, 5, and 6 by Doo ........................ 28
`Doo’s Disclosure with respect to Independent Claim 1 ....... 29
`1.
`(a)
`“A method for augmenting real-time, non-
`image actual views of a patient with three-
`dimensional (3D) data” (Preamble of Claims 1,
`7, and 11) ........................................................................ 29
`“identifying 3D data for the patient, the 3D data
`including an outer layer of the patient and
`multiple inner layers of the patient”
`(“identifying” step of Claims 1, 7, and 11) .................... 30
`“displaying, in an augmented reality (AR)
`headset, one of the inner layers of the patient
`from the 3D data projected onto real-time, non-
`image actual views of the outer layer of the
`patient” (“displaying … inner layer[]” step of
`Claims 1, 7, and 11) ........................................................ 31
`
`(b)
`
`(c)
`
`ME1 42794222v.2
`
`ii
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`B.
`
`2.
`
`(d)
`
`“the projected inner layer of the patient from
`the 3D data being confined within a volume of
`a virtual 3D shape” (“confined” limitation of
`Claim 1) .......................................................................... 33
`Doo’s Disclosure with respect to Dependent Claims 5
`and 6 ......................................................................................... 36
`(a) Claim 5: “The method of claim 1, wherein lines
`of the virtual 3D shape are hidden.” ............................... 36
`(b) Claim 6: “One or more non-transitory
`computer-readable media storing one or more
`programs that are configured, when executed,
`to cause one or more processors to perform the
`method as recited in claim 1.” ........................................ 36
`Ground 2: Obviousness of Claims 1-6 and 11-20 Over Doo in
`view of Amira ...................................................................................... 37
`Disclosure with respect to Claims 1-6 ................................... 37
`1.
`(a) Claim 1 limitations including “the projected
`inner layer of the patient from the 3D data
`being confined within a volume of a virtual 3D
`shape”.............................................................................. 37
`(b) Claim 2: “The method of claim 1, wherein: the
`virtual 3D shape is a virtual box; and the virtual
`box includes a top side, a bottom side, a left
`side, a right side, a front side, and a back side.” ............ 39
`(c) Claim 3: “The method of claim 1, [3a]
`wherein: the virtual 3D shape is configured to
`be controlled to toggle between displaying and
`hiding lines of the virtual 3D shape; and [3b]
`the virtual 3D shape is configured to be
`controlled to reposition two-dimensional (2D)
`slices and/or 3D slices of the projected inner
`layer of the patient from the 3D data.” ........................... 39
`
`ME1 42794222v.2
`
`iii
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`2.
`
`3.
`
`(f)
`
`(d) Claim 4: “The method of claim 1, wherein
`lines of the virtual 3D shape are displayed.” .................. 40
`(e) Claim 5: “The method of claim 1, wherein lines
`of the virtual 3D shape are hidden.” ............................... 40
`Claim 6: “One or more non-transitory
`computer-readable media storing one or more
`programs that are configured, when executed,
`to cause one or more processors to perform the
`method as recited in claim 1” ......................................... 40
`Disclosure with respect to Independent Claim 11 ............... 40
`(a)
`“altering the original color gradient of the
`multiple inner layers to be lighter than the
`original color gradient in order to be better
`visible when projected onto real-time, non-
`image actual views of the outer layer of the
`patient” (“altering” step of Claim 11) ............................ 41
`“the projected inner layer of the patient from
`the 3D data being having the altered color
`gradient” (“altered color” limitation of Claim
`11) ................................................................................... 45
`Disclosure with respect to Dependent Claims 12-20 ............ 45
`(a) Claim [12/13/14/15/16/17/18/19]: “The method
`as recited in claim 11, wherein the altered color
`gradient represents a tissue
`[hardness/relaxivity/echogenicity/enhancement
`amount/enhancement speed/radioactivity/water
`content] tissue property of the multiple inner
`layers of the patient. ........................................................ 45
`(b) Claim 20: “One or more non-transitory
`computer-readable media storing one or more
`programs that are configured, when executed,
`
`(b)
`
`ME1 42794222v.2
`
`iv
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`C.
`
`2.
`
`to cause one or more processors to perform the
`method as recited in claim 11.” ...................................... 47
`4. Motive to Combine Doo and Amira ....................................... 47
`Ground 3: Obviousness of Claims 1-6 and 11-20 Over Chen in
`view of 3D Visualization and 3D Slicer .............................................. 47
`Chen’s Disclosure with respect to Independent Claim
`1.
`1 ................................................................................................. 50
`(a)
`preamble of Claim 1 ....................................................... 50
`(b)
`“identifying” step of Claim 1.......................................... 50
`(c)
`“displaying … inner layer[]” step of Claims 1 ............... 51
`(d)
`“confined” limitation of Claim 1 .................................... 53
`Disclosure with respect to Claims 2-6 ................................... 55
`(a) Claim 2: “The method of claim 1, wherein: the
`virtual 3D shape is a virtual box; and the virtual
`box includes a top side, a bottom side, a left
`side, a right side, a front side, and a back side.” ............ 55
`(b) Claim 3: “The method of claim 1, [3a]
`wherein: the virtual 3D shape is configured to
`be controlled to toggle between displaying and
`hiding lines of the virtual 3D shape; and [3b]
`the virtual 3D shape is configured to be
`controlled to reposition two-dimensional (2D)
`slices and/or 3D slices of the projected inner
`layer of the patient from the 3D data.” ........................... 55
`(c) Claim 4: “The method of claim 1, wherein
`lines of the virtual 3D shape are displayed.” .................. 56
`(d) Claim 5: “The method of claim 1, wherein lines
`of the virtual 3D shape are hidden.” ............................... 56
`(e) Claim 6: “One or more non-transitory
`computer-readable media storing one or more
`programs that are configured, when executed,
`
`ME1 42794222v.2
`
`v
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`3.
`
`4.
`
`to cause one or more processors to perform the
`method as recited in claim 1” ......................................... 56
`Chen’s Disclosure with respect to Independent Claim
`11 ............................................................................................... 57
`(b)
`“altering” step of Claim 11 ............................................. 58
`(c)
`“displaying … altered color” step of Claim 11 .............. 60
`Disclosure with respect to Dependent Claims 12-20 ............ 61
`(a) Claim [12/13/14/15/16/17/18/19]: “The method
`as recited in claim 11, wherein the altered color
`gradient represents a tissue
`[hardness/relaxivity/echogenicity/enhancement
`amount/enhancement speed/radioactivity/water
`content] tissue property of the multiple inner
`layers of the patient. ........................................................ 61
`(b) Claim 20: “One or more non-transitory
`computer-readable media storing one or more
`programs that are configured, when executed,
`to cause one or more processors to perform the
`method as recited in claim 11.” ...................................... 62
`
`
`
`
`ME1 42794222v.2
`
`vi
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`I, Peter Kazanzides, Ph.D., declare as follows:
`1.
`I have been retained by Medivis and its attorneys, McCarter &
`
`Attorney Docket No. 127971-00012
`
`
`
`English, LLP, to provide expert analysis with respect to Medivis’s inter partes
`
`review challenge to U.S. Patent No. 11,004,271 (“the ‘271 Patent”). I am being
`
`paid my consulting rate of $500 per hour for my consulting activities. My fees
`
`strictly depend on the time I spend on this matter and do not depend on the
`
`substance of my opinions nor the outcome of the inter partes review.
`
`I.
`
`QUALIFICATIONS
`
`2.
`
`I am a Research Professor in the Department of Computer Science at
`
`The Johns Hopkins University. I have been an employee of The Johns Hopkins
`
`University since 2002 and a Research Professor at The Johns Hopkins University
`
`since 2014.
`
`3. My research
`
`includes computer-assisted surgery encompassing
`
`human/machine interfaces, real-time sensing to account for uncertainty, and
`
`systems engineering for deployment in the real world. My research has focused on
`
`the integration of information from various sources, such as preoperative and
`
`intraoperative imaging and force sensing. I have done research on augmented
`
`reality interfaces for medical applications.
`
`4.
`
`Prior to my employment at JHU, I was employed in the medical
`
`device industry for 12 years, as a co-founder of Integrated Surgical Systems, Inc.
`
`ME1 42794222v.2
`
`1
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`(ISS). I have extensive experience in field service and customer support including
`
`Attorney Docket No. 127971-00012
`
`
`
`installation of systems at clinical trial sites, training of personnel, and support
`
`during surgical procedures.
`
`5.
`
`As a Research Professor or Associate or Assistant Research Processor
`
`at The Johns Hopkins University, I have been mentoring and advising graduate and
`
`undergraduate students since 2004. For example, I was the Advisor for Qian
`
`Long’s April 2020 Ph.D. Thesis entitled “Augmented Reality Assistance for
`
`Surgical Interventions using Optical See-Through Head-Mounted Displays” and
`
`for Ehsan Azimi’s July 2020 Ph.D. Thesis entitled “Interactive Platform for
`
`Medical Procedures in Mixed Reality.” I was the Secondary Advisor or Reader
`
`for Seth Billings’s August 2015 Ph.D. Thesis entitled “Probabilistic Feature-Based
`
`Registration for Interventional Medicine,” and for Byeol Star Kim’s November
`
`2021 Ph.D. Thesis entitled “Advances in Diagnosis and Surgery of Congenital
`
`Heart Disease through Novel Virtual Reality Systems for Design, Simulation, and
`
`Planning Methods.” I was an Advisor for Pramod Verma’s October 2010 M.S.
`
`Thesis entitled “Augmented Reality Based 3D UI for Medical Interventions,”
`
`Computer Science.
`
`6.
`
`I was on the Editorial Board of the INTERNATIONAL JOURNAL OF
`
`MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY in 2019.
`
`ME1 42794222v.2
`
`2
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`7.
`
`I co-authored the “Systems Integration” chapter of the HANDBOOK OF
`
`MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (Academic
`
`Press 2020).
`
`8.
`
`Before that, I co-authored the “Medical Robotics and Computer-
`
`Integrated Interventional Medicine” chapter of the ADVANCES IN COMPUTERS (M.
`
`Zelkowitz ed., Academic Press 2008).
`
`9.
`
`I have authored or co-authored over 200 journal articles and
`
`conference proceedings in the field of computer-assisted surgery. For example, I
`
`co-authored a paper entitled “ARAMIS: Augmented Reality Assistance for
`
`Minimally Invasive Surgery Using a Head-Mounted Display” that was published
`
`in Medical Image Computing and Computer-Assisted Intervention (MICCAI) on
`
`Oct. 2019. I co-authored a paper entitled “Modeling Physical Structure as
`
`Additional Constraints for Stereoscopic Optical See-Through Head-Mounted
`
`Display Calibration” that was published in the IEEE Intl. Symp. on Mixed and
`
`Augmented Reality (ISMAR) in Sept. 2016. I co-authored a paper entitled
`
`“Augmented Reality Goggles with an Integrated Tracking System for Navigation
`
`in Neurosurgery,” that was published in IEEE Virtual Reality in March 2012.
`
`10. From 2004 to 2010, I was involved in ASTM working group F04.05
`
`developing standards for Computer Assisted Orthopaedic Surgery (CAOS)
`
`Systems.
`
`ME1 42794222v.2
`
`3
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`11. My credentials, my publications, and other qualifications are more
`
`thoroughly detailed in my curriculum vitae, attached as Ex. 1013.
`
`12. Additionally, I am an inventor of more than a dozen issued U.S.
`
`patents directed to computer-assisted surgery. Some of these patents issued before
`
`the 2017 date I have been asked to apply in my analysis. For example, I am an
`
`inventor of U.S. Patent No. 8,398,541, issued March 19, 2013, entitled “Interactive
`
`User Interfaces for Robotic Minimally Invasive Surgical Systems.” More recently,
`
`I am an inventor of U.S. Patent No. 11,244,508, entitled “Augmented Reality
`
`Display for Surgical Procedures.”
`
`13. Based on
`
`the above highlighted credentials, and as further
`
`documented in my attached curriculum vitae, I believe that I have the necessary
`
`education, training, research, scholarship, and experience to analyze the ‘271
`
`Patent and opine on how a person of ordinary skill in the art (as further defined
`
`below) would have understood it and its validity in the context of the relevant prior
`
`art.
`
`II.
`
`LEGAL PRINCIPLES
`
`14. As seen above, I have some familiarity with patents, having applied
`
`for and obtained a number of them. I am thus generally aware, at least at a very
`
`high level, of the patent process and related concepts. Nonetheless, I am not a
`
`lawyer nor a patent specialist. Accordingly, the McCarter & English lawyers have
`
`ME1 42794222v.2
`
`4
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`informed me about some specific legal issues that may arise in this matter. In
`
`Attorney Docket No. 127971-00012
`
`
`
`particular, they have informed me that anticipation of a patent claim requires that
`
`every limitation of that claim is disclosed expressly or inherently in a single prior
`
`art reference, arranged as in the claim.
`
`15.
`
`I have also been informed that to assess whether a claim is anticipated
`
`or obvious in light of the prior art, I must analyze the challenged patent claim and
`
`the prior art from the perspective of a hypothetical “person of ordinary skill in the
`
`art” at the relevant time. A person of ordinary skill in the art, I am told, is deemed
`
`to be familiar with the state of the art, including all of the relevant prior art
`
`references, at the relevant time. I am further told that this person of ordinary skill
`
`in the art is not necessarily an inventor but, being familiar with the state of the art
`
`and the technology, would at least be able to read and understand the patent and
`
`then make or use the claimed invention.
`
`16.
`
`I have also been informed that when considering the obviousness of a
`
`patent claim, one may consider whether there existed at the relevant time a
`
`teaching, suggestion, or motivation that would have led a person of ordinary skill
`
`in the art to modify the prior art or to combine prior art teachings to arrive at the
`
`claimed invention. I understand that other rationales that may support a conclusion
`
`of obviousness include, but are not limited to, combining prior art elements
`
`according to known methods to yield predictable results; the simple substitution of
`
`ME1 42794222v.2
`
`5
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`one known element for another to obtain predictable results; the use of a known
`
`Attorney Docket No. 127971-00012
`
`
`
`technique to improve similar devices or products in the same way; the application
`
`of a known technique to improve a known device or product ready for
`
`improvement to yield predictable results; choosing from a finite number of
`
`identified, predictable solutions, with a reasonable expectation of success; and
`
`variations of known work based on design incentives or other market forces where
`
`the variations are predictable to one skilled in the art. I have been informed that
`
`hindsight is impermissible and that the obviousness or non-obviousness of a claim
`
`must be determined on the basis of the facts gleaned from the prior art and the
`
`knowledge of a person of ordinary skill at the relevant time (that is, before the
`
`priority date of the challenged patent). I have been informed that I must, in view
`
`of all of the factual information, determine whether the claimed invention “as a
`
`whole” would have been obvious at the relevant time to the hypothetical person of
`
`ordinary skill in the art. I have been informed that in an inter partes review, a prior
`
`art reference must be read and evaluated in its entirety, including portions that
`
`would teach away from the claimed invention.
`
`III. MATERIALS CONSIDERED
`
`17.
`
`I have read and analyzed the ‘271 Patent (Ex. 1001) and excerpts of
`
`its prosecution history (Exs. 1002, 1003, 1004). I have also reviewed and relied on
`
`the following prior art references and other documents:
`
`ME1 42794222v.2
`
`6
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`Description
`Exhibit
`Ex. 1001 US Patent No. 11,004,271, claiming priority to March 30, 2017 (the
`‘271 Patent)
`Ex. 1002 Excerpts of File history of Application No. 16/574,524, now the
`‘271 Patent
`Ex. 1003 Excerpts of File history of Application No. 15/894,595, now U.S.
`Patent No. 10,475,244, through which the ‘271 Patent claims
`priority
`Ex. 1004 Excerpts of File history of Application No. 15/474,702, filed on
`March 30, 2017, and now U.S. Patent No. 9,892,564, through which
`the ‘271 Patent claims priority
`Ex. 1005 Excerpt of Amira 5 User’s Guide title through Chapter 2 (Visual
`Imaging 2009) (“Amira”)
`Ex. 1006 US Publication No. WO 2016/0191887 A1 to Casas, published on
`June 30, 2016 (“Casas”)
`
`Ex. 1007 S. Pujol, Ph.D. et al., 3D Visualization of DICOM Images for
`Radiological Applications (Surgical Planning Laboratory, Brigham
`and Women’s Hospital, Boston, Massachusetts 2014) (“3D
`Visualization”)
`
`Ex. 1008 International Publication No. WO 2015/164402 A1 to Doo et al.,
`published on October 29, 2015 (“Doo”)
`
`Ex. 1009 X. Chen et al., “Development of a Surgical Navigation System
`Based On Augmented Reality Using an Optical See-Through Head-
`Mounted Display,” 55 JOURNAL OF BIOMEDICAL INFORMATICS 124-
`131 (2015) (“Chen”)
`Ex. 1010 Main Application GUI for 3D Slicer
`<https://www.slicer.org/wiki/Documentation/4.6/Slicer/Application/
`MainApplicationGUI> (last edited 7 November 2016) (“3D Slicer”)
`
`ME1 42794222v.2
`
`7
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`Description
`Exhibit
`Ex. 1011 E. Azimi et al., “Augmented Reality Goggles with an Integrated
`Tracking System for Navigations in Neurosurgery,” IEEE
`VIRTUAL REALITY 123-124, 123 (IEEE 2012) (“AR Goggle for
`Neurosurgery”).
`
`
`
`Intentionally Omitted
`
`Ex. 1013 Curriculum Vitae of Peter Kazanzides Ph.D.
`
`
`
`IV. OPINIONS AND ANALYSIS
`
`18.
`
`I have been asked to provide my opinions regarding (a) the relevant
`
`art for the ‘271 Patent and the hypothetical person of ordinary skill in that art;
`
`(b) an overview of the technology of the ‘271 Patent and the state of the art before
`
`the filing of that patent; (c) the understanding by a person of ordinary skill in the
`
`art of claims of the ‘271 Patent including certain claim terms in the patent, as well
`
`as an understanding of the prior art; (d) whether Doo anticipates Claims 1, 5, and 6
`
`of the ‘271 Patent; (e) whether Doo, in combination with Amira renders Claims 1-6
`
`and 11-20 of the ‘271 Patent obvious; and (f) whether Chen anticipates Claims 1,
`
`5, and 6 of the ‘271 Patent; (e) whether Chen in combination with 3D Visualization
`
`or 3D Slicer renders Claims 1-6 and 11-20 of the ‘271 Patent obvious.
`
`ME1 42794222v.2
`
`8
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`A.
`
`The Relevant Field of Invention and the Person of Ordinary Skill
`in the Art
`
`19. The “Background” of the ‘271 Patent states that “[a]ugmented reality
`
`(AR) systems generally take a user’s live view with computer-generated virtual
`
`elements such as video, sound, or graphics.” Ex. 1001, ‘271 Patent, 1:17-20. It
`
`further states that, “[a]s a result, AR systems functions to enhance a user’s current
`
`perception of reality.” Ex. 1001, ‘271 Patent, 1:20-21.
`
`20. The ‘271 Patent asserts that “[o]ne common problem faced by AR
`
`systems is accurately aligning the position of a virtual elements with a live view of
`
`a real-world environment.” Ex. 1001, ‘271 Patent, 1:22-24.
`
`21. The ‘271 Patent states that “[c]onventional medical imagining systems
`
`may create three-dimensional (3D) data for a patient and then display that 3D data
`
`as an image or images on a computer display.” Ex. 1001, ‘271 Patent, 2:49-51. It
`
`states that “viewing images of a patient on a computer display … may be useful in
`
`training, research, diagnosis, and treatment ….” Ex. 1001, ‘271 Patent, 2:51-55.
`
`22. All of the claims of the ‘271 Patent are directed to “augmenting real-
`
`time, non-image actual views of a patient with three-dimensional (3D) data.” Ex.
`
`1001, ‘271 Patent, Claims 1, 7, and 11. “FIG. 1 illustrates an example augmented
`
`reality (AR) environment in which real-time views of a patient may be augmented
`
`with the three-dimensional (3D) data.” Ex. 1001, ‘271 Patent, 2:25-27. Figure 1
`
`illustrates “an operating room with an operative table 103.” Ex. 1001, ‘271 Patent,
`
`ME1 42794222v.2
`
`9
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`4:8-11. Figure 1 illustrates the patient as “a living human adult” on the operative
`
`Attorney Docket No. 127971-00012
`
`
`
`table. Ex. 1001, ‘271 Patent, 4:28-31. The ‘271 Patent states “the patient 106 may
`
`… ha[ve] been rendered unconscious in order to undergo a medical procedure ….”
`
`Ex. 1001, ‘271 Patent, 4:28-31.
`
`23. Based on my reading of the ‘271 Patent, as well as the prior art
`
`discussed below, it is clear, in my opinion, that the relevant art is systems and
`
`methods for using augmented reality during medical procedures.
`
`24.
`
`I have been asked to assume that the relevant date for my analysis of
`
`how a person of ordinary skill in the art would understand the substance of the
`
`‘271 Patent and the prior art is on or before March 30, 2017.
`
`25.
`
`In my opinion, based on my reading of the ‘271 Patent, the
`
`hypothetical person of ordinary skill in the art would be a person with a bachelor’s
`
`degree in computer science, electrical engineering, or a related field with several
`
`years of experience in the design, development, and study of augmented reality
`
`devices and either (a) familiar with conventional medical imaging data and
`
`visualization of data for medical procedures or (b) working with a team including
`
`someone with such familiarity.
`
`B. Challenged Claims
`
`26. As I discuss above, the ‘271 Patent claims certain methods for
`
`augmenting real-time, non-image actual views of a patient with three-dimensional
`
`ME1 42794222v.2
`
`10
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`(3D) data. All of the claimed methods have two common steps. Claim 1, for
`
`Attorney Docket No. 127971-00012
`
`
`
`example, is detailed below:
`
`1. A method for augmenting real-time, non-image actual views
`of a patient with three-dimensional (3D) data, the method comprising:
`identifying 3D data for the patient, the 3D data including an
`outer layer of the patient and multiple inner layers of the
`patient; and
`displaying, in an augmented reality (AR) headset, one of the
`inner layers of the patient from the 3D data projected onto
`real-time, non-image actual views of the outer layer of the
`patient,
`the projected inner layer of the patient from the 3D data being
`confined within a volume of a virtual 3D shape.
`
`The other challenged independent claim—claim 11—includes all of the
`
`limitations of claim 1 except the requirement that “the projected inner layer of the
`
`patient from the 3D data being confined within a volume of a virtual 3D shape.”
`
`27. The method of claim 11 further requires the step of “altering the
`
`original color gradient of the multiple inner layers to be lighter than the original
`
`color gradient in order to be better visible when projected onto real-time, non-
`
`image actual views of the outer layer of the patient.” Claim 11 also requires “the
`
`projected inner layer of the patient from the 3D data being having the altered color
`
`gradient.” Most of the method claims that depend from claim 11 limit what the
`
`altered color gradient represents.
`
`ME1 42794222v.2
`
`11
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`C.
`
`28.
`
`Claim Interpretation
`
`I have been told that, on August 3, 2022, the patentee asserted that “all
`
`of the claims terms … should be given their plain and ordinary meaning as
`
`understood by one having ordinary skill in the art” and that “no court claim
`
`construction is necessary.”
`
`29. The phrases “three-dimensional (3D) data” and “3D data” appear in
`
`each of the independent claims of the ‘271 Patent. The ‘271 Patent states that
`
`“[c]onventional medical imagining systems may create three-dimensional (3D)
`
`data for a patient and then display that 3D data as an image or images on a
`
`computer display.” Ex. 1001, ‘271 Patent, 2:49-51. In my opinion, the person of
`
`ordinary skill in the art would understand “three-dimensional (3D) data” and “3D
`
`data” to be synonymous because the ‘271 patent uses those terms synonymously.
`
`30. The ‘271 Patent states “3D data of the patient 106 may include, but is
`
`not limited to, MRI images, Computerized Tomography (CT) scan images, X-ray
`
`images, Positron Emission Tomography (PET) images, ultrasound images,
`
`fluorescence images, Infrared Thermography (IRT) images, and Single-Photon
`
`Emission Computer Tomography (SPECT) scan image, or some combination
`
`thereof.” Ex. 1001, 11:45-51. It further states: “Any of these images may be in
`
`the form of still images or video images.” Ex. 1001, 11:51-53.
`
`ME1 42794222v.2
`
`12
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`
`
`
`Attorney Docket No. 127971-00012
`
`31. Like claim 1 of the ‘271 patent, claim 1 of the original parent
`
`application (Application No. 15/474,702, filed on March 30, 2017) introduces
`
`“three-dimensional (3D) data” in the preamble. Like claim 1 of the ‘271 patent,
`
`claim 1 of the original parent application is directed to “[a] method for augmenting
`
`real-time, non-image actual views of a patient with three-dimensional (3D) data.”
`
`Also like claim 1 of the ‘271 patent, claim 1 of the original parent application
`
`includes the step of “identifying 3D data for a patient, the 3D data including an
`
`outer layer of the patient and multiple inner layers of the patient.”
`
`32. Claim 6 of
`
`the original parent application (Application No.
`
`15/474,702, filed on March 30, 2017) depends directly on original parent
`
`application claim 1. Ex. 1004, original claim 6. Original parent application claim
`
`6 states “the 3D data includes one or more of MRI images, Computerized
`
`Tomography (CT) scan images, X-ray images, Positron Emission Tomography
`
`(PET) images, ultrasound images, fluorescence images, Infrared Thermography
`
`(IRT) images, and Single-Photon Emission Computer Tomography (SPECT) scan
`
`image.” Ex. 1004, original claim 6. Based on the foregoing, in my opinion, the
`
`person of ordinary skill in the art would understand that “3D data” may be one or
`
`more of any of the foregoing types of images.
`
`33. The phrase “virtual 3D shape” only appears in the ‘271 Patent in
`
`claim 1 and claims that depend from claim 1. Figure 1 illustrates element 116,
`
`ME1 42794222v.2
`
`13
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`which the specification references as “a virtual spatial difference box.” Ex. 1001,
`
`Attorney Docket No. 127971-00012
`
`
`
`FIG. 1 and 3:63-4:7. The ‘271 patent states that element 116 is “generated by the
`
`AR headset 108 and only viewable by the user 104 through the AR headset 108.”
`
`Ex. 1001, 4:1-7. It states “the virtual spatial difference box 116 may be generated
`
`by the AR headset 108 to confine within a volume of the virtual spatial difference
`
`box 116 the projected inner layer of the patient 106 from the 3D data. For example,
`
`the projected bones 106b of the patient 106 may be confined within the virtual
`
`spatial difference box 116 in FIG. 1.” Ex. 1001, 6:5-11. But the ‘271 patent also
`
`discloses “the AR headset 108 [may] analyz[e] the 3D data in order to determine
`
`the size and shape of the outer layer of the patient [and] may … generat[e] a point
`
`cloud of the outer layer that represents the size and shape of the outer layer. For
`
`example, where the outer layer is represented by triangles or other polygonal
`
`shapes, the point cloud may include some or all of the vertices of the polygonal
`
`shapes.” Ex. 1001, 12:39-48. Accordingly, the ‘271 patent discloses virtual 3D
`
`shapes as simple as a box and as complex as the outer layer of the patient, all of
`
`which would confine a projected inner layer of the patient.
`
`34. The term “being having” only appears in the ‘271 Patent in claim 11
`
`within the larger phrase “the projected inner layer of the patient from the 3D data
`
`being having the altered color gradient.” To a person of ordinary skill in the art,
`
`the term “being having” appears to be an error. “Being having” is not a term of art.
`
`ME1 42794222v.2
`
`14
`
`Medivis Exhibit 1012
`
`

`

`Patent No. 11,004,271
`
`“Being having” is not plain English. For purposes of my analysis, I interpret the
`
`Attorney Docket No. 127971-00012
`
`
`
`term “being having” as “being or having.”
`
`D. Reasons for Allowance
`
`35. The Notice of Allowability states: “Reasons for allowance remain the
`
`same as discussed in the non-final office action dated 8/06/2020.” Ex. 1002,
`
`Excerpts of File History, 9. The office action provides reasons for allowance of
`
`claims 1-6 and claims 11-20, but not claims 7-10.
`
`36. As the reason for allowance of claims 1-6, the office action indicates
`
`that none of three cited references “suggest projection onto real-time actual views
`
`of the patient, layers confined to a 3D shape.” Ex. 1002, 37.
`
`37. As the reason for allowance of claims 11-20, the office action
`
`indicates that none of three cited references “suggest application to projection on to
`
`real-time actual views by lightening the layer to improve visibility.” Ex. 1002, 38.
`
`38. The office rejects claims 7-10 and therefore provides no reason for
`
`allowance of claims 7-10. Ex. 1002, 32-36.
`
`E. Overview of the Technology and State of the Art
`
`39. Neither the general concepts disclosed in nor t