`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`Medivis, Inc.
`Petitioner
`
`v.
`
`Novarad Corp.
`Patent Owner
`
`
`
`Case IPR2023-00042
`US Patent No. 11,004,271
`
`
`
`DECLARATION OF MAHESH S. MULUMUDI, M.D.
`
`1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 1 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`TABLE OF CONTENTS
`
`
`I. QUALIFACTIONS ................................................................................................................. 7
`
`II. MY EXPERTISE AND THE PERSON OF ORDINARY SKILL IN THE ART ................ 11
`
`III.
`
`LEGAL STANDARDS ..................................................................................................... 12
`
`IV. OVERVIEW OF THE USE OF AUGMENTED REALITY (AR) IN MEDICAL
`PRACTICE AT THE TIME OF THE ’271 PATENT .................................................................. 15
`
`V. THE ’271 PATENT AND THE CHALLENGED CLAIMS ................................................ 20
`
`A. The ’271 Patent Requires Direct Volume Rendered 3D Data. .......................................... 20
`
`B. The ’271 Patent Provides a Novel Method for Navigating, in Real-Time, Direct Volume
`Rendered Data While It is Projected onto Non-Image Actual Views of a Patient. .................. 23
`
`VI. CONSTRUCTION OF CERTAIN CLAIM TERMS ........................................................ 25
`
`1. “three-dimensional (3D) data … including an outer layer of the patient and multiple inner
`layers of the patient” ................................................................................................................. 27
`
`2. “inner layer(s) of the patient” ............................................................................................ 29
`
`3. “confined within a virtual 3D shape”................................................................................. 31
`
`4. “being having” ................................................................................................................... 34
`
`5. Summary Table of Claim Interpretation ............................................................................ 34
`
`VII. ANALYSIS OF THE PRIOR ART ................................................................................... 35
`
`A. Doo (Ex. 1008)................................................................................................................... 35
`
`B. Chen (Ex. 1009) ................................................................................................................. 41
`
`C. 3D Slicer (Ex. 1010) .......................................................................................................... 42
`
`D. 3D Visualization (Ex. 1007) .............................................................................................. 44
`
`E. Amira (Ex. 1005) ............................................................................................................... 45
`
`VIII. GROUNDS OF PATENTABILITY .................................................................................. 45
`
`
`
`2
`
`Page 2 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`A. Ground 1: Doo Does Not Anticipate Claims 1, 5, and 6. .................................................. 45
`
`1. Doo does not anticipate independent claim 1................................................................. 46
`
`2. Doo does not anticipate dependent claims 5 and 6. ....................................................... 54
`
`B. Ground 2: Doo Alone or in View of Amira Does Not Render Claims 1-6 Obvious. ........ 55
`
`1. Non-disclosure with respect to claims 1-6 ..................................................................... 55
`
`2. Non-disclosure with respect to independent claim 11 ................................................... 61
`
`3. Non-Disclosure with respect to claims 12-20 ................................................................ 63
`
`4. No motive to combine Doo and Amira .......................................................................... 64
`
`C. Ground 3: Chen in View of 3D Visualization and 3D Slicer Does Not Render Claim 1-6
`and 11-20 Obvious. ................................................................................................................... 66
`
`1. Chen’s non-disclosure with respect to independent claim 1 .......................................... 67
`
`2. Chen’s disclosure with respect to dependent claims 2-6 ................................................ 70
`
`3. Chen’s disclosure with respect to independent claim 11 ............................................... 72
`
`4. Non-disclosure with respect to dependent claims 12-20 ................................................ 75
`
`5. No motive to combine Chen, 3D Visualization, and 3D Slicer. .................................... 76
`
`
`
`
`
`
`
`
`
`
`
`3
`
`Page 3 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`I, Mahesh S. Mulumudi, M.D., declare as follows:
`
`1.
`
`I am over 21 years of age and otherwise competent to make this
`
`declaration. I make this declaration based upon facts and matters within my own
`
`knowledge and on information provided to me by others.
`
`2.
`
`I have been retained as an expert witness to provide testimony on
`
`behalf of Novarad Corp. (“Patent Owner”) as part of the above-captioned inter
`
`partes review proceeding (“IPR”).
`
`3. More specifically, I have been asked primarily to assist in evaluating
`
`the grounds raised in the Petition (Paper 3) and in Dr. Kazanzides’ declaration (Ex.
`
`1012) in support of that Petition.
`
`4.
`
`I reserve the right to supplement this Declaration in response to
`
`additional evidence that may come to light.
`
`5.
`
`I am not currently, nor have I ever been, employed by Patent Owner.
`
`Nor have I previously served as an expert witness on behalf of Patent Owner.1
`
`6.
`
`I understand that Patent Owner owns U.S. Patent No. 11,004,271
`
`(“the ’271 Patent”), entitled AUGMENTING REAL-TIME VIEWS OF A
`
`PATIENT WITH THREE-DIMENSIONAL DATA,” the validity of which
`
`Medivis, Inc. (“Petitioner”) challenges.
`
`
`1 Nor have I performed work for, met, or ever spoken with Dr. Steven Cvetko or Dr. Wendell
`Gibby, the named inventors in the ’271 Patent.
`
`4
`
`
`
`Page 4 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`7.
`
`I have reviewed the specification of the ’271 Patent as well as its
`
`prosecution history. I am familiar with the ’271 Patent. A copy of the ’271 Patent
`
`was provided as Exhibit 1001.
`
`8.
`
`I am familiar with the technology at issue at the time of the ’271
`
`Patent, which Petitioner has assumed to be on or before March 30, 2017. Petition
`
`at 10.
`
`9.
`
`I have also reviewed the Petition. The Petition presents several
`
`grounds for challenging the claims of the ’271 Patent, which appear deficient to me
`
`for reasons I address in more detail below. See infra at ¶¶ 98-179.
`
`10.
`
`I have also reviewed the declaration of Dr. Kazanzides (Ex. 1012),
`
`which I also address.
`
`11.
`
`I have also reviewed the following references cited by Petitioner as
`
`prior art and other documents included in the exhibit list below:
`
`Description
`
`Exhibit
`
`Ex. 1001 U.S. Patent No. 11,004,271 (the ’271 Patent)
`
`Ex. 1002 Excerpts of file history of Application No. 16/574,524, now the ’271
`Patent
`
`Ex. 1003 Excerpts of file history of Application No. 15/894,595, now U.S.
`Patent No. 10,475,244.
`
`Ex. 1004 Excerpts of file history of Application No. 15/474,702, now U.S.
`Patent No. 9,892,564.
`
`
`
`
`5
`
`Page 5 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Ex. 1005 Excerpt of Amira 5 User’s Guide (“Amira”)
`
`Ex. 1006 U.S. Publication No. US 2016/0191887 A1 to Casas (“Casas”)
`
`Ex. 1007 S. Pujols, Ph.D. et al., 3D Visualization of DICOM Images for
`Radiological Applications (“3D Visualization”)
`
`Ex. 1008 International Publication No. WO 2015/164402 A1 to Doo et al.
`(“Doo”)
`
`Ex. 1009 X. Chen et al., “Development of Surgical Navigation System Based
`On Augmented Reality Using an Optical See-Through Head-Mounted
`Display,” 55 JOURNAL OF BIOMEDICAL INFORMATICS 124-131 (2015)
`(“Chen”)
`
`Ex. 1010 Main Application GUI for 3D Slicer
`<https://www.slicer.org/wiki/Documentation/4.6/Slicer/Application/
`MainApplicationGUI> (“3D Slicer”)
`Ex. 1011 E. Azimi et al. “Augmentation Reality Goggles with an Integrated
`Tracking System for Navigations in Neurosurgery,” IEEE VIRTUAL
`REALITY 123-124, 123 (IEEE 2012) (“AR Goggle for Neurosurgery”)
`
`Ex. 1012 Declaration of Peter Kazanzides, Ph.D.
`
`Ex. 1013 Curriculum Vitae of Peter Kazanzides Ph.D.
`
`
`
`12.
`
`I am being compensated for my time in connection with the IPR at a
`
`rate of $700 per hour. I am also being compensated for any out-of-pocket
`
`expenses for my work in this review. My compensation as an expert is in no way
`
`dependent upon the results of any investigations I undertake, the substance of any
`
`opinion I express, or the ultimate outcome of the review proceedings.
`
`
`
`
`
`
`
`6
`
`Page 6 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`I.
`
`QUALIFACTIONS
`
`13.
`
`I believe I am qualified to serve as a technical expert in this
`
`proceeding based on my educational and work experience, including my 20+ years
`
`of experience in imaging, virtual reality, augmented reality with biofeedback, and
`
`treating peripheral vascular and coronary artery diseases from 2000 through today.
`
`14. This declaration sets forth my opinions, which I formed based on my
`
`study of the evidence; my understanding as an expert in the field; and my training,
`
`education, research, knowledge, and personal and professional experience. All of
`
`my opinions stated in this declaration are based on my own personal knowledge,
`
`expertise, training, education, and professional judgment. In forming my opinions,
`
`I have relied on my knowledge and experience in human factors, user interface
`
`design, user interaction design, human-computer interaction, and software
`
`engineering.
`
`15.
`
`If I am called upon to do so, I would be competent to testify to the
`
`matters set forth herein. My qualifications to testify about the technical subject
`
`matter in this case and the relevant technology are outlined in my curriculum vitae
`
`and this section of this declaration. A copy of my current curriculum vitae is
`
`provided for this proceeding as Exhibit 2003.
`
`16.
`
`I earned my Bachelor of Medicine and Bachelor of Surgery, the
`
`equivalent to a Doctor of Medicine degree, from Kurnool Medical College in 1990.
`
`
`
`7
`
`Page 7 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`After graduation, I completed an internship at the Government General Hospital in
`
`India.
`
`17. Upon completion of my internship, I continued my studies at the
`
`University of Texas at Austin as a PhD student in the Department of Pharmacology
`
`and Toxicology from 1992 to 1994. As a PhD student, I researched the effects of
`
`multiple oral doses of ICI D2138 on allergen induced bronchoconstriction,
`
`conducting Phase I and Phase II of the clinical studies at Health Quest Research.
`
`18.
`
`I completed my residency in Internal Medicine at Detroit Medical
`
`Center in 1997. Following my residency, in 1998, I was the primary investigator
`
`for the Phase IIIb clinical trial of a cyclooxygenase inhibitor versus naproxen
`
`sodium in the treatment of osteoarthritis.
`
`19. From 2000 to 2005, I was engaged in two cardiology fellowships at
`
`the Oschner Clinic Foundation in New Orleans, Louisiana. During these
`
`fellowships, I organized clinical research investigating the role of abciximab in
`
`improving myocardial blush after primary coronary intervention for acute ST
`
`segment elevation myocardial infarction. I also researched the concept of kidney
`
`frame counts in the renal angiography and the relationship of tissue myocardial
`
`perfusion to the intravascular ultrasound of the epicardial coronary arteries in
`
`orthotopic heart transplant patients.
`
`
`
`8
`
`Page 8 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`20. Following my fellowships, I became certified in Interventional
`
`Cardiology through the American Board of Internal Medicine in 2005. In 2013, I
`
`became board certified in Cardiovascular Disease through the American Board of
`
`Internal Medicine.
`
`21.
`
`I have filed for nineteen patents involving the invention of electronic
`
`devices in conjunction with anatomic regions.
`
`22.
`
`I am the author of fifteen research papers in professional journals and
`
`proceedings in cardiac and renal assessment and functions.
`
`23.
`
`I am the inventor of a virtual reality application that leverages
`
`biofeedback to offer mindfulness exercises in an effort to reduce stress and
`
`enhance focus. I also created a virtual reality telemedicine solution that enhances
`
`remote healthcare delivery by enabling purposeful and engaging interactions
`
`between patients and their healthcare providers.
`
`24.
`
`I currently serve as President and CEO of StratoScientific, Inc.;
`
`Division Chief of Medicine; Section Head of Cardiology; and Medical Director of
`
`Interventional Cardiology and Chest Pain Programs at Providence Regional
`
`Medical Center in Washington. I am also the Director of Vascular Laboratory and
`
`Heart & Vascular Department at the Everett Clinic. From 2006 through 2012, I
`
`served as Facilities Medical Director I the Department of Heart & Vascular,
`
`Neurology, and Nephrology at the Everett Clinic.
`
`9
`
`
`
`Page 9 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`25. More information about my educational and professional background
`
`can be found in my curriculum vitae, Exhibit 2003.
`
`26. The combination of my education, research, training, and work
`
`experience in cardiovascular health and virtual reality technology in healthcare as
`
`it relates to improving the patient experience enables me to provide analysis and
`
`opinions on the subject matter of this litigation.
`
`27.
`
`I am being compensated for my work on this case and my fee is not
`
`contingent on the outcome of this case or on any of my opinions or the technical
`
`positions I explain in this report. In addition, I have no financial interest in the
`
`outcome of this case, or any of the parties involved this case. I have worked on a
`
`number of litigation cases, several of which were related to the role technology
`
`plays in the standard of care received by patients.
`
`28. Based on the above credentials and experience, and as further
`
`documented in my attached CV, I believe that I have the necessary education,
`
`training, research, scholarship, and experience to analyze the ’271 Patent and opine
`
`on how a person of ordinary skill in the art (as further defined below) would have
`
`understood it and its validity in the context of the relevant prior art.
`
`
`
`
`
`
`
`10
`
`Page 10 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`II. MY EXPERTISE AND THE PERSON OF ORDINARY SKILL IN
`THE ART
`
`29. My opinions in this declaration are based on my experience in this
`
`technical field. More specifically, my over 20+ years of experience in imaging,
`
`virtual reality, augmented reality with biofeedback, and treating peripheral vascular
`
`and coronary artery diseases from 1983 through today. I am qualified to provide
`
`expert opinions on some aspects of the technology described in the ’271 Patent and
`
`on the references cited by Petitioner and Dr. Kazanzides.
`
`30.
`
`I understand that my opinion must be taken from the perspective of
`
`what would have been known or understood by a person of ordinary skill in the art
`
`(“POSITA”) at the time of the invention.
`
`31.
`
`I understand that Dr. Kazanzides has concluded that the relevant art is
`
`“systems and methods for using augmented reality during medical procedures.”
`
`Ex. 1012, ¶ 23.
`
`32.
`
`I further understand that Petitioner has defined POSITA at the time of
`
`the ’271 Patent as “a person with a bachelor’s degree in computer science,
`
`electrical engineering, or a related field with several years of experience in the
`
`design, development, and study of augmented reality devices and either (a) familiar
`
`with conventional medical imaging data and visualization of data for medical
`
`
`
`11
`
`Page 11 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`procedures or (b) working with a team including someone with such familiarity.”
`
`Ex. 1012, ¶ 25.
`
`33.
`
`I understand, and have been informed, that for purposes of assessing
`
`the obviousness of a claimed invention, the level of ordinary skill possessed by a
`
`POSITA is informed by, among other things, the type of problems encountered in
`
`the relevant art, the prior art solutions to those problems, the rapidity with which
`
`innovations are made in the relevant art, the sophistication of the relevant
`
`technology, and the educational level of active workers in the field.
`
`III. LEGAL STANDARDS
`
`34.
`
`I am not an attorney. My analysis and opinions are based on my
`
`expertise as an interventional cardiologist, as well as the instructions I have been
`
`given by counsel for the legal standards relating to patent validity.
`
`35.
`
`I understand that a claim of an issued patent may be found to be
`
`unpatentable if the claim is not novel or would have been obvious in view of the
`
`prior art. I understand that this determination is made from the perspective of a
`
`person having ordinary skill in the art who is presumed to be aware of all prior art.
`
`36.
`
`I understand that a patent claim will not be unpatentable due to what
`
`is called anticipation unless all elements of a claim, arranged in the same way, are
`
`found in a single reference. I have also been informed that for anticipation, each
`
`
`
`12
`
`Page 12 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`and every claim limitation must be explicitly or inherently disclosed in the prior
`
`art.
`
`37.
`
`I have also been informed and understand that anticipation by inherent
`
`disclosure is appropriate only when the reference necessarily includes the unstated
`
`limitation. Thus, it has been explained to me that inherency may not be established
`
`by probabilities or possibilities, and the mere fact that a certain thing may result
`
`from a given set of circumstances is insufficient.
`
`38.
`
`I understand that a patent claim will not be unpatentable due to
`
`obviousness unless the differences between the claimed invention and the prior art
`
`are such that the subject matter as a whole would have been obvious to a POSITA
`
`at the time of the invention.
`
`39.
`
`I understand that the following factors are considered in assessing
`
`obviousness: (1) the scope and content of the prior art; (2) the differences between
`
`the prior art and the claimed invention; (3) the level of ordinary skill in the art at
`
`the time of the invention; and (4) “objective indicia of non-obviousness,” also
`
`referred to as secondary considerations of non-obviousness. Those objective
`
`indicia include considerations such as whether a product covered by the claims is
`
`commercially successful due to the merits of the claimed invention, whether there
`
`was a long felt need for the solution provided by the claimed invention, whether
`
`others failed to find the solution provided by the claimed invention, and whether
`
`
`
`13
`
`Page 13 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`there was acceptance by others of the claimed invention as shown by praise from
`
`others in the field.
`
`40.
`
`I also understand that one should analyze whether the prior art
`
`references disclose each and every element of the claim as those references would
`
`have been read by a POSITA. I understand that obviousness cannot be proven by
`
`mere conclusory statements.
`
`41.
`
`I understand that a claim is not proved obvious merely by showing
`
`that each of the individual elements existed somewhere in the prior art.
`
`42.
`
`I understand that when a combination of prior art references is relied
`
`upon to assert obviousness, the party asserting obviousness must also identify a
`
`reason or motivation that would have prompted one of ordinary skill in the art to
`
`combine the elements of the references in the same way as the claimed invention
`
`and with a reasonable expectation of success.
`
`43.
`
`I also understand that in assessing obviousness, it is improper to rely
`
`upon hindsight and that one should avoid using the claimed invention as a roadmap
`
`to piece together disclosures from the prior art.
`
`44.
`
`I understand that in this inter partes review proceeding, the claims of
`
`the ’271 Patent are generally given their ordinary and customary meaning as
`
`understood by a POSITA in light of the patent’s specification and the prosecution
`
`history.
`
`
`
`14
`
`Page 14 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`45.
`
`I also understand that claim terms should be given their broadest
`
`reasonable interpretation to a POSITA, which interpretation must be consistent
`
`with the patent’s specification.
`
`46. Similarly, I understand that the prosecution history contains the
`
`complete record of all the proceedings before the Patent and Trademark Office and
`
`may contain contemporaneous exchanges between the patent applicant and the
`
`PTO about what the claims mean. Thus, I understand that the prosecution history
`
`can inform the meaning of the claim language by demonstrating how the inventor
`
`understood the invention and whether the inventor limited the invention in the
`
`course of prosecution.
`
`47.
`
`I understand that where a patent expressly defines a claim term, that
`
`definition may also inform the understanding of a POSITA. I have applied these
`
`standards in my analysis in this declaration.
`
`IV. OVERVIEW OF THE USE OF AUGMENTED REALITY (AR) IN
`MEDICAL PRACTICE AT THE TIME OF THE ’271 PATENT
`
`48. One common source of medical imaging is a computed tomography
`
`(CT) scan, which uses X-rays to create detailed images of the inside of the body.
`
`When the X-ray source and detectors rotate around the patient, they record from
`
`many different angles. Conventionally, a computer then uses this 3D data to
`
`reconstruct a set of 2D images or slices. This set of 2D slices are conventionally
`
`
`
`15
`
`Page 15 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`reconstructued along one of three planes: coronal (front to back), sagittal (left to
`
`right), or axial (top to bottom):
`
`Coronal Slices
`
`
`
`Sagittal Slices
`
`
`
`
`Each of these 2D images represents distribution of X-ray attenuation in the
`
`Axial Slices
`
`
`
`patient’s body.
`
`49. Each 2D CT scan image consists of a grid of 2D pixels. The value of
`
`the pixel corresponds to the radiodensity of the tissue within the pixel. While each
`
`pixel represents a small volume element, or voxel2, of the patients body, a 2D CT
`
`scan image actually provides no information about the patient anatomy between
`
`the slices.
`
`
`
`Pixel on 2D Plane
`
`
`
`2 A voxel is the 3D equivalent of a pixel.
`
`3D Voxel
`
`
`
`16
`
`
`
`Page 16 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`50. Another common source of medical imaging is a magnetic resonance
`
`imaging (MRI) scan, which uses a strong magnetic field and radio waves to create
`
`images of the inside of the body. The output of an MRI scan, conventionally, is
`
`one of three sets of 2D images or slices (coronal, sagittal, and axial), similar to a
`
`CT scan. Again, each 2D image consists of a grid of 2D pixels. The value of the
`
`pixel corresponds to the strength of the MRI signal received at that pixel, which
`
`depends on the type of tissue in the pixel. The pixel value also depends on the
`
`MRI sequence used, among other factors. While each pixel represents a small
`
`volume of element of the patient’s body, a 2D MRI scan image provides no
`
`information about layers of the patient anatomy between the 2D slices.
`
`51. There are various methods for visualizing CT and MRI scan data,
`
`including “slice-by-slice viewing,” “multiplanar reconstruction (MPR), surface
`
`rendering, and direct volume rendering (DVR). One of the earliest, and still most
`
`common, methods for visualizing medical imaging data is “slice-by-slice viewing.”
`
`This is where individual 2D slices are viewed sequentially, one at a time, in the
`
`direction they were compiled. Traditionally, CT and MRI slices are acquired and
`
`viewed along only one axis, typically the axial or transverse plane (horizontal
`
`slices as if looking from the feet towards the head). This “slice-by-slice viewing”
`
`method does not provide a complete 3D view of anatomical structures within the
`
`volume of a patient.
`
`
`
`17
`
`Page 17 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`52. Another method for visualizing a specific anatomical feature from CT
`
`and/or MRI data is “surface rendering.” This method involves creating a 3D
`
`surface model of an anatomical feature of interest from the medical imaging data.
`
`This is done by first performing a segmentation step to identify the voxels within
`
`the original 3D dataset that belong to an anatomical structure of interest. After
`
`segmentation, a polygonal model of the surface of the identified anatomical
`
`structure, i.e., a shell, is generated. Since surface rendering only provides a view
`
`of the modeled surface of the previously identified anatomical structure, it is used
`
`when the focus is on a specific structure of interest. But it does not provide
`
`information about the antomical areas of interest within the structure to which a
`
`modeled surface was rendered.
`
`53. Surface rendering can be time-consuming and requires a high-level of
`
`expertise to perform the segmentation step accurately. Also, surface rendering
`
`may not accurately represent complex or internal structures because it relies on
`
`identifying and rendering the surfaces of structures. As a result, minor errors or
`
`noise in the data can cause significant errors in the final surface rendering. In
`
`surface rendering, it is common to apply thresholding to distinguish between
`
`different types of tissues or materials in the data. However, if thresholds are not
`
`set correctly, this can lead to errors or omissions in the final surface rendering. It
`
`might also oversimplify the complexity of the data. Also, surface rendering is
`
`
`
`18
`
`Page 18 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`unable to represent semi-transparent or overlapping structures accurately which
`
`can be a disadvantage with medical imaging where these types of structures are
`
`common.
`
`54. Direct volume rendering (DVR) is very different from the other
`
`visualization methods described above. DVR is a method for visualizing medical
`
`imaging data, including CT and MRI scan or ultrasound data, in three-dimensions.
`
`DVR gives physicians a more realistic and intuitive view of complex anatomical
`
`situations avoiding geometric assumptions. It can offer perspectives that are not
`
`limited to the traditional planes (e.g., axial, sagittal, and coronal), enabling a view
`
`of the anatomical structures within the volume of a patient from any direction.
`
`55. Unlike surface rendering, DVR is not feature specific, but takes into
`
`account the complete volume of the patient data. While surface models depict the
`
`external surfaces or boundaries of an object, DVR creates a 3D image by
`
`considering every individual voxel in the dataset allowing for the visualization of
`
`internal stuctures, and not just outer surfaces of a particular anatomical feature.
`
`56. Also, unlike surface rendering, DVR does not require the creation of
`
`an intermediate surface representation and does not require segmentation of the
`
`data into different structures or tissues. Instead, DVR operates directly on the
`
`original 3D dataset (i.e., grid of voxels that make up the raw volume data). Each
`
`voxel within the 3D dataset is assigned a color and opacity based on transfer
`
`
`
`19
`
`Page 19 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`fuctions. All the voxels are then composited, or blended together, to create a 3D
`
`image of the entire volume of the patient anatomy.
`
`57. DVR is incredibly beneficial in surgerical applications where the
`
`anatomical features of interest do not align along conventional axial, sagittal, or
`
`coronal planes or are intertwined with other structures. DVR is not limited to
`
`conventional planes. It provides improved depth perception and spatial
`
`relationships between different structures, which is crucial in complex surgeries.
`
`The other visualization methods described above do not provide these same
`
`benefits.
`
`58. Although DVR is more computationally intensive than other
`
`visualization methods, it provides a unique comprehensive and flexible
`
`visualization of 3D medical imaging. At the same time, because DVR provides a
`
`comprehensive and flexible visualization of 3D medical imaging, controls are
`
`needed to help isolate and identify areas of interest within a DVR volume—
`
`especially in an augmented reality (AR) environment. For example, when
`
`projected onto real, non-image, actual views of a patient in an operating room.
`
`V. THE ’271 PATENT AND THE CHALLENGED CLAIMS
`
`A. The ’271 Patent Requires Direct Volume Rendered 3D Data.
`
`59. The ’271 Patent discloses “[a]ugmenting real-time views of a patient
`
`with three-dimensional (3D) data.” Ex. 1001, Abstract. This “3D data” includes
`
`
`
`20
`
`Page 20 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`“an outer layer of the patient and multiple inner layers of the patient.” Ex. 1001,
`
`1:46-49. The ’271 Patent further discloses displaying the full volume of the “3D
`
`data,” and any lesser portion of the full volume, including a single slice. See Ex.
`
`1001, 5:34-25 (“display a slice of the 3D data instead of a volume of the 3D data”).
`
`The 3D slices can be cut along any axis. Ex. 1001, 5:38-40 (“display different
`
`slices of the 3D data including, but not limited to, axial slices, coronal slices,
`
`sagittal slices, and oblique slices.”); 6:17-21 ("Slices may be … 3D slices [and] …
`
`[m]ay include curved slices, such as curved slices that follow the natural curve of
`
`an anatomical feature, or slices that have depth as well as height and width”). The
`
`’271 Patent further discloses that the user can navigate, e.g., move between, the
`
`slices cut along any axis in real time. Ex. 1001, 8:44-47 (“the user may … change
`
`from viewing the axial slice to a sagittal slice, and then from the sagittal slice to a
`
`coronal slice, respectively.”). The projected 3D data may be viewed from any
`
`angle. Ex. 1001, 4:55-56. (“the 3D data of the patient 106 may be viewed by the
`
`user 104 from any angle”). The “multiple inner layers may be layers that go all the
`
`way through the patient 106, or may be layers that only go to a certain partial depth
`
`into the patient.” Ex. 1001: 12:13-16.
`
`60.
`
`In view of the foregoing, a POSITA understands that the ’271 Patent
`
`describes direct volume rendered 3D data because only a DVR image is 3D,
`
`includes an outer layer and multiple inner layers of the patient, and can be cut
`
`
`
`21
`
`Page 21 of 79
`
`Ex. 2002
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`along any axis and viewed from any angle in real-time. A POSITA understands
`
`that only DVR provides the comprehensiveness, detail, and flexibility to visualize
`
`and navigate the internal layers of a projected 3D volume of a patient in real-time:
`
`“all the way through the patient,” at a “certain partial depth into the patient,” and
`
`along multiple axes, as described and claimed in the ’271 Patent.
`
`61. A POSITA understands that this cannot be done using any of the other
`
`known methods. For example, as mentioned above, “slice-by-slice viewing” and
`
`“multiplanar reconstruction (MPR)” do not provide a full 3D view. They also
`
`cannot be viewed or navigated in real-time along multiple axes. Conventional
`
`“slice-by-slice viewing” is limited to the axis of the original scan (e.g., axial,
`
`sagittal, or coronal). Similarly, MPR is limited to the plane selected for “re-
`
`slicing.” Additionally, anatomical structures do not always align themselves along
`
`the conventionally defined axial planes (e.g., axial, sagittal, or coronal). A POSITA
`
`appreciates that anatomical structures are very unique to each patient and tend to
`
`traverse in and out of conventional planes as they course through the human body.
`
`62. A POSITA understands that “surface rendering” is also limited. For
`
`example, surface rendering only provides a view of a surface model, i.e., shell, of a
`
`specific feature of interest and nothing else. A surface model does not include any
`
`of the surrounding