`
`Transcript of Peter Kazanzides,
`Ph.D (271) and Peter Kazanzides,
`Ph.D (807)
`
`Date: June 22, 2023
`Case: Medivis, Inc. -v- Novarad Corp. (PTAB)
`
`Planet Depos
`Phone: 888.433.3767
`Email: transcripts@planetdepos.com
`www.planetdepos.com
`
`WORLDWIDE COURT REPORTING & LITIGATION TECHNOLOGY
`
`Page 1 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 1 (1 to 4)
`Conducted on June 22, 2023
`
`3
`
`4
`
` A P P E A R A N C E S
`
` ON BEHALF OF THE PETITIONER:
`
` KIA L. FREEMAN, ESQUIRE
`
` McCARTER & ENGLISH, LLP
`
` 265 Franklin Street
`
` Boston, Massachusetts 02110
`
` 617-449-6549
`
`
`
` ON BEHALF OF THE PATENT OWNER:
`
`0
`
` JOSEPH HARMER, ESQUIRE
`
`1 2 3 4 5 6 7 8 9 1
`
`11
`
` JED HANSEN, ESQUIRE
`
`12
`
` THORPE NORTH & WESTERN LLP
`
`13
`
` 175 South Main, Suite 900
`
`14
`
` Salt Lake City, Utah 84111
`
`15
`
` 801-566-6633
`
`16
`
`17
`
`
`
`
`
`18
`
`Also Present: William Pantoja, Technician
`
`1
`
`19
`
`20
`
`21
`
`22
`
`
`
`
`
`
`
`
`
` UNITED STATES PATENT AND TRADEMARK OFFICE
`
` ----------------------------------------
`
` BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
` ----------------------------------------
`
` MEDIVIS, INC.,
`
` Petitioner,
`
` v.
`
` NOVARAD CORP.,
`
` Patent Owner.
` ----------------------------------------
` U.S. PATENT 11,004,271
` ----------------------------------------
` IPR2023-00042
`
`1 2 3 4 5 6 7 8 9
`
`10
`
`11
`
` ----------------------------------------
`
`12
`
` U.S. PATENT 10,945,807 B2
`
`13
`
` ----------------------------------------
`
`14
`
` IPR2023-00045
`
`15
`
`
`
`16
`
` Deposition of PETER KAZANZIDES
`
`17
`
` CONDUCTED VIRTUALLY
`
`18
`
` Thursday, June 22, 2023
`
`19
`
` 9:59 a.m. EST
`
`20
`
`
`
`21
`
`Job No.: 496331
`
`22
`
`Pages: 1 - 134
`
` C O N T E N T S
`
`EXAMINATION PAGE
`
` By Mr. Harmer 5,69
`
`
`
`
`
` E X H I B I T S
`
`EXHIBIT PAGE
`
`Exhibit '807 1010 16,80
`
`Exhibit '807 1009 41,105
`
`0
`
`Exhibit '271 1012 53,117
`
`1 2 3 4 5 6 7 8 9 1
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Reported By: Brooklyn E. Schweitzer, RPR, CRR
`
` Deposition of PETER KAZANZIDES, conducted
`
`2
`
`virtually.
`
`
`
`
`
` ** ALL PARTIES ATTENDED REMOTELY. **
`
`
`
`
`
` Pursuant to Notice, before Brooklyn E.
`
`0
`
`Schweitzer, Registered Professional Reporter,
`
`1 2 3 4 5 6 7 8 9 1
`
`11
`
`Certified Realtime Reporter, and Notary Public in
`
`12
`
`and for the Commonwealth of Pennsylvania.
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`Page 2 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 2 (5 to 8)
`Conducted on June 22, 2023
`5
`
`7
`
`courtroom?
` A Yes.
` Q Are there any reasons why you can't give
`truthful testimony today?
` A No.
` Q And you're not aware of any illness or
`medication that can affect your ability to provide
`truthful testimony?
` A That's correct.
` Q Is there anybody in the room with you
`today?
` A No.
` Q Do you understand that no one can assist
`you in answering questions?
` A Yes.
` Q Are you using any device or software to
`receive assistance answering questions?
` A No.
` Q And you agree not to accept any
`assistance --
` A Excuse me. I want to exit my email.
`Sorry for interrupting. I forgot to do that.
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
` P R O C E E D I N G S
` PETER KAZANZIDES,
` was called, and having been duly sworn,
` testified as follows:
` EXAMINATION
`BY MR. HARMER:
` Q Good morning, Dr. Kanzanzides.
` A Good morning.
` Q Can you please state your full name and
`address for the record?
` A Peter Kanzanzides, 10 Forest Ridge Ct,
`Lutherville, Maryland 21093.
` Q Thank you. My name is Joseph Harmer. I'm
`an attorney for Novarad Corporation, and I want to
`thank you for your time this morning.
` Have you ever had your deposition taken
`before?
` A No, I have not.
` Q One of the objectives of a deposition is
`to create a clean record. If we all talk at the
`same time, we make it difficult for our court
`reporter. It may seem like a small thing, but if
`
`8
`
`1234567891
`
`we can all try not to talk at the same time, it
`will really help.
` Will you try to wait and respond until
`after I'm done asking a question?
` A Yes.
` Q Thank you. And I will try not to ask
`another question until you've finished answering.
` It's also important to provide verbal
`responses. Other responses including a head nod
`or other gestures are more difficult to record, as
`you can imagine. Does that make sense?
` A Yes.
` Q And there may be times when Ms. Freeman,
`Medivis' counsel, might make an objection. If
`that happens, please wait until the objection is
`made, and then go ahead and answer to the best of
`your ability unless you are specifically
`instructed not to answer. Can you do that?
` A Yes.
` Q Thank you. And you understand that you
`are under oath today to tell the truth the same as
`if you were in front of a judge and a jury in a
`
` In fact, I'll close everything on this
`computer except for Zoom. Sorry. I meant to do
`that beforehand.
` Okay. There we go.
` Q Thank you.
` A Nothing running.
` Q And you agree not to accept any assistance
`answering questions today?
` A Yes.
` Q If you don't understand a question, please
`0
`let me know. Otherwise I will assume that you
`11
`understood the question. Is that fair?
`12
` A Yes.
`13
` Q Have you ever given testimony at trial
`14
`before?
`15
` A No.
`16
` Q Have you ever provided a declaration in an
`17
`inter partes proceeding other than the two IPR
`18
`proceedings we're currently involved in?
`19
` A No.
`20
` Q Rather than say inter partes review, I may
`21
`use IPR. Do you understand that IPR stands for
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`6
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 3 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 3 (9 to
`Conducted on June 22, 2023
`9
`
`12)
`
`11
` Q Do you remember who at Medivis reached out
`to you?
` A I'm not certain of who it was, but I
`believe it would have been -- I don't recall his
`full name, but one of -- one of the founders of
`the company.
` Q Have you provided any other expert
`services to Medivis?
` A No.
` Q Do you have any prior relationship with
`the founders of Medivis?
` A I've spoken to them, but I don't have any
`prior relationship with them.
` Q Have you spoken to them about either of
`the '807 and '271 IPRs?
` A Prior to when they contacted me about it,
`no.
` Q Have you had any business dealings at all
`with Medivis?
` A I am not clear what you mean by business
`dealings.
` Q Have you conducted business with Medivis
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`inter partes review?
` A Yes.
` Q You've been retained by Medivis, Inc., to
`provide expert testimony in connection with two
`IPR proceedings; correct?
` A Yes.
` Q One is related to U.S. Patent No.
`10,945,807; is that right?
` A Yes.
` Q For clarification, is it okay if we refer
`to that patent as the '807 patent?
` A Yes.
` Q And is it okay if we refer to that IPR
`proceeding as the '807 IPR?
` A Yes.
` Q The other IPR proceeding is U.S. -- pardon
`me. Yes, the other IPR proceeding is related to
`U.S. Patent No. 11,004,271; is that right?
` A That's right.
` Q And may we refer to that patent as the
`'271 patent and that IPR proceeding as the '271
`IPR?
`
`12
`
`1234567891
`
` A Yes.
` Q You've provided a declaration in each of
`the '807 and '271 IPR proceedings. Can we refer
`to those as the '807 and '271 declarations
`respectively?
` A Yes.
` Q Do you understand that you're here today
`to provide testimony with respect to each of the
`'807 and '271 declarations?
` A Yes.
` Q Thank you. When were you first engaged to
`provide services in connection with the '807 and
`'271 IPR proceedings?
` A I don't recall the exact date, but it was
`sometime in late 2022. I'm guessing somewhere
`August -- between August and October.
` Q And who engaged you to perform those
`services?
` A Who engaged me? The company Medivis in
`conjunction with McCarter & English.
` Q Did Medivis reach out to you directly?
` A Yes, that's my recollection.
`
`in any way?
` A I would say no. Clarify: I did once
`contact them about possibly partnering with a
`grant proposal, but they declined, so I would not
`consider that a business dealing.
` Q And what was the subject of the grant?
` A It was -- it was a DOD solicitation on,
`like, 5D for telemedicine, along those lines.
` Q Thank you. Did it have anything to do
`with augmented reality?
`0
` A The grant?
`11
` Q Yes.
`12
` A So the -- the DOD grant would have
`13
`involved augmented reality.
`14
` Q In what way?
`15
` A My best recollection is that it involved
`16
`telementoring of having a -- basically remote
`17
`assistance.
`18
` Q Can you explain to me what that is? I'm
`19
`sorry. I didn't understand.
`20
` A It's hard for me to say because I did go
`21
`ahead with another entity and we did refine the
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`10
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 4 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 4 (13 to
`Conducted on June 22, 2023
`13
`grant further. So really at the time I approached
`Medivis, it was rather vague. The DOD was just
`looking for ways to use 5G to enable long-distance
`sort of telemedicine.
` So there weren't really any specific plans
`at the time at which I contacted Medivis, and as I
`said, they have declined to participate.
` Q When was the first time you reached out to
`Medivis about the grant proposal?
` A It was a couple years ago. I don't
`recall.
` Q Can you approximate? A couple years, you
`said?
` A Yeah. Right. I can't really pin it down.
`Within the last five years, anyway.
` Q And when was the last time you spoke to
`them about the proposal?
` A About the same time. It really was a very
`brief "are you interested or no," that kind of
`thing.
` Q Why did you reach out to them in the first
`place? How did you know about them? Sorry.
`
`16
`
`Those are two questions. Let me ask it --
` (Exhibit '807 1010 was marked for
` A Thank you -- thank you for clarifying.
`identification and is attached to the transcript.)
` Q How did you know about Medivis before you
`BY MR. HARMER:
`reached out to them?
` Q Dr. Kanzanzides, is this a copy of the
` A One of my students who graduated is
`curriculum vitae or CV that you attached to your
`employed there.
`'807 declaration?
` A Yes. At least the first one or two pages
` Q Is he still your student?
` A No, no. He had graduated.
`that you've shown, yes.
` Q At the time. I see.
` MR. HARMER: William, can you give
` A He graduated, and then he -- yeah, he took
`Dr. Kanzanzides control so he can scroll through
`0
`a position there at some point after he graduated.
`it real quick?
`11
`It was not his first position after graduation.
` TECHNICIAN: No problem.
`12
` Q Does that student still work for Medivis?
` THE WITNESS: All right.
`13
` A To the best of my knowledge, yes.
` TECHNICIAN: You now have control.
`14
` Q What is that student's name?
` THE WITNESS: Okay. Okay, this works.
`15
` A Long Qian. Long --
` Yes, this is my entire CV.
`16
` Q Spell that. Thank you.
`BY MR. HARMER:
`17
` A L-O-N-G Q-I-A-N.
` Q Did you attach the same document to your
`18
` Q Have you spoken with anyone at Medivis
`'271 declaration as Exhibit 1013?
`19
` A Let me check the exhibit number. Which --
`specifically about your deposition today?
`20
` A No.
`which -- I'm sorry, which did you ask about? Was
`21
`it attached to which one?
` Q And have you had any other conversations
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`1234567891
`
`14
`
`16)
`
`15
`
`with anyone at Medivis?
` A Not since beginning this process, no.
` Q When was the last time you spoke with them
`before this process began?
` MS. FREEMAN: Objection.
` THE WITNESS: Okay. Could you --
` Q Yes, I'll rephrase the question.
` A Yeah.
` Q When was the last time you spoke to anyone
`at Medivis prior to being retained as an expert in
`these IPR proceedings, the '271 and '807 IPRs?
` A It was probably in May of 2021.
` Q And what was the purpose of that contact
`with Medivis in May of 2021?
` A Yeah. My student attended the graduation
`ceremony to receive his PhD, the one who's
`employed at Medivis.
` Q This is Long Qian?
` A Yes.
` MR. HARMER: William, can you please put
`up Exhibit '807 1010?
` TECHNICIAN: Sure. One moment.
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 5 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 5 (17 to
`Conducted on June 22, 2023
`17
`
`20)
`
`19
`
` All right. So it looks like you've been
`an employee at Johns Hopkins since 2002; is that
`right?
` A Yes.
` Q Did you speak permission from Johns
`Hopkins before you were retained as an expert
`witness in these IPR proceedings?
` A The process at Johns Hopkins is that we
`don't seek permission in advance, but we submit
`a -- what's it called, a conflict of interest
`disclosure, and that gets reviewed and approved.
` Q Who reviews it?
` A The process. I don't have a name as to
`who reviews it, but someone at the -- at Johns
`Hopkins reviews it.
` Q And what's the purpose of the conflict of
`interest check?
` A Well, there's several goals. One is
`that -- you know, they want to make sure that what
`you're doing is not -- well, some of the concerns
`they have when they review conflict of interest
`disclosures is whether you could abuse your
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
` Q This one that we're looking at now is your
`'807 declaration.
` A Okay.
` Q I just want to make sure it's the same CV
`you attached to the '271 declaration, and it was
`Exhibit 1013.
` A Okay. In my notes, it's the same as
`Exhibit 1010.
` Q '807 is Exhibit 1010?
` A That is my -- that is what my notes say.
` Q And for the '271 declaration, what exhibit
`is it?
` A '271 -- 1013.
` Q Is this CV current?
` A I have the date on the bottom. It is
`current as of that date.
` Q July 24th, 2022?
` A Yes.
` Q Is there anything that you would like to
`add to it?
` A There's not any significant change. I
`probably have a few more publications and some
`18
`changes in sponsored research, but I don't think
`that's extremely important to add.
` Q You're currently a research professor at
`Johns Hopkins University?
` A Yes.
` Q Do you have full confidence in a college
`that doesn't know how to spell John? I'm just
`joking.
` Who said that?
` A I'm sorry. Who said what?
` Q Do you remember who said "I don't have
`full confidence in a college that doesn't know how
`to spell John"?
` A I mean, you said it; right?
` Q It's a famous quote from Mark Twain. It's
`on Johns Hopkins' website. I just thought it was
`funny.
` A It's on the website?
` Q It is. It's not funny. I apologize. Bad
`joke.
` A Okay. It's good to break the ice, though.
` Q Yeah, yeah.
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`20
`
`1234567891
`
`position of authority over a student. For
`example, if you do consulting work and you have
`your students do the work and get paid for it,
`that would be a bad case.
` They also look for conflict of interest,
`conflict of commitment, factors such as that.
` Q On Page 2 of your CV, you state that
`you've done research in augmented reality
`head-mounted displays. What does that research
`entail?
`0
` A So --
`11
` MS. FREEMAN: Objection; foundation.
`12
` THE WITNESS: I'm sorry. Was there --
`13
` MS. FREEMAN: I just said objection;
`14
`foundation. Feel free to answer.
`15
` THE WITNESS: Okay.
`16
` So I've done research both in sort of
`17
`fundamental technology involving augmented reality
`18
`displays. For example, calibration of optical
`19
`see-through head-mounted displays.
`20
` I have done applications in the use of
`21
`augmented reality -- not necessarily with
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`Page 6 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`24)
`
`23
`
`software.
` Q And what are the purposes of those
`algorithms?
` A So one example is the calibration of
`the -- when you have an optical see-through
`head-mounted display as an example of a HoloLens,
`the -- you're overlaying virtual objects on
`essentially a little transparent or
`semitransparent lens.
` And so the alignment of that virtual
`object with the real world really depends on how
`the user perceives it. So you essentially have to
`calibrate the display system to the retina of the
`user.
` And so in that calibration step, you're
`collecting data. You're trying to identify
`parameters, and there's different techniques that
`you can use for doing that.
` Q What are the challenges with doing that?
` A So challenges are essentially getting it
`to be accurate enough, depending on what your
`application requirements are, without making it
`
`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 6 (21 to
`Conducted on June 22, 2023
`21
`head-mounted displays, but with different kinds of
`display technology for applications in the medical
`domain as well as in space -- space robotics
`applications.
` I can get into more detail if you wish. I
`could also point out some of the publications that
`I have as examples. Let's see.
` Okay. So here's one where -- again, I can
`get into as much detail as you'd like in these,
`but this is an augmented reality including
`magnification. Going kind of quickly here, so --
`let's see.
` This one refers to some work in the use of
`mixed reality for space and space application.
`This one technically could qualify as well as far
`as telerobotics and sort of remote control.
` I think there's a lot more in the
`conferences. Okay, here's another one, a review
`paper that we did on the use of augmented reality
`in robot-assisted surgery.
` Here's another journal paper on restoring
`awareness of the visual field. Here's one on the
`24
`22
`too much of a burden for the user. People don't
`use of augmented reality on a head-mounted display
`want to do -- every time they put on a headset, do
`for an assistant for robotic surgery.
`a calibration for, you know, ten minutes, for
` This is one using augmented virtuality,
`example. So...
`which is on the sort of a mixed-reality continuum
` Q Does any of your research have application
`for telerobotic satellite servicing.
`in the medical field with regard to AR headsets,
` I think I go now to the -- okay. This is
`AR technology?
`one -- this was just a comparison of different
` A Yes.
`display technologies.
` Q Does it involve the use of augmented
` So those are just the journals. I have
`reality head-mounted displays by medical
`even more in conference publications. Do you wish
`0
`professionals during surgery?
`me to continue?
`11
` A Could you clarify whether -- are you
` Q No, that's okay.
`12
`asking if it's being used in actual surgeries or
` A Okay.
`13
`if it's -- or if that's the application area.
` Q Does your research, does it involve the
`14
` Q If that's the application area.
`software side of these AR applications or hardware
`15
` A Yes.
`or both?
`16
` A It's been primarily software and
` Q As part of your research, do you ever work
`17
`algorithms.
`with physicians at Johns Hopkins?
`18
` A Yes.
` Q And when you say algorithms, what do you
`19
` Q Which ones?
`mean?
`20
` A You know, they're listed as co-authors on
` A I mean, let's say some mathematical
`21
`some of these papers.
`constructions which can be implemented in
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 7 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 7 (25 to
`Conducted on June 22, 2023
`25
`
`27
`
` A Okay. Okay. So Paper No. 19 on the
`screen here is about using augmented reality --
`where's the cursor -- for minimally invasive
`surgery using a head-mounted display, and what
`that did is -- the goal there was to provide
`essentially see-through vision for the surgeon,
`for example, in a laparoscopic procedure.
` And the way that was done, as disclosed in
`the publication, was to in reality build a model
`or a 3D point cloud of the internal anatomy of the
`patient, project the endoscope video onto that
`point cloud, and then render that on a
`head-mounted display.
` Q When you say 3D point cloud, what do you
`mean?
` A Yes. What I mean is a collection of
`points that have three dimensional coordinates;
`that is, X, Y, and Z locations.
` Q How is that 3D point cloud generated?
` A In the paper, No. 19, we generated it from
`basically looking at the disparity between images
`from a left and right camera in a stereo
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
` Q Can you identify some of those for us?
` A Sure. Robin Yang and Cecil Qiu, maybe.
`Camilo Molina and Judy Huang. Also here Camilo
`and Judy Huang are in this one. Greg Osgood, I
`think, was in one of the journal papers that was
`listed above, and -- augmented reality -- let's
`see.
` There may be others, but those are the
`ones that came to mind.
` Q Thank you. Do you know who Dr. Timothy
`Witham is?
` A I don't think so. Could you spell the
`last name.
` Q W-I-T-H-A-M?
` A W-I-T-H-A-M? I do not.
` Q Do you know who -- let's see -- the
`director of Johns Hopkins neurosurgery spinal
`fusion laboratory is?
` A I do not. I may know the person, but I
`don't know who holds that position.
` Q Thank you. Are these all physicians that
`work at Johns Hopkins medical -- or it's Johns
`
`28)
`
`1234567891
`
`26
`
`28
`endoscope. But there are other techniques in the
`literature.
` Q In this particular application, what was
`the purpose of generating the 3D point cloud?
` A So to be able to render the internal view
`of the patient from a different perspective; that
`is, from the perspective of the person wearing the
`head-mounted display rather than from the
`perspective of the endoscope that's actually
`taking the image.
`0
` Q And the endoscope has two cameras?
`11
` A Yes. In this -- in our implementation, we
`12
`used a stereo endoscope.
`13
` Q Can you please identify the next paper
`14
`that has surgical applications for AR?
`15
` A All right. I will go to Paper 21, which
`16
`is called augmented reality-assisted instrument
`17
`insertion and tool manipulation for the first
`18
`assistant in robotic surgery.
`19
` Q And what is this app -- what is this paper
`20
`about?
`21
` A So first -- some knowledge. First
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`Hopkins Hospital, I think?
` A Yeah, that's fine.
` MS. FREEMAN: Objection to form.
` THE WITNESS: I -- are you -- could you
`please clarify the question?
` Q Yes. Are the physicians that you
`identified in these papers, do they all work at
`the Johns Hopkins Hospital?
` A No, not currently.
` Q Do any of them have a business
`relationship with Medivis?
` A Not that I'm aware of.
` Q Besides calibration, which you mentioned
`in your research, is there any other functions of
`the use of AR during your surgery that you've
`investigated in your research?
` A Yes.
` Q Can you identify some of those that have
`use in surgical applications by a medical
`professional?
` A Do you want me to identify papers?
` Q If that's easiest.
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 8 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 8 (29 to
`Conducted on June 22, 2023
`29
`
`32)
`
`31
`
`they were also able to see the anatomy.
` Q How are they able to see the anatomy?
` A In the case in this paper, it was as a --
`basically a 2D image that was correctly oriented
`inside -- you know, what would appear to be at the
`correct location inside the patient.
` Q It's a 2D image, you said?
` A In that case, it was 2D. I'm going
`through two things, I guess, in reverse
`chronological order. That was one of the
`motivations for the Paper No. 19 that I discussed.
`It was basically being able to do that at 3D. But
`in No. 21, it was a 2D image of the anatomy.
` Q In Paper No. 19, was that the first time
`you had done 3D patient anatomy?
` MS. FREEMAN: Objection; form.
` A So I need to think about that a moment.
` Q No problem.
` A No.
` Q Okay. When did you use a 3D point cloud
`before Paper 19?
` A A 3D point cloud is not the only 3D
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`assistant is a term used in da Vinci robotic
`surgery. If you're familiar with the da Vinci
`surgical robot, there's a robot pulling
`instruments inside the patient controlled by a
`surgeon sitting at a console.
` First assistant is another surgeon who's
`actually at the patient's bedside and is
`assisting, doing things like exchanging
`instruments, feeding materials, say, such as
`suture through another port, retracting --
`applying retractions, suction, irrigation, those
`kinds of tasks.
` And that person doesn't have the
`visualization that's provided to the surgeon
`sitting at the console there operating similar to
`a laparoscopic surgeon looking currently at a 2D
`display of, you know, the endoscope view, and that
`can be difficult due to some, you know, hand-eye
`coordination challenges.
` And so the -- what we proposed and
`implemented was give that assistant a head-mounted
`display, and by registering that head-mounted
`
`30
`entity. So -- okay. So 3D entities could also
`display to the da Vinci robot, we could project,
`include, like, a wire mesh, which is a point cloud
`you know, augmented images on the head-mounted
`but also with edges to produce a more closed
`display where the robotic instruments are inside
`object.
`the patient, where the endoscope is, also the
`endoscope field of view, which we rendered as a --
` Q What do you mean by closed object?
` A Well, a point cloud has both --
`as a three-dimensional outline, like a ThruSTEM,
`essentially just a cloud of points. So there's
`as well as they could also view the endoscope
`gaps between the points. If you define shapes
`image in its correct location physically, you
`through the points; for example, taking three
`know, inside the patient.
`points, forming a triangle, that now defines --
` So it gave them -- to summarize, it gives
`0
`and you do that for all of the points. That
`the assistant sort of see-through capability to
`11
`defines a shape. Could be a -- well, some sort of
`look inside the patient, which helps them do their
`12
`3D shape.
`task because now they can more easily, when
`13
` Often it's called a 3D mesh.
`they're inserting devices or instructs to assist,
`14
`they can -- they have better awareness of where
` Q How is a 3D mesh generated?
`15
` A Well, there are a number of ways.
`they need to go.
`16
`Probably the most famous was a technique called
` Q So the assistant can see through the
`17
`marching cubes, which was patented by GE I think
`augmented reality display the anatomy of the
`18
`in about 1990 or '91. But it was just a technique
`patient as well as the instrument?
`19
`of creating a 3D mesh from voxelized data such as
` A The -- yeah. The anatomy of the -- that
`20
`you would find in a CT or an MRI.
`is correct. Yes. We focused more on the
`21
`instruments and the endoscope field of view, but
` Q Okay. We'll come back to that. Can you
`22
`PLANET DEPOS
`888.433.3767 | WWW.PLANETDEPOS.COM
`
`1234567891
`
`32
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`Page 9 of 64
`
`Ex. 2006
`Medivis v. Novarad
`IPR 2023-00042
`
`

`

`Transcript of Peter Kazanzides, Ph.D (271) and Peter Kazanzides, Ph.D (807) 9 (33 to
`Conducted on June 22, 2023
`33
`
`point out the next paper that has surgical --
`excuse me. Strike that.
` Can you point out the next paper that has
`AR applications or AR use in surgical
`applications?
` A Okay. I think what I'll do is I'm going
`to go kind of in chronological order, so I'll go
`down towards the bottom in -- let's see. So are
`you asking, as a point of clarification, augmented
`reality only on head-mounted displays or on other
`displays as well?
` Q Just head-mounted, and we can just do one
`more. We don't have to do every one.
` A Okay. Let's see. There's one, I know --
`one or two in 2012, head-mounted displays. Almost
`there.
` Okay. I believe this one is one of the
`exhibits, No. 80 -- Paper 82, which is augmented
`reality goggles with an integrated tracking system
`for navigation in neurosurgery.
` Q Great. Thank you.
` Does Johns Hopkins own the research that
`
`36)
`
`35
`
` THE WITNESS: Sorry. What was the
`objection?
` MS. FREEMAN: Objection; relevance.
` THE WITNESS: Okay.
` MS. FREEMAN: I apologize. I think I
`might have thrown off Dr. Kanzanzides with that
`objection. Would you mind having -- repeating the
`question?
` MR. HARMER: Sure.
`BY MR. HARMER:
` Q Which of these patents is Johns Hopkins
`University the applicant or the assignee?
` A Mm-hmm. Okay. So for Patents 6 -- let's
`see -- so from 6 through 14, I was a Johns Hopkins
`employee, and so therefore they -- at least my
`rights -- some of these are joined patents with
`other entities.
` I believe Patent No. 5 might have Hopkins
`involved through one of my co-inventors. At the
`time, I was -- I was not at Johns Hopkins, so my
`assignment would not be to Johns Hopkins or No. 5
`or any of the ones -- you know, 1 through 4
`
`1234567891
`
`0
`11
`12
`13
`14
`15
`16
`17
`18
`19
`20
`21
`22
`
`1234567891
`
`34
`
`36
`
`you conduct since you've been employed there?
` A You mean -- yes. They have -- any IP that
`I develop is assigned to Johns Hopkins.
` Q Including IP related to augmented reality
`head-mounted displays?
` A If it's developed during the course of my
`employment, yes.
` Q Dr. Kanzanzides, can you turn to page --
`let's see -- 26? Kanzanzides, I apologize.
` A That's okay. You can just call me
`whatever, Peter or Mr. K or whatever.
` Q I was calling you Mr. K initially,
`actually. It was easier.
` A Yes, 26.
` Q So here you list 14 patents.
` A Yes.
` Q Are you a named inventor on each of these
`patents?
` A Yes.
` Q Is Johns Hopkins the applicant or assignee
`in any of these?
` MS. FREEMAN: Objection; relevance.
`
`either.
` Q Thank you. I believe that on a few of
`these patents Intuitive Surgical Operations is a
`joint applicant with Johns Hopkins.
` A That's correct.
` Q What is Intuitive Surgical Operations?
` A That is the company that makes the da
`Vinci surgical robot.
` Q And what's its relationship with Johns
`Hopkins?
`0
` A They -- well, they've done some -- I think
`11
`these particular patents were a result of a joint
`12
`research grant from NSF, but they've -- they've --
`13
`they're a large company. They've sponsored
`14
`research. They've been partners on funded
`15
`research.
`16
` So they, I believe, have -- they haven't
`17
`licensed any of my IP from Johns Hopkins, but they
`18
`may have licensed other IP.
`19
` Q Does Intuitive Surgical Operations provide
`20
`tools for augmented reali