`IPR2023-00042
`U.S. Pat. No. 11,004,271
`
`Joseph Harmer
`Representing Patent Owner
`
`January 30, 2024
`
`
`
`fi
`
`102
`
`|
`
`118a
`
`106b
`
`'
`
`y
`
`
`a
`
`
`Ex. 1001, FIG. 1 (Sur-reply, 16)
`Ex. 1001, FIG. 1 (Sur-reply, 16)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 1
`Slide 1
`
`FIG. 1
`
`
`
`106
`
`116~.
`
`107~118a
`
`106
`
`——e
`
`ht
`
`* *B ste ee ee ee
`
`.
`
`Ex. 1001, FIG. 1 (Sur-reply, 16)
`Ex. 1001, FIG. 1 (Sur-reply, 16)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`=w
`
`Slide 2
`Slide 2
`
`
`
`etetateta
`
`%.
`
`es
`
`Peetare,
`ay
`
`teens.
`
`etaSateterata:
`
`%
`
`OSaaeae
`saatah,
`nanaaane,
`
`Ex. 1001, FIG. 2C (Sur-reply, 8)
`2C (Sur-
`8)
`reply,
`Ex. 1001, FIG
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 3
`Slide 3
`
`oe
`
`
` SeaewttOX, is
`x
`a
`‘a,%,
`ees eveniry ie
`ant
`rad
`nn
`raat
`yi
`iat
`x,ixis
`
`eittteeeeeeeekeeee
` Jrrentchavamnbnceantinntttvatatatatetatatctatelatatctata!
`
`
`aiataAatUW8aweeeawN88eee
`RRRCRNERROR.
`
`atatetetatatatatetatatenatiadakatataNaNaheaNgNgMghtgtgeee!
`SatatetatstatatatahRoane7
`
`aNglaretwtetalenatetaatatarsrarn
`iieaeaeae
`eiatanrcaraeceneentteeetes
`
`nbaSeats
`
`icestytete
`
`ratchetteeaneee!
`
`
`rarahatatatatattates!
`tetabataPctetetetattatade?,
`08taeaN
`ORRIN
`
`
`avnainaereatenataretes
`
`ieiecncmeteriteratemehemsfe
`aaaadyedecireceneeesncrrer
`fasigateateneaatdeatvals
`eeyiwee
`rtetejieteemteetate
`RRR
`issata!Bae
`
`s f
`
`tan
`
`
`
`retatex
`sehateetatatatatetatatataats’
`
`caweittetasSn688eeeee
`FeaaaaaaaatateetaaaanaaataTeaaaNa
`atoaNeatats
`amtahNate!ra
`Wetrtatatatetventntatatstatatatatstuesceu®
`
`eatatatavanaatanseatanae
`
`sevictoeneneens
`resvenearacenyeueuavariene
`aiabatatctabatabatstatcts
`
`aa
`Netetiavatatatatavacate
`lavenarenciech
`
`edSetatate
`
`aNaatatatntas
`Nactarcaenennens
`senesenaeanees
`masneayevaaan
`
`!
`
`eee
`r
`aa nae:
`ae
`Sea a ate
`anata ty!
`ae
`tanta ty
`tate
`4rte!
`at aa!ara tatate’ teaaate:
`
`
`
`a5a
`roots:
`ateNatataty
`OG
`5
`ane
`neataatatatat abate
`kal
`
`0ite:stata:poate,atatatata yt ata a ta tateage arate tata tetera ater ate,tye,
`rata,
`
`atatelatacat 'saaeae arate ta tate:
`
`
`tetatatetyatatete’
`ate
`araae,
`2,
`
`
`
`aN gt!
`cata tat
`
`2,5"a5
`
`a|e
`
`e]
`
`x$
`
`ene
`
`=va
`
`
`
`
`
`eeyrtstenensearmen
`
`nies
`nt
`
`Sayakatnt
`
` iueerteWa?i
`
`
`ote
`sas
`
`eeeeees
`
`anesenh,rat
`
`5
`rattete
`Satie
`Sanwa
`‘i
`niraieanee
`
`ena
`
`ee stetaterete!
`tata yt tata ta
`attataa
`tat ate’
`har ata ata ae,
`nna a at ata te!
`55
`tate
`x
`eiateteteatet,
`wnt ee eat ta tet ete
`anatataNetaCalatatatatatenala,
`eat
`23
`aa ane aN
`ek Nya tata tats!
`a!
`rata eta Care Neca te ana
`aah at
`ore!
`ann
`aaah
`tePeo
`
`a'*ate’
`5
`
`aantet ary'2)
`rarer!
`o2!oS
`
`aneanata’
`sears!
`
`x,
`
`cata
`atex,
`reer ene5
`a,
`ela
`
`anarai
`rf
`nf tetwereae
`
`
`arnti
`ie
`ae
`Sf, Mo
`rae!
`ns nies “i
`
`Slide 4
`Slide 4
`
`
`
`
`
`
`
`atanenenanaananatataeeveeeekatma
`
`Ex. 1001, FIG. 2D (Sur-reply, 8)
`Ex. 1001, FIG. 2D (Sur-reply, 8)
`
`FIG
`. 2D
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`ee ee
`ae hh
`Piatalatatate
`Netatatitatetatetey
`catesaraverecveeetreeuvieunikrwrbyhet
`‘
`reevesieie
`wat
`AA
`
`yee
`eraser
`
`etaNR
`entries
`x
`m
`niatatnietatnistaatetitenaty
`
`cr ace aca
`aie
`aca
`a
`ee
`AImre
`ra
`retetatates
`ratemrae
`Walataten rf
`ae
`aeeetee)
`ratten
`
`an eaeawe
`ey
`eae,42.
`
`SOGwert‘arm
`
`omneatesi gee
`aeatate
`ageee aee
`sie!
`.
`a
`
`teaaatatatate
`neaaa
`*%
`an
`ana!
`
`rately,atatatatataty
`/ nee
`atata tea tata tate
`ea!
`*
`seaeeata
`reat
`gy
`‘aeee88!
`re
`NetatatateN
`NR
`wataty,rat
`ee
`saatte!
`aesctatacatatate
`hatatabetahehrtel
`
`
`ahatatahex‘‘
`anti
`AttNatasa
`aren
`stata!
`aa
`i
`
`actedronan
`ra
`1oeaee es
`
`&5
`
`3 a
`
`xa
`
`
`
`
`
`Ex. 1008, Doo, FIG. 2 (Sur-reply, 16)
`Ex. 1008, Doo,FIG. 2 (Sur-reply, 16)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 5
`Slide 5
`
`
`
`
`
`Ex. 1008, Doo, FIG. 7 (Sur-reply, 17)
`Ex. 1008, Doo,FIG. 7 (Sur-reply, 17)
`
`FIG. 7
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 6
`Slide 6
`
`
`
`peLeasBeat
`
`“
`
`“a z
`
`Og
`
`eRREWeURONaeMaMNbhBeaeemawe
`
`Ex. 1008, Doo, FIGS. 7, 9 (Sur-reply, 17)
`Ex. 1008, Doo,FIGS. 7, 9 (Sur-reply, 17)
`
`FIG. 7
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 7
`Slide 7
`
`
`
`
`
`Ex. 1008, Doo, FIG. 11 (Sur-reply, 18)
`Ex. 1008, Doo,FIG. 11 (Sur-reply, 18)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 8
`Slide 8
`
`
`
`
`
`Ex. 1008, Doo, FIG. 6 (PO Resp., 32)
`Ex. 1008, Doo, FIG. 6 (PO Resp., 32)
`
`FIG. 6
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 9
`Slide 9
`
`
`
`ing Sources:
`eo&¢&€&&€&&F
`
`
`Radiography
`MRI
`Uiirasqund
`Elastograpny
`Photeacaustic Imaging
`Thermography
`echocardiography
`Funciional Near-Infrared
`Spectrascapy
`
`
`image
`
`Control
`Unit
`
`
`
`Parpnerals:
` » Touchpad or Touch Screen
`
`e Microphone
`
`
`* Eye Tracking
`» Camera
`Ss
`A pn eb rarypcevers
`bee xe
`Fy
`i
`“HE
`
`Accelerometer
`
`
`Gutput Imagery to
`Display on Cornmand
`
`
`
`Ex. 1008, Doo, FIG. 3 (Sur-reply, 16)
`Ex. 1008, Doo,FIG. 3 (Sur-reply, 16)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 10
`Slide 10
`
`
`
`
`
`Figure 8.2: Sketch of a single-screen passive stereo projection system.
`
`Ex. 2023 at 294, Fig. 8.2 (Sur-reply, 21)
`Ex. 2023 at 294,Fig. 8.2 (Sur-reply, 21)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 11
`Slide 11
`
`
`
`
`
`Figure 8.3: Sketch of a super-wide projection system with soft-edge blending.
`
`Ex. 2023 at 298, Fig. 8.3 (Sur-reply, 21)
`Ex. 2023 at 298,Fig. 8.3 (Sur-reply, 21)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 12
`Slide 12
`
`
`
`
`
`Figure 8.4: A single-screen 3D projection system.
`
`Ex. 2023 at 304, Fig. 8.4 (Sur-reply, 21)
`Ex. 2023 at 304,Fig. 8.4 (Sur-reply, 21)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 13
`Slide 13
`
`
`
`
`
`Uadate||Suatpoeined)Viewer regik Views
`
`46i oe
`05 SudeceVeelVenn werkVeni 0-0]
`
`M sctwe [] opine [] treme oct
`
`Ex. 1005, 80/92 (PO Resp., 38)
`Ex. 1005, 80/92 (PO Resp., 38)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 14
`Slide 14
`
`
`
`ARAMEAODPRL Ve 2X DaAame ZLK MEAN AlPalCoae 4
`
`Calib/Registration\/STL Property = | &
`
`esate es |
`er eat
`
`airtual drill trajectory
`
`r ae
`
`3D Titanium miniscrews File Tracking Edit View/Measurement
`
`3D bladder model
`
`Fig. 3. A 3D pelvis model and a bladder imported into the AR-SNS.
`
`Ex. 1009, 3 (PO Resp., 53)
`Ex. 1009, 3 (PO Resp., 53)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 15
`Slide 15
`
`
`
`1. A method for augmenting real-time, non-image actual
`views of a patient with three-dimensional (3D) data, the
`method comprising:
`identifying 3D data for the patient, the 3D data including
`an outer layer ofthe patient and multiple inner layers of
`the patient: and
`displaying, in an augmented reality (AR) headset, one of
`the inner layers of the patient from the 3D data pro-
`jected onto real-time, non-image actual views of the
`outer layer of the patient, the projected inner layer of
`the patient from the 3D data being confined within a
`volume of a virtual 3D shape.
`
`Ex. 1001, 18:54-65 (Claim 1)
`Ex. 1001, 18:54-65 (Claim 1)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 16
`Slide 16
`
`
`
`3. The method of claim 1, wherein:
`the virtual 3D shape is configured to be controlled to
`toggle between displaying and hiding lines of the 5
`virtual 3D shape: and
`the virtual 3D shape is configured to be controlled to
`reposition two-dimensional
`(2D)
`slices and/or 3D
`slices of the projected innerlayer ofthe patient from the
`3D data.
`
`10
`
`Ex. 1001, 19:3-10 (Claim 3)
`Ex. 1001, 19:3-10 (Claim 3)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 17
`Slide 17
`
`
`
`11. A method for augmenting real-time, non-image actual
`views of a patient with three-dimensional (31D) data, the
`method comprising:
`identifying 3D data for the patient, the 3D data including
`an outer layer of the patient and multiple inner layers of
`the patient,
`the multiple inner layers of the patient
`having an original color gradient:
`altering the original color gradient of the multiple inner
`layers to be lighter than the original color gradient in
`order to be better visible when projected onto real-time,
`non-imageactual views ofthe outer layer of the patient;
`and
`displaying, in an augmented reality (AR) headset. one of
`the inner layers of the patient from the 3D data pro-
`jected onto real-time, non-image actual views ofthe
`outer layer of the patient, the projected inner layer of
`the patient from the 3D data being having the altered
`color gradient.
`
`Ex. 1001, 20:11-28 (Claim 11)
`Ex. 1001, 20:11-28 (Claim 11)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 18
`Slide 18
`
`
`
`12. The method asrecited in claim 11, wherein the altered
`color gradient represents a tissue hardnesstissue property of
`the multiple inner layers of the patient.
`13. The method as recited in claim 11. wherein the altered
`color gradient represents a tissue relaxivity tissue property
`of the multiple inner layers of the patient.
`14. The methodas recited in claim 11, wherein the altered
`color gradient represents a tissue echogenicity tissue prop-
`erty of the multiple inner layers of the patient.
`15. The methodas recited in claim 11, wherein the altered
`color gradient represents a tissue enhancement amounttis-
`sue property ofthe multiple inner layers ofthe patient.
`16. The method asrecited in claim 11, wherein the altered
`color gradient represents a tissue enhancement speed tissue
`property of the multiple inner layers of the patient.
`
`17. The method as recited in claim 11, whereinthe altered
`color gradient represents a tissue density tissue property of
`the multiple inner layers of the patient.
`18. The method as recited in claim 11, wherein the altered
`color gradient represents a tissue radioactivity tissue prop-
`erty of the multiple inner layers of the patient.
`19. The method as recited in claim 11, wherein the altered
`color gradient represents a tissue water content tissue prop-
`erty of the multiple inner layers of the patient.
`
`Ex. 1001, 20:29-56 (Claims 12-19)
`Ex. 1001, 20:29-56 (Claims 12-19)
`
`DEMONSTRATIVE EXHIBIT – NOT EVIDENCE
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Slide 19
`Slide 19
`
`