Towards More Capable and Less Invasive
`Robotic Surgery in Orthopaedics
`
`R.V. O'Toole 1111,2, D.A. Simon2,
`B. Jaramaz 1 ,2, O. Ghattas3 , M.K. Blackwe1l2 , L. Kallivokas1 ,
`F. Morgan 2 , C. Visnic1 , A.M. DiGioia 1111,2,3, and T. Kanade2
`
`1 Center for Orthopaedic Research, Shadyside Hospital
`2 The Robotics Institute, Carnegie Mellon University
`3 Department of Civil Engineering, Carnegie Mellon University
`Pittsburgh, PA
`
`Abstract. Current surgical robotic systems in orthopaedics lack realis(cid:173)
`tic pre-operative simulations and utilize invasive methods to register bone
`intra-operatively. A multidisciplinary group of researchers is addressing
`these deficiencies in the context of robotic cementless hip replacement
`surgery. In this paper we outline our current research progress and a
`road-map for the short-term future of our research agenda. This paper
`addresses four components of this effort: (1) realistic anatomical model(cid:173)
`ing, (2) biomechanics-based simulations, (3) surface-based registration,
`and (4) surgical robotics. We are integrating these components with the
`goal of developing more capable and less invasive robotic systems for use
`in orthopaedic surgery.
`
`1
`
`Introduction
`
`The field of orthopaedics presents excellent opportunities for the incorporation
`of robotic and computer-based technologies to improve surgical techniques. Pro(cid:173)
`cedures such as total joint replacement are performed in large numbers and at
`significant cost each year. Over 300,000 total hip and knee replacements occur
`annually in the U.S. alone [3]. The short and long term clinical success of these
`surgical procedures depends strongly on the proper alignment, placement, and
`fit of the implant within the bony structure [9]. The clinical importance of pre(cid:173)
`cision and accuracy, along with the large number and high cost of the surgical
`procedures, indicates that important contributions can be made through the use
`of surgical robots and computer-based pre-operative planning and simulation in
`orthopaedics.
`Figure 1 outlines the four basic components of our research effort: (1) realis(cid:173)
`tic anatomical modeling, (2) biomechanics-based simulations, (3) surface-based
`registration, and (4) surgical robotics. We are integrating these components with
`the goal of developing more capable and less invasive robotic systems for use in
`orthopaedic surgery.
`In biomechanics, our goal is to allow a surgeon to simulate the mechanical
`consequences of a proposed surgery, and to change surgical strategies based
`
`N. Ayache (ed.), Computer Vision, Virtual Reality and Robotics in Medicine
`© Springer-Verlag Berlin Heidelberg 1995
`
`Mako Exhibit 1008 Page 1
`
`

`

`124
`
`Surgical Robotics _ ..
`~usingtramer.$S --
`
`"cr*nll'fI(f" IUr;IcII
`
`'1 ~.'f , '
`
`...,...., ~
`~rab B
`
`Fig. 1. Interaction of Research Topics
`
`upon these consequences. By coupling this realistic simulation and planning
`capability with precise surgical robots, the surgeon can not only plan an "ideal"
`surgery, but also ensure that it is carried out. To execute a surgical plan with
`a robot, the system must possess the ability to register (determine the position
`and orientation) a bone in the clinical environment. Surface-based registration is
`desirable because it does not require fiducials to properly align the pre-operative
`plan with the patient's anatomy. The success of both surgical registration and
`pre-operative simulation is highly dependent upon the realism of geometric and
`physical models. As such, anatomic modeling has also emerged as a distinct
`research area. Our work attempts to join these four seemingly disparate research
`topics into one integrated effort to improve techniques in orthopaedic robotics.
`
`2 Modeling
`
`While algorithms exist for the creation of geometric models from volumetric
`medical data, little work has been published on the validation of these models.
`One reason for this lack of research is that until recently, geometric surface mod(cid:173)
`els have been used primarily for visualization tasks that do not demand highly
`accurate models. With the increased use of surface models for pre-operative
`planning and intra-operative guidance, geometric model accuracy has taken on
`new importance. Physical modeling issues subsume those associated with geo(cid:173)
`metric modeling. In addition to difficulties in generating the geometry of the
`bone and implant, physical models must appropriately represent the underlying
`constitutive laws.
`
`Mako Exhibit 1008 Page 2
`
`

`

`125
`
`2.1 Geometric Model Creation and Validation
`
`The primary geometric model in our work is the polyhedral surface mesh. It is
`used for visualizing 3D surface geometries and for surface-based intra-surgical
`registration. A variety of techniques are available for reconstructing polyhedral
`surface meshes directly from CT or MRI data [6]. We currently generate surface
`meshes using several of these methods [2][8][1] .
`Geometric surface model validation is complicated since errors can be intro(cid:173)
`duced at several stages of model creation: during imaging, segmentation, and
`surface creation. Furthermore, there are multiple measures of error that can be
`used (e.g. Hausdorff distances, difference volumes, surface normal differences).
`Since different tasks place different requirements upon the underlying model,
`validation criteria should be application dependent. For example, a geometric
`model used to specify pre-operative prosthesis placement may have very differ(cid:173)
`ent accuracy requirements than one used for surface-based registration.
`Geiger [2] provides an excellent discussion of surface model validation assum(cid:173)
`ing idealized input data. He uses an analytical model of a torus instead of actual
`CT data to evaluate surfaces generated using several approaches. This work pro(cid:173)
`vides a first step towards the validation of surface models derived from clinically
`realistic CT data. The following list suggests a progression of experiments (in
`order of increasing complexity and realism) that could be performed to reach
`this goal:
`
`no CT data
`1. Analytic model of a solid (cylinder, torus) -
`2. CT images of physical objects that can be analytically modeled (cylinder,
`torus), constructed from bone analog
`3. CT images of anatomical phantoms (femur, pelvis) made from bone analog
`4. CT images of cadaver anatomy (femur, pelvis)
`
`A fundamental issue in model validation is determining the "ground truth" to
`which the reconstructed model will be compared. Ground truth can either be
`determined by accurately sensing an existing object, or by accurately manufac(cid:173)
`turing an object based on an existing model. In Experiments 2 and 3, we can
`use either method for obtaining ground truth, whereas in Experiment 4 we must
`rely on accurate sensing.
`An important measure of clinical realism in the above experiments is the ease
`with which an object of interest can be segmented from surrounding objects. In
`order to study this issue at early stages of experimentation, we are using CT soft
`tissue analogs. By surrounding an object with soft tissue analog during imaging,
`we complicate the segmentation process and make the resulting images closer to
`clinical reality. A second measure of experimental realism is the spatial density
`variation within an object. Density variations, like that in real bone, compli(cid:173)
`cate segmentation by reducing the effectiveness of simple thresholding schemes.
`Cadaver studies allow us to study both of the above effects. In cadaver stud(cid:173)
`ies, bones would be imaged within the cadaver and then dissected out and used
`to build highly accurate ground truth models. We are currently investigating
`Experiment 2 and plan to progress to Experiments 3 and 4.
`
`Mako Exhibit 1008 Page 3
`
`

`

`126
`
`2.2 Physical Model Creation
`
`The geometric modeling of biological surfaces is a fairly well understood process.
`This is not the case for the accurate physical modeling of complex biological sys(cid:173)
`tems such as the bone-implant systems in total joint replacements. Fundamental
`research is still needed to create computer models that realistically mimic the
`biomechanical properties of the bone-implant construct.
`In contrast to bone, physical modeling of an implant is relatively straightfor(cid:173)
`ward since the material properties are well known and a geometric description
`can be obtained from CAD models. The situation is much more difficult for a
`biological material such as bone. To simulate the mechanical consequences of
`a total hip replacement surgery, the bone model must accurately represent the
`geometric complexity and spatial distribution of material properties of a pa(cid:173)
`tient's femur and pelvis. Furthermore, bone exhibits a complex constitutive law
`with significant anisotropy, viscoelasticity, and nonlinearity. Additional nonlin(cid:173)
`earity arises from the contact between the implant and bone, with the interface
`depending on numerous geometric, material, and loading parameters.
`When developing biomechanical models, there are a variety of options which
`progressively add realism to the resulting physical models. The types of models
`we are currently investigating fall into one of the following four broad categories:
`
`1. Idealized (e.g axisymmetric) geometry, and complex material properties -
`both based upon the literature.
`2. Idealized geometry, and complex material properties - both derived directly
`from CT scan data.
`3. Full 3D geometry based on CT data, but with idealized material properties.
`4. Full 3D geometry, and complex material properties -
`both derived directly
`from CT scan data.
`
`The categories range from least (1) to most (4) clinically realistic. There is
`a tradeoff between model complexity and the computational resources required
`to perform a simulation with a given model. The simplest model that accurately
`captures patient-specific mechanics is the best; however, more detailed models
`are still useful to help validate simplified approaches. As such, we are examining
`models that fall into all of the above categories.
`Our initial approach was to develop simplified axisymmetric models to simu(cid:173)
`late the implantation offemoral and acetabular components (Category 1 above).
`These models incorporated idealized material properties with bi-linear elastic
`stress response and contact elements [10] [16]. While the bone geometries and
`material properties were not derived from CT data, these models incorporated
`many of the biomechanical complexities. Next, to help validate the results de(cid:173)
`rived with the first set of models, full 3D irregular mesh FEM models were de(cid:173)
`veloped with geometry based upon CT scan data, but using material properties
`from the literature (Category 3 above).
`Ideally, we wish to create realistic physical models directly from CT scans
`(Category 4). One option for such models is an irregular tetrahedral mesh that is
`grown directly from the CT data. Although potentially yielding a very realistic
`
`Mako Exhibit 1008 Page 4
`
`

`

`127
`
`simulation, the disadvantage of this approach is the difficulty in growing and
`solving large irregular meshes. A second option is the use of a regular grid to
`represent the CT data, while embedding the unstructured implant model within
`the regular bone grid. Material properties can be based on the CT numbers
`using experimentally derived relations. The disadvantage of this approach is
`that it may yield a mesh that is too large unless the resolution is lowered. On
`the other hand, special fast algorithms that exploit the regular structure can be
`derived. Currently we are pursuing both the irregular and regular mesh options.
`
`3 Simulations
`
`Software systems exist that allow a surgeon to plan a surgery on the computer
`before entering the operating room. In orthopaedics, a clinical system now ex(cid:173)
`ists that can carry out a pre-operative plan precisely for the femoral part of
`total hip replacements [15]. In such systems, the surgeon can no longer rely on
`intra-operative feedback to determine the proper placement and fit of a press(cid:173)
`fit implant. Instead, the surgeon must make these decisions by interacting with
`geometric implant models and the CT data to plan the surgery on a computer.
`By adding simulations of the mechanical consequences of a proposed surgery, it
`may be possible to compensate for the lack of intra-operative feedback in the
`current scenario.
`We have simulated the press-fit insertion of cement less acetabular and femoral
`components for total hip replacements. Unlike previous efforts which have as(cid:173)
`sumed an initialline-to-line fit between the implant and bone, our models and
`simulations have included contact coupling to simulate the actual implantation
`process [10] [16] . We argue that since the short term success of the implants
`depends strongly on the post-operative mechanical environment, this type of
`simulation provides valuable information in planning the surgery.
`Currently, we are working towards simulations using more realistic models
`of the bone-implant system, as described above. These simulations will also in(cid:173)
`corporate contact coupling between bone and implant that occurs during the
`forceful insertion of the oversized implant into the bone cavity. The choice of
`model will dictate which solvers we ultimately use. For unstructured meshes we
`are developing preconditioned conjugate gradiant methods. We are also devel(cid:173)
`oping fast multigrid methods that exploit the structure of the regular meshes.
`A surgical simulation must be realistic, but must also run fast enough that the
`information is useful clinically. Currently our analyses require between several
`hours (simple models), to several weeks (full 3D models), using commercial code
`and low-end workstations. By incorporating the latest algorithms and using high(cid:173)
`end workstations, we anticipate performing biomechanical simulations in near
`real-time.
`Once realistic surgical simulations exist, work can progress on parametric
`studies of total joint replacement. Variables include the implant size and place(cid:173)
`ment, and the shape of the bone cavity. The long-term goal of this work is not
`
`Mako Exhibit 1008 Page 5
`
`

`

`128
`
`only to model the consequences of a surgery, but to create a system that can
`suggest optimal parameters to the surgeon.
`
`4 Registration
`
`Intra-surgical registration is the process of establishing a common reference
`frame between pre-surgical data and the corresponding patient anatomy. Once
`a common reference frame is established, pre-surgical data can be used to guide
`robotic tool movements [7], superimpose graphical overlays of internal anatomy
`upon a surgeon's view of the patient [4], position radiosurgical equipment [13],
`or guide a surgeon's tool movements [12]. Recent clinical approaches to intra(cid:173)
`surgical registration assume a known correspondence between points in the two
`data sets being registered [15]. This is usually achieved by attaching fiducials
`to the underlying object, and extracting the locations of these markers in both
`data sets. Unfortunately, attachment of fiducials typically require an additional
`surgical procedure prior to the collection of pre-surgical data. Furthermore, these
`fiducials are invasive and cause added trauma to the patient in sites far from the
`primary surgical field.
`An alternative to fiducial-based registration is to use surfaces that are in(cid:173)
`trinsic to the data itself. If data from the· bounding surface of an object can be
`extracted pre- and intra-surgically, these data sets can be matched to perform
`registration. Several research groups have investigated such surface-based meth(cid:173)
`ods in medical registration [14][4][11][5]. A benefit of these techniques is that
`they do not require the use of costly and invasive external markers. Surface(cid:173)
`based methods, however, place a heavy burden on sensing and modeling tech(cid:173)
`nology since accurate pre- and intra-surgical surface data are needed. This is a
`much more difficult sensing task than acquiring 3D fiducial locations as required
`by previous approaches.
`As we demonstrated in [14], the accuracy resulting from surface-based reg(cid:173)
`istration depends highly on the underlying data. These data include the geo(cid:173)
`metric surface models from pre-operative CT scans, as well as data collected
`intra-surgically using digitizing probes, ultrasound, fluoroscopes or CT. We are
`currently developing methods for planning the acquisition of potentially costly
`intra-surgical registration data using pre-operative geometric models as input.
`The goal of this work is to select data that will result in the best possible regis(cid:173)
`tration accuracy, while minimizing the quantity of data required.
`
`5 Robotics
`
`In total hip replacement surgery, the modeling, simulation, and registration com(cid:173)
`ponents manifest themselves in the robotic milling of a bone cavity. The focus
`of our work is on the milling of the acetabulum (socket in the pelvis) to prepare
`for the implantation of a cement less acetabular component. We have developed
`a robotic testbed to demonstrate registration and cutting strategies. The test(cid:173)
`bed is not designed for clinical use, so issues of sterility, sensor redundancy, and
`
`Mako Exhibit 1008 Page 6
`
`

`

`129
`
`clinical safety have not been addressed. Our system consists of a 5 DOF SCARA
`direct drive manipulator (Adept, San Jose, CA) with an attached 6 axis force(cid:173)
`torque sensor (JR3 Inc, Woodland, CA) and pneumatic milling tool. The overall
`system configuration is conceptually similar to that used by Taylor [15].
`A pre-operative plan is defined using our custom planner and models of the
`implants. There are three reference frames of interest: (1) the CT reference frame
`(where the pre-operative plan is defined), (2) the intra-operative sensor reference
`frame (where bone surface data are defined), and (3) the robot reference frame
`(where the actual robot paths are defined). Intra-operative surface data and
`a surface model from the CT data are used to determine the transformation
`between sensor and CT reference frames (as described in Section 4). The trans(cid:173)
`formation between the sensor and the robot frames is derived by repeatedly
`moving the robot and sensor to the same point in space, recording the locations,
`and then calculating an average transformation matrix.
`The above process yields the pose of the bone and a cutting plan in the
`robot's coordinates. Shapes in the bone can be milled by first making broad cuts
`to remove large amounts of material in a short time, and then returning to finish
`the cavity with finer passes. We have demonstrated (on pelvis bone phantoms)
`milling of hemispherical cavities that match the shape of the implants, but are
`undersized to allow varying amounts of press-fit. These shapes are high precision
`versions of the cavities created with standard hand-held tools, but arbitrary
`shapes could be generated should the biomechanics simulations predict that a
`non-hemispherical cavity is preferable.
`Current work centers on milling less-rigidly fixed bones. With high speed
`tracking it may be possible to do away with rigid fixation and allow some small
`(millimeters) motion of the bone. The robotic system would compensate for this
`motion. The accuracy and bandwidth requirements of such a system are stren(cid:173)
`uous, and much work is needed to evaluate the clinical efficacy of the concept.
`
`6 Conclusion
`
`The value of surgical robots is greatly enhanced by realistic surgical simula(cid:173)
`tions, less invasive registration methods, and accurate modeling. Surgical sim(cid:173)
`ulations will enhance the surgeon's ability to investigate the implications of
`pre-operative plans. Surface-based registration will eliminate invasive and costly
`fiducials which limit the clinical application of orthopaedic robotics. Accurate
`modeling is a prerequisite for realistic simulations and precise intra-operative
`registration. Each of these components are worthy research efforts on their own;
`together, they form a framework for advancing the clinical utility of medical
`robotics.
`
`References
`
`1. R.N. Christiansen and T.W. Sederberg. Conversion of complex contour line def(cid:173)
`initions into polygonal element mosaics. Computer Graphics, 8:658-660, August
`1978.
`
`Mako Exhibit 1008 Page 7
`
`

`

`130
`
`2. B. Geiger. Three-dimensional modeling of human organs and its application to
`diagnosis and surgical planning. PhD thesis, Ecole des Mines de Paris, April 1993.
`3. E.J. Graves. 1992 summary: National hospital discharge survey. In Advance Data
`from Vital Health and Statistics, page 45. National Center for Health Statistics,
`1992.
`4. E Grimson, T. Lozano-Perez, W. Wells, G. Ettinger, S. White, and R. Kikinis.
`Automated registration for enhanced reality visualization in surgery. In AAAI
`1994 Spring Symposium Series, Applications of Computer Vision in Medical Im(cid:173)
`age Processing, pages 26-29. AAAI, March 1994.
`5. H. Jiang, R.A. Robb, and K.S. Holton. A new approach to 3-d registration of
`multimodality medical images by surface matching. In Visualization in Biomedical
`Computing - SPIE Vol 1808, pages 196-213. SPIE, 1992.
`6. A.D. Kalvin. A survey of algorithms for constructing surfaces from 3D volume
`data. IBM Reseach Report RC 17600, IBM, January 1992.
`7. J.T. Lea, D. Watkins, A. Mills, M.A. Preshkin, T.C. Kienzle, and S.D Stulberg.
`Registration and immobilization for robot-assisted orthopaedic surgery. In Pro(cid:173)
`ceedings of the First International Symposium on Medical Robotics and Computer
`Assisted Surgery, pages 63-68, Pittsburgh, PA, September 1994.
`8. W.E. Lorensen and H.E. Cline. Marching cubes: A high resolution 3D surface
`construction algorithm. Computer Graphics, 21:163-169, July 1987.
`9. V.C. Mow and W.C. Hayes. Basic Orthopaedic Biomechanics. Raven Press, 1991.
`10. R.V. O'Toole, B. Jaramaz, A.M. DiGioia, C.D. Visnic, and R.H. Reid. Biomechan(cid:173)
`ics for pre-operative planning and surgical simulation in orthopaedics. Computers
`in Biology and Medicine, 1994.
`11. o. Peria, A. Francois-Joubert, S. Lavallee, G. Champleboux, P. Cinquin, and
`S. Grand. Accurate registration of SPECT and MR brain images of patients suf(cid:173)
`fering from epilepsy or tumor. In Proceedings of the First International Symposium
`on Medical Robotics and Computer Assisted Surgery, pages 58-62, Pittsburgh, PA,
`September 1994.
`12. K. Radermacher, H.W. Staudte, and G. Rau. Computer assisted orthopedic
`surgery by means of individual templates - aspects and analysis of potential appli(cid:173)
`cations. In Proceedings of the First International Symposium on Medical Robotics
`and Computer Assisted Surgery, pages 42-48, Pittsburgh, PA, September 1994.
`13. A. Schweikard, R. Tombropoulos, J.R. Adler, and J. Latombe. Planning for image(cid:173)
`In AAAI 1994 Spring Symposium Series, Applications of
`guided radiosurgery.
`Computer Vision in Medical Image Processing, pages 96-101. AAAI, March 1994.
`14. D.A. Simon, M. Hebert, and T. Kanade. Techniques for fast and accurate intra(cid:173)
`surgical registration. . In Proceedings of the First International Symposium on
`Medical Robotics and Computer Assisted Surgery, pages 90-97, Pittsburgh, PA,
`September 1994.
`15. R.H. Taylor, B.D. Mittelstadt, H.A. Paul, W. Hanson, P. Kazanzides, J.F. Zuhars,
`B Williamson, B.L. Musits, E. Glassman, and W.L. Bargar. An image-directed
`robotic system for precise orthopaedic surgery. IEEE Transactions on Robotics
`and Automation, 10(3):261-275, June 1994.
`16. C.D. Visnic, R.H. Reid, A.M. DiGioia, B. Jaramaz, and O. Ghattas. Finite element
`pre-operative simulation of cementless hip replacement. In Proceedings of the 1994
`Winter Simulation Conference, 1994.
`
`Mako Exhibit 1008 Page 8
`
`