Medical Image Analysis (1998) volume 2, number 1, pp 1–36
`c(cid:1) Oxford University Press
`
`A survey of medical image registration
`
`J. B. Antoine Maintz
`
`∗
`
`and Max A. Viergever
`
`Image Sciences Institute, Utrecht University Hospital, Utrecht, The Netherlands
`
`Abstract
`The purpose of this paper is to present a survey of recent (published in 1993 or later) publications
`concerning medical image registration techniques. These publications will be classified according
`to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic
`methods. The statistics of the classification show definite trends in the evolving registration
`techniques, which will be discussed. At this moment, the bulk of interesting intrinsic methods
`is based on either segmented points or surfaces, or on techniques endeavouring to use the full
`information content of the images involved.
`
`Keywords: matching, registration
`
`Received May 25, 1997; revised October 10, 1997; accepted October 16, 1997
`
`1.
`
`INTRODUCTION
`
`Within the current clinical setting, medical imaging is a vital
`component of a large number of applications. Such appli-
`cations occur throughout the clinical track of events; not
`only within diagnostic settings, but prominently in the areas
`of planning, carrying out and evaluating surgical and radio-
`therapeutical procedures. The imaging modalities employed
`can be divided into two global categories: anatomical and
`functional. Anatomical modalities, i.e. depicting primarily
`morphology, include X-ray, CT (computed tomographya),
`MRI (magnetic resonance imagingb), US (ultrasoundc), portal
`images and video sequences obtained by various catheter
`‘scopes’, e.g. by laparoscopy or laryngoscopy. Some promi-
`nent derivative techniques are so detached from the original
`modalities that they appear under a separate name, e.g. MRA
`(magnetic resonance angiography), DSA (digital subtraction
`angiography, derived from X-ray), CTA (computed tomogra-
`phy angiography) and Doppler (derived from US, referring
`to the Doppler effect measured). Functional modalities, i.e.
`depicting primarily information on the metabolism of the
`underlying anatomy, include (planar) scintigraphy, SPECT
`
`∗
`Corresponding author
`(e-mail: Twan.Maintz@cs.ruu.nl)
`aAlso formerly and popularly CAT, computed axial tomography.
`bAlso referred to as NMR, nuclear magnetic resonance, spin imaging and
`various other names.
`cAlso echo(graphy).
`
`(single-photon emission computed tomographyd), PET
`(positron emission tomographye), which together make
`up the nuclear medicine imaging modalities and fMRI
`(functional MRI). With a little imagination, spatially sparse
`techniques like, EEG (electro-encephalography) and MEG
`(magneto-encephalography) can also be called functional
`imaging techniques. Many more functional modalities can
`be named, but these are either little used, or still in the
`pre-clinical research stage, e.g. pMRI (perfusion MRI), fCT
`(functional CT), EIT (electrical impedance tomography) and
`MRE (magnetic resonance elastography).
`Since information gained from two images acquired in the
`clinical track of events is usually of a complementary nature,
`proper integration of useful data obtained from the separate
`images is often desired. A first step in this integration process
`is to bring the modalities involved into spatial alignment, a
`procedure referred to as registration. After registration, a
`fusion step is required for the integrated display of the data
`involved. Unfortunately, the terms registration and fusion, as
`well as matching, integration, correlation and others, appear
`polysemously in the literature, either referring to a single step
`or to the whole of the modality integration process. In this
`paper, only the definitions of registration and fusion as defined
`above will be used.
`An eminent example of the use of registering different
`modalities can be found in the area of epilepsy surgery.
`
`dAlso SPET, single-photon emission tomography.
`eSPECT and PET together are sometimes referred to as ECAT (emission
`computerized axial tomography).
`
`-1-
`
`Smith & Nephew Ex. 1015
`IPR Petition - USP 7,534,263
`
`

`
`2
`
`J. B. A. Maintz and M. A. Viergever
`
`Patients may undergo various MR, CT and DSA studies for
`anatomical reference; ictal and interictal (during and between
`seizures) SPECT studies; MEG and extra and/or intra-cranial
`(subdural or depth) EEG, as well as 18FDG and/or 11C-
`Flumazenil PET studies. Registration of the images from
`practically any combination will benefit the surgeon. A sec-
`ond example concerns radiotherapy treatment, where both CT
`and MR can be employed. The former is needed to compute
`the radiation dose accurately, while the latter is usually better
`suited for delineation of tumour tissue.
`Besides multimodality registration, important application
`areas exist
`in monomodality registration.
`Examples
`include treatment verification by comparison of pre- and
`post-intervention images, comparison of ictal and inter-ictal
`SPECT images, and growth monitoring, e.g. using time series
`of MR scans on tumours, or X-ray time series on specific
`bones. Because of the high degree of similarity between
`these images, solving the registration is usually significantly
`easier than in the multimodality applications.
`This paper aims to provide a survey of recent literature
`concerning medical image registration. Because of the sheer
`volume of available papers, the material presented is by ne-
`cessity heavily condensed, and except for a few interesting
`and ‘classic’ cases no papers written before 1993 are referred
`to. Concerning publications pre-dating 1993, we refer the
`reader to review papers such as van den Elsen et al. (1993) and
`Maurer and Fitzpatrick (1993). No complete review papers
`of a later date exist to our knowledge, except for the field of
`computer-aided surgery (Lavall´ee, 1996). To narrow the field
`of available publications in such a way does not, however,
`impede us in reaching our primary goal, which is to paint a
`comprehensive picture of current medical image registration
`methods.
`
`2. CLASSIFICATION OF REGISTRATION
`METHODS
`
`The classification of registration methods used in this paper
`is based on the criteria formulated by van den Elsen et al.
`(1993). A considerably augmented and detailed version is
`presented. Nine basic criteria are used, each of which is
`again subdivided into one or two levels. The nine criteria and
`primary subdivisions are:
`
`I. Dimensionality
`
`II. Nature of registration basis
`
`a. Extrinsic
`b. Intrinsic
`c. Non-image based
`
`III. Nature of transformation
`
`a. Rigid
`
`b. Affine
`
`c. Projective
`
`d. Curved
`
`IV. Domain of transformation
`
`V. Interaction
`
`VI. Optimization procedure
`
`VII. Modalities involved
`
`a. Monomodal
`
`b. Multimodal
`
`c. Modality to model
`
`d. Patient to modality
`
`VIII. Subject
`
`a. Intrasubject
`
`b. Intersubject
`
`c. Atlas
`
`IX. Object
`
`A registration procedure can always be decomposed into three
`major parts: the problem statement, the registration paradigm
`and the optimization procedure. The problem statement and
`the choice of paradigm and optimization procedure together
`provide a unique classification according to the nine criteria
`mentioned. Although parts and criteria are heavily inter-
`twined and have many cross-influences, it can be said that
`the problem statement determines the classification according
`to criteria VII, VIII and IX, and has a direct bearing on the
`criteria I and III. The paradigm influences the criteria II, III,
`IV and V most directly, while the optimization procedure
`influences criterion V and controls VI. It is often helpful to
`remember that the three pillars are independent, since many
`papers do not describe them as such, often presenting the
`problem statement, paradigm and optimization procedure in
`a compounded way.
`In the following sections, we will discuss the separate
`criteria in more detail.
`
`-2-
`
`

`
`A survey of medical image registration
`
`3
`
`3. DIMENSIONALITY
`
`I. Dimensionality
`
`a. Spatial dimensions only:
`
`1. 2-D–2-D
`2. 2-D–3-D
`3. 3-D–3-D
`
`b. Time series (more than two images), with spatial dimen-
`sions:
`
`1. 2-D–2-D
`2. 2-D–3-D
`3. 3-D–3-D
`
`3.1. Spatial registration methods
`The main division here is whether all dimensions are spatial,
`or that time is an added dimension. In either case, the prob-
`lem can be further categorized depending on the number of
`spatial dimensions involved. Most current papers focus on
`the 3-D–3-D registration of two images (no time involved).
`3-D–3-D registration normally applies to the registration of
`two tomographic datasets, or the registration of a single to-
`mographic image to any spatially defined information, e.g. a
`vector obtained from EEG data. 2-D–2-D registration may
`apply to separate slices from tomographic data, or intrinsically
`2-D images such as portal images. Compared with 3-D–3-D
`registration, 2-D–2-D registration is far less complex both
`where the number of parameters and the volume of the data are
`concerned, so obtaining a registration is in many cases easier
`and faster than in the 3-D–3-D case. We reserve 2-D–3-D reg-
`istration for the direct alignment of spatial data to projective
`data (e.g. a pre-operative CT image to an intra-operative X-ray
`image), or the alignment of a single tomographic slice to spa-
`tial data. Some applications register multiple 2-D projection
`images to a 3-D image, but since a usual preprocessing step is
`to construct a 3-D image from the 2-D projection images, such
`applications are best categorized as 3-D–3-D applications.
`Since most 2-D–3-D applications concern intra-operative pro-
`cedures within the operating theatre, they are heavily time-
`constrained and consequently have a strong focus on speed
`issues connected with the computation of the paradigm and
`the optimization. The majority of applications outside the
`operating theatre and radiotherapy setting allow for off-line
`registration, so speed issues need only be addressed as con-
`strained by clinical routine.
`
`3.2. Registration of time series
`Time series of images are acquired for various reasons, such
`as monitoring of bone growth in children (long time interval)
`
`monitoring of tumour growth (long to medium interval), post-
`operative monitoring of healing (short interval), observing the
`passing of an injected bolus through a vessel tree (ultra-short
`interval) or evaluation of drug effects (various time intervals),
`e.g. the evaluation of multiple sclerosis drugs using MR. If
`two time series need to be compared, registration will be
`necessary except in some instances of ultra-short time series,
`where the patient does not leave the scanner between the
`acquisition of two images. The same observations as for
`spatial-only registrations apply.
`
`4. NATURE OF REGISTRATION BASIS
`
`II. Nature of registration basis
`
`a. Extrinsic
`
`1. Invasive
`A. Stereotactic frame
`B. Fiducials (screw markers)
`2. Non-invasive
`A. Mould, frame, dental adapter etc.
`B. Fiducials (skin markers)
`
`b. Intrinsic
`
`1. Landmark based
`A. Anatomical
`B. Geometrical
`2. Segmentation based
`A. Rigid models (points, curves, surfaces)
`B. Deformable models (snakes, nets)
`3. Voxel property based
`A. Reduction to scalars/vectors (moments, prin-
`cipal axes)
`B. Using full image content
`
`c. Non-image based (calibrated coordinate systems)
`
`4.1. Extrinsic registration methods
`Image-based registration can be divided into extrinsic, i.e.
`based on foreign objects introduced into the imaged space,
`and intrinsic methods, i.e. based on the image information as
`generated by the patient.
`Extrinsic methods rely on artificial objects attached to the
`patient, objects which are designed to be well visible and
`accurately detectable in all of the pertinent modalities. As
`such, the registration of the acquired images is comparatively
`easy, fast, can usually be automated, and, since the registration
`
`-3-
`
`

`
`4
`
`J. B. A. Maintz and M. A. Viergever
`
`parameters can often be computed explicitly, has no need for
`complex optimization algorithms. The main drawbacks of
`extrinsic registration are the prospective character, i.e. provi-
`sions must be made in the pre-acquisition phase, and the often
`invasive character of the marker objects. Non-invasive mark-
`ers can be used, but as a rule are less accurate. A commonly
`used fiducial object is a stereotactic frame (Lunsford, 1988;
`Vandermeulen, 1991; Lemieux and Jagoe, 1994; Lemieux
`et al., 1994b; Strother et al., 1994; Hemler et al., 1995c;
`Vandermeulen et al., 1995; Peters et al., 1996) screwed rigidly
`to the patient’s outer skull table, a device which until recently
`provided the ‘gold standard’ for registration accuracy. Such
`frames are used for localization and guidance purposes in neu-
`rosurgery. Since neurosurgery is one of the main application
`areas of registration, the use of a stereotactic frame in the
`registration task does not add an additional invasive strain to
`the patient. However, the mounting of a frame for the sole
`purpose of registration is not permissible. Sometimes other
`invasive objects are used, such as screw-mounted markers
`(Gall and Verhey, 1993; Leung Lam et al., 1993; Maurer
`et al., 1993, 1994, 1995a, b; S. Li et al., 1994; Simon et al.,
`1995b; Ellis et al., 1996), but usually non-invasive marking
`devices are reverted to. Most popular amongst these are
`markers glued to the skin (Evans et al., 1991; Maguire et al.,
`1991; Malison et al., 1993; Wahl et al., 1993; Bucholz et al.,
`1994; S. Li et al., 1994; Wang et al., 1994b, 1995; Edwards
`et al., 1995a, b; Leslie et al., 1995; Stapleton et al., 1995;
`Fuchs et al., 1996), but larger devices that can be fitted snugly
`to the patient, like individualized foam moulds, head holder
`frames and dental adapters have also been used, although they
`are little reported on in recent literature (Greitz et al., 1980;
`Laitinen et al., 1985; Schad et al., 1987; Evans et al., 1989,
`1991; Hawkes et al., 1992).
`
`include
`Since extrinsic methods by definition cannot
`patient-related image information,
`the nature of
`the
`registration transformation is often restricted to being rigid
`(translations and rotations only).
`Furthermore,
`if they
`are to be used with images of low (spatial) information
`content such as EEG or MEG, a calibrated video image
`or spatial measurements are often necessary to provide
`spatial information as a basis for the registration. Because
`of the rigid-transformation constraint and various practical
`considerations, use of extrinsic 3-D–3-D methods is largely
`limited to brain and orthopedic (Simon et al., 1995b; Ellis
`et al., 1996) imaging, although markers can often be used
`in projective (2-D) imaging of any body area. Non-rigid
`transformations can in some cases be obtained using markers,
`e.g. in studies of animal heart motion, where markers can be
`implanted into the cardiac wall.
`
`Intrinsic registration methods
`4.2.
`Intrinsic methods rely on patient-generated image content
`only. Registration can be based on a limited set of identified
`salient points (landmarks), on the alignment of segmented bi-
`nary structures (segmentation based), most commonly object
`surfaces, or directly onto measures computed from the image
`grey values (voxel property based).
`
`4.2.1. Landmark-based registration methods
`Landmarks can be anatomical,
`i.e. salient and accurately
`locatable points of the morphology of the visible anatomy,
`usually identified interactively by the user (Evans et al., 1989,
`1991; Hill et al., 1991a, b, 1993b; Maguire et al., 1991; Zubal
`et al., 1991, 1995; Henri et al., 1992; Bijhold, 1993; Ding
`et al., 1993; Fright and Linney, 1993; Gluhchev and Shalev,
`1993; Morris et al., 1993; Neelin et al., 1993; Wahl et al.,
`1993; Ge et al., 1994, 1995; Harmon et al., 1994; Moseley
`and Munro, 1994; Pietrzyk et al., 1994; Strother et al., 1994;
`Edwards et al., 1995a, b; Hamadeh et al., 1995b, c; Leslie
`et al., 1995; McParland and Kumaradas, 1995; Meyer et al.,
`1995; Savi et al., 1995; Soltys et al., 1995; Stapleton et al.,
`1995; Vandermeulen et al., 1995; Christensen et al., 1996;
`Erbe et al., 1996; Evans et al., 1996a, b; Fang et al., 1996;
`Peters et al., 1996; Rubinstein et al., 1996), or geometrical,
`i.e. points at the locus of the optimum of some geometric
`property, e.g. local curvature extrema, corners etc., generally
`localized in an automatic fashion (He et al., 1991; Fontana
`et al., 1993; Ault and Siegel, 1994, 1995; Eilertsen et al.,
`1994; Thirion, 1994, 1996a; Uenohara and Kanade, 1995;
`Amit and Kong, 1996; Chua and Jarvis, 1996). Technically,
`the identification of landmark points is a segmentation pro-
`cedure, but we reserve the classification segmentation-based
`registration for methods relating to segmentation of structures
`of higher order, i.e. curves, surfaces and volumes. Landmark-
`based registration is versatile in the sense that it, at least in
`theory, can be applied to any image, no matter what the object
`or subject is. Landmark-based methods are mostly used to
`find rigid or affine transformations. If the sets of points are
`large enough, they can theoretically be used for more complex
`transformations. Anatomical landmarks are also often used
`in combination with an entirely different registration basis
`(Evans et al., 1989, 1991, 1996b; Wahl et al., 1993; Moseley
`and Munro, 1994; Hamadeh et al., 1995c; McParland and Ku-
`maradas, 1995; Zubal et al., 1995; Christensen et al., 1996):
`methods that rely on optimization of a parameter space that is
`not (nearly) convex are prone to sometimes getting stuck in
`local optima, possibly resulting in a large mismatch. By con-
`straining the search space according to anatomical landmarks,
`such mismatches are unlikely to occur. Moreover, the search
`procedure can be sped up considerably. A drawback is that
`
`-4-
`
`

`
`A survey of medical image registration
`
`5
`
`user interaction is usually required for the identification of the
`landmarks.
`In landmark-based registration, the set of identified points
`is sparse compared with the original image content, which
`makes for relatively fast optimization procedures. Such al-
`gorithms optimize measures such as the average distance (L2
`norm) between each landmark and its closest counterpart (the
`Procrustean metric), or iterated minimal landmark distances.
`For the optimization of the latter measure the iterative closest
`point (ICP) algorithm (Besl and McKay, 1992) and derived
`methods are popular. Its popularity can be accredited to its
`versatility (it can be used for point sets, and implicitly and ex-
`plicitly defined curves, surfaces and volumes), computational
`speed and ease of implementation. The Procrustean optimum
`can sometimes be computed, e.g. using Arun et al.’s method
`(1987), but is more commonly searched for using general op-
`timization techniques. Such techniques are referred to in Sec-
`tion 7. Yet other methods perform landmark registration by
`testing a number of likely transformation hypotheses, which
`can, for example, be formulated by aligning three randomly
`picked points from each point set involved. Common opti-
`mization methods here are quasi-exhaustive searches, graph
`matching and dynamic programming approaches.
`
`4.2.2. Segmentation-based registration methods
`Segmentation-based registration methods can be rigid-model
`based (Chen et al., 1987; Levin et al., 1988; Gu´eziec and Ay-
`ache, 1992; Jiang et al., 1992b; Ayache et al., 1993; Collignon
`et al., 1993a, 1994; Fritsch, 1993; Gee et al., 1993, 1994,
`1995a, b; Gilhuijs and van Herk, 1993; Hill et al., 1993a;
`Kittler et al., 1993; Miller et al., 1993; Rusinek et al., 1993;
`Tsui et al., 1993; Turkington et al., 1993, 1995; Zhao et al.,
`1993; Ettinger et al., 1994a, b, 1996; Feldmar and Ayache,
`1994, 1996; Fritsch et al., 1994a, b; Grimson et al., 1994a,
`b, c, 1995, 1996; Hata et al., 1994; Hemler et al., 1994a, b,
`1995a, b, c, 1996; Henderson et al., 1994; Huang and Cohen,
`1994; Kanatani, 1994; Kooy et al., 1994; Krattenthaler et al.,
`1994; Lavall´ee et al., 1994, 1996a, b; Liu et al., 1994; Maurer
`et al., 1994; Mendonc¸a et al., 1994; P´eria et al., 1994; Petti
`et al., 1994; Philips, 1994; Serra and Berthod, 1994, 1995;
`Simon et al., 1994, 1995a, b; Scott et al., 1994, 1995; Staib
`and Xianzhang, 1994; Strother et al., 1994; Szelisky and
`Lavall´ee, 1994a, b, 1996; Taneja et al., 1994; van Herk and
`Kooy, 1994; Wang et al., 1994a, 1996c; Zuk et al., 1994;
`Andersson, 1995; Andersson et al., 1995; Ardekani et al.,
`1995; Betting and Feldmar, 1995; Betting et al., 1995; Burel
`et al., 1995; Christmas et al., 1995; Feldmar et al., 1995;
`Hamadeh et al., 1995a, b, c; Henri et al., 1995; Kruggel and
`Bartenstein, 1995; Lavall´ee and Szeliski, 1995; Leszczynski
`et al., 1995; Maurer et al., 1995a; Pallotta et al., 1995; Pajdla
`and van Gool, 1995; Pellot et al., 1995; Pennec and Thirion,
`
`1995; Rizzo et al., 1995; Ryan et al., 1995; Sull and Ahuja,
`1995; Troccaz et al., 1995; Vandermeulen et al., 1995; Vassal
`et al., 1995; Xiao and Jackson, 1995; Zubal et al., 1995;
`Declerc et al., 1996; Evans et al., 1996b; Ge et al., 1996;
`Gee and Haynor, 1996; Gilhuijs et al., 1996; Goris et al.,
`1996; Jain et al., 1996; Qian et al., 1996), where anatomi-
`cally the same structures (mostly surfaces) are extracted from
`both images to be registered, and used as sole input for the
`alignment procedurea. They can also be deformable model
`based (Bajcsy et al., 1983; Gu´eziec, 1993; Taubin, 1993;
`Davatzikos and Prince, 1994; MacDonald et al., 1994; Sandor
`and Leahy, 1994; Tom et al., 1994; Bainville et al., 1995; Bro-
`Nielsen, 1995; Mangin et al., 1995; Sandor and Leahy, 1995;
`Thirion, 1995, 1996b; Cuisenaire et al., 1996; Davatzikos,
`1996; Davatzikos et al., 1996; McInerney and Terzopoulos,
`1996), where an extracted structure (also mostly surfaces and
`curves) from one image is elastically deformed to fit the sec-
`ond image. The rigid-model-based approaches are probably
`the most popular methods currently in clinical use. Their
`popularity relative to other approaches is probably for a large
`part due to the success of the ‘head-hat’ method as introduced
`by Pelizzari and co-workers (Chen et al., 1987; Levin et al.,
`1988; Pelizzari et al., 1989; Chen and Pelizzari, 1989), which
`relies on the segmentation of the skin surface from CT, MR
`and PET images of the head. Since the segmentation task
`is fairly easy to perform and the computational complex-
`ity is relatively low, the method has remained popular and
`many follow-up papers aimed at automating the segmentation
`step, improving the optimization performance or otherwise
`extending the method have been published. Another rea-
`son for its popularity is the fast Chamfer-matching technique
`for alignment of binary structures by means of a distance
`transform, introduced by Borgefors (1988). A drawback of
`segmentation-based methods is that the registration accuracy
`is limited to the accuracy of the segmentation step. In theory,
`segmentation-based registration is applicable to images of
`many areas of the body, yet in practice the application areas
`have largely been limited to neuroimaging and orthopedic
`imaging. The methods are commonly automated except for
`the segmentation step, which is performed semi-automatically
`most of the time.
`With deformable models, however, the optimization crite-
`rion is different: it is always locally defined and computed,
`and the deformation is constrained by elastic modelling con-
`straints (by a regularization term) imposed onto the segmented
`curve or surface. Deformable curves appear in the literature
`as snakes or active contours; 3-D deformable models are
`sometimes referred to as nets. To ease the physical modelling,
`
`aNote that in this case the term rigid applies to the segmentation procedure
`only. This does not necessarily imply that the registration transformation is
`also rigid.
`
`-5-
`
`

`
`6
`
`J. B. A. Maintz and M. A. Viergever
`
`the data structure of deformable models is not commonly
`a point set.
`Instead, it is often represented using localized
`functions such as splines. The deformation process is always
`done iteratively, small deformations at a time. Deformable
`model approaches are based on a template model that needs to
`be defined in one image. After this, two types of approaches
`can be identified: the template is either deformed to match
`a segmented structure in the second image (Taubin, 1993;
`Davatzikos and Prince, 1994; Sandor and Leahy, 1994, 1995;
`Tom et al., 1994; Bainville et al., 1995; Bro-Nielsen, 1995;
`Thirion, 1995, 1996b; Cuisenaire et al., 1996; Davatzikos,
`1996; Davatzikos et al., 1996), or the second image is used
`unsegmented (Bajcsy et al., 1983; Gu´eziec, 1993; MacDonald
`et al., 1994). In the latter case, the fit criterion of the template
`can be, for example, to lie on an edge region in the second
`image.
`In contrast to registration based on extracted rigid
`models, which is mainly suited for intrasubject registration,
`deformable models are in theory very well suited for inter-
`subject and atlasa registration, as well as for registration of a
`template obtained from a patient to a mathematically defined
`general model of the templated anatomy. A drawback of
`deformable models is that they often need a good initial posi-
`tion in order to converge properly, which is generally realized
`by (rigid) pre-registration of the images involved. Another
`disadvantage is that the local deformation of the template can
`be unpredictably erratic if the target structure differs suffi-
`ciently from the template structure. A typical error is that the
`deformable model matches the anatomy perfectly, except in
`the one interesting image area where a large tumour growth
`has appeared. In intrasubject matching of, for example, the
`cortical surface, this may result in entire gyri being missed or
`misplaced. The solution may lie in locally adapting the elas-
`ticity constraints (Bro-Nielsen, 1995; Little et al., 1996). De-
`formable models are best suited to finding local curved trans-
`formations between images, and less so for finding (global)
`rigid or affine transformations. They can be used on almost
`any anatomical area or modality, and are usually automated
`except for the segmentation step. In the current literature the
`major applications are registration of bone contours obtained
`from CTb, and cortical registration of MR images (Bajcsy
`et al., 1983; Davatzikos and Prince, 1994; MacDonald et al.,
`1994; Sandor and Leahy, 1994, 1995; Thirion, 1995, 1996b;
`Cuisenaire et al., 1996; Davatzikos, 1996; Davatzikos et al.,
`1996). Deformable models are ideally suited for the former
`application, as the bone contours are easily extracted from the
`CT, and there are often no other contours near that disturb
`the proper deformation convergence. The latter application is
`important because if a cortical registration between two brains
`
`aIntersubject and atlas registration is covered in Section 9.
`bFor example see Fang et al. (1996).
`
`can be found, a segmentation of one cortex can be instantly
`transfered to the other.
`
`4.2.3. Voxel property-based registration methods
`The voxel-property-based registration methods stand apart
`from the other intrinsic methodsc by the fact that they op-
`erate directly on the image grey values, without prior data
`reduction by the user or segmentation. There are two distinct
`approaches: the first is immediately to reduce the image grey
`value content to a representative set of scalars and orienta-
`tions, the second is to use the full image content throughout
`the registration process.
`Principal-axes and moments-based methods are the prime
`examples of reductive registration methods. Within these
`methods the image centre of gravity and its principal orien-
`tations (principal axes) are computed from the image zeroth-
`and first-order moments. Registration is then performed by
`aligning the centre of gravity and the principal orientations
`(Alpert et al., 1990; Banerjee and Toga, 1994; Ettinger et al.,
`1994a, b; Pav´ıa et al., 1994; Wang and Fallone, 1994; Slomka
`et al., 1995; Dong and Boyer, 1996; Wang et al., 1996a).
`Sometimes, higher-order moments are also computed and
`used in the process. The result is usually not very accu-
`rate, and the method is not equipped to handle differences
`in scanned volume well, although some authors attempt to
`remedy this latter problem. Despite its drawbacks, principal-
`axes methods are widely used in registration problems that
`do not require high accuracy, because of the automatic and
`very fast nature of its use, and the easy implementation. The
`method is used primarily in the re-alignment of scintigraphic
`cardiac studies (even intersubject) (Slomka et al., 1995), and
`as a coarse pre-registration in various other registration areas
`(Banerjee and Toga, 1994; Ettinger et al., 1994a, b; Pav´ıa
`et al., 1994; Slomka et al., 1995; Dong and Boyer, 1996).
`Moment-based methods also appear as hybridly classified
`registration methods that use segmented or binarized image
`data for input.
`In many applications, pre-segmentation is
`mandatory in order for moment-based methods to produce
`acceptable results.
`Voxel property-based methods using the full image content
`are the most interesting methods of current research. Theoret-
`ically, these are the most flexible of the registration methods,
`since, unlike all other methods mentioned, they do not start
`by reducing the grey-level image to relatively sparse extracted
`information, but use all of the available information through-
`out the registration process. Although voxel-property-based
`methods have been around for a long time, their use in ex-
`tensive 3-D–3-D clinical applications has been limited by
`the considerable computational costs. An increasing clinical
`call for accurate and retrospective registration, along with
`
`cExcept some instances of geometric landmark registration.
`
`-6-
`
`

`
`
`
`A survey of medical image registration
`
`7
`
`the development of ever-faster computers with large internal
`memories, have enabled full-image-content methods to be
`used in clinical practice, although they have not yet been intro-
`duced in time-constrained applications such as intra-operative
`2-D–3-D registration. Methods using the full image content
`can be applied in almost any medical application area, using
`any type of transformation. However, such a statement is
`largely merited by the fact that ‘full-image-content based’ is a
`very gross classifier. The real versatility of a method can only
`be established on an individual basis. Many recent papers
`report on applications that are tailored for rigid or affine global
`registration of 3-D images of the head. Nearly all presented
`methods are automatic, although hybrid approaches (e.g. in-
`cluding an interactive landmark-based pre-registration) are
`being suggested. While the methods theoretically support
`curved transformations and intersubject registration, we have
`encountered only few publications on this.
`As concerns full-image-content-based voxel property reg-
`istration methods,
`the literature reports on the following
`paradigms being used (∗ denotes methods most likely to be
`restricted to monomodal applications):
`• Cross-correlation (of original images or extracted feature
`images) (Junck et al., 1990; Bacharach et al., 1993;
`Bettinardi et al., 1993; Hill, 1993; Hua and Fram, 1993;
`M¨unch and R¨uegsegger, 1993; Radcliffe et al., 1993,
`1994; van den Elsen and Viergever, 1993; Banerjee and
`Toga, 1994; Collins et al., 1994a, b, 1995; Lemieux
`et al., 1994a; Maintz et al., 1994, 1995, 1996b, c; Mose-
`ley and Munro, 1994; Pav´ıa et al., 1994; van den Elsen,
`1994; van den Elsen et al., 1994, 1995; Andersson, 1995;
`Andersson et al., 1995; Cideciyan, 1995; Hemler et al.,
`1995c; McParland and Kumaradas, 1995; Perault et al.,
`1995; Studholme et al., 1995a, b; Dong and Boyer, 1996;
`Gottesfeld Brown and Boult, 1996; Hristov and Fallone,
`1996; Lehmann et al., 1996).
`• Fourier-domain-based cross-correlation, and phase-only
`correlation (de Castro and Morandi, 1987; Leclerc and
`Benchimol, 1987; Chen, 1993; Lehmann et al., 1996;
`Shekarforoush et al., 1996; Wang et al., 1996b).
`• Minimization of variance of intensity ratios (Hill, 1993;
`Hill et al., 1993a; Woods et al., 1993; Ardekani et al.,
`1994; Studholme et al., 1995a, b; Zuo et al., 1996).
`• Minimization of variance of grey values within segments
`(Cox and de Jager, 1994; Ardekani et al., 1995).
`∗ Minimization of the histogram entropy of difference im-
`ages (Buzug and Weese, 1996).
`• Histogram clustering and minimization of histogram dis-
`persion (Hill, 1993; Hill and Hawkes, 1994; Hill et al.,
`1994; Collignon et al., 1995b; Hawkes et al., 1995;
`Studholme et al., 1995a, b; Lehmann et al., 1996).
`
`• Maximization of mutual information (relative entropy)
`of the histogram (Collignon et al., 1995a; Viola, 1995;
`Viola and Wells III, 1995; Wells III et al., 1995, 1996;
`Maes et al., 1996; Pokrandt, 1996; Studholme et al.,
`1996; Viola et al., 1996).
`∗ Maximization of zero crossings in difference images
`[stochastic sign change (SSC) and deterministic si