Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 9)

Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 9) 9 LandmarkBased Registration of MedicalImage Data J. RuizAlzola, E. SuarezSantana, C. AlberolaLopez, and CarlFredrik Westin CONTENTS 9.1 Introduction 9.2 Deformation Maps 9.3 LandmarkBased Image Analysis 9.4 Landmark Detection and Location 9.5 Our Approach to LandmarkBased Registration 9.6 Deformation Model Estimation 9.6.1 IntensityBased Registration 9.6.1.1 Template Matching 9.6.1.2 Multiresolution Pyramid 9.6.1.3 Local Structure 9.6.1.4 EntropyBased Similarity Measure 9.6.2 Variogram Estimation 9.7 LandmarkBased Local Registration 9.7.1 Displacement Field Model 9.7.2 Ordinary Kriging Prediction of Displacement Fields 9.8 Results 9.9 Conclusions Appendix 9.1 Geostatistical Spatial Modeling Acknowledgment References 9.1 INTRODUCTION Image registration consists of finding the geometric (coordinate) transformation that relates two different images, source and target. Hence, when the transformation is applied to the source image, an image with the same geometry as the target one is obtained. Should both images be obtained with the same acquisition modality and illumination conditions, the transformed source image would ideally become identical Copyright 2005 by Taylor Francis Group, LLC 342 Medical Image Analysis to the target one. Image registration is a crucial element of computerized medicalimage analysis that is also present in other nonmedical applications of image processing and computer vision. In computer vision, for example, it appears as the socalled correspondence problem for stereo calibration 1 and for motion estimation 2, which is also of paramount importance in video coding 3. In remote sensing, registration is needed to equalize image distortion 4, and in the broader area of geographic information systems (GIS), registration is needed to accommodate different maps in a common reference system 5. In this chapter we propose a geostatistical framework for the registration of medical images. Our motivation is to provide the highest possible accuracy to computeraided clinical systems in order to estimate the geometric (coordinate) transformation between two multidimensional, possibly multimodal, datasets. Hence, in addition to being accurate, the approach must be fast if it is to operate in clinically acceptable times. Even though the framework presented here could be applied to several fields, such as the ones mentioned above, this chapter focuses on the application of image registration to the medical field. Registration of medical (both two and threedimensional) images, from the same or different imaging modalities, is needed by computeraided clinical systems for diagnosis, preoperative planning, intraoperative procedures, and postoperative followup. Registration is also needed to perform comparisons across a population, for deterministic and statistical atlas construction, and to embed anatomic knowledge in segmentation algorithms. A good review of the current state of the art for medicalimage registration can be found in the literature 6. Our framework is based on the reconstruction of a dense arbitrary displacement field by interpolating the displacements measured from control points 7. To this extent, the statistical secondorder characterization of the displacement field is estimated from the result of a generalpurpose intensitybased registration algorithm, and it is used to make the best linear unbiased estimation of the displacement in every point using a fast implementation of universal Kriging, an optimal estimation scheme customarily used in geostatistics. Several schemes have been proposed in the past to interpolate sparse displacement fields for medicalimage registration. Most of them fit in one of the two next categories, i.e., PDE and splinebased. As for PDEbased approaches 8, 9, they rely on a mechanical dynamic model stated as a set of partial differential equations, where the sparse displacements are associated with actuating forces. The mechanical model provides an ad hoc regularization to the interpolation problem that produces a physically feasible result. However, the assumption that the physical difference between the source and the target image can actually be represented by some specific model is by no means evident. Moreover, mechanical properties must also be endowed to the anatomic structures in order to obtain a proper model. With respect to splinebased approaches, they usually make an independent interpolation for each of the components of the vector field. Interpolating or smoothing thinplate splines 10–12 are used, depending on whether the sparse displacements are considered to be noiseless or not. The former need the order of the spline to be specified in advance, while the latter also need the regularization parameter to be specified. Adaptiveness can be obtained by spatially changing the spline order and the regularization term. Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 343 The bending term in the spline energy functional could, in principle, also be modified to account for nonisotropic behavior, and even a set of covariables could also be added to the coupling term of the functional. None of these improvements are usually implemented, possibly because of the difficulty of obtaining an objective design from data. Our framework departs from the previous two approaches by adopting a geostatistical approach. Related work in the field of statistical shape analysis has been previously reported by Dryden and Mardia 13. The underlying idea is to use an experimental approach that makes the fewest a priori assumptions by statistically analyzing the available data, i.e., the displacement field obtained from approximate intensitybased image registration. Our method consists of locally applying the socalled universal Kriging estimator 14 to obtain the best linear unbiased estimator (BLUE) of the displacement at every point from the displacements initially obtained at the control points. Central to this approach is the estimation of the secondorder characterization of the displacement field, now modeled as a vector random process model. The estimated variogram 14 (a statistics related to the spatial covariance function or covariogram) plays the role of the spline kernel, though now they are directly obtained from data and not from an a prioridynamic model. Remarkably, thinplate splines can be considered as a special case of universal Kriging estimation 15. 9.2 DEFORMATION MAPS Consider two multidimensional images I 1 (x) (source) and I 2 (x′) (target). Registration consists of finding the mapping (9.1) that geometrically transforms the source image onto the target image. The components of the mapping can be made explicit as (9.2) The vector field Y(x) is commonly termed deformation or warp. Sometimes the displacement field is considered instead, i.e., D(x) = Y(x) − x (9.3) A deformation mapping should count on two basic properties: Y xxYx : () ℜ→ℜ → ′= DD ′= ′ ′         = ,, ,, x Y Y 1 1 1 1 x x x…x x…x D D D D

() ())         Copyright 2005 by Taylor Francis Group, LLC 344 Medical Image Analysis 1. Bijective: a onetoone and onto mapping, which means that the inverse mapping exists 2. Differentiable: continuous and smooth, ideally a diffeomorphism, so that the inverse mapping is also differentiable, thus ensuring that no foldings are present In addition, the construction method of the mapping must be equivariant with respect to some global transformations. For example, to be equivariant with respect to affine transformations, if both (source and target) images are affinely transformed, the mapping should be consistently affinely transformed too. Any deformation must also accommodate both global and local differences, i.e., the mapping can be decomposed in a global and a local component. Global differences are largescale trends, such as an overall polynomial, affine, or rigid transformation. Local differences are on a smaller scale, highlighting changes in a local neighborhood, and are less smooth. Local differences are the reminder of the deformation once the global difference has been compensated. The definition of global and local components depends on whether they are composed or added to form the total map Y(x) = YG(x) + YL (x) = TL TG(x) (9.4) where YG, YL and TG, TL refer to the global and local components of the mapping, in the addition and in the composition forms, respectively. Most commonly, the global deformation consists of a polynomial map (of moderate order to avoid oscillations). Translations, rotations (i.e., Euclidean maps), and affine maps are the most usual global maps. The global polynomial map can be expressed as YG(x) = c0 + C1x + x tC2x… = Λ(x)a (9.5) where a contains all the unknown coefficients in c0 , C1 , C2 , etc. Registration algorithms must estimate the deformation from the corresponding source and target images. This process can be done in one step by obtaining directly both the global and the local deformation, usually decomposed as an addition. Alternatively, many registration algorithms use a twostep approach by which the global map is first obtained and, then, the local map is obtained from the globally transformed source image and the target one, leading to the composition formulation. Both forms are equivalent, and it is possible to switch easily between them, i.e.: YT GG() () xx= TY GG() () xx= YTT T LLG G () () () xxx = − TYY LLG () () xx x =+    −1 Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 345 9.3 LANDMARKBASED IMAGE ANALYSIS Landmarks are singular points of correspondence in objects with a highly descriptive power. They are commonly used in morphometrics 13 to describe shape and to analyze intra and interpopulation statistical differences. In particular, local differences of shape between two objects are commonly studied by reconstructing a deformation that maps both objects from their homologous landmark correspondences. The most popular approach to making the reconstruction of the deformation is based on independent interpolating thinplate splines for each coordinate 10. Approximating thinplate splines can also be used when a tradeoff between actual confidence on landmark positions and smoothness of the deformation is desired. The tradeoff is controlled with a smoothing parameter that can be either estimated by crossvalidation (something usually involving a nonstraightforward optimization) or just by an ad hoc guess 13. The former approach has also been applied to image registration 10, 12. In this case two twodimensional (2D) or threedimensional (3D) images contain the corresponding objects, and the deformation is the geometric mapping between both images. Hence, registration consists of finding this mapping from both images. In this case, landmarks are extracted from the images. Landmarks are referred to in the literature in different ways, e.g., control points, fiducials, markers, vertices, sampling points, etc. Different applications and communities, as ever, usually have different jargons. This is also true for different classifications on landmark types. For example: A usual classification Anatomical landmark: point assigned by an expert that corresponds between organisms in some biologically meaningful way Mathematical landmark: points located on an object according to some mathematical property (e.g., curvature maximum) Pseudolandmark: points located in between anatomical or mathematical landmarks to complete a description (They can also lie along outlines. Continuous curves and surfaces can be approximated by a large number of pseudolandmarks.) Another usual classification Type I landmark: a point whose location is supported by the strongest evidence, such as the joins of tissuebone or a small patch of some unusual histology Type II landmark: a point whose location is defined by a local geometric property Type III landmark: a landmark having at least one deficient coordinate, for instance, either end of a longest diameter, or the bottom of a concavity (Type III landmarks characterize more than one region.) A useful classification for image registration Normal landmark: point with a unique position or with an approximately isotropic uncertainty around a mean position Copyright 2005 by Taylor Francis Group, LLC 346 Medical Image Analysis Quasi (or semi) landmark: point with one or more degrees of freedom, i.e., it can slide along some direction or, with a highly anisotropic location uncertainty, around a mean position Yet another classification Unlabeled landmark: a point for which no natural labeling is available Labeled landmark: a point for which a natural and unique identification exists 9.4 LANDMARK DETECTION AND LOCATION Before any deformation map can be reconstructed, landmarks must be detected and located. These are not easy tasks, even for human experts. On the one hand, no general detection paradigm (i.e., answering the question: is there any landmark around?) can be used because the definition of landmarks varies from application to application. On the other hand, locating landmarks accurately (once a landmark has been detected it is necessary to estimate its exact position) on images is extremely difficult because digital images are defined on discrete grids, and quite often they are quasilandmarks defined on smooth boundaries (and consequently with a high uncertainty along these boundaries). For a human expert, things become even more complicated when the images are 3D, no matter what interaction approach with the data is implemented to clickpoint on the landmark locations. Therefore, it is important to count on reconstruction schemes of the deformation map that are able to deal with the uncertainty in the extracted landmark positions. A first step toward this goal is the use of approximating thinplate splines mentioned previously. Nevertheless, this scheme only considers isotropic noise models for the landmark positions. A remarkable extension due to Rohr 16, 17 allows the incorporation of anisotropic noise models and, hence, quasilandmarks, something important in order to deal with the registration of smooth boundaries. Anisotropic noise models correspond to nondiagonal covariance matrices, with the obvious consequence of coupling the thinplate splines formerly acting on each coordinate independently. The location of N landmarks, extracted by any means from both images, can be modeled as realizations of independent Gaussian random vectors (Xl and Xl ′, l = 1, …, N) with means equal to the correct landmark positions and covariance matrices and . Notice that nondiagonal covariance matrices account for anisotropic uncertainty. Another remarkable achievement of Rohr, which will be used in this chapter extensively, is the derivation of the CramerRao lower bound for the estimation of a point landmark position 12 from discrete images of arbitrary dimensionality in additive white Gaussian noise, (9.6) CXl C Xl ′ ΣI kMm kkt I m II () ( ) () () xxx = ∇⋅∇       ∈ − ∑ σN 2 1 Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 347 where denotes the variance of the noise, and M(m) is a neighborhood around the landmark with m elements. We will also assume this result to model the covariance of the manually extracted landmarks directly from the image data. 9.5 OUR APPROACH TO LANDMARKBASED REGISTRATION We will consider that the deformation that puts into correspondence the source and target images is a realization of a vector random field. The global component of the deformation corresponds to the trend (mean) of the random field, whereas the local component of deformation is modeled by an intrinsically stationary random field. The field is sampled by means of landmark correspondences, i.e., to each landmark in the source image corresponds a landmark in the target one, which are then used to reconstruct the whole realization of the random deformation field. The geostatistical method tries to honor actual observations by estimating the model spatial variability directly from available data. This essentially consists of estimating the variogram of the field, which is a difficult problem, especially if it is to be done from landmarks displacements, because there are usually just a few. This has possibly prevented Kriging’s method from being used in landmarkbased registration. Here we propose a practical way to circumvent these difficulties by splitting the approach into three steps: 1. Imagebased global registration: Estimating the variogram of the displacement field requires detrending of the data. To avoid introducing any subsequent bias into the variogram estimation, we propose to make an intensitybased global (i.e., rigid or affine) registration first to remove the trend effect, with a variety of algorithms being available. For example, rigid registration by maximization of mutual information is a wellknown algorithm 18 that can be used when image intensities in both images are different. 2. Model estimation: Estimating the variogram structure of the detrended displacement field is still a difficult task. The number of available landmarks in most practical applications is almost never enough to make good variogram estimations, and trying to extract a significant number from the images would render the method impractical. We propose to use a fast, generalpurpose, nonrigid registration algorithm to obtain an approximate dense displacement field. Again, a number of algorithms are available, although we are using, with excellent results, a regularized blockmatching scheme with mutual information (and others) similarity measure that was developed by our team 19. The variogram is then readily estimated from this field. 3. Landmarkbased local registration: Landmarks are extracted from the registered image pair and used to reconstruct a realization of a zeromean random deformation field using ordinary Kriging, with the variogram structure just estimated. σNI 2 Copyright 2005 by Taylor Francis Group, LLC 348 Medical Image Analysis 9.6 DEFORMATION MODEL ESTIMATION 9.6.1 INTENSITYBASEDREGISTRATION The model estimation, as noted previously, relies on a fast, generalpurpose, intensitybased, nonrigid registration algorithm to obtain an approximate dense displacement field. This registration framework is presented in the following subsections. To understand the design criteria of our algorithm, general properties of registration algorithms are shown. To simplify the exposition, we will restrict the discussion to threedimensional medical images. Let I 1 and I 2 be two medical images, i.e., two scalar functions defined on two regions of the space. We will use two different coordinate systems x and x′ for each one. The registration problem consists of finding the transformation x′ = Y(x) that relates every point x in the coordinate system of I 1 with a point x′ in the coordinate system of I 2 . The criteria of correspondence are usually set by means of highlevel information, for example anatomical knowledge. However, when coding the correspondence into a registration algorithm, some properties should be satisfied. Invertibility of the solution: A registration algorithm should provide an invertible solution. Invertibility implies the existence of an inverse transformation x = Y(x′) that relates every point on I 2 back to a point on I1 , where Y = Y−1 . It is satisfied if the Jacobian of the transformation is positive. No boundary restriction: A registration algorithm should not impose any boundary condition. Boundary restrictions, sometimes in the model, sometimes in the representation of the warping, are usually set to help either implementation or convergence of the search technique. However, boundaries are acquisition dependent, not data dependent, so they are a fictitious matching in the solution. Thus, ideal registration should provide freeform warpings. Intensity channel insensitivity: Another desirable property of a registration algorithm is the insensitivity to noise or to a bias field in the acquisitions. These variations are usually dealt with by an entropybased similarity measure. Possibility of large deformations: Some registration schemes are based on models such as linear elastic models, which are not thought to be useful for large deformations. The theory of linear elasticity is successful whenever relative displacements are small. Hence, mechanical models should be used with care when trying to register tissue deformations. 9.6.1.1 Template Matching Intensitybased registration methods, i.e., those using directly the full content of the image and not simplifying it to a set of features to steer the registration, usually correspond to one of two important families: template matching and variational. The former was popular years ago because of its conceptual simplicity 20. Nevertheless, Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 349 in its conventional formulation, it is not powerful enough to address the challenging needs of medicalimage registration. Variational methods rely on the minimization of a functional (energy) that is usually formulated as the addition of two terms: data coupling and regularization, the former forcing the similarity between both data sets (target and source deformed with the estimated field) to be high, and the latter enforcing the estimated field to fulfill some constraint (usually enforcing spatial coherencesmoothness). As opposed to variational methods, template matching does not impose any constraint on the resulting fields, which, moreover, due to the discrete movement of the template, turn out to be discrete as well. These facts have led to an increasing popularity of variational methods for registration, while template matching has been losing ground in this arena. Template matching finds the displacement for every voxel in a source image by minimizing a local cost measure that is obtained from a small neighborhood of the source image and a set of potential correspondent neighborhoods in a target image. The main disadvantage of template matching is that it estimates the displacement field independently in every voxel, and no spatial coherence is imposed to the solution. Another disadvantage of template matching is that it needs to test several discrete displacements to find a minimum. There are several optimizationbased templatematching solutions that provide a real solution for every voxel, although they are slow 21. Therefore, most templatematching approaches render discrete displacement fields. Another problem associated with template matching is commonly denoted as the aperture problem in the computervision literature 22. This essentially consists of the inability of making a good match when no discriminant structure is available, such as in homogeneous regions, surfaces, and edges. When this fact is not taken into account, the matching process is steered by noise and not by the local structure, because it is not available. The modelestimation registration algorithm that we present here maintains the simplicity of template matching while addressing its drawbacks. It consists of a weighted regularization of the templatematching solution, where weights are obtained from the local structure, to render spatially coherent real deformation fields. Thanks to the multiscale nature of our approach, only displacements of one voxel on every scale are necessary when matching the local neighborhoods. 9.6.1.2 Multiresolution Pyramid The algorithm works in a way that is similar to the Kovaˇ ciˇ cand Bajcsy elastic warping 23, in which images are decomposed on Gaussian multiresolution pyramids. On the highest level, the deformation field is estimated by regularized template matching steered by local structure (details in the following subsections). On the next level, the source data set is deformed with a deformation field obtained by spatial interpolation of the one obtained on the first level. The deformed source and the target data sets on the current level are then registered to obtain the deformation field corresponding to the current level of resolution. This process is iterated on every level. The algorithm implementation is summarized in Figure 9.1. Copyright 2005 by Taylor Francis Group, LLC 350 Medical Image Analysis 9.6.1.3 Local Structure Local structure measures the quantity of discriminant spatial information on every point of an image, and it is crucial for templatematching performance: the higher the local structure, the better is the result obtained on that region with template matching. To quantify local structure, a structure tensor is defined as T(x) = (∇I(x)⋅⋅ ⋅∇I(x) t ) σ , where the subscript σ indicates a local smoothing. The structure tensor consists of a symmetric positivesemidefinite 3×3 matrix that can be associated with ellipsoids, i.e., eigenvectors and eigenvalues correspond to the ellipsoids’ axes directions and lengths, respectively. A scalar measure of the local structure can be obtained as 16, 17, 24. (9.7) Figure 9.2 shows an MRI T1weighted axial slice of the brain and the estimated structure tensors overlaid as ellipsoids. Small eigenvalues indicate a lack of gradient variation along the associated principal direction, and therefore, high structure is indicated by big (large eigenvalues), round (no eigenvalue is small) ellipsoids. The color coding represents the scalar structure measure, with hot colors indicating higher structure. FIGURE 9.1 Algorithm pipeline for pyramidal level (i). FIGURE 9.2 (Color figure follows p. 274.)MRI T1weighted axial slice of human brain and its structure tensors. (Hot color represents high structure.) Image 1 Transformed Data Matching 1 2 Step Deformation Deformation 1 2 Global (i) Local Structure Image 1 Image 2 (i) (i) (i) (i) Next scale level (i+1) Previous scale level (i1) structure trace () det ( ) () x x x = T T Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 351 Figure 9.3 shows crosssections of a T1weighted MRI dataset of a human brain (top row) and the scalar measure of local structure obtained from them, represented with a logarithmic histogram correction (bottom row). Note how anatomical landmarks have the highest measure of local structure, corresponding to the points indicated by the arrows on the top row. Curves are detected with lower intensity than points, and surfaces have even lower intensity. Homogeneous areas have almost no structure. Template matching provides a discrete deformation field where no spatial coherence constraints have been imposed. In the discussion in this subsection, this field is regularized so as to obtain a mathematically consistent continuous mapping. We will consider the deformation field to be a diffeomorphism, i.e., an invertible continuously differentiable mapping. To be invertible, the Jacobian of the deformation field must be positive. On every scale level, the displacement is small enough to guarantee such a condition. For every level of the pyramid, the mapping is obtained by composing the transformation on a higher level than the one on the current level, so that the positive Jacobian condition is preserved. Spatial regularization is achieved by locally projecting the deformation field provided by template matching on an appropriate signal subspace, and simultaneously taking into account the quality of the matching as indicated by the scalar measure of local structure. We propose here to use normalized convolution 25, 26, a popular refinement of weightedleast squares that explicitly deals with the socalled signalcertainty philosophy. Essentially, the scalar measure of structure is FIGURE 9.3 (Top) MRI T1weight crosssections; (bottom) local structure measure (arrows point at higher structure regions). Copyright 2005 by Taylor Francis Group, LLC 352 Medical Image Analysis incorporated as a weighting function in a least squares fashion. The field obtained from template matching is then projected onto a vector space described by a nonorthogonal basis, i.e., the dot products between the field and every element of the basis provide covariant components that must be converted into contravariants by an appropriate metric tensor. Normalized convolution provides a simple implementation of this operation. Moreover, an applicability function is enforced on the basis elements to guarantee a proper localization and avoid highfrequency artifacts. This essentially corresponds to weighting each basis element with a Gaussian window. The desired transformation is related to the displacement field by the simple relation shown in Equation 9.3. Because the transformation is differentiable, we can write the function in different orders of approximation (9.8) (9.9) Equation 9.8 and Equation 9.9 consist of linear decompositions of bases of size 3 and 12 basis elements, respectively. We have not found relevant experimental improvement of the registration algorithm by using the linear approximation instead of the zeroorder one, probably due to the local nature of the algorithm. The basis set used is then (9.10) Figure 9.4 shows a 2D discrete deformation field that has been regularized using the certainty on the left side and a 2D Gaussian applicability function with σ = 0.8. FIGURE 9.4 (Left) certainty, (center) discrete matching deformation, (right) weightfiltered deformation. Yx Yx () ( ) ≈ 0 Yx Yx Jx x x () ( ) ()( ) ≈ + ⋅− 00 0 bb 1 1 2 3 2 1 2 1 0 0 0 = = = =      = = Y Y Y Y Y () () () () ( x x x x x)) () () () () = =      = = = =    1 0 0 0 1 3 3 1 2 3 Y Y Y Y x x x x b   Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 353 9.6.1.4 EntropyBased Similarity Measure In a work by Suarez et al. 19, the registration framework was tested using square blocks that were matched using the sum of squared differences and correlation coefficient as similarity measures. In the current work, we introduce entropybased similarity measures into this framework, although it can be used by any algorithm based on template matching. A similarity measure can be interpreted as a function defined on the joint probability space of two random variables to be matched. In the case of block matching, each block represents a set of samples from each random variable. When this probability density function (PDF) is known, mutual information can be computed as (9.11) where I 1 , I 2 are the images to register, and Ω is the joint probability function space. A discrete approximation is to compute the mutual information from the PDF and a small number N of samples (i 1 k, i 2 k) (9.12) where ƒp is a coupling function defined on Ω. Therefore, the local evaluation of the mutual information for a displaced block containing N voxels can be computed just by summing the coupling function ƒp on the k samples that belong to this block. We propose to compute a set of multidimensional images, each of them containing at each voxel the local similarity measure corresponding to a single displacement applied to the whole target image. A decision will be made for each voxel, depending on which displacement renders the greatest similarity. A problem associated with local entropybased similarity measures is the local estimation of the joint PDF of both blocks, because there are never enough samples available. We propose to overcome this problem by using the joint PDF corresponding to the whole displaced source image and the target one. The PDF to be used for a given displacement is the global jointintensity histogram of the reference image with the displaced target image. This is crucial for higher pyramidal levels, where one voxel displacement drastically changes the PDF estimation. It is straightforward to compute the local mutual information for a given discrete displacement in the whole image. This requires only the convolution of a square kernel representing the block window and the evaluation of the coupling function for every pair of voxels. Furthermore, because the registration framework only needs discrete deformation fields, no interpolation is needed in this step. Any similarity measure that can be computed as a kernel convolution can be implemented this way. MI I I p i i pi i pi pi di () ()log() () ( ) 12 12 12 12 ,= , , ∫Ω 112di MI I I pi k i k pi k pi k N () log( ) () ( 12 1 12 1 , , = ∑  22 1 12 ) ( ) k fikik k N p =,= ∑ Copyright 2005 by Taylor Francis Group, LLC 354 Medical Image Analysis A small sketch of this technique is shown in Figure 9.5. For smoothness and locality reasons, we have chosen to convolve using Gaussian kernels instead of square ones. To achieve a further computational saving, Equation 9.12 can be written as (9.13) The displacement field defines the displacement of a voxel in the source image. The similarity measure will be referred to as the sourceimage reference system (image 1). For a given voxel in the source image, the comparison of Equation 9.13 for different displacement will always contain the same terms, depending on p(i 1 k). Thus, we can take this term off and modify accordingly the coupling function to reduce computational cost. Any other entropybased similarity measure can be estimated in a similar way. The computational cost is then very similar to any other similarity measure not based on entropy. 9.6.2 VARIOGRAMESTIMATION The variogram is estimated under the assumption of intrinsic stationarity (i.e., the mean of the displacement field must be constant) from the displacement field obtained by intensitybased image registration. Should intrinsic stationarity not be the case, a trend model must be preestimated so that it can be substrated from the field prior to estimating the variogram. This process is undesirable because it introduces bias in the variogram estimation due to its inherent circularity: the probabilistic characterization of the random component of the field must be known to estimate the trend, but the trend must also be known to estimate the probabilistic characterization of the random component. Nevertheless, this issue is present in any model with a trend and a random component, and, in fact, estimating the sample variogram instead of the sample autocovariance has several advantages 14 from this point of view: FIGURE 9.5 (Left) target image to be matched, (center) reference image where similarity measure is going to be estimated for every discrete displacement, (right) for every discrete displacement, the similarity measure is computed for every voxel by performing a convolution. Image 1 Image 2 Image 2 Image 1 MI I I p i k i k p i k k N () (log( ) log ( 12 1 12 1 ,,− = ∑  ) log ( )) − pi k 2 Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 355 If the mean value of the field is an unknown constant, it is not necessary to preestimate it because the variogram sample estimator is based on differences. Hence, in this case, the sample variogram can be estimated unbiasedly. The sample variogram estimator is more robust against mean model mismatch than the sample autocovariance one. The sample variogram estimator is less biased than the sample autocovariance one when the mean model is preestimated and subtracted from the field realization to make the spatialdependence model estimation. 9.7 LANDMARKBASED LOCAL REGISTRATION 9.7.1 DISPLACEMENTFIELDMODEL The reconstruction of the local displacement field DL (x), can be cast as the optimal prediction of the displacement at every location x from our set of observations. These observations are obtained by measuring the displacement between pairs of point landmarks extracted from both images. The observation process is then Z(x) = X′(x) − X(x) = D(x) + Nz (x) (9.14) where X, X′ are the landmark position random processes, D is the stochastic characterization of the local displacement field, and NZconsists of a zeromean Gaussian random noise field with autocovariance independent of D. From the model, it follows that µZ (x) = µD(x) (9.15) CZ (x) = CX′ (x) + CX (x) (9.16) CZ (xi , xj ) = CD(xi , xj ) (9.17) Furthermore, Equation 9.16 can be rewritten for the sampled landmarks (xl , x′ l ) as (9.18) where the CramerRao lower bound introduced in Section 9.4 has been used. Hereinafter, the L subscript will be omitted. CCC Z () () () xxx XX lIlIl ll =+= + ΣΣ12 Copyright 2005 by Taylor Francis Group, LLC 356 Medical Image Analysis 9.7.2 ORDINARYKRIGINGPREDICTION OFDISPLACEMENTFIELDS The mean for each component of the displacement field, µD(x), is assumed to be an unknown constant. We have found that this is a very convenient model, even after the global preregistration that should render zeromean values for the resulting displacement components. The reason is that usually a locally varying mean structure can model much of the local deformation. Therefore, in this case we will not use all the samples but a limited number around the prediction location. This has the added benefit of reducing the computational burden. For the sake of simplicity, positions of the observed landmarks will be denoted by the set O = {x1 , …, xN}, and the observation vector is denoted Zr (O) = Z r (x1) … Z r (xN ) t (9.19) The ordinary coKriging (i.e., multivariate Kriging) predictor takes the form (9.20) If there is no secondorder probabilistic dependence among the field components, each of them is dealt with independently, leading to a blockdiagonal K(x,O) matrix and resulting in the conventional ordinary Kriging predictor for each component. The ordinary Kriging coefficients must minimize the mean square prediction error (9.21) ˆ () ˆ () ˆ () () Dx x x k x k ≡         = , 1 1 1 D D d t t d
… O 11 1 1 () () () x k x k x , ,,         O OO

… t d t d d Z(() () O O
Z d         = , ,         t t d k x k x 1 () () () O O O
Z =,Kx()() OOZ MSPE E D rrt r ()(() ()) xxk ,= − OOZ 2 = −− − EDr D r r t ( ( ) ( ) ( ( ) ( ))) xxk µµZ Z OO2 = − ,+ , σ D r D rr r t r t 2 2 () ( ) ( ) xk xk k CCZ Z OO Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 357 subject to the unbiasedness constraint (9.22) Closedform equations for the coefficients and for the achieved squared error can be readily obtained after some algebra (see, for example, Cressie 14). Because of space constraints, we only present the coefficients’ equation, expressed in terms of covariances. The matrix Λ is block diagonal, with each diagonal block equal to a column vector of ones, and the vector λ r is a zero row vector with a single 1 in the r position: (9.23) Extensions of ordinary Kriging are possible by incorporating more complex mean structure models. Though this could seem in principle appealing, it has the serious drawback of hindering the estimation of the spatial variability model, because the mean structure has to be filtered out before the covariance structure can be estimated. Notice that estimating the variogram does not require preestimation of the mean, as this is constant. 9.8 RESULTS We are currently using the proposed framework in a number of applications. To better illustrate its behavior, we have selected two simple experiments. Figure 9.6(a) shows a T1w MRI axial slice of a multiple sclerosis patient, and Figure 9.6(b) a corresponding T2w axial slice of a different patient. Ellipsoids representing landmark covariances have been overlaid (seven landmarks in the brain and four on the skull). Figure 9.6(d) and Figure 9.6(e) show two T1w midsagittal slices of MS patients, also with covariance landmark ellipsoids overlaid (11 landmarks in the brain and 3 on the skull). In each case, the second image is to be warped onto the first one. In both cases the images are first globally registered. Then a forward displacement field is obtained for each one using our generalpurpose general registration scheme 19 to estimate the variograms. Sample variograms and their weightedleast squares fit to theoretical models (linear combination of Gaussian and power models) are shown in Figure 9.6(g) and Figure 9.6(h). For this purpose, 5000 displacements were sampled, which makes the estimation highly accurate. Registration results are shown in Figure 9.6(c) and Figure 9.6(f) by ordinary Kriging prediction of the displacement field, using only the displacements from the landmarks on the images. Notice how even with so few landmarks, a good result is achieved, especially in areas closer to the landmarks, because of the proper estimation of the random displacement field. The opensource software Gstat 27 was used in these experiments. EE ˆ () () DDxx= kOO O rttr = − −−−− CC C C Z Z ZZ 1111 () (,) ( ())( D x ΛΛ Λ Λ (() (,) ) OOC ZD r r x −λ Copyright 2005 by Taylor Francis Group, LLC 358 Medical Image Analysis 9.9 CONCLUSIONS We have presented a practical approach to the statistical prediction of displacement fields from pairs of landmarks. The method is grounded on the solid theory of ordinary Kriging, and it also provides a way of estimating the spatialdependence models from image data, thus circumventing some of the hurdles found when using Kriging. The fact that the statistical relation between both geometries is successfully used makes the method highly accurate and particularly well suited for image(a) (b) (c) (d) (e) (f) (g) (h) FIGURE 9.6 Experimental results: (a) axial T1, (b) axial T2, (c) warped axial T2, (d) first T1 sagittal, (e) second T1 sagittal, (f) warped second sagittal, (g) displacement variograms (axial), and (h) displacement variograms (sagittal). 0 10 20 30 40 50 60 70 80 90 0 5 10 15 20 25 30 35 40 45 h semivariogram dy: sample variogram dy: 6.78018 Gau(16.3546) + 0.0254245 Pow(1.32512) dx: sample variogram dy: 27.886 Gau(32.4634) + Pow 0.380956 Pow(0.788065) 0 10 20 30 40 50 60 70 80 90 –10 0 10 20 30 40 50 60 70 80 90 dx: sample variogram dx: 10.2934 Gau(18.4780) + 0.0355 Pow(1.2746) dy: 24.7626 Gau(33.4645) + 0.2789 Pow(1.1992) dy: sample variogram Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 359 registration and shapeanalysis applications. It is remarkable to note that thinplate splines can be considered a particular case of Kriging, and in this sense, our approach generalizes this popular registration method. APPENDIX 9.1 GEOSTATISTICAL SPATIAL MODELING Consider a random field Zr (x) (the superscript ris meant to consider several random fields, such as the components of a vector random field) such that (9.24) The function , with h = xi − xj , is called the variogram of the random field Z r (x) and, assuming it exists, is the central parameter to model the spatial dependence of the random field in the geostatistical method. The variable (without the 2 factor) is usually called the semivariogram. The variogram can be easily related to the variance and covariance from the relation (9.25) The shape of a variogram is summarized by the following parameters: Nugget: it is the size of the discontinuity of the semivariogram at the origin. Note that the presence of a nugget other than zero indicates that the random field is not continuous. The presence of a nugget effect is usually attributed to measurement noise and to a very local random component of the field that appears as uncorrelated at the working resolution. Both effects are usually superimposed and modeled with white noise. Sill: if the variogram is bounded, the sill is the value of the bound. A sill indicates total noncorrelation as, for example, with white noise. Usually, random fields become uncorrelated for big lags, reaching a sill. Partial sill: it is the difference between the sill and the nugget. Range: it is the lag for which the sill is reached, of course assuming there is a sill in the variogram. Various approaches for constructing valid theoretical variogram models are available 14, 27–30. Most often, existing variogram models such as nugget (white field), spherical, linear, exponential, power, etc. are used as building blocks in a linear combination of valid variogram models, making use of the convexity of the set of valid variograms. The variogram can be extended for the multivariate case 14. The pseudocrossvariogram function is defined as (9.26) var ( ) ( ) ( ) ZZr i r j Z ij ij r xx xxxx − = − , ∀ , ∈ 2γΩ 2γ Z r()h γ Z r()h var ( ) ( ) ( ) ( ) ( ZZr i r j Z i Z j Z rr r xx x x x − =+− σσ222C iij,x) 2γ ZZ ij r i s j rs ZZ ()()() xx x x − = − var Copyright 2005 by Taylor Francis Group, LLC 360 Medical Image Analysis A9.1.1 INTRINSICSTATIONARITY The scalar random field Z r (x) is said to be intrinsically stationary if it has a constant mean and its variogram exists. Moreover, any conditionally negativedefinite function 2γ(h) is the variogram of an intrinsically stationary random field. The variogram of an intrinsic random field Z r (x) is (9.27) A9.1.2 RELATION BETWEENINTRINSIC ANDSECONDORDERSTATIONARITIES Note that the family of intrinsic stationary fields is larger than the secondorder stationary one. In particular, unbounded valid variograms, i.e., variograms without a sill, do not have a corresponding autocovariance function. For secondorder stationary fields, there is a simple relation between the variogram and the autocovariance, i.e., (9.28) It is clear that in the common situation for secondorder stationary fields where the covariance approaches zero for large space lags, the sill of the variogram is . ACKNOWLEDGMENT This work has been partially funded by the Spanish Government (MCyT) under research grant TIC20013808C0201. REFERENCES 1. Faugeras, O., ThreeDimensional Computer Vision: a Geometric Viewpoint, MIT Press, Cambridge, MA, 1993. 2. Shah, M. and Jain, R., Eds., MotionBased Recognition, Vol. 9, Computational Imaging and Vision, Kluwer, Dordrecht, Netherlands, 1997. 3. Tekalp, A.M., Digital Video Processing, Signal Processing Series, Prentice Hall, Upper Saddle River, NJ, 1995. 4. Lillesand, T.M. and Kiefer, R.W., Remote Sensing and Interpretation, 4th ed., John Wiley Sons, New York, 1999. 5. Burrough, P.A. and McDonell, R.A., Principles of Geographic Information Systems (Spatial Information Systems and Geostatistics), 2nd ed., Oxford University Press, Oxford, U.K., 1988. 6. Maintz, J.B.A. and Viergever, M.A., A survey of medicalimage registration, Medical Image Anal., 2, 1–36, 1998. 7. RuizAlzola, J., Suárez, E., AlberolaLópez, C., Warfield, S.K., and Westin, C.F., Geostatistical medicalimage registration, in Lecture Notes in Computer Science, no. 2879, SpringerVerlag, New York, 2003, pp. 894–901. E ( ) Z r Z r x =µ 2 2 γ Z rr r ZZ () ( ( ) ()) hxhx =+− E 22γ ZZZ rrr ( )(() ()) hh= − CC0 2C Z r()0 Copyright 2005 by Taylor Francis Group, LLC LandmarkBased Registration of MedicalImage Data 361 8. Bajcsy, R. and Kovacˇiˇ c, S., Multiresolution elastic matching, Computer Vision, Graphics, Image Process., 46, 1–21, 1989. 9. Christensen, G.E., Joshi, S.C., and Miller, M.I., Volumetric transformation of brain anatomy, IEEE Trans. Medical Imaging, 16, 864–877, 1997. 10. Bookstein, F.L., Principal warps: thinplate splines and the decomposition of deformations, IEEE Trans. Pattern Anal. Machine Intelligence, 11, 567–585, 1989. 11. Rohr, K., Image registration based on thinplate splines and local estimates of anisotropic landmark localization uncertainties, in Lecture Notes in Computer Science, no. 1496, SpringerVerlag, Heidelberg, 1998, pp. 1174–1183. 12. Rohr, K., Landmarkbased image analysis (using geometry and intensity models), Vol. 21, Computational Imaging and Vision, Kluwer, Dordrecht, Netherlands, 2001. 13. Dryden, I.L. and Mardia, K.V., Statistical Shape Analysis, Wiley Series in Probability and Statistics, John Wiley Sons, New York, 1998. 14. Cressie, N.A.C., Statistics for Spatial Data, Wiley Series in Probability and Mathematical Statistics, John Wiley Sons, New York, 1993. 15. Matheron, G., Splines and Kriging: their formal equivalence, in DowntoEarth Statistics: Solutions Looking for Geological Problems, Syracuse University Geological Contributions, Syracuse, NY, 1981, pp. 77–95. 16. Rohr, K., Differential operators for detecting point landmarks, Image Vision Computing, 15, 219–233, 1997. 17. Harris, C. and Stephens, M., A combined corner and edge detector, in Proc. Fourth Alvey Vision Conference, 1988, pp. 147–151. 18. Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., and Kikinis, R., Multimodal volume registration by maximization of mutual information, Medical Image Anal., 1, 35–51, 1996. 19. Suárez, E., Westin, C.F., Rovaris, E., and RuizAlzola, J., Nonrigid registration using regularized matching weighted by local structure, in Lecture Notes in Computer Science, no. 2489, SpringerVerlag, Heidelberg, 2002, pp. 581–589. 20. Duda, R.O. and Hart, P.E., Pattern Classification and Scene Analysis, John Wiley Sons, New York, 1973. 21. Suárez, E., Cárdenes, R., Alberola, C., Westin, C.F., and RuizAlzola, J., A general approach to nonrigid registration: decoupled optimization, in 23rd Ann. Int. Conf. IEEE Eng. Med. Biol. Soc., IEEE, Washington, DC, 2000. 22. Poggio, T., Torre, V., and Koch, C., Computational vision and regularization theory, Nature, 317, 314–319, 1985. 23. Kovacic, S. and Bajcsy, R.K., Multiscalemultiresolution representations, in Brain Warping, Academic Press, New York, 1999, pp. 45–65. 24. RuizAlzola, J., Kikinis, R., and Westin, C.F., Detection of point landmarks in multidimensional tensor data, Signal Process., 81, 2243–2247, 2001. 25. Westin, C.F., A Tensor Framework for Multidimensional Signal Processing, Ph.D. Thesis, Linköping University, Sweden, 1994. 26. Knutsson, H. and Westin, C.F., Normalized and differential convolution: methods for interpolation and filtering of incomplete and uncertain data, in Proc. Computer Vision and Pattern Recognition, IEEE, New York, 1993, pp. 515–523. 27. Pebesma, E.J. and Wesseling, C.G., Gstat: a program for geostatistical modelling, prediction and simulation, Comput. Geosci., 24, 17–31, 1998. 28. Chiles, J.P. and Delfiner, P., Geostatistics: Modeling Spatial Uncertainty, Wiley Series in Applied Probability and Statistics, WileyInterscience, New York, 199. 29. Ripley, B.D., Statistical Inference for Spatial Processes, repr., Cambridge University Press, Cambridge, U.K., 1991. 30. Arlinghaus, S.L. and Griffith, D.A., eds., Practical Handbook of Spatial Statistics, rev. ed., CRC Press, Boca Raton, FL, 1995. Copyright 2005 by Taylor Francis Group, LLC