1 Chapter Introduction 1.1 Introduction This thesis presents methods for analyzing 3-D maxillofacial image data that are used for orthodontic1 treatment and maxillofacial surgery. The term “maxillofacial” in a narrow sense literally involves only the maxilla (upper jawbone) and the face, but in a broad sense, it may also contain the mandible (lower jawbone) and the dentition (the set of teeth). We use the term in the latter sense, which is closely related to the more specialized medical term “orthognathic”. The term “orthognathic” originates from the Greek words “orthos” and “gnathos” that mean “straight” and “jaw”, respectively. The two terms “maxillofacial” and “orthognathic” are often used interchangeably and we not particularly differentiate between the two in this thesis. Orthodontic treatment is aimed at correcting the malocclusion (incorrect bite) between the upper and lower teeth. Maxillofacial (or orthognathic) surgery combines orthodontic treatment with surgery of the jaw to correct or establish a stable, functional balance between the teeth, jaws, and facial structures. The goal of maxillofacial surgery is to treat any jaw imbalance and the resulting malocclusion, which could adversely affect the proper functioning of the teeth as well as the aesthetic appearance. Of or related to orthodontics, the dental specialty and practice of preventing and correcting irregularities of the teeth. Traditionally, maxillofacial image data have been provided in the form of 2-D images such as photographs and X-ray radiographs. In recent years, however, rapid advances in medical imaging technology have made the use of 3-D maxillofacial image data increasingly more common. 3-D data are obtained in two forms, surface-scan data and volumetric data. Surface-scan data is generally provided as a wireframe model or a range image that is also referred to as a depth map. The surface of a wireframe model is represented by polygons (typically, triangular patches). A range image (also called a depth map) is an array of pixels in which each pixel value (gray level) represents the height or depth of a sample point on an object from a reference surface. Thus a range image carries viewpoint-dependent depth information of an object. Volumetric data, on the other hand, is provided as a stack of 2-D images. A significant advantage over surface-scan data is that volumetric data delivers internal information of an object. For example, computed tomography (CT) data comprises a series of 2-D cross-sectional images of an object such as the head, providing vital information of the internal anatomy. There is a great demand for computerized systems that perform analysis of 3D maxillofacial image data. These systems would enable not only 3-D visualization to provide the clinician with a visual aid for diagnosis but also 3-D measurement with consistent quality, which allows the quantitative assessment of the patient. With these systems, it is also possible to manipulate 3-D data for the purposes of treatment planning and simulation. Dental study models (plaster casts of the dentition) are widely used for orthodontic treatment. They are obtained by taking an impression of the dentition and pouring dental plaster into the impression. At present, orthodontists manually perform analysis of the dental cast using rulers and calipers, which is inaccurate and limited to linear measurements. In this thesis, we first deal with surface-scan data of the dental study model (Figure 1.1). We digitize the dental models with a commercially available laser scanner that provides high resolution and ease of use [1]. Our research team at the Department of Electrical and Computer Engineering, NUS, has previously developed a computer-aided system that allows visualization, computer-assisted measurement, and manipulation of the image data [2]. With the system, the orthodontist can, for instance, simulate tooth rearrangement for the treatment of malocclusion, but this requires prior segmentation of individual teeth by manual means. With the aim of enhancing the system, we studied the problem of automated tooth segmentation and developed suitable algorithms. Figure 1.1: Surface-scan data of a dental study model. CT provides detailed 3-D information of the interior of the human body. CT data is a volumetric data set used not only for orthodontic treatment but also for craniofacial surgery and maxillofacial surgery. The term “craniofacial” is derived from the word “cranio”, referring to the skull or cranium, and “facial”, referring to the face. Thus, craniofacial surgery is concerned with the anomalies of the head and facial bones, whereas, as stated above, maxillofacial surgery is concerned with the correction of a wide range of jaw and facial irregularities. One of the most successful applications of CT data in orthodontic treatment is in dental implantology, in which an artificial root is surgically inserted into the jawbone to provide anchorage for a dental prosthesis. Treatment of tooth loss with dental implants is today a routine specialty procedure in dentistry. In many medical imaging applications, 3-D data sets have to be segmented and segmentation plays an important role in biomedical image processing [3]. We present methods for automatically extracting the skull, segmenting the mandible (lower jawbone), and also extracting the nerve canal that passes through the mandible (i.e., the mandibular nerve canal). The nerve canal is a vital structure required to be located prior to dental implant surgery. 1.2 Previous Work In recent years, much effort has been expended in developing computerized systems for clinical and research applications in dentistry. Expert systems have been proposed for automatic diagnosis in orthodontics [4]−[6]. Computer-aided design (CAD) and computer-aided manufacturing (CAM) in dentistry are examples of the introduction of computer technology to dentistry with successful clinical applications [7]−[12]. One such application is the automatic manufacturing of dental fillings such as crowns and inlays [7]. CAD and fabrication of dental restorations have been proposed to speed production, eliminate labor-intensive steps, and provide consistent quality [8]. Pre- surgery simulation systems have been implemented for procedures such as tooth rearrangement and quantitative evaluation of 3-D tooth movement [11], [12]. Tooth segmentation is an important step in many automated and semi-automated computer-based systems that require the accurate demarcation of individual teeth prior to the detection and measurement of dental features. Applications include the measurement of orthodontic parameters [4]−[6], simulation of tooth rearrangement for correction of malocclusion [2], and pose estimation of teeth [13]. In an early attempt at tooth segmentation [4]−[6], the interstices (contact areas) between the teeth were detected along the U-shaped axis of a dental wax imprint [4], [5]. Orthodontic features such as the cutting edges of the incisors, tips of the cuspids (canines), and cusps2 of the bicuspids (premolars) and molars were located using the watershed algorithm [6] (Figure 1.2). However, this system is not designed to deal with severe malocclusions but is targeted at the mild cases that are more likely to be encountered in epidemiological studies. In a recent work on tooth segmentation, line segments are detected using the Sobel filter and the completion to a closed contour is achieved by dynamic programming [7]. Since the authors were concerned with the restoration of the occlusal (biting) surfaces of the teeth, their work was limited to the posterior (back) teeth. The above approaches to tooth segmentation, which make use of only the plan-view range image of the teeth, have had limited success due to the small incisor interstices and the deep fissures on the occlusal (biting) surface of the molars. An attempt to segment the teeth by processing a wireframe model is reported in [14], [15], but only The high points on the occlusal (biting or chewing) surfaces of the posterior (back) teeth. preliminary results using a few models are given. Similar to the works of D. Laurendeau et al. [4], J. Cote et al. [5], and M. Mokhtari et al. [6], the system of S.M. Yamany et al [14], [15] is only designed to handle cases of mild malocclusion. Tooth segmentation is, in general, a difficult task because teeth have different shapes and their arrangements vary substantially from one individual to another. The difficulty is exacerbated when the teeth are malaligned, which is a common occurrence in clinical cases. Figure 1.2: Names of the teeth in the panoramic view. CT data has been widely employed for craniofacial surgery and maxillofacial surgery [16]−[19]. The segmentation of the mandible is an important step for maxillofacial surgery. For instance, assessment of mandibular asymmetry is considered to be an appropriate means of determining the effectiveness of maxillofacial and orthodontic treatment [16], [19]. Currently, manual operation is necessary for segmenting the mandible, which is time-consuming and labor intensive; an hour or two of manual refinement is typically required [17]. CT data has been increasingly used for the applications in orthodontic treatment because it provides 3-D information of the jaw without geometrical distortion [20]−[25]. This trend has been accelerated since the advent of techniques that can generate various cross-sectional images by reformatting CT data. Dental CT programs that can reformat CT data are now commercially available, though they require extensive human intervention [24]−[26]. For the assessment and planning of dental implant surgery, panoramic CT images (or panoramics) are particularly useful because they exhibit a series of cross-sections along curved planes through the mandible (Figure 1.3). A pre-surgery simulation system has also been developed for computeraided surgery in dental implantology [21], [23]. Figure 1.3: Panoramic CT image. 1.3 Objectives The goal of our research work is to develop a computer-based system that can perform analysis of 3-D maxillofacial image data. Feature detection is an essential early step in image processing and analysis algorithms. The first objective of this thesis is to devise an algorithm that can be applied to the detection of such features as edges and tubular structures, and also the segmentation of dental features in 3-D image data. The computer-aided system that has been developed by our research group allows the orthodontist to simulate tooth rearrangement for the treatment of malocclusion [2]. In the system, it is necessary to segment individual teeth manually. To enhance the capability of the system, the second objective of this thesis is to automate the tooth segmentation step. We focus especially on the development of a technique for tooth segmentation that can deal robustly with dental study models exhibiting a variety of malocclusions. In addition to the surface-scan data of the dental study models, we also deal with CT data so that our computer-aided system is further extended to applications in craniofacial surgery, maxillofacial surgery, and orthodontic treatment (typically, dental implant surgery). The third objective of the thesis is to develop a method for selecting the appropriate threshold value for extracting the skull in CT data. The method can be used for any medical applications in which the object of interest is a bony structure. The fourth objective of the thesis is to devise an automated method for segmenting the mandible from the skull. For a smooth and successful implant treatment, it is crucial to locate vital internal structures that need to be avoided during surgery. One such internal structure is the mandibular nerve canal. It is difficult to locate the nerve canal in CT data because the nerve canal passes through a number of CT images and thus appears only as a small dark spot in each CT image. Mental reconstruction is required to grasp the global structure of the nerve canal. To ease these limitations, panoramic CT images are nowadays often utilized for the assessment and planning of dental implant surgery. The panoramics can be obtained through a dental CT program that usually comes with the CT scanner. The dental CT programs, however, require frequent human intervention for computing the panoramics. In addition, none of the existing programs is capable of tracing out the nerve canal automatically, especially in 3-D space. The accurate segmentation of the nerve canal is difficult because the structure of the nerve canal is not well defined; it is similar to that of a tube with many side openings. The fifth objective of the thesis is to develop a technique that can compute panoramics with minimum human intervention and extract the nerve canal accurately and automatically. 1.4 The Thesis In this section, an overview of the thesis is provided in order to give the scope of each chapter. The thesis consists of chapters, including this introductory chapter. In Chapter 2, we introduce a novel technique that we call gradient orientation analysis (GOA). This technique detects features such as crease edges and tubular structures in an image by examining the discontinuity in gradient orientation. Its most significant characteristic is that it focuses on only gradient orientation and neglects gradient magnitude, unlike most feature detection strategies. The concept of GOA is based on the fact that the gradient orientation is highly structured in the vicinity of a local feature. An important strength is that, unlike gradient magnitude, gradient orientation is insensitive to variations in intensity levels [27], [28]. We will discuss the properties of GOA through comparative studies with other closely related methods. In Chapter 3, we present an automated method for segmenting the teeth of the dental study model. The input data are given as wireframe models of the dentition obtained by digitizing dental plaster casts with a laser scanner. We generate a plan-view (top- 10 view) range image of the dentition from the wireframe model first. Then GOA is used for extracting dental features in the plan-view range image, which leads to automatic determination of the dental arch that describes the arrangement of the teeth. Using the dental arch as the reference, we generate another range image, i.e., a panoramic range image. Tooth segmentation is performed in the two range images separately, with the results from both images subsequently combined to obtain the locations and the orientations of the interstices between teeth. In Chapter 4, we present methods for extracting the skull, segmenting the mandible, and extracting the mandibular nerve canal in CT data. The extraction of the skull is achieved by thresholding, in which an appropriate threshold value is selected automatically according to an input data set. The mandible is often in contact with the maxilla at the teeth. The mandible is segmented by separating the tooth enamel of the upper and lower teeth. This is based on the fact that the biting surfaces of the teeth are made of enamel. In order to extract the mandibular nerve canal, we first generate a set of panoramic CT images, cross-sectional images along curved planes through the mandible. The 3-D extension of GOA is applied to the panoramic CT images for detecting tubular hollow spaces. We then extract the mandibular nerve canal using a novel 3-D line tracing technique. Finally, in Chapter 5, we give an overview of our achievements together with recommendations for future work. . 1 Chapter 1 Introduction 1. 1 Introduction This thesis presents methods for analyzing 3- D maxillofacial image data that are used for orthodontic 1 treatment and maxillofacial. advances in medical imaging technology have made the use of 3- D maxillofacial image data increasingly more common. 3- D data are obtained in two forms, surface-scan data and volumetric data. Surface-scan. 1. 1: Surface-scan data of a dental study model. CT provides detailed 3- D information of the interior of the human body. CT data is a volumetric data set used not only for orthodontic treatment