EXTRACTION OF MAN-MADE FEATURES FROM HIGH RESOLUTION SATELLITE IMAGERY SOWMYA SELVARAJAN (B.E., Anna University, India) A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF ENGINEERING DEPARTMENT OF CIVIL ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2002 ACKNOWLEDGEMENTS The author would like to thank her supervisor, Associate Professor Chan Weng Tat for initiating and encouraging her interest in application-based remote sensing and for providing advice and direction The author is also thankful to her parents and all her friends who have been providing moral support all along the way i TABLE OF CONTENTS Acknowledgements i Table of Contents ii List of Figures vi List of Tables ix List of Acronyms x Chapter Chapter Introduction 1.1 Background of the research 1.2 Scope 1.3 Objective of the study 1.4 Organization of the thesis Literature Review 2.1 Introduction 2.2 Satellite remote sensing of urban areas 2.2 Wavelet approaches to image processing 2.3 Current edge detection methods 11 2.4 Template matching methods 14 2.5 Summary 16 ii Chapter Existing methods and underlying theories for urban remote sensing 18 3.1 Urban remote sensing 18 3.1.1 Fundamentals of remote sensing 18 3.1.2 Extraction of information through digital image processing 3.2 3.3 19 Wavelet Theory 21 3.2.1 Introduction to wavelets 21 3.2.2 Comparison of Fourier and wavelet transforms 23 3.2.3 Discrete Wavelet Transform 26 3.2.4 Multi-resolution Wavelet Analysis 27 Edge detection approach 29 3.3.1 Purpose of edge enhancement 29 3.3.2 Types of edge operators 29 3.3.3 The Canny edge algorithm 30 3.4 Template matching 3.5 Proposed wavelet-edge template matching technique for man-made object extraction 32 33 3.5.1 Detection of local intensity variation 34 3.5.2 Edge based segmentation approach 36 3.5.3 Shape classification 37 iii Chapter Details of implementation 38 4.1 Focus of the study 38 4.2 Imagery and Study Area 38 4.3 Parameters for wavelet analysis 41 4.3.1 Choice of wavelet type 41 4.3.2 Approximations and details of the wavelet analysis 42 4.3.3 Wavelet resolution 4.4 Chapter 44 Parameters for Canny edge detection 46 4.4.1 Threshold parameter 47 4.4.2 Standard Deviation 51 4.5 Morphological operations 54 4.6 Parameters for template matching 55 4.5.1 Shape models 55 4.5.2 Block Matching 56 4.5.3 Edge-based Matching 58 Results and discussions 61 5.1 Results of feature extraction 61 5.2 Assessment of accuracy 64 5.3 Key features of the proposed method 66 5.4 Validation of the algorithm 73 iv Chapter Conclusions 79 6.1 Conclusions 79 6.2 Future Improvements 81 References 82 Appendix 87 v LIST OF FIGURES Fig 3.1 Fourier basis functions, time-frequency tiles, and coverage of the time-frequency plane Fig 3.2 26 Daubechies wavelet basis functions, time-frequency tiles, and coverage of the time-frequency plane 27 Fig 3.3 Flow chart depicting the proposed algorithm 34 Fig 4.1 IKONOS 1m PAN imagery 41 Fig 4.2 Training site 42 Fig 4.3 The Daubechies family 43 Fig 4.4.a Approximation Xa1 44 Fig 4.4.b Horizontal Detail Xh 44 Fig 4.4.c Vertical Detail Xv1 45 Fig 4.4.d Diagonal Detail Xd1 45 Fig 4.5.a Level image 46 Fig 4.5.b Edges detected at level 46 Fig 4.6.a Level image 46 Fig 4.6.b Edges detected at level 46 Fig 4.7.a Level image 47 Fig 4.7.b Edges detected at level 47 Fig 4.8 Decomposed 2nd level Wavelet image 49 Fig 4.9 Edge map at 0.25 threshold 49 vi Fig 4.10 Edge map from Canny edge algorithm (threshold = 0.2) 50 Fig 4.11 Edge map from Canny edge algorithm (threshold = 0.01) 50 Fig 4.12 Edge map from Canny edge algorithm (threshold = 0.07) 51 Fig 4.13 Threshold plot 51 Fig 4.14 Edge map (threshold = 0.13) 52 Fig 4.15 Edge map (standard deviation = 4) 52 Fig 4.16 Edge map (standard deviation = 2) 53 Fig 4.17 Edge map (standard deviation = 0.2) 53 Fig 4.18 Edge map (standard deviation = 0.05) 54 Fig 4.19 Standard deviation plot 54 Fig 4.20 Edge map 55 Fig 4.21 Final edge map 56 Fig 4.22 Block template matching (accuracy parameter: 92%) 58 Fig 4.23 Block template matching (accuracy parameter: 75%) 58 Fig 4.24 Edge template matching (accuracy parameter: 95%) 59 Fig 4.25 Edge template matching (accuracy parameter: 90%) 60 Fig 4.26 Edge template matching (accuracy parameter: 80%) 60 Fig 4.27 Edge template matching (accuracy parameter: 80%) 61 Fig 5.1 Training Site 62 Fig 5.2 Schematic map of the training site with house plots 63 Fig 5.3 Block Template Matching 64 Fig 5.4 Edge template matching 64 Fig 5.5 Prewitt Edge Detector 68 Fig 5.6 Sobel Edge Detector 68 vii Fig 5.7 Zero-crossing Edge Detector 69 Fig 5.8 Canny Edge Detector without wavelet analysis 69 Fig 5.9 Canny Edge Detector with wavelet analysis 70 Fig 5.10 Sobel Edge Map Fig 5.11 Wavelets + Sobel Edge Map 72 Fig 5.12 Sobel Edge Detector + Template Matching 73 Fig 5.13 Wavelets + Sobel Edge Detector + Template Matching 73 Fig 5.14 Canny Edge Detector + Template Matching 74 Fig 5.15 Experiment Site I 75 Fig 5.16 Edge Map of Experiment Site I 75 Fig 5.17 Final Map of Experiment Site I 75 Fig 5.18 Experiment Site II 76 Fig 5.19 Schematic map of Experiment Site II 77 Fig 5.20 Edge Map of Experiment Site II 78 Fig 5.21 Final Map of Experiment Site II 79 viii LIST OF TABLES Table 5.1 Accuracy assessment of buildings 65 Table 5.2 Comparison between Canny & Sobel edge detectors 71 ix Chapter Results and Discussions here the buildings are tall and its facets are also seen Fig 5.19 shows the setup of the tall buildings in a map These are categorized under the private housing section Each building has 14 floors Fig 18 Experiment Site Patch of tall buildings 75 Chapter Results and Discussions MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN RD RD RD AMP Singapore Geylang Methodist Pr Sch MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN RD RD RD MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN RD RD RD AMBER AMBER AMBER RD RD RD RD MOUNTBATTEN MOUNTBATTEN MOUNTBATTEN RD RD TG KATONG RD MOUNTBATTEN RD AMBER AMBER AMBER AMBER RD RD RD RD D1 AMBER AMBER AMBER GDNS GDNS GDNS TG TG KATONG RD TG KATONG KATONG RD RD SOUTH SOUTH SOUTH AMBER AMBER AMBER RD RD RD Fig 19 Setup of the Experiment Site as seen on a map 76 Chapter Results and Discussions The resulting edge map is seen in fig 5.20 The next problem was choosing a template shape and size for this patch Due to the problem of un-ortho-rectified imagery, resulting in the non-rectification of the relief displacement, the templates to be chosen are different from the previous examples After many attempts the templates were chosen and applied on the scene The resulting image is shown in fig 5.21 It shows that all buildings were identified along with one more template matched The error was due to a half building seen in the scene Since its face is also in a similar shape, the technique identified the face also as a building The sixth building (half part of the tall building) was introduced to see the algorithm’s affect and as expected, it identified the sixth building also The other small building seen in the scene is not extracted, due to the template size Fig 20 Edge Map 77 Chapter Results and Discussions Fig 21 Final Map Concluding all the above experiments, it is noted that the proposed WavEDIS and template matching algorithm has successfully extracted the buildings Further conclusions are discussed in detail in the next chapter 78 Chapter Conclusions CHAPTER CONCLUSIONS 6.1 Conclusions An algorithm for finding building outlines in high resolution satellite imagery such as IKONOS was implemented and tested on an urban area The proposed technique can be used to detect objects in the imagery in semi-automated way The application of wavelet based edge detection and template matching was successfully demonstrated on building detection The application of wavelets brings about a better change in the recognition of the characteristics of the edges A possible explanation is that the wavelets try to localize the signals which in turn tend to decrease the number of spurious edges In most cases, the spurious edges are those which help to connect the weak and strong edges Comparing the results of the Sobel edge detector and the Canny edge detector, the latter produced much better results than the former The Sobel method finds edges using the Sobel approximation to the derivative and it returns edges at those points where the gradient of image is the maximum The Sobel edge detection algorithms convolves masks with the image to detect both the horizontal and vertical edge components; the resulting outputs are simply added to give a gradient map The Canny method finds edges by looking for local maxima of the gradient of the image The gradient is calculated using the derivative of a Gaussian filter The method uses two thresholds, to detect strong and weak edges, and 79 Chapter Conclusions includes the weak edges in the output only if they are connected to strong edges This method is therefore less likely to be "fooled" by noise, and more likely to detect true weak edges There are also a few shortcomings to be noted The imagery is an un-orthorectified imagery, which results in relief displacement for high rise buildings and other tall features Simpler template shapes can be used on low rise buildings More complicated shapes are used for tall condominiums and Housing Development Board (HDB) flats if orthorectified images are not used If such images are available, then simpler shapes can be used Automating the entire image analysis process is a complex problem In the tests conducted, the main cause for the false figures was the edge detection algorithm Though the edge detector used was one of the best available, a few edges were still missed Although the number of edges missed may be small, the effect could be significant This error was compromised by using templates for matching objects It can be concluded that the proposed edge-based recognition of man-made features will produce better results if combined with region-based techniques The computational time of the algorithm is quite lengthy, due to the number of operations involved and the computing equipment used However there is much opportunity for parallelism to be exploited and this could make the computation much shorter With the increased availability of high resolution imagery, better resolutions than 80 Chapter Conclusions 1-meter will soon be abundantly available Since the proposed algorithm can detect buildings even at coarser resolution, it should be able to recognize additional details within buildings and other man-made features when using higher-resolution imagery As the study is a preliminary one, the results of the present study cannot be generalized due to the small training area But the results are definitely promising and pave way for future studies 6.2 Future Improvements Future improvements to make the algorithm more useful and practical include: Combining edge maps The edge detection algorithm can be further modified as to combine edge maps from different threshold hystersis This will enable to accommodate edges from different thresholds and lead to a better edge map than from a single threshold Using more template shapes and sizes In the shape classification technique, more template shapes and sizes can be added to the database This would allow for a wider range of building object matches Improvising on the above areas will make the project a powerful and practically useful algorithm to detect building features from high spatial resolution imagery 81 References REFERENCES Abyoto Kun Wardana, Wirdjosoedirdjo Sri Jatno, Tadashi Watanabe, ‘Unsupervised Segmentation using Multiresolution Analysis for Feature Extraction’ Bergholm, F., ‘Edge Focusing’, IEEE Trans Pattern Analysis and Machine Intelligence, vol 9, 1987, pp 726-741 Canny, J., ‘A Computational Approach to Edge Detection’, IEEE Trans Pattern Analysis and Machine Intelligence, vol 8, no 6, 1986, pp 679-678 Clark, J J., ‘Authenticating Edges Produced by Zero Crossing Algorithms’, IEEE Trans Pattern and Analysis and Machine Intelligence, vol 11, no 1, 1989, pp 4357 Daubechies, I., ‘Ten Lectures on Wavelets’, SIAM, 1992 Draper, B.A., ‘Modeling object recognition as a Markov decision process’, Proceedings of the 13th International Conference on Pattern Recognition, 1996, vol 4, pp 95 -99 Dunn, D., Higgin, W.E., Wakeley, J., ‘Texture Segmentation Using 2-D Gabor Elementary Functions’, IEEE Trans on Pattern Analysis and Machine Intelligence, Feb.1994, vol.16, no.2, pp.130-149 Elder, J H., Zucker, S W, ‘Local Scale Control for Edge Detection and Blur Estimation’, IEEE Trans Pattern Analysis and Machine Intelligence, vol 20, no 7, 1998, pp 699-716 82 References Fatemi-Ghomi, N., Palmer, P L., Petrou, M., ‘Performance Evaluation of Texture Segmentation Algorithms based on Wavelets’, Paper presented at the Workshop on Performance Characteristics of Vision Algorithms, Cambridge, April 19, 1996 10 Friha, S., Boulemden, M., ‘Meteorological Image Processing with Wavelets’, System Theory, 1998 Proceedings of the Thirtieth Southeastern Symposium, 1998, pp 208 -212 11 Goshtasby, A, Shyu, H-L, ‘Edge Detection by Curve Fitting’, Image and Vision Computing, vol 13, no 3, 1995, pp 169-177 12 Haralick, R., ‘Digital Step Edges from Zero Crossing of Second Directional Derivatives’, IEEE Trans Pattern Analysis and Machine Intelligence, vol 6, 1984, pp 58-68 13 Jawerth, B., Sweldens, W., ‘An overview of wavelet based multiresolution analyses’, SIAM Review, 36 (1994), pp 377-412 14 Jothimani, P., ‘Operational Urban Sprawl Monitoring using Satellite Remote Sensing: Excerpts from the Studies of Ahmedabad, Vadodara and Surat, India’ – Proceedings of the 18th Asian Conference on Remote Sensing 15 Karras, D.A.; Karkanis, S.A.; Mertzios, B.G., ‘A classification approach to image structure segmentation based on the wavelet transform’, EUROMICRO 97 Proceedings of the 23rd Euromicro Conference on 'New Frontiers of Information Technology' Short Contributions, 1997, pp 56 -59 16 Li Deren, Shao Juliang, ‘The Wavelet and its Application in Image Edge Detection’, ISPRS Journal of Photogrammetry and Remote Sensing, 1994, vol 49, no 3, pp 4-11 83 References 17 Li, S Z., ‘Closed-Form Solution and Parameter Selection for Convex Minimization-Based Edge Preserving Smoothing’, IEEE Trans Pattern Analysis and Machine Intelligence, vol 20, no 9, 1998, pp 916-932 18 Lijun Ding, Goshtasby Ardeshir, ‘On the Canny edge detector’, Pattern Recognition, vol 34 issue (3) March 2001 pp 721-725 19 Luiz Alberto Lisoba da Silva Cardoso, ‘Computer Aided Recognition of Man-made Structures in Aerial Photography’, Master’s Thesis, Naval Postgraduate School, Monterey, CA 1999 20 Mah Abdullam, ‘Urban planning and monitoring changes using ER Mapper’ – Proceedings of the 21st Asian Conference on Remote Sensing 21 Mallat, S., Zhong, S., ‘Characterization of Signals from Multiscale Edges’, IEEE Trans on Pattern Analysis and Machine Intelligence, July 1992, vol.14, no.7, pp.710-732 22 Mandal, D.P.; Murthy, C.A.; Pal, S.K., ‘Analysis of IRS imagery for detecting man-made objects with a multivalued recognition system’, IEEE Transactions on Systems, Man and Cybernetics, Part A, March 1996, vol 26, issue 2, pp 241 247 23 Manjunath, B.S., Ma, W.Y., ‘Texture Features for Browsing and Retrieval of Image Data’, IEEE Trans on Pattern Analysis and Machine Intelligence, Aug.1996, vol.18, no.8, pp.837-842 24 Marr, D., Hildreth, E., ‘Theory of Edge Detection’, Proc Royal Society of London, vol B-207, 1980, pp 187-217 84 References 25 Mueller, M and Segl, K., ‘Object recognition based on high spatial resolution panchromatic satellite imagery’, Joint workshop of ISPRS on Sensors and Mapping from Space September 1999, 27-30, Hannover, Proceedings as electronic publication 26 Pereira, M.S.; Manolakos, E.S ‘Hierarchical neural network for multiresolution image analysis’, International Conference on Image Processing, 1996, Proceedings, 1996, vol.1, pp 261 -264 27 Rifaat, R., Kinsner, W., ‘Experiments with Wavelet and other Edge Detection Techniques’, 1997 Conference on Communications, Power and Computing Proceedings, Winnipeg, MB, May 22-23, 1997, pp 322-326 28 Rowe Neil C., Grewe Lynne L., ‘Change Detection for Linear Features in Aerial Photographs Using Edge-Finding’, IEEE Transactions On Geoscience And Remote Sensing, vol 39, no 7, July 2001, pp 1608-1612 29 Rusyn, B., ‘The investigation of singling algorithms for small-dimensioned object recognition for remote Earth sensing’, 6th International Conference on Mathematical Methods in Electromagnetic Theory, 1996, pp 136 -138 30 Mallat, S.G., ‘A Theory for Multiresolution Signal Decomposition: The Wavelet Representation’, IEEE Trans on Pattern Analysis and Machine Intelligence, July 1989, vol.11, no.7, pp.674-693 31 Shi, Z.; Shibasaki, R., ‘An approach to image segmentation using multiresolution analysis of wavelets’, SMC '99 Conference Proceedings, IEEE International Conference on Systems, Man, and Cybernetics, 1999, vol 6, pp 810 -815 85 References 32 Siddique, J.I.; Barner, K.E, ‘Wavelet-based multiresolution edge detection utilizing gray level edge maps’, Image Processing, ICIP 98 Proceedings 1998 International Conference, vol 2, pp 550-554 33 Singh, S.; Markou, M.; Haddon, J., ‘FLIR image segmentation and natural object classification’, Pattern Recognition, 2000 Proceedings 15th International Conference, vol 1, pp 681 -684 34 Ulupinar, F., Medioni, G., ‘Redefining Edges Detected by a LoG Operator’, Computer Vision, Graphics, and Image Processing, vol 51, 1990, pp 275-298 35 Wu Xiuqing; Zhou Rong; Xu Yunxiang, ‘A method of wavelet-based edge detection with data fusion for multiple images’, Intelligent Control and Automation, Proceedings of the 3rd World Congress, vol 4, 2000, pp 2691 2694 36 Xiuwen Liu; Ke Chen; DeLiang Wang, ‘Extraction of hydrographic regions from remote sensing images using an oscillator network with weight adaptation’, IEEE Transactions on Geoscience and Remote Sensing, vol 39, issue 1, Jan 2001, pp 207 -211 37 Yingcheng Li, Liangcal Chu, Tongying Guo, Yanli Xue, Xueyou Li, Xiaobo Ding, Xiaolong Liu, ‘Landuse Change and Urban Growing Monitoring in China’, Proceedings of the 20th Asian Conference on Remote Sensing 38 HongJiang Zhang, ‘Automatic Tracking Ice Floe from Satellite Imagery Abstract via Invariant Moment Matching’, IGARSS '92 International Geoscience and Remote Sensing Symposium, 1992, pp 582 -584 86 Appendix APPENDIX Orthonormal compactly supported wavelets A brief description of orthonormal, compactly supported wavelet bases is discussed here; detailed information can be found, for example, in Daubechies [38] and Jawerth and Sweldens [39] An orthonormal, compactly supported wavelet basis of dilation and translation of a single function is formed by the , called the wavelet function: where is the set of integers In equation (1), the function has moments up to order , and it satisfies the following ``two-scale'' difference equation, The wavelet function has a companion, the scaling function forms a set of orthonormal bases of The scaling function vanishing , which also , satisfies, 87 Appendix and the ``two-scale difference'' equation, In equations (2) and (5), two coefficient sets and have the same finite length for a certain basis, where is related to the number of vanishing moments For example, equals representation of signals, in in the Daubechies wavelets In the wavelet behaves as a low-pass filter and behaves as a high-pass filter to signals These two filters are related by Dilated wavelets are related by a scaling equation Rescaling can be interpreted as discrete filtering Vanishing moments, support, regularity and symmetry of the wavelet and scaling function are determined by the scaling filter A wavelet has m vanishing moments if and only if its scaling function can generate polynomials of degree smaller than or equal to M While this property is used to describe the approximating power of scaling functions, in the wavelet case it has a "dual" usage, e.g the possibility to characterize the order of isolated singularities The number of vanishing moments is entirely determined by the coefficients h[n] of the filter h which is featured in the scaling equation 88 Appendix The scaling function is compactly supported if and only if the filter h has a finite support, and their supports are the same If the support of the scaling function is [N1, N2], then the wavelet support is [(N1-N2+1)/2,(N2-N1+1)2] Daubechies has proved that, to generate an orthogonal wavelet with p vanishing moment, a filter h with minimum length 2p had to be used Daubechies filters, which generate Daubechies wavelets, have a length of 2p 89 ... of remote sensing, the rapid development of technology and the opening of the image market have made high spatial resolution data available The new high spatial resolution satellite imagery from. .. many resolutions were experimented As the resolution decreases, the contrast between the features also decreases It is found from the study that for the extraction of man- made features, the high. .. the feasibility of auto-detection of manmade features like buildings in the local context using recently available high resolution Chapter Introduction IKONOS imagery Satellite imagery and aerial