Image Processing: The Fundamentals Image Processing: The Fundamentals, Second Edition Maria Petrou and Costas Petrou © 2010JohnWiley&Sons,Ltd. ISBN: 978-0-470-74586-1 www.it-ebooks.info Image Processing: The Fundamentals Maria Petrou Costas Petrou A John Wiley and Sons, Ltd., Publication www.it-ebooks.info This edition first published 2010 c  2010 John Wiley & Sons Ltd Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com. The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congr ess Cataloging-in-Publication Data Petrou, Maria. Image processing : the fundamentals / Maria Petrou, Costas Petrou. – 2nd ed. p. cm. Includes bibliographical references and index. ISBN 978-0-470-74586-1 (cloth) 1. Image processing–Digital techniques. TA1637.P48 2010 621.36  7–dc22 2009053150 ISBN 978-0-470-74586-1 A catalogue record for this book is available from the British Library. Set in 10/12 Computer Modern by Laserwords Private Ltd, Chennai, India. Printed in Singapore by Markono www.it-ebooks.info This book is dedicated to our mother and grandmother Dionisia, for all her love and sacrifices. www.it-ebooks.info Contents Preface xxiii 1 Introduction 1 Whydoweprocessimages? 1 Whatisanimage? 1 Whatisadigitalimage? 1 Whatisaspectralband? 2 Why do most image processing algorithms refer to grey images, while most images wecomeacrossarecolourimages? 2 Howisadigitalimageformed? 3 If a sensor corresponds to a patch in the physical world, how come we can have more than one sensor type corresponding to the same patch of the scene? . . . . . 3 What is the physical meaning of the brightness of an image at a pixel position? . . 3 Why are images often quoted as being 512 × 512, 256 × 256, 128 × 128 etc? . . . . 6 Howmanybitsdoweneedtostoreanimage? 6 Whatdeterminesthequalityofanimage? 7 Whatmakesanimageblurred? 7 Whatismeantbyimageresolution? 7 Whatdoes“goodcontrast”mean? 10 Whatisthepurposeofimageprocessing? 11 Howdowedoimageprocessing? 11 Doweusenonlinearoperatorsinimageprocessing? 12 Whatisalinearoperator? 12 Howarelinearoperatorsdefined? 12 What is the relationship between the point spread function of an imaging device andthatofalinearoperator? 12 Howdoesalinearoperatortransformanimage? 12 Whatisthemeaningofthepointspreadfunction? 13 Box1.1.Theformaldefinitionofapointsourceinthecontinuousdomain 14 Howcanweexpressinpracticetheeffectofalinearoperatoronanimage? 18 Canweapplymorethanonelinearoperatorstoanimage? 22 Does the order by which we apply the linear operators make any difference to the result? 22 Box 1.2. Since matrix multiplication is not commutative, how come we can change theorderbywhichweapplyshiftinvariantlinearoperators? 22 vii www.it-ebooks.info viii Contents Box1.3.Whatisthestackingoperator? 29 What is the implication of the separability assumption on the structure of matrix H?38 Howcanaseparabletransformbewritteninmatrixform? 39 What is the meaning of the separability assumption? . . . 40 Box1.4.Theformalderivationoftheseparablematrixequation 41 Whatisthe“takehome”messageofthischapter? 43 What is the significance of equation (1.108) in linear image processing? . 43 Whatisthisbookabout? 44 2 Image Transformations 47 Whatisthischapterabout? 47 Howcanwedefineanelementaryimage? 47 Whatistheouterproductoftwovectors? 47 Howcanweexpandanimageintermsofvectorouterproducts? 47 How do we choose matrices h c and h r ? 49 Whatisaunitarymatrix? 50 Whatistheinverseofaunitarytransform? 50 Howcanweconstructaunitarymatrix? 50 How should we choose matrices U and V so that g can be represented by fewer bits than f? 50 What is matrix diagonalisation? 50 Can we diagonalise any matrix? . 50 2.1 Singular value decomposition 51 How can we diagonalise an image? . 51 Box2.1.Canweexpandinvectorouterproductsanyimage? 54 How can we compute matrices U, V and Λ 1 2 needed for image diagonalisation? . . 56 Box 2.2. What happens if the eigenvalues of matrix gg T arenegative? 56 Whatisthesingularvaluedecompositionofanimage? 60 Canweanalyseaneigenimageintoeigenimages? 61 HowcanweapproximateanimageusingSVD? 62 Box2.3.WhatistheintuitiveexplanationofSVD? 62 WhatistheerroroftheapproximationofanimagebySVD? 63 Howcanweminimisetheerrorofthereconstruction? 65 Are there any sets of elementary images in terms of which any image may be expanded? 72 Whatisacompleteandorthonormalsetoffunctions? 72 Arethereanycompletesetsoforthonormaldiscretevaluedfunctions? 73 2.2 Haar, Walsh and Hadamard transforms 74 HowaretheHaarfunctionsdefined? 74 HowaretheWalshfunctionsdefined? 74 Box 2.4. Definition of Walsh functions in terms of the Rademacher functions . . . 74 HowcanweusetheHaarorWalshfunctionstocreateimagebases? 75 How can we create the image transformation matrices from the Haar and Walsh functionsinpractice? 76 WhatdotheelementaryimagesoftheHaartransformlooklike? 80 Can we define an orthogonal matrix with entries only +1 or −1? 85 Box2.5.WaysoforderingtheWalshfunctions 86 WhatdothebasisimagesoftheHadamard/Walshtransformlooklike? 88 www.it-ebooks.info Contents ix What are the advantages and disadvantages of the Walsh and the Haar transforms? 92 WhatistheHaarwavelet? 93 2.3 Discrete Fourier transform 94 WhatisthediscreteversionoftheFouriertransform(DFT)? 94 Box2.6.WhatistheinversediscreteFouriertransform? 95 HowcanwewritethediscreteFouriertransforminamatrixform? 96 Is matrix U usedforDFTunitary? 99 Which are the elementary images in terms of which DFT expands an image? . . . 101 Why is the discrete Fourier transform more commonly used than the other transforms? 105 Whatdoestheconvolutiontheoremstate? 105 Box 2.7. If a function is the convolution of two other functions, what is the rela- tionshipofitsDFTwiththeDFTsofthetwofunctions? 105 HowcanwedisplaythediscreteFouriertransformofanimage? 112 What happens to the discrete Fourier transform of an image if the image isrotated? 113 What happens to the discrete Fourier transform of an image if the image isshifted? 114 What is the relationship between the average value of the image and its DFT? . . 118 WhathappenstotheDFTofanimageiftheimageisscaled? 119 Box2.8.WhatistheFastFourierTransform? 124 WhataretheadvantagesanddisadvantagesofDFT? 126 CanwehavearealvaluedDFT? 126 CanwehaveapurelyimaginaryDFT? 130 CananimagehaveapurelyrealorapurelyimaginaryvaluedDFT? 137 2.4 The even symmetric discrete cosine transform (EDCT) 138 Whatistheevensymmetricdiscretecosinetransform? 138 Box2.9.Derivationoftheinverse1Devendiscretecosinetransform 143 Whatistheinverse2Devencosinetransform? 145 What are the basis images in terms of which the even cosine transform expands an image? 146 2.5 The odd symmetric discrete cosine transform (ODCT) 149 Whatistheoddsymmetricdiscretecosinetransform? 149 Box2.10.Derivationoftheinverse1Dodddiscretecosinetransform 152 Whatistheinverse2Dodddiscretecosinetransform? 154 What are the basis images in terms of which the odd discrete cosine transform expandsanimage? 154 2.6 The even antisymmetric discrete sine transform (EDST) 157 Whatistheevenantisymmetricdiscretesinetransform? 157 Box2.11.Derivationoftheinverse1Devendiscretesinetransform 160 Whatistheinverse2Devensinetransform? 162 What are the basis images in terms of which the even sine transform expands an image? 163 What happens if we do not remove the mean of the image before we compute its EDST? 166 2.7 The odd antisymmetric discrete sine transform (ODST) 167 Whatistheoddantisymmetricdiscretesinetransform? 167 www.it-ebooks.info x Contents Box2.12.Derivationoftheinverse1Dodddiscretesinetransform 171 Whatistheinverse2Doddsinetransform? 172 What are the basis images in terms of which the odd sine transform expands an image? 173 Whatisthe“takehome”messageofthischapter? 176 3 Statistical Description of Images 177 Whatisthischapterabout? 177 Whydoweneedthestatisticaldescriptionofimages? 177 3.1 Random fields 178 Whatisarandomfield? 178 Whatisarandomvariable? 178 Whatisarandomexperiment? 178 Howdoweperformarandomexperimentwithcomputers? 178 Howdowedescriberandomvariables? 178 What is the probability of an event? . . . 179 Whatisthedistributionfunctionofarandomvariable? 180 What is the probability of a random variable taking a specific value? . 181 What is the probability density function of a random variable? 181 Howdowedescribemanyrandomvariables? 184 What relationships may n randomvariableshavewitheachother? 184 Howdowedefinearandomfield? 189 How can we relate two random variables that appear in the same random field? . . 190 How can we relate two random variables that belong to two different random fields? 193 If we have just one image from an ensemble of images, can we calculate expectation values? 195 Whenisarandomfieldhomogeneouswithrespecttothemean? 195 When is a random field homogeneous with respect to the autocorrelation function? 195 Howcanwecalculatethespatialstatisticsofarandomfield? 196 How do we compute the spatial autocorrelation function of an image in practice? . 196 Whenisarandomfieldergodicwithrespecttothemean? 197 When is a random field ergodic with respect to the autocorrelation function? . . . 197 Whatistheimplicationofergodicity? 199 Box 3.1. Ergodicity, fuzzy logic and probability theory 200 How can we construct a basis of elementary images appropriate for expressing in an optimalwayawholesetofimages? 200 3.2 Karhunen-Loeve transform 201 WhatistheKarhunen-Loevetransform? 201 Why does diagonalisation of the autocovariance matrix of a set of images define a desirablebasisforexpressingtheimagesintheset? 201 How can we transform an image so its autocovariance matrix becomes diagonal? . 204 What is the form of the ensemble autocorrelation matrix of a set of images, if the ensembleisstationarywithrespecttotheautocorrelation? 210 How do we go from the 1D autocorrelation function of the vector representation of animagetoits2Dautocorrelationmatrix? 211 How can we transform the image so that its autocorrelation matrix is diagonal? . . 213 www.it-ebooks.info Contents xi HowdowecomputetheK-Ltransformofanimageinpractice? 214 How do we compute the Karhunen-Loeve (K-L) transform of an ensemble of images? 215 Istheassumptionofergodicityrealistic? 215 Box 3.2. How can we calculate the spatial autocorrelation matrix of an image, when itisrepresentedbyavector? 215 Isthemeanofthetransformedimageexpectedtobereally0? 220 HowcanweapproximateanimageusingitsK-Ltransform? 220 What is the error with which we approximate an image when we truncate its K-L expansion? 220 What are the basis images in terms of which the Karhunen-Loeve transform expands animage? 221 Box 3.3. What is the error of the approximation of an image using the Karhunen- Loevetransform? 226 3.3 Independent component analysis 234 WhatisIndependentComponentAnalysis(ICA)? 234 Whatisthecocktailpartyproblem? 234 Howdowesolvethecocktailpartyproblem? 235 Whatdoesthecentrallimittheoremsay? 235 What do we mean by saying that “the samples of x 1 (t) are more Gaussianly dis- tributed than either s 1 (t)ors 2 (t)” in relation to the cocktail party problem? Are we talking about the temporal samples of x 1 (t), or are we talking about all possible versions of x 1 (t)atagiventime? 235 Howdowemeasurenon-Gaussianity? 239 Howarethemomentsofarandomvariablecomputed? 239 Howisthekurtosisdefined? 240 Howisnegentropydefined? 243 Howisentropydefined? 243 Box 3.4. From all probability density functions with the same variance, the Gaussian hasthemaximumentropy 246 Howisnegentropycomputed? 246 Box 3.5. Derivation of the approximation of negentropy in terms of moments . . . 252 Box3.6.Approximatingthenegentropywithnonquadraticfunctions 254 Box 3.7. Selecting the nonquadratic functions with which to approximate the ne- gentropy 257 How do we apply the central limit theorem to solve the cocktail party problem? . . 264 HowmayICAbeusedinimageprocessing? 264 Howdowesearchfortheindependentcomponents? 264 Howcanwewhitenthedata? 266 Howcanweselecttheindependentcomponentsfromwhiteneddata? 267 Box3.8.HowdoesthemethodofLagrangemultiplierswork? 268 Box3.9.Howcanwechooseadirectionthatmaximisesthenegentropy? 269 HowdoweperformICAinimageprocessinginpractice? 274 HowdoweapplyICAtosignalprocessing? 283 Whatarethemajorcharacteristicsofindependentcomponentanalysis? 289 What is the difference between ICA as applied in image and in signal processing? . 290 Whatisthe“takehome”messageofthischapter? 292 www.it-ebooks.info xii Contents 4 Image Enhancemen t 293 Whatisimageenhancement? 293 Howcanweenhanceanimage? 293 Whatislinearfiltering? 293 4.1 Elements of linear filter theory 294 Howdowedefinea2Dfilter? 294 How are the frequency response function and the unit sample response of the filter related? 294 Whyareweinterestedinthefilterfunctionintherealdomain? 294 Are there any conditions which h(k, l) must fulfil so that it can be used as a convo- lutionfilter? 294 Box 4.1. What is the unit sample response of the 2D ideal low pass filter? . . . . . 296 Whatistherelationshipbetweenthe1Dandthe2Dideallowpassfilters? 300 How can we implement in the real domain a filter that is infinite in extent? 301 Box 4.2. z-transforms 301 Canwedefineafilterdirectlyintherealdomainforconvenience? 309 Can we define a filter in the real domain, without side lobes in the frequency domain? 309 4.2 Reducing high frequency noise 311 Whatarethetypesofnoisepresentinanimage? 311 Whatisimpulsenoise? 311 WhatisGaussiannoise? 311 Whatisadditivenoise? 311 Whatismultiplicativenoise? 311 Whatishomogeneousnoise? 311 Whatiszero-meannoise? 312 Whatisbiasednoise? 312 Whatisindependentnoise? 312 Whatisuncorrelatednoise? 312 Whatiswhitenoise? 313 What is the relationship between zero-mean uncorrelated and white noise? . . . . 313 Whatisiidnoise? 313 Isitpossibletohavewhitenoisethatisnotiid? 315 Box 4.3. The probability density function of a function of a random variable . . . 320 Whyisnoiseusuallyassociatedwithhighfrequencies? 324 Howdowedealwithmultiplicativenoise? 325 Box4.4.TheFouriertransformofthedeltafunction 325 Box4.5.Wiener-Khinchinetheorem 325 IstheassumptionofGaussiannoiseinanimagejustified? 326 Howdoweremoveshotnoise? 326 Whatisarankorderfilter? 326 Whatismedianfiltering? 326 Whatismodefiltering? 328 HowdowereduceGaussiannoise? 328 Canwehaveweightedmedianandmodefilterslikewehaveweightedmeanfilters? 333 CanwefilteranimagebyusingthelinearmethodswelearntinChapter2? 335 Howdowedealwithmixednoiseinimages? 337 www.it-ebooks.info [...]... blurring This is called image restoration • To make explicit certain characteristics of the image which can be used to identify the contents of the image This is called feature extraction How do we do image processing? We perform image processing by using image transformations Image transformations are performed using operators An operator takes as input an image and produces another image In this book... 387 393 5 Image Restoration 395 What is image restoration? 395 Why may an image require restoration? 395 What is image registration? 395 How is image restoration performed? 395 What is the difference between image enhancement and image restoration? 395 5.1 Homogeneous linear image restoration:... not calibrated What is the purpose of image processing? Image processing has multiple purposes • To improve the quality of an image in a subjective way, usually by increasing its contrast This is called image enhancement • To use as few bits as possible to represent the image, with minimum deterioration in its quality This is called image compression • To improve an image in an objective way, for example... the brightness of the image in the location of the sensor and in the band of the sensor This figure shows the sensitivity curves of three different sensor types Why do most image processing algorithms refer to grey images, while most images we come across are colour images? For various reasons 1 A lot of the processes we apply to a grey image can be easily extended to a colour image by applying them... Introduction 7 What determines the quality of an image? The quality of an image is a complicated concept, largely subjective and very much application dependent Basically, an image is of good quality if it is not noisy and (1) it is not blurred; (2) it has high resolution; (3) it has good contrast What makes an image blurred? Image blurring is caused by incorrect image capturing conditions For example, out... multispectral images? 759 How do we rank vectors? 760 How do we deal with mixed noise in multispectral images? 760 How do we enhance a colour image? 761 How do we restore multispectral images? 767 How do we compress colour images? 767 How do we segment multispectral images?... that are special to multispectral images? 669 What is this chapter about? 670 7.1 Image preprocessing for multispectral images 671 Why may one wish to replace the bands of a multispectral image with other bands? 671 How do we usually construct a grey image from a multispectral image? 671 How can we construct... chapter? 526 6 Image Segmentation and Edge Detection 527 What is this chapter about? 527 What exactly is the purpose of image segmentation and edge detection? 527 6.1 Image segmentation 528 How can we divide an image into uniform regions? 528 What do we mean by “labelling” an image? ... techniques were developed around the type of image that was available These techniques have been well established in image processing Nevertheless, colour is an important property of the natural world, and so we shall examine its role in image processing in a separate chapter in this book www.it-ebooks.info 1 Introduction 3 How is a digital image formed? Each pixel of an image corresponds to a part of a physical... the image is a power of 2 We shall see some examples in Chapter 2 How many bits do we need to store an image? The number of bits, b, we need to store an image of size N × N , with 2m grey levels, is: b=N ×N ×m (1.6) So, for a typical 512 × 512 image with 256 grey levels (m = 8) we need 2,097,152 bits or 262,144 8-bit bytes That is why we often try to reduce m and N , without significant loss of image . 1 Whydoweprocessimages? 1 Whatisanimage? 1 Whatisadigitalimage? 1 Whatisaspectralband? 2 Why do most image processing algorithms refer to grey images, while most images wecomeacrossarecolourimages? 2 Howisadigitalimageformed?. 387 Whatisthe“takehome”messageofthischapter? 393 5 Image Restoration 395 Whatisimagerestoration? 395 Whymayanimagerequirerestoration? 395 Whatisimageregistration? 395 Howisimagerestorationperformed? 395 Whatisthedifferencebetweenimageenhancementandimagerestoration?. 56 Whatisthesingularvaluedecompositionofanimage? 60 Canweanalyseaneigenimageintoeigenimages? 61 HowcanweapproximateanimageusingSVD? 62 Box2.3.WhatistheintuitiveexplanationofSVD? 62 WhatistheerroroftheapproximationofanimagebySVD?