Image Processing: Fundamentals.Maria Petrou and Panagiota Bosdogianni The Copyright 1999 John Wiley & Sons Ltd Print ISBN 0-471-99883-4 Electronic ISBN 0-470-84190-7 Chapter Statistical Description of lrnages T What is this chapter about? This chapter provides the necessarybackground for the statistical description of images from the signal processing point of view Why we need the statistical description of images? In various applications, we often haveto deal with sets of images of a certain type; for example,X-rayimages, traffic sceneimages,etc.Eachimage in the set may be different from all the others, but at the same time all images may share certain common characteristics We need the statistical description of images so that we capture these common characteristics and use them in order to represent an image with fewer bits and reconstruct it with the minimum error “on average” The first idea is then to try to minimize the mean square error in the reconstruction of the image, if the same image or a collection of similar images were to be transmitted and reconstructed several times, as opposed to minimizing the square error of each image separately The second idea is that the data with which we would like to r present the image must be uncorrelated Both these ideas lead to the statistical description of images Is there an image transformationthat allows its representation in terms of uncorrelated data that can be used to approximate the image in the least mean square error sense? Yes It is called Karhunen-Loeve or Hotelling transform It is derived by treating the image as an instantiation of a random field 90 Image Processing: The Fundamentals What is a random field? A random field is a spatial function that assigns a random variable at each spatial position What is a random variable? A random variable is the value we assign to the outcome of a random experiment How we describe random variables? Random variables are described in terms of their distribution functions which in turn are defined in terms of the probability of an event happening An event is a collection of outcomes of the random experiment What is the probability of an event? The probability of an event happening is a non-negative number which has the following properties: (A) The probability of the event which includes all possible outcomes of the experiment is (B) The probability of two events which not have any common outcomes is the sum of the probabilities of the two events separately What is the distribution function of a random variable? The distribution function of a random variable f is a function which tells us how likely it is for f to be less than the argument of the function: function of f variable Clearly, Pf(-co) = and Pf(+co)= Example 3.1 If 21 2 , show that P f ( z )5 Pf(22) Suppose that A is the event (i.e the set of outcomes) which makes f z1 and B is the event which makes f 2 Since z1 z2, A B + B = ( B - A ) U A; i.e the events ( B - A) and A not have common outcomes (see the figure on the next page) c Statistical Description of Images 91 fd Zld z2 f6Z2 Then by property (B) in the definition of the probability of an event: - ?(B) = P ( B - A ) + P ( A ) + Pf(z2) = '?'(B - A ) Pf(z1)+ + non-negative number Example 3.2 Show that: According to the notation of Example 3.1, z1 f z2 when the outcome of the random experiment belongs t o B - A (the shaded area in the above figure); i.e P(z1 f I z2) = P f ( B - A ) Since B = ( B - A ) U A , P f ( B - A )= P f ( B ) - P f ( A ) and the result follows What is the probability of a random variable taking a specific value? If the randomvariable takes values from the setof real numbers, it has probability zero of taking a specific value (This can be seen if in the result of example 3.2 we set f = z1 = 2 ) However, it may have non-zero probability of taking a value within an infinitesimally small range of values This is expressed by its probability density function 92 Image Processing: The Fundamentals What is the probability density function of a random variable? The derivative of the distributionfunction of a random variable is called the probability density functionof the random variable: The expected or mean value of the random variable f is defined by: m (3.4) and the variance by: The standard deviation is the positive square root of the variance, i.e af How we describe many random variables? If we have n random variables we can define their joint distribution function: Pflf2 (z1,22, ,zn) fn = P{fl I Zl,f2 z2, .,fn I Zn} (3.6) We can also define their joint probability density function: What relationships may n random variables have with each other? If the distribution of n random variables can be written as: Pflf2 f (21, n z2, ., zn) = Pfl (Z1)Pfz (z2) Pfn (Zn) (3.8) then these random variables are called independent They are called uncorrelated if E{fifj) = E{fi)E{fj>,K j ,i # j (3.9) Any two random variables are orthogonal to each other if E{fifj) = (3.10) The covariance of any two random variables is defined as: cij = E{(fi - Pfi)(fj - P f j H (3.11) Description Statistical of Images 93 Example 3 Show that if the covariance cij of two random variables is zero, the two variables are uncorrelated Expanding the right hand side of the definition of the covariance we get: + Pfi Pfj } E U i f j ) - PfiE{fj) - PfjE{fi) + PfiPfj E U i f j ) - PfiPfj - PfjPfi + PfiPfj = E1fi.t-j - P f i f j - Pfj fi cij = = = E U i f j ) - PfiPfj (3.12) Notice that the operation of taking the expectation value of a fixed number has no effect on it; i.e E { p f i ) = p f i If cij = 0, we get: E t f i f j ) = PfiPfj = E { f i ) E { f j > which shows that fi and fj (3.13) are uncorrelated How we then define a random field? If we define a random variable at every point in a 2-dimensional space we say that we have a 2-dimensional random field The position of the space where a random variable is defined is like a parameter of the random field: f (r;W i ) (3.14) This function for fixed r is a random variable but for fixed wi (outcome) is a 2dimensional function in the plane, animage, say As wi scans all possible outcomes of the underlying statistical experiment, the random field represents a series of images On the other hand, for a given outcome, (fixed w i ) , the random field gives the grey level values at the various positions in an image Example 3.4 Using an unloaded die, we conducted a series of experiments Each experiment consisted of throwing the die four times The outcomes { w l , w2, w3, wq) of sixteen experiments aregiven below: 94 Image Processing: The Fundamentals If r is a 2-dimensional vector taking values: 1(1,1>7 (1,217 (1,317 (1,417 (2,117 (2,217 31, (2741, (27 (37 l),(37 2), (373), (374),(47 l),(47 2), (4,3)7 (4,4)1 give the series of images defined by the random field f (r;w i ) The first image is formed by placing the first outcome of each experiment in the by of corresponding position, the second using the second outcome each experiment, and so on The ensemble of images we obtain is: (2 3 3 1 1 5 2 ( 2 ) (3 ) 1 5 6 4 (l ) 2 6 56 ) (3‘15) How can we relate two random variables that appear in the same random field? For fixedr a random field becomes a random variable with an expectation value which depends on r: (3.16) Since for different values of r we have different random variables, f ( r l ;w i ) and f (1-2;w i ) , we can define their correlation, called autocorrelation (we use “auto” because the two variables come from the same random field) as: s_,L +m Rff(r1, r2) = E ( f (r1;w2)f (r2;W i ) } = +m Zlz2pf(z17 z2; r l , r2)dzldz2 (3.17) The autocovariance C(rl,r2) is defined by: =E{[f(rl;wi) -pf(rl)l[f(r2;Wi) -pf(r2)l} cff(rl?r2) (3.18) 95 Statistical Description of Images Example 3.5 Show that: Starting from equation (3.18): How can we relate tworandomvariables random fields? that belong to twodifferent If we have two random fields, i.e two series of images generated by two different underlying random experiments, represented by f and g , we can define their cross correlation: and their cross covaraance: Two random fields are called uncorrelated if for any rl and This is equivalent to: r2: Processing: Image 96 The Fundamentals Example 3.6 Show that for two uncorrelated random fields we have: E{f(rl;Wi)g(r2;wj)) E { f ( r l ; w i > > E { g ( r ; w j ) ) = It follows trivially from the definition ( C f g ( r l , = ) and the expression: r2) of uncorrelated random Cfg(rl,r2)= E{f(rl;wi)g(ra;wj)) C L f h ) C L & ) - fields (3.24) which can be proven in a similar way as Example 3.5 Since we always have just one version of an image how we calculate the expectation values that appear in all previous definitions? We make the assumption that the image we have is a homogeneous random field and ergodic The theorem of ergodicity which we then invoke allows us to replace the ensemble statistics with the spatial statistics of an image When is a random field homogeneous? If the expectation value of a random field does not depend on r, and if its autocorrelation function is translation invariant, then the field is called homogeneous A translation invariant autocorrelation function depends on only one argument, the relative shifting of the positions at which we calculate the values of the random field: Example Show thattheautocorrelation function R(rl,r2) of a homogeneous random field depends only on the difference vector rl - r2 random field is translation The autocorrelation function of a homogeneous invariant Therefore, for any translation vector ro we can write: + R f f ( r l , r ) E{f(rl;wi)f(r2;Wi)) E{f(r1+ ro;wi)f(r2 ro;wi)) = = = Rff(r1 ro, r2 ro) Qro (3.26) + + 97 Statistical Description of Images How can we calculate the spatial statistics of a random field? Given a random field we can define its spatial average as: Jo mO , S, f (r;wi)dxdy (3.28) ss is the integral over the whole space S with area S and r = ( X , y) The result p(wi) is clearly a function of the outcome on which f depends; i.e p(wi) is a random where variable The spatial autocorrelation function of the random field is defined as: (3.29) This is another random variable When is a random field ergodic? A random field is ergodic when it is ergodic with respectto themean and with respect to the autocorrelation function When is a random field ergodic with respect to the mean? A random field is said to be ergodic with respect to the mean, if it is homogeneous and its spatial average, defined by (3.28), is independent of the outcome on which f depends; i.e it is a constant and is equal to theensemble average defined by equation (3.16): f (r;wi)dxdy = p = a constant When is a random field function? (3.30) ergodic with respect to the autocorrelation A random field is said to be ergodic with respect to the autocorrelation function if it is homogeneous and its spatial autocorrelation function, defined by (3.29), is independent of the outcome of the experiment on which f depends, and depends 98 Image Processing: The Fundamentals only on the displacement ro, and it is equal to the ensemble autocorrelationfunction defined by equation (3.25): E { f ( r ;w i ) f ( r 'S + ro; Q)} = s+ms lim - f(r;wi)f(r S + ro; wi)dxdy = R(r0) (3.31) Example 3.8 Assuming ergodicity, following image: compute the autocorrelation matrix of the A X image has the form: (3.32) To compute its autocorrelation function we writeit as a column vector by stacking its columns one under the other: g = (911 g12 g23g22g21 g13 g31 g32 g33 ) T (3.33) The autocorrelation matrix is given by: C = E{ggT} Instead of averaging Over all possible versions of the image, we average over all pairs ofpixels at the Same relative position in the image since ergodicity is assumed Thus, the autocorrelation matrix will have the following structure: g g g g g g g g32 g22 g12 g31 g21 911 l l A B C D B A B J l C B A L D J L A 2 E D J B F E D C G K M D H G K E 3 I H G F g13 g33 g23 E F G H D E K G J D M K B C D E A B J D B A L J J L A B D J B A E D C B I H G F E D C B A (3.34) The top row and the left-most column of this matrix show which elements of the image are associated with which in order to produce the corresponding entry in the matrix A is the average square element: Statistical Description of Images What are the basis images in transform expands an image? 111 terms of which the Karhunen-Loeve Since g = A(g - pg) and A is an orthogonal matrix, the inverse transformation is given by g - pg = ATg We can write this expression explicitly: UN1 31 aN2 321 3N1 321 aN,2N 2N aN,NZ-N+l 3N1 Processing: Image 112 The Fundamentals (3.47) This expression makes it obvious that the eigenimages in terms of which the K-L transform expands an image are formedfrom the eigenvectors of its spatial autocorrelation matrix, by writing them in matrix form; i.e by using the first N elements of an eigenvector to form the first column of the corresponding eigenimage, the next N elements to form the next column and so on The coefficients of this expansion are the elements of the transformed image Example 3.11 Consider a X image with column representation g Write down an expression for the K-L transform of the image in terms of the elements of g and the elements aij of the transformation matrix A Calculate an approximation to the image g by setting the last six rows of A to zero Show that the approximation will be a X vector with the first three elements equal to those of the full transformation of g and the remaining six elements zero Assume that pg is the average grey value of image g Then the transformed image will have the form: '311 321 331 312 322 832 313 323 333 (3.48) 113 Statistical Description of Images If we set a41 = a42 = = a49 = a51 = = a = = a 9 = 0, clearly the last six rows of the above vector will be and the truncated transformation of the image will be vector g' = (311 321 331 0 0 o)T (3.49) According to formula (3.47) the approximation of the image is then: g13 g12 g11 (921 (E; E;)E; PSS P 922 g.3) = g33 g32 g31 a37 a34 PS a27 + 311 all a14 a17 ( ; :; :;;) W ; a24 a29 a26 a23 Example 3.12 (B) UT Show that if A is an N X N matrix the ith row of which is vector and C2 an N X N matrix with all its elements zero except the element at position (2,2) which is equal to c2, then: ATC2A = C Assume that uij T U ~ U ~ indicates the j t h component of vector U21 U22 ATC2A ~ U23 U22 U32 UN22 U2N2 U12 U; Then: 0 c2 0 0 i Image Processing: The Fundamentals 114 Example 3.13 (B) Assuming a X image, and accepting that we approximate it retaining only the first three eigenvalues of its autocovariance matrix, show that: Using the result of Example 3.12 concerning the truncated transform of the image g‘, we have: I p11 ) 821 831 82 E{ 822 832 83 823 (911 921 3I 0 0 l16 Image Processing: The Fundamentals Therefore, the mean square error is: We can exchange the order of taking the expectation value and taking the trace because trace means nothing more than the summing of the diagonal elements of the matrix: [ { = trace E (ATg-A’Tg’)(gTA- = trace[E{ArggTA-ATgg‘TA‘-A‘Tg‘gTA+A’Tg’g’~’}] (3.57) Matrices A and A’ are fixed, so the expectation operator does not affect them Therefore: E{llg - g‘ll} = trace[ATE(ggT}A - ATE{gg’T}A’ -A’TE{g‘gT}A + A’TE{g’g’T}A‘] (3.58) In this expression we recognize E { @ T } and E{g’g’T} as thecorrelation matrices of the two transforms: Cgg and C‘gg The gg’ matrix is the product of a vector and its transpose but with the last N - K components of the transposereplaced by The expectation operator will make all the off-diagonal elements of @’ zero anyway (since the transform is such that its autocorrelation matrix has0 all the off-diagonal elements) The fact that the last N - K elements of g’T are too will also make the last N - K diagonal elements (see Example 3.13) So, the result is: - -I E k g T = C‘gg (3.59) Similar reasoning leads to: (3.60) so: 117 Statistical Description of Images E{ Ilg - g’l I} = trace[ATCggA - ATC’ggA’ - AfTC’ggA+ A’TC’ggA’] (3.61) gg Consider the sum: -ATC’ggA’ + AJTC’ A’ = -(A - A’)TC’ggA’ We can partition A in two sections, a K X N2 submatrix AI and an (N2- K ) X N2 submatrix A2 A’ consists of AI and an ( N 2- K ) X N submatrix with all its elements zero: A= A’= ( i -A1 -2) (-i-) + A1 Then ( A - A’)TC‘ggA’ = ( I A T ) C‘ggA’ C ’ g g can be partitioned into four submatrices: ClPL= (3.63) where C is K X K diagonal Then the product is: (0 I AT) (5 I - -) =(O) (3.64) Using this result in (3.61) we obtain: E{ I I - g‘l I} = trace[ATCggA - A‘TC‘ggA] g (3.65) Consider the term ATCggA We mayassume that Cgg is the sum of N2 matrices, each one being N2 X N2 and having only one non zero element: X1 0 0 0 0 AN2 0 (3.66) A is made up of rows of eigenvectors while AT is made up of columns of eigenvectors Then we can write: Statistical Description of Images 119 Example 3.14 The autocovariance matrix of a X image is given by: 0 - -1 Calculate the transformation matrix A for the image, which when used for the inverse transform will approximate image with mean square the error equal to We must find the eigenvalues of the matrix b y solving the equation: 13-X I I 0 -1 -1 3-XI (3 - X) [(3- X)3 - (3 - X)] - [(3- X)2 - (-1)2] = =3 (3 - X)2 [(3- X)2 - l] - [(3 - X)2 - l] = * [(3 - - l] = =3 (3 - X - 1)2(3- X + =0 + (2 - = =3 X1 = 4, X2 = 4, X3 = 2, X4 = (3.70) The corresponding eigenvectors for X = are: - 3x1 - X3 = 4x1 3x2 - X4 = 4x2 -21 3x3 = 4x3 -22 3x4 = 4x4 + + -1 X3 = -21, X4 = -22, X1 = -23, Choose: x1 = = 0, x2 = = -L Jz Jz17 Or choose: x1 = -, x3 = , x2 = = Jz Jz eigenvectors, two first therefore, The are: (0 l ( 0 ) , which are orthogonal to each For other have: 5 X2 = -24 (3.71) a and X = we Image Processing: The Fundamentals 120 - (-:: i -1 -n) 3x1 - X3 = 221 322 - X4 = 222 -xl 3x3 = 2x3 -x2 3x4 = 2x4 + + (i:)=2(i)+ X4 21 =23, 22 =~ XI , =23, ~2 + = (3.72) Choose: XI = = 0, 2 = x4 = - &i W e not need to calculate thefourtheigenvector becauseweare interested in an approximate transformation matrix By setting eigenvectors some to ( 0 0 ) themean squareerrorwecommitwhenreconstructingtheimage is equal to the sum of the corresponding eigenvalues In this case, if we consider as transformation matrix the matrix A: (3.73) \ o the error will be equal to X4 0 = Example 3.15 Show the different stages of the Karhunen-Loevetransformof image in Example 2.14 the Figure 3.2 showsthe 64 eigenimages of theoriginalimage of Example 2.14 according to the Karhunen-Loeve transform in Figure 3.3 are thereconstructedimageswhen Theeightimagesshown 8,16,24,32,40,48,56and 64 terms were used for the reconstruction The sums of the mean squared errors for each reconstructed image are: Errorforimage(a): 5360 Errorforimage(b): 3846 Errorforimage 2715 (c): (g (2 (2 x i = 5365) Xi = 3850) Xi = 2718) Statistical Description of Images Errorforimage (d): 121 1850 Errorforimage(e): 1194 Error image for (f): 715 Error image for (g): 321 Error image for (h): (E Xi = 1852) (2 (g (2 Xi = 1195) Xi = 715) Xi = 321) Note that the mean squared errors of the reconstructions agree very well with the sum of the omitted eigenvalues in each case Figure 3.2: The 64 eigenimages, each scaled separately to have values from to 255 Image Processing: The Fundamentals 122 I l l l L I r l (h) Figure 3 Reconstructed image when the first 8,16,24,32,40,48,56and 64 : eigenimages shown in Figure 3.2 were used (from top left to bottom right respectively) Statistical Description of Images 123 Example 3.16 The autocovariance matrix of a X image is given by: 0-1 0 -1 Calculate the transformation matrix A for the image, which,when used for the inverse transform, will approximatethe image with meansquare-error equal to Find the eagenvalues of the matrix first: 4-X -1 (4-4 4-X -1 4-X -1 -1 - -1 4-X 0 4-X (4 - X)[(4 - -1 4-X -1 4-X -1 4-X 4-X -1 - (4 - X)] - [(4 - =o* =o* - l] = =3 [(4 - X)2 - l] = j(4 - X - 1)2(4- X + X1 = X2 = X3 = X4 = =0 j Since we allow error of image reconstruction of , we not need to calculate the eigenvectors that correspond to X = Eigenvectors for X = 5: -1 -21 -1 * * * * 4x1 - X3 = 5x1 =5x2 - x = 5x3 -4x4 =5x4 x - X4 -22 = X3 = -X4 X1 = -X3 X2 = -X4 X1 X2 Choose x1 = = 0, 2 = - = -2 For X2 choose an orthogonaleigen4’ vector, i.e x = = 0, XI = -, x = - Then the transformation matrix d2 d2 whichallowsreconstructionwith mean squareerror (equal to the sum of the omitted eigenvalues) is: 124 Image Processing: The Fundamentals What is the “take home” message of this chapter? If we view an image as an instantiation a whole lot of images which are theresult of of a random process, then we can try to represent it as thelinear superposition of some eigenimages which are appropriatefor representing the whole ensemble of images For an N X N image there may be as many as N such eigenimages, while in the SVD approach there are only N The difference is that with these N eigenimages we are supposed to be able to represent the whole ensemble of images, while in the case of SVD the N eigenimages are appropriate only for representing the one image If the set of eigenimages is arranged in decreasing order of the corresponding eigenvalues, truncating theexpansion of the image in terms of them, approximates the image with the minimum mean square error, over the whole ensemble of images In the SVD case similar truncation leads to the minimum square error approximation The cruxof K-L expansion is the assumptionof ergodicity This assumption states that the spatial statistics a single image are thesame as theensemble statistics over of the whole set of images If a restricted type of image is considered, this assumption is clearly unrealistic: Images are not simply the outcomes of a random process; there is always a deterministic underlying componentwhich makes the assumption invalid So, in such case, the K-L transform effectively puts more emphasis on the random component of the image; i.e the noise, rather than the component of interest However, if many different images are considered, the average greyvalue over the ensemble even of the deterministic component may be the same from pixel to pixel, and the assumption of ergodicity may be nearly valid On the other hand,if one has available a collection of images representative of the type of image of interest, the assumption of ergodicity may not need to be made: The K-L transform can be calculated using ensemble statistics andused to define a basis of images taylor-madefor the particular type of image ... Errorforimage 2715 (c): (g (2 (2 x i = 5365) Xi = 3850) Xi = 2718) Statistical Description of Images Errorforimage (d): 121 1850 Errorforimage(e): 1194 Error image for (f): 715 Error image for... of a single image, rather than from the statistics of a whole ensemble of images Average across images at a single position = Average over all positions on the same image / I of images Ensemble... N X N image: NI l