PCA and ICA 267 F IGURE 9.13 Scree plot of eigenvalues from the data set of Figure 9.12. Note the shape break at N = 3, indicating that there are only three independent variables in the data set of five waveforms. Hence, the ICA algorithm will be requested to search for only three components. % % Do PCA and plot Eigenvalues figure; [U,S,pc]= svd(X,0); % Use single value decomposition eigen = diag(S).v2; % Get the eigenvalues plot(eigen,’k’); % Scree plot labels and title % nu_ICA = input(’Enter the number of independent components’); % Compute ICA W = jadeR(X,nu_ICA); % Determine the mixing matrix ic = (W * X)’; % Determine the IC’s from the % mixing matrix figure; % Plot independent components plot(t,ic(:,1)-4,’k’,t,ic(:,2),’k’,t,ic(:,3)؉4,’k’); labels and title The original source signals are shown in Figure 9.12. These are mixed together in different proportions to produce the five signals shown in Figure 9.14. The Scree plot of the eigenvalues obtained from the five-variable data set does show a marked break at 3 suggesting that there, in fact, only three separate components, Figure 9.13. Applying ICA to the five-variable mixture in Figure TLFeBOOK F IGURE 9.14 Five signals created by mixing three different waveforms and noise. ICA was applied to this data set to recover the original signals. The results of applying ICA to this data set are seen in Figure 9.15. 268 TLFeBOOK PCA and ICA 269 F IGURE 9.15 The three independent components found by ICA in Example 9.3. Note that these are nearly identical to the original, unmixed components. The presence of a small amount of noise does not appear to hinder the algorithm. 9.14 recovers the original source signals as shown in Figure 9.15. This figure dramatically demonstrates the ability of this approach to recover the original signals even in the presence of modest noise. ICA has been applied to biosignals to estimate the underlying sources in an multi-lead EEG signal, to improve the detection of active neural areas in functional magnetic resonance imaging, and to uncover the underlying neural control components in an eye movement motor control system. Given the power of the approach, many other applications are sure to follow. TLFeBOOK 270 Chapter 9 PROBLEMS 1. Load the two-variable data set, X, contained in file p1_data . Assume for plotting that the sample frequency is 500 Hz. While you are not given the di- mensions or orientation of the data set, you can assume the number of time samples is much greater than the number of measured signals. (A) Rotate these data by an angle entered through the keyboard and output the covariance (from the covariance matrix) after each rotation. Use the function rotate to do the rotation. See comments in the function for details. Continue to rotate the data set manually until the covariances are very small (less than 10 -4 ). Plot the rotated and unrotated variables as a scatter plot and output their variances (also from covariance matrix). The variances will be the eigenvalues found by PCA and the rotated data the principal components. (B) Now apply PCA using the approach given in Example 9.1 and compare the scatter plots with the manually rotated data. Compare the variances of the princi- pal components from PCA (which can be obtained from the eigenvalues) with the variances obtained by manual rotation in (A) above. 2. Load the multi-variable data set, X, contained in file p2_data . Make the same assumptions with regard to sampling frequency and data set size as in Problem 1 above. (A) Determine the actual dimension of the data using PCA and the scree plot. (B) Perform an ICA analysis using either the Jade or FastICA algorithm limiting the number of components determined from the scree plot. Plot independent components. TLFeBOOK 10 Fundamentals of Image Processing: MATLAB Image Processing Toolbox IMAGE PROCESSING BASICS: MATLAB IMAGE FORMATS Images can be treated as two-dimensional data, and many of the signal process- ing approaches presented in the previous chapters are equally applicable to im- ages: some can be directly applied to image data while others require some modification to account for the two (or more) data dimensions. For example, both PCA and ICA have been applied to image data treating the two-dimen- sional image as a single extended waveform. Other signal processing methods including Fourier transformation, convolution, and digital filtering are applied to images using two-dimensional extensions. Two-dimensional images are usually represented by two-dimensional data arrays, and MATLAB follows this tradi- tion;* however, MATLAB offers a variety of data formats in addition to the standard format used by most MATLAB operations. Three-dimensional images can be constructed using multiple two-dimensional representations, but these multiple arrays are sometimes treated as a single volume image. General Image Formats: Image Array Indexing Irrespective of the image format or encoding scheme, an image is always repre- sented in one, or more, two dimensional arrays, I(m,n) . Each element of the *Actually, MATLAB considers image data arrays to be three-dimensional, as described later in this chapter. 271 TLFeBOOK 272 Chapter 10 variable, I , represents a single picture element, or pixel. (If the image is being treated as a volume, then the element, which now represents an elemental vol- ume, is termed a voxel.) The most convenient indexing protocol follows the traditional matrix notation, with the horizontal pixel locations indexed left to right by the second integer, n , and the vertical locations indexed top to bottom by the first integer m (Figure 10.1). This indexing protocol is termed pixel coor- dinates by MATLAB. A possible source of confusion with this protocol is that the vertical axis positions increase from top to bottom and also that the second integer references the horizontal axis, the opposite of conventional graphs. MATLAB also offers another indexing protocol that accepts non-integer indexes. In this protocol, termed spatial coordinates, the pixel is considered to be a square patch, the center of which has an integer value. In the default coordi- nate system, the center of the upper left-hand pixel still has a reference of (1,1), but the upper left-hand corner of this pixel has coordinates of (0.5,0.5) (see Figure 10.2). In this spatial coordinate system, the locations of image coordi- nates are positions on a (discrete) plane and are described by general variables x and y. The are two sources of potential confusion with this system. As with the pixel coordinate system, the vertical axis increases downward. In addition, the positions of the vertical and horizontal indexes (now better though of as coordinates) are switched: the horizontal index is first, followed by the vertical coordinate, as with conventional x,y coordinate references. In the default spatial coordinate system, integer coordinates correspond with their pixel coordinates, remembering the position swap, so that I(5,4) in pixel coordinates references the same pixel as I(4.0,5.0) in spatial coordinates. Most routines expect a specific pixel coordinate system and produce outputs in that system. Examples of spatial coordinates are found primarily in the spatial transformation routines described in the next chapter. It is possible to change the baseline reference in the spatial coordinate F IGURE 10.1 Indexing format for MATLAB images using the pixel coordinate sys- tem. This indexing protocol follows the standard matrix notation. TLFeBOOK Fundamentals of Image Processing 273 F IGURE 10.2 Indexing in the spatial coordinate system. system as certain commands allow you to redefine the coordinates of the refer- ence corner. This option is described in context with related commands. Data Classes: Intensity Coding Schemes There are four different data classes, or encoding schemes, used by MATLAB for image representation. Moreover, each of these data classes can store the data in a number of different formats. This variety reflects the variety in image types (color, grayscale, and black and white), and the desire to represent images as efficiently as possible in terms of memory storage. The efficient use of memory storage is motivated by the fact that images often require a large numbers of array locations: an image of 400 by 600 pixels will require 240,000 data points, each of which will need one or more bytes depending of the data format. The four different image classes or encoding schemes are: indexed images, RGB images, intensity images, and binary images. The first two classes are used to store color images. In indexed images, the pixel values are, themselves, in- dexes to a table that maps the index value to a color value. While this is an efficient way to store color images, the data sets do not lend themselves to arithmetic operations (and, hence, most image processing operations) since the results do not always produce meaningful images. Indexed images also need an associated matrix variable that contains the colormap , and this map variable needs to accompany the image variable in many operations. Colormaps are N by 3 matrices that function as lookup tables. The indexed data variable points to a particular row in the map and the three columns associated with that row TLFeBOOK 274 Chapter 10 contain the intensity of the colors red, green, and blue. The values of the three columns range between 0 and 1 where 0 is the absence of the related color and 1 is the strongest intensity of that color. MATLAB convention suggests that indexed arrays use variable names beginning in x (or simply x ) and the sug- gested name for the colormap is map . While indexed variables are not very useful in image processing operations, they provide a compact method of storing color images, and can produce effective displays. They also provide a conve- nient and flexible method for colorizing grayscale data to produce a pseudocolor image. The MATLAB Image Processing Toolbox provides a number of useful prepackaged colormaps. These colormaps can implemented with any number of rows, but the default is 64 rows. Hence, if any of these standard colormaps are used with the default value, the indexed data should be scaled to range between 0 and 64 to prevent saturation. An example of the application of a MATLAB colormap is given in Example 10.3. An extension of that example demonstrates methods for colorizing grayscale data using a colormap. The other method for coding color image is the RGB coding scheme in which three different, but associated arrays are used to indicate the intensity of the three color components of the image: red, green, or blue. This coding scheme produces what is know as a truecolor image. As with the encoding used in indexed data, the larger the pixel value, the brighter the respective color. In this coding scheme, each of the color components can be operated on separately. Obviously, this color coding scheme will use more memory than indexed im- ages, but this may be unavoidable if extensive processing is to be done on a color image. By MATLAB convention the variable name RGB , or something similar, is used for variables of this data class. Note that these variables are actually three-dimensional arrays having dimensions N by M by 3. While we have not used such three dimensional arrays thus far, they are fully supported by MATLAB. These arrays are indexed as RGB(n,m,i) where i = 1,2,3. In fact, all image variables are conceptualized in MATLAB as three-dimensional arrays, except that for non-RGB images the third dimension is simply 1. Grayscale images are stored as intensity class images where the pixel value represents the brightness or grayscale value of the image at that point. MATLAB convention suggests variable names beginning with I for variables in class intensity. If an image is only black or white (not intermediate grays), then the binary coding scheme can be used where the representative array is a logical array containing either 0’s or 1’s. MATLAB convention is to use BW for variable names in the binary class. A common problem working with binary images is the failure to define the array as logical which would cause the image variable to be misinterpreted by the display routine. Binary class variables can be specified as logical (set the logical flag associated with the array) using the command BW = logical(A) , assuming A consists of only zeros and ones. A logical array can be converted to a standard array using the unary plus operator: TLFeBOOK Fundamentals of Image Processing 275 A = ؉BW . Since all binary images are of the form “logical,” it is possible to check if a variable is logical using the routine: isa(I, ’logical’) ; which will return a1 if true and zero otherwise. Data Formats In an effort to further reduce image storage requirements, MATLAB provides three different data formats for most of the classes mentioned above. The uint8 and uint16 data formats provide 1 or 2 bytes, respectively, for each array ele- ment. Binary images do not support the uint16 format. The third data format, the double format, is the same as used in standard MATLAB operations and, hence, is the easiest to use. Image arrays tha t use the double format can be treated as regular MATLAB matrix variables subject to all the power of MATLAB and its many functions. The problem is that this format uses 8 bytes for each array element (i.e., pixel) which can lead to very large data storage requirements. In all three data formats, a zero corresponds to the lowest intensity value, i.e., black. For the uint8 and uint16 formats, the brightest intensity value (i.e., white, or the brightest color) is taken as the largest possible number for that coding scheme: for uint8, 2 8-1 , or 255; and for uint16, 2 16 , or 65,535. For the double format, the brightest value corresponds to 1.0. The isa routine can also be used to test the format of an image. The routine, isa(I,’type’) will return a 1 if I is encoded in the format type , and a zero otherwise. The variable type can be: unit8 , unit16 ,or double . There are a number of other assessments that can be made with the isa routine that are described in the associated help file. Multiple images can be grouped together as one variable by adding an- other dimension to the variable array. Since image arrays are already considered three-dimensional, the additional images are added to the fourth dimension. Multi-image variables are termed multiframe variables and each two-dimen- sional (or three-dimensional) image of a multiframe variable is termed a frame. Multiframe variables can be generated within MATLAB by incrementing along the fourth index as shown in Example 10.2, or by concatenating several images together using the cat function: IMF = cat(4, I1, I2, I3, ); The first argument, 4, indicates that the images are to concatenated along the fourth dimension, and the other arguments are the variable names of the images. All images in the list must be the same type and size. Data Conversions The variety of coding schemes and data formats complicates even the simplest of operations, but is necessary for efficient memory use. Certain operations TLFeBOOK 276 Chapter 10 require a given data format and/or class. For example, standard MATLAB oper- ations require the data be in double format, and will not work correctly with Indexed images. Many MATLAB image processing functions also expect a spe- cific format and/or coding scheme, and generate an output usually, but not al- ways, in the same format as the input. Since there are so many combinations of coding and data type, there are a number of routines for converting between different types. For converting format types, the most straightforward procedure is to use the im2xxx routines given below: I_uint8 = im2uint8(I); % Convert to uint8 format I_uint16 = im2uint16(I); % Convert to uint16 format I_double = im2double(I); % Convert to double format These routines accept any data class as input; however if the class is indexed, the input argument, I , must be followed by the term indexed . These routines also handle the necessary rescaling except for indexed images. When converting indexed images, variable range can be a concern: for example, to convert an indexed variable to uint8, the variable range must be between 0 and 255. Converting between different image encoding schemes can sometimes be done by scaling. To convert a grayscale image in uint8, or uint16 format to an indexed image, select an appropriate grayscale colormap from the MATLAB’s established colormaps, then scale the image variable so the values lie within the range of the colormap; i.e., the data range should lie between 0 and N, where N is the depth of the colormap (MATLAB’s colormaps have a default depth of 64, but this can be modified). This approach is demonstrated in Example 10.3. However, an easier solution is simply to use MATLAB’s gray2ind function listed below. This function, as with all the conversion functions, will scale the input data appropriately, and in the case of gray2ind will also supply an appro- priate grayscale colormap (although alternate colormaps of the same depth can be substituted). The routines that convert to indexed data are: [x, map] = gray2ind(I, N); % Convert from grayscale to % indexed % Convert from truecolor to indexed [x, map] = rgb2ind(RGB, N or map); Both these routines accept data in any format, including logical, and pro- duce an output of type uint8 if the associated map length is less than or equal to 64, or uint16 if greater that 64. N specifies the colormap depth and must be less than 65,536. For gray2ind the colormap is gray with a depth of N ,orthe default value of 64 if N is omitted. For RGB conversion using rgb2ind ,a colormap of N levels is generated to best match the RGB data. Alternatively, a TLFeBOOK [...]... image I % Adds image I and J (images and/ % or constants) to form image % I_add % Subtracts J from image I % Divides image I by J % Multiply image I by J For the last four routines, J can be either another image variable, or a constant Several arithmetical operations can be combined using the imlincomb function The function essentially calculates a weighted sum of images For example to add 0.5 of image. .. contrast enhancement of the image, inverting the image, slicing the image, windowing, and thresholding the image TLFeBOOK Fundamentals of Image Processing 287 FIGURE 10.5 Montage display of 27 frames of magnetic resonance images of the brain plotted in Example 10.4 These multiframe images were obtained from MATLAB’s mri.tif file in the images section of the Image Processing Toolbox Used with permission from... and vertically by a Hanning window function, and it is thresholded and converted to a binary image % % % % % % % % Example 10.4 and Figures 10.5 and 10.6 Demonstration of various image functions Load all frames of the MRI image in mri.tif from the the MATLAB Image Processing Toolbox (in subdirectory imdemos) Select one frame based on a user input Process that frame by: contrast enhancement of the image, ... the ColorType, FileSize, and BitDepth Alternatively, the image data and map can be loaded using imread and the format image data determined from the MATLAB whos command The whos command will also give the structure of the data variable (uint8, uint16, or double) Basic Arithmetic Operations If the image data are stored in the double format, then all MATLAB standard mathematical and operational procedures... introduction to image processing and basic MATLAB formats and operations In subsequent chapters we use this foundation to develop some useful image processing techniques such as filtering, Fourier and other transformations, and registration (alignment) of multiple images PROBLEMS 1 (A) Following the approach used in Example 10.1, generate an image that is a sinusoidal grating in both horizontal and vertical... zeros(1,10)]; TLFeBOOK 282 Chapter 10 FIGURE 10.4 Montage of sinewave gratings created by Example 10.2 I _8 = im2uint8(I_sin); % Convert to a uint8 vector for i = 1:N if i < 10 * I > 90 % Extend to N (100) vertical lines % Insert black space at top and % bottom % I(i,:,1:j) = 0; else TLFeBOOK Fundamentals of Image Processing 283 I(i,:,1,j) = I _8; end end end montage(I); % Display image as montage title(’Sinewave... input image, I, can be either intensity, RGB, or indexed,* and in any format (uint8, uint16, or double) While most functions output binary images in uint8 format, im2bw outputs the image in logical format *As with all conversion routines, and many other routines, when the input image is in indexed format it must be followed by the colormap variable TLFeBOOK 2 78 Chapter 10 In this format, the image. .. Such images are often used as stimuli in experiments on vision TLFeBOOK Fundamentals of Image Processing 281 % Generate a single horizontal line of the image in a vector of % 400 points % % Generate sin; scale between 0&1 I_sin(1,:) = 5 * sin(2*pi*x) ؉ 5; I _8 = im2uint8(I_sin); % Convert to a uint8 vector % for i = 1:N % Extend to N (400) vertical lines I(i,:) = I _8; end % imshow(I); % Display image. .. 20 from the MRI image (mri.tif) and code it in pseudocolor by coding the image into green and the inverse of the image into blue Then take a threshold and plot pixels over 80 % maximum as red 4 Load the image of a cancer cell (from rat prostate, courtesy of Alan W Partin, M.D., Johns Hopkins University School of Medicine) cell.tif and apply a correction to the intensity values of the image (a gamma correction... previously The program first loads a set of MRI (magnetic resonance imaging) images of the brain from the MATLAB Image Processing Toolbox’s set of stock images This image is actually a multiframe image consisting of 27 frames as can be determined from the command imifinfo One of these frames is selected by the operator and this image is then manipulated in several ways: the contrast is increased; it is . independent components. TLFeBOOK 10 Fundamentals of Image Processing: MATLAB Image Processing Toolbox IMAGE PROCESSING BASICS: MATLAB IMAGE FORMATS Images can be treated as two-dimensional data, and many of the signal process- ing. image I I_add = imadd(I, J); % Adds image I and J (images and/ % or constants) to form image % I_add I_sub = imsubtract(I, J); % Subtracts J from image I I_divide = imdivide(I, J) % Divides image. classes or encoding schemes are: indexed images, RGB images, intensity images, and binary images. The first two classes are used to store color images. In indexed images, the pixel values are, themselves,