2.2 LabVIEW For Image Processing 27 the block diagram. The results of the operation are then passed back to the front panel indicators. A typical block diagram is shown in Fig. 2.1(b). Within the block diagram, you have access to a full-featured graphical programming language that includes all the standard features of a general-purpose programming environment, such as data structures, looping structures, event handling, and object-oriented programming. ■ Icon and connector pane—identifies the interface to the VI so that you can use the VI in another VI. A VI within another VI is called a sub-VI. Sub-VIs are analogous to subroutines in conventional programming languages. A sub-VI is a virtual instr ument and can be run as a program, with the front panel serving as a user interface, or, when dropped as a node onto the block diagram, the front panel defines the inputs and outputs for the given node through the connector pane. This allows you to easily test each sub-VI before being embedded as a subroutine into a larger program. LabVIEW also includes debugging tools that allow you to watch data move through a program and see precisely which data passes from one function to another along the wires, a process known as execution highlighting. This differs from text-based languages, which require you to step from function to function to trace your program execution. An excellent introduction to LabVIEW is provided in [4, 5]. 2.2.2 Image Processing and Machine Vision in LabVIEW LabVIEW is widely used for programming scientific imaging and machine vision appli- cations because engineers and scientists find that they can accomplish more in a shorter period of time by working with flowcharts and block diagrams instead of text-based function calls. The NI Vision Development Module [6] is a software package for engineers and scientists who are developing machine vision and scientific imaging applications. The development module includes NI Vision for LabVIEW—a library of over 400 func- tions for image processing and machine vision and NI Vision Assistant—an interactive environment for quick prototyping of vision applications without programming. The development module also includes NI Vision Acquisition—software with support for thousands of cameras including IEEE 1394 and GigE Vision cameras. 2.2.2.1 NI Vision NI Vision is the image processing toolkit, or library, that adds high-level machine vision and image processing to the LabVIEW environment. NI Vision includes an extensive set of MMX-optimized functions for the following machine vision tasks: ■ Grayscale, color, and binary image display ■ Image processing—including statistics, filtering, and geometric transforms ■ Pattern matching and geometric matching 28 CHAPTER 2 The SIVA Image Processing Demos ■ Part icle analysis ■ Gauging ■ Measurement ■ Object classification ■ Optical character recognition ■ 1D and 2D barcode reading. NI Vision VIs are divided into three categories: Vision Utilities, Image Processing, and Machine Vision. Vision Utilities VIs Allow you to create and manipulate images to suit the needs of your application. This category includes VIs for image management and manipulation, file management, calibration, and region of interest (ROI) selection. You can use these VIs to: – create and dispose of images, set and read attributes of an image, and copy one image to another; – read,write, and retrieve image file information. The file formats NIVision supports are BMP, TIFF, JPEG, PNG, AIPD (internal file format), and AVI (for multiple images); – display an image, get and set ROIs, manipulate the floating ROI tools window, configure an ROI constructor window, and set up and use an image browser; – modify specific areas of an image. Use these VIs to read and set pixel values in an image, read and set values along a row or column in an image, and fill the pixels in an image with a particular value; – overlay figures,text, and bitmaps onto an image without destroying the image data. Use these VIs to overlay the results of your inspection application onto the images you inspected; – spatially calibrate an image. Spatial calibration converts pixel coordinates to real- world coordinates while compensating for potential perspective er rors or nonlinear distortions in your imaging system; – manipulate the colors andcolorplanes of an image. UsetheseVIs to extract different color planes from an image, replace the planes of a color image with new data, convert a color image into a 2D array and back, read and set pixel values in a color image, and convert pixel values from one color space to another. Image Processing VIs Allow you to analyze, filter, and process images according to the needs of your application. This category includes VIs for analysis, grayscale and 2.2 LabVIEW For Image Processing 29 binary image processing, color processing, frequency processing, filtering, morphology, and operations. You can use these VIs to: – transform images using predefined or custom lookup tables, change the contrast information in an image, invert the values in an image, and segment the image; – filter images to enhance the information in the image. Use these VIs to smooth your image, remove noise, and find edges in the image. You can use a predefined filter kernel or create custom filter kernels; – perform basic mor phological operations, such as dilation and erosion, on grayscale and binary images. Other VIs improve the quality of binary images by filling holes in particles, removing particles that touch the border of an image, removing noisy particles, and removing unwanted particles based on different characteristics of the particle; – compute the histogram information and grayscale statistics of an image, retrieve pixel information and statistics along any 1D profile in an image, and detect and measure particles in binary images; – perform basic processing on color images; compute the histogram of a color image; apply lookup tables to color images; change the brightness, contrast, and gamma information associated with a color image; and threshold a color image; – perform arithmetic and bit-wise operations in NI Vision; add, subtract, multiply, and divide an image with other images or constants or apply logical opera- tions and make pixel comparisons between an image and other images or a constant; – perform frequency processing and other tasks on images; convert an image from the spatial domain to the frequency domain using a 2D Fast Fourier Transform (FFT) and convert an image from the frequency domain to the spatial domain using the inverse FFT. These VIs also extract the magnitude, phase, real, and imaginary planes of the complex image. Machine Vision VIs Can be used to perform common machine vision inspection tasks, including checking for the presence or absence of parts in an image and measuring the dimensions of parts to see if they meet specifications. You can use these VIs to: – measure the intensity of a pixel on a point or the intensity statistics of pixels along a line or in a rectangular region of an image; – measure distances in an image, such as the minimum and maximum horizontal separation between two vertically or iented edges or the minimum or maximum vertical separation between two horizontally oriented edges; 30 CHAPTER 2 The SIVA Image Processing Demos – locate patterns and subimages in an image. These VIs allow you to perform color and grayscale pattern matching as well as shape matching; – derive results from the coordinates of points returned by image analysis and machine vision algorithms; fit lines, circles, and ellipses to a set of points in the image; compute the area of a polygon represented by a set of points; mea- sure distances between points; and find angles between lines represented by points; – compare images to a golden template reference image; – classify unknown objects by comparing significant features to a set of features that conceptually represent classes of known objects; – read text and/or characters in an image; – develop applications that require reading from seven-segment displays, meters or gauges, or 1D barcodes. 2.2.2.2 NI Vision Assistant NIVisionAssistant isa tool for prototyping and testing image processing applications.You can create custom algorithms with the Vision Assistant scripting feature, which records every step of your processing algorithm. After completing the algorithm, you can test it on other images to check its reliability. Vision Assistant uses the NI Vision library but can be used independently of LabVIEW. In addition to being a tool for prototyping vision systems, you can use Vision Assistant to learn how different image processing functions perform. The Vision Assistant interface makes prototyping your application easy and efficient because of features such as a reference window that displays your original image, a script window that stores your image processing steps, and a processing window that reflects changes to your images as you apply new parameters (Fig. 2.2). The result of prototyping an application in Vision Assistant is usually a script of exactly which steps are necessary to properly analyze the image. For example, as shown in Fig. 2.2, the prototype of bracket inspection application to determine if it meets specifications has basically five steps: find the hole at one end of the bracket using pattern matching, find the hole at the other end of the bracket using pattern matching, find the center of the bracket using edge detection, and measure the distance and angle between the holes from the center of the bracket. Once you have developed a script that correctly analyzes your images, you can use Vision Assistant to tell you the time it takes to run the script. This information is extremely valuable if your inspection has to finish in a certain amount of time. As shown in Fig. 2.3, the bracket inspection takes 10.58 ms to complete. After prototyping and testing, Vision Assistant automatically generates a block diagram in LabVIEW. 2.3 Examples from the SIVA Image Processing Demos 31 1 Reference window 2 Processing window 3 Navigation buttons 4 Processing functions palette 5 Script window FIGURE 2.2 NI Vision Assistant, part of the NI Vision Development Module, prototypes vision applications, benchmarks inspections and generates ready-to-run LabVIEW Code. 2.3 EXAMPLES FROM THE SIVA IMAGE PROCESSING DEMOS The SIVA gallery includes demos for 1D signals, image, and video processing. In this chapter, we focus only on the image processing demos. The image processing gallery of SIVA contains over 40 VIs (Table 2.1) that can be used to visualize many of the image processing concepts described in this book. In this section, we illustrate a few of these demos to familiarize the reader with SIVA’s simple, intuitive interface and show the results of processing images using the VIs. ■ Image Quantization and Sampling: Quantization and sampling are fundamental operations performed by any digital imag e a cquisition device. Many people are familiar with the process of resizing a digital image to a smaller size (for the pur- pose of emailing photos or uploading them to social networking or photography Web sites). While a thorough mathematical analysis of these operations is rather 32 CHAPTER 2 The SIVA Image Processing Demos FIGURE 2.3 The Performance Meter inside NI Vision Assistant allows you to benchmark your application and help identify bottlenecks and optimize your vision code. (b) (c)(a) FIGURE 2.4 Grayscale quantization. (a) Front panel; (b) Original “Eggs” (8 bits per pixel); (c) Quantized “Eggs” (4 bits per pixel). involved and difficult to interpret, it is nevertheless very easy to visually appreciate the effects and artifacts int roduced by these processes using the VIs provided in the SIVA galler y. Figure 2.4, for example, illustrates the “false contouring” effect of grayscale quantization. While discussing the process of sampling any signal, students are introduced to the importance of “Nyquist sampling” and warned of “aliasing”or “false frequency” artifacts introduced by this process. The VI shown in 2.3 Examples from the SIVA Image Processing Demos 33 TABLE 2.1 A list of image and video processing demos available in the SIVA gallery. Basics of Image Processing: Nonlinear Filtering: Image quantization Median filtering Image sampling Gray level morphological filters Image histogram Trimmed mean filters Peak and valley detection Binary Image Processing: Homomorphic filters Image thresholding Image complementation Digital Image Coding & Compression: Binary morphological filters Block truncation image coding Image skeletonization Entropy reduction via DPCM JPEG coding Linear Point Operations: Full-scale contrast stretch Edge Detection: Histogram shaping Gradient-based edge detection Image differencing Laplacian-of-Gaussian Image interpolation Canny edge detection Double thresholding Discrete Fourier Analysis: Contour thresholding Digital 2D sinusoids Anisotropic diffusion Discrete Fourier transform (DFT) DFTs of important 2D functions Digital Video Processing: Masked DFTs Motion compensation Directional DFTs Optical flow calculation Block motion estimation Linear Filtering: Low, high, and bandpass filters Other Applications: Ideal lowpass filtering Hough transform Gaussian filtering Template matching Noise models Image quality using structural similarity Image deblurring Inverse filter Wiener filter Fig. 2.5 demonstrates these artifacts caused by sampling. The patterns in the scarf, the books in the bookshelf, and the chair in the background of the “Barbara”image clearly change their orientation in the sampled images. ■ Binary Image Processing: Binary images have only two possible “gray levels” and are therefore represented using only 1 bit per pixel. Besides the simple VIs used for thresholding grayscale images to binary images, SIVA has a demo that demonstrates the effects of various morphological operations on binary images, such as Median, Dilation, Erosion, Open, Close, Open-Clos, Clos-Open, and other 34 CHAPTER 2 The SIVA Image Processing Demos (a) (b) (c) (d) FIGURE 2.5 Effects of sampling. (a) Front panel; (b) Original “Barbara” image (256 ϫ 256); (c) “Barbara” subsampled to 128 ϫ 128; (d) Image c resized to 256 ϫ 256 to show details. binary operations including skeletonization. The user has the option to vary the shape and the size of the structuring element. The interface for the Morphology VI along with a binary image processed using the Erode, CLOS, and Majority operations is shown in Fig. 2.6. ■ Linear Point Operations and their Effects on Histograms: Irrespective of their familiarity with the theory of DIP, most computer and digital camera users are familiar, i f not proficient, with some form of an image editing software, such as Adobe Photoshop, Gimp, Picasa, or iPhoto. One of the frequently p erformed oper- ations (on-camera or using software packages) is that of changing the brightness and/or contrast of an underexposed or overexposed photograph. To illustrate how these operations affect the histogram of the image, a VI in SIVA provides the user with controls to perform linear point operations, such as adding an offset, 2.3 Examples from the SIVA Image Processing Demos 35 (a) (b) (c) (d) (e) FIGURE 2.6 Binary morphological operations. (a) Front panel; (b) Original image; (c) Erosion using X-shaped window; (d) CLOS operation using square window; (e) Median (majority) operation using square window. scaling the pixel values by scalar multiplication, and performing full-scale contrast stretch. Figure 2.7 shows a simple example where the histogram of the input image is either shifted to the right (increasing brightness), compressed while retaining shape, flipped to create an image negative, or stretched to fill the range (corres- ponding to full-scale contrast stretch). Advanced VIs allow the user to change the shape of the input histogram—an operation that is useful in cases where full-scale contrast stretch fails. ■ Discrete Fourier Transform: Most of introductory DIP is based on the theory of linear systems. Therefore, a lucid understanding of frequency analysis tech- niques such as the Discrete Fourier Transform (DFT) is important to appreciate more advanced topics such as image filtering and spectral theory. SIVA has many VIs that provide an intuitive understanding of the DFT by first introducing the concept of spatial frequency using images of 2D digital sinusoidal gratings. The DFT VI can be used to compute and display the magnitude and the phase of the DFT for gray level images. Masking sections of the DFT using zero-one masks 36 CHAPTER 2 The SIVA Image Processing Demos (a) (b) (c) (d) (e) (f) FIGURE 2.7 Linear point operations. (a) Front panel; (b) Original “Books” image; (c) Brightness enhanced by adding a constant; (d) Contrast reduced by multiplying by 0.9; (e) Full-scale contrast stretch; (f) Image negative. of different shapes and then performing inverse DFT is a very intuitive way of understanding the granularity and directionality of the DFT (see Chapter 5 of this book). To demonstrate the directionality of the DFT, the VI shown in Fig. 2.8 was implemented. As shown on the front panel, the input parameters, Theta 1 and Theta 2,are used to control the angle of the wedge-like zero-one mask in Fig. 2.8(d). It is instructive to note that zeroing out some of the oriented components in the DFT results in the disappearance of one of the tripod legs in the “Cameraman” image in Fig. 2.8(e). ■ Linear and Nonlinear Image Filtering: SIVA includes several demos to illustrate the use of linear and nonlinear filters for image enhancement and restoration. Low- pass filters for noise smoothing and inverse, pseudo inverse, and Wiener filters for deconvolving images that have been blurred are examples of some demos for linear image enhancement. SIVA also includes demos to illustrate the power of nonlin- ear filters over their linear counterparts. Figure 2.9, for example, demonstrates the result of filtering a noisy image corrupted with“salt and pepper noise” with a linear filter (average) and with a nonlinear (median) filter. ■ Image Compression: Given the ease of capturing and publishing digital images on the Internet, it is no sur prise most people are familiar with the terminology of compressed image formats such as JPEG. SIVA incorporates demos that highlight [...]... version of the image f (n) Since spatial relationships between pixels are unaffected, the appearance of the image will otherwise be essentially the same Likewise, if L < 0, then g (n) will be a dimmed version of the f (n) The histograms of the two images have a simple relationship: Hg (k) ϭ Hf (k Ϫ L) (3.7) Thus, an offset L corresponds to a shift of the histogram by amount L to the left or to the right,... contains basic histogram computation, manipulation, and display routines 3.3 Image Histogram Since the histogram represents a reduction of dimensionality relative to the original image f , information is lost the image f cannot be deduced from the histogram Hf except in trivial cases (when the image is constant-valued) In fact, the number of images that share the same arbitrary histogram Hf is astronomical... spike in the histogram to the right at gray level 255 Figures 3.6 and 3.7 show the result of applying an additive offset to the images “students” and “books” in Figs 3.2 and 3.4, respectively In both cases, the overall visibility of the images has been somewhat increased, but there has not been an improvement in the contrast Hence, while each image as a whole is easier to see, the details in the image. .. entry in the matrix, therefore, correspond to the strongest straight line in the image Figure 2.11 shows the result of applying the Hough transform in the SIVA gallery on the edges detected in the “Tower” image As seen from Fig 2.11(d), the simple algorithm presented above will be unable to distinguish partial line segments from a single straight line We have illustrated only a few VIs to whet the reader’s... Operations on Images Hf (k) KϪ1 0 LϾ0 LϽ0 Hg (k) Hg (k) 0 0 KϪ1 KϪ1 FIGURE 3.5 Effect of additive offset on the image histogram Top: original image histogram; bottom: positive (left) and negative (right) offsets shift the histogram to the right and to the left, respectively 3000 2000 1000 0 50 100 150 200 250 FIGURE 3.6 Left: Additive offset of the image “students” in Fig 3.2 by amount 60 Observe the clipping... for estimating the center of an image s gray level distribution A target value for the AOD might be specified when designing a point operation to change the overall gray level distribution of an image Figure 3.1 depicts two hypothetical image histograms The one on the left has a heavier distribution of gray levels close to zero (and a low AOD), while the one on the right is skewed toward the right (a... FIGURE 3.8 Effects of multiplicative image scaling on the histogram If P > 1, the histogram is expanded, leading to more complete use of the grayscale range If P < 1, the histogram is contracted, leading to possible information loss and (usually) a less striking image Figure 3.8 depicts the effect of multiplicative scaling on a hypothetical histogram For P > 1, the histogram is expanded (and hence, saturation... distribution toward one end of the grayscale or that the histogram is unusually compressed An example would be an image of the night sky, which is dark nearly everywhere In such a case, the appearance of the image may be normal but the histogram will be very skewed In some situations, it may still be of interest to attempt to enhance or reveal otherwise difficult -to- see details in the image by application of... We make the shorthand notation FSHS, since (3.19) will prove to be commonly useful as an addendum to other algorithms The operation in (3.19) can produce dramatic improvements in the visual quality of an image suffering from a poor (narrow) grayscale distribution Figure 3.12 shows the result of applying the full-scale histogram stretch to the image “books.” The contrast and visibility of the image was,... for free from the Web site mentioned in [2] We hope that the intuition provided by the demos will make the reader’s experience with image processing more enjoyable Perhaps, the reader’s newly found image processing lingo will compel them to mention how they “Designed a pseudo-inverse filter for deblurring” in lieu of “I photoshopped this image to make it sharp.” ACKNOWLEDGMENTS Most of the image processing . change the contrast information in an image, invert the values in an image, and segment the image; – filter images to enhance the information in the image. Use these VIs to smooth your image, . in the matrix, therefore,correspond to the strongest straight line in the image. Figure 2.11 shows the result of applying the Hough transform in the SIVA gallery on the edges detected in the “Tower”. demonstrates these artifacts caused by sampling. The patterns in the scarf, the books in the bookshelf, and the chair in the background of the “Barbara image clearly change their orientation in the sampled