CHAPTER 8 ■ FILES AND DATA 200 open(SORT, '|-', 'perl sort2.pl'); Now we can print the data out: while (my ($item, $quantity) = each %inventory) { We use each() to get each key/value pair from the hash, as explained in Chapter 5. if ($quantity > 1) { $item =~ s/(\w+)/$1s/ unless $item =~ /\w+s\b/; } This makes the output a little more presentable. If there is more than one of the current item, the name should be pluralized unless it already ends in an “s”. \w+ gets the first word in the string, the parentheses will store that word in $1, and we then add an “s” after it. Last of all, we print this out by printing to the sort2.pl filehandle. That filehandle is in turn connected to the standard input of the sort2.pl program so the output is in sorted order. Bidirectional Pipes It is possible to pipe into the standard input of a process and then pipe the resulting standard output. The syntax to accomplish this is a bit beyond the scope of this book, but you can quench your curiosity by checking out perldoc IPC::Open2. File Tests So far, we’ve just been reading and writing files, and die()ing if anything bad happens. For small programs, this is usually adequate; but if we want to use files in the context of a larger application, we should really check their status before we try to open them and, if necessary, take preventive measures. For instance, we may want to warn the user if a file we’re going to overwrite already exists, giving them a chance to specify a different file. We also want to ensure that, for instance, we’re not trying to read a directory as if it were a file. ■ Tip This sort of programming—anticipating the consequences of future actions—is called defensive programming. Just like defensive driving, you assume that everything is out to get you. Just because this is paranoid behavior does not mean they are not out to get you—files will not exist or not be writable when you need them, users will specify things inaccurately, and so on. Properly anticipating, diagnosing, and working around such obstacles is the mark of a top-class programmer. Perl provides us with file tests, which allow us to check various characteristics of files. Most of these tests act as logical operators and return a true or false value. For instance, to check if a file exists, we write this: if (-e "somefile.dat") { } CHAPTER 8 ■ FILES AND DATA 201 The test is -e and it takes a file name (or filehandle) as its argument. Just like open(), this file name can also be specified from a variable. You can just as validly say if (-e $filename) { } where $filename contains the name of the file you want to check. Table 8-1 shows the most common file tests. For a complete list of file tests, see perldoc perlfunc. Table 8-1. File Test Operators Test Meaning -e True if the file exists -f True if the file is a plain file—not a directory -d True if the file is a directory -z True if the file has zero size -s True if the file has nonzero size—returns size of file in bytes -r True if the file is readable by you -w True if the file is writable by you -x True if the file is executable by you -o True if the file is owned by you The last four tests will only make complete sense on operating systems for which files have meaningful permissions, such as Unix and Windows. If this isn’t the case, they’ll frequently all return true (assuming the file or directory exists). So, for instance, if we’re going to write to a file, we should check to see whether the file already exists, and if so, what we should do about it. ■ Tip Note that on systems that don’t use permissions comprehensively, -w is the most likely of the last four tests to have any significance, testing for read-only status. This program does all it can to find a safe place to write a file: #!/usr/bin/perl # filetest.pl use warnings; CHAPTER 8 ■ FILES AND DATA 202 use strict; my $target; while (1) { print "What file should I write to? "; $target = <STDIN>; chomp $target; if (-d $target) { print "No, $target is a directory.\n"; next; } if (-e $target) { print "File already exists. What should I do?\n"; print "(Enter 'r' to write to a different name, "; print "'o' to overwrite or\n"; print "'b' to back up to $target.old)\n"; my $choice = <STDIN>; chomp $choice; if ($choice eq "r") { next; } elsif ($choice eq "o") { unless (-o $target) { print "Can't overwrite $target, it's not yours.\n"; next; } unless (-w $target) { print "Can't overwrite $target: $!\n"; next; } } elsif ($choice eq "b") { if ( rename($target, $target.".old") ) { print "OK, moved $target to $target.old\n"; } else { print "Couldn't rename file: $!\n"; next; } } else { print "I didn't understand that answer.\n"; next; } } last if open(OUTPUT, '>', $target); print "I couldn't write to $target: $!\n"; # and round we go again. } print OUTPUT "Congratulations.\n"; print "Wrote to file $target\n"; close OUTPUT; CHAPTER 8 ■ FILES AND DATA 203 So, after all that, let’s see how the program handles our input. First of all, what happens with a text file that doesn’t exist? $ perl filetest.pl What file should I write to? test.txt Wrote to file test.txt $ Seems OK. What about if we “accidentally” give it the name of a directory? Or give it a file that already exists? Or give it a response it’s not prepared for? $ perl filetest.pl What file should I write to? work No, work is a directory. What file should I write to? filetest.pl File already exists. What should I do? (Enter 'r' to write to a different name, 'o' to overwrite or 'b' to back up to filetest.pl.old) r What file should I write to? test.txt File already exists. What should I do? (Enter 'r' to write to a different name, 'o' to overwrite or 'b' to back up to test.txt.old) g I didn't understand that answer. What file should I write to? test.txt File already exists. What should I do? (Enter 'r' to write to a different name, 'o' to overwrite or 'b' to back up to test.txt.old) b OK, moved test.txt to test.txt.old Wrote to file test.txt $ There is a lot going on with this program. Let’s look at it in detail. The main program takes place inside an infinite loop—the only way we can exit the loop is via the last statement at the bottom: last if open(OUTPUT, '>', $target); That last will happen only if we’re happy with the file name and we can successfully open the file. In order to be happy with the file name, though, we have a gauntlet of tests to run: if (-d $target) { We need to first see whether what has been specified is actually a directory. If it is, we don’t want to go any further, so we go back and get another file name from the user: print "No, $target is a directory.\n"; next; We print a message and then use next to take us back to the top of the loop. CHAPTER 8 ■ FILES AND DATA 204 Next, we check to see whether the file already exists. If so, we ask the user what we should do about this. if (-e $target) { print "File already exists. What should I do?\n"; print "(Enter 'r' to write to a different name, "; print "'o' to overwrite or\n"; print "'b' to back up to $target.old\n"; my $choice = <STDIN>; chomp $choice; If he wants a different file, we merely go back to the top of the loop: if ($choice eq "r") { next; If he wants us to overwrite the file, we see if this is possible: } elsif ($choice eq "o") { First, we see if the user actually owns the file: it’s unlikely he’ll be allowed to overwrite a file he doesn’t own. unless (-o $target) { print "Can't overwrite $target, it's not yours.\n"; next; } Next we check to see if there are any other reasons we can’t write to the file; if there are, we report them and go around for another file name: unless (-w $target) { print "Can't overwrite $target: $!\n"; next; } If the user wants to back up the file—that is, rename the existing file to a new name—we see if this is possible: } elsif ($choice eq "b") { The rename() function renames a file; it takes two arguments: the current file name, and the new name. if ( rename($target, $target.".old") ) { print "OK, moved $target to $target.old\n"; } else { If we couldn’t rename the file, we explain why and start from the beginning again: print "Couldn't rename file: $!\n"; next; CHAPTER 8 ■ FILES AND DATA 205 } Otherwise, if the user said something we weren’t prepared for, we say: } else { print "I didn't understand that answer.\n"; next; } You may think this program is excessively paranoid—after all, it’s 50 lines just to print a message to a file. In fact, it isn’t paranoid enough: it doesn’t check to see whether the backup file already exists before renaming the currently existing file. This just goes to show you can never be too careful when dealing with the operating system. Later, we’ll see how to turn big blocks of code like this into reusable elements so we don’t have to reinvent the wheel every time we want to safely write to a file. Summary Files give our data permanence by allowing us to store the data on disk. It’s no good having the best accounting program in the world, say, if it loses all your accounts every time the computer is switched off. What we’ve seen here are the fundamentals of getting data in and out of Perl. Files are accessed through filehandles. Perl gives us three filehandles when our program executes: standard input (STDIN), standard output (STDOUT), and standard error (STDERR). We can open other filehandles, either for reading or for writing, with the open() function, and we should always remember to check the return value of the open() function. Wrapping the filehandle in angle brackets, <FILEHANDLE>, reads from the specified filehandle. We can read in scalar context (one line at a time) or list context (all remaining lines until end of file). Writing to a file is done with the print() function. By default, this writes to standard output, so the filehandle must be specified. The diamond, <>, allows us to write programs that read from the files provided on the command line, or from STDIN if no files are given. Pipes can be used to talk to programs outside of Perl. We can read in and write out data to them as if we were looking at the screen or typing on the keyboard. We can also use them as filters to modify our data on the way in or out of a program. File test operators can be used to check the status of a file in various ways, and we’ve seen an example of using file test operators to ensure that there are no surprises when we’re reading or writing a file. Exercises 1. Read each line of gettysburg.txt. Ignore all blank lines in the file. For all other lines, break the line into all the text separated by whitespace (keeping all punctuation) and write each piece of text to the output file ex1out.txt on its own line. 2. Write a program that, when given files as command-line arguments, displays their contents. For instance, if the program is invoked as CHAPTER 8 ■ FILES AND DATA 206 $ perl ex2.pl file1.dat it displays the contents of file1.dat. If invoked as $ perl ex2.pl file2.dat file3.dat it displays the contents of file2.dat followed by file3.dat. However, if invoked with no arguments like so: $ perl ex2.pl it always displays the contents of file1.dat followed by file2.dat followed by file3.dat. 3. Modify the file backup facility in filetest1.pl so that it checks to see if a backup already exists before renaming the currently existing file. When a backup does exist, the user should be asked to confirm that she wants to overwrite it. If not, she should be returned to the original query. C H A P T E R 9 ■ ■ ■ 207 String Processing Perl was created to be a text processing language, and it is arguably the most powerful text processing language around. As discussed in Chapter 7, one way that Perl displays its power in processing text is through its built-in regular expression support. Perl also has many built-in string operators (such as the string concatenation operator • and the string replication operator x) and string functions. In this chapter you will explore several string functions and one very helpful string operator. Character Position Before getting started with some of Perl’s built-in functions, let’s talk about the ability to access characters in a string by indexing into the string. The numeric position of a character in a string is known as its index. Recall that Perl is 0-based—it starts counting things from 0, and this applies to character indexing as well. So, for this string: "Wish You Were Here" here are the characters of the string and their indexes: character 0: W character 1: i character 2: s character 3: h character 4: <space> character 5: Y character 17: e You can also index characters by beginning at the rightmost character and starting from index –1. Therefore, the characters in the preceding example string can also be accessed using the following negative indices: character -1: e character -2: r character -3: e character -4: H character -5: <space> CHAPTER 9 ■ STRING PROCESSING 208 character -6: e character -18: W String Functions Perl has many string functions built into the language. This section will discuss several of the most common built-in functions used to process text. The length() Function To determine the length of a string, you can use the length() function. length(string) This function returns the number of characters in its argument. If no argument is given, length() returns the number of characters in Perl’s default variable $_. An example of the code follows: #!/usr/bin/perl # length.pl use warnings; use strict; my $song = 'The Great Gig in the Sky'; print 'length of $song: ', length($song), "\n"; # the *real* length is 4:44 $_ = 'Us and Them'; print 'length of $_: ', length, "\n"; # this one is 7:40 Running the code produces this result: $ perl length.pl length of $song: 24 length of $_: 11 $ The index() Function The index()function locates substrings in strings. Its syntax is index(string, substring) It returns the starting index (0-based) of where the substring is located in the string. If the substring is not found, it returns –1. This invocation: index('Larry Wall', 'Wall') CHAPTER 9 ■ STRING PROCESSSING 209 would return 6 since the substring “Wall” is contained within the string “Larry Wall” starting at position 6 (0-based, remember?). This invocation: index('Pink Floyd', 'ink'); would return 1. The index() function has an optional third argument that indicates the starting position from which it should start looking. For instance, this invocation: index('Roger Waters', 'er', 0) tells index() to try to locate the substring “er” in “Roger Waters” (http://en.wikipedia.org/ wiki/Roger_Waters) and to start looking from position 0. Position 0 is the default, so it is not necessary to include it, but it is OK if you do. This function returns 3. If you provide another starting position as in index('Roger Waters', 'er', 5) it tells index() to search for the substring “er” in “Roger Waters” but to start searching from index 5. This returns 9 because it finds the “er” in Roger’s last name. The following is an example illustrating the use of the index() function. It prompts the user for a string and then a substring and determines if the string contains any instance of the substring. If so, index() returns something other than –1, so you print that result to the user. Otherwise, you inform the user that the substring was not found. #! /usr/bin/perl # index.pl use warnings; use strict; print "Enter a string: "; chomp(my $string = <STDIN>); print "Enter a substring: "; chomp(my $substring = <STDIN>); my $result = index($string, $substring); if ($result != -1) { print "the substring was found at index: $result\n"; } else { print "the substring was not found\n"; } Here is an example of running this program: $ perl index.pl Enter a string: Perl is cool! Enter a substring: cool the substring was found at index: 8 $ perl index.pl Enter a string: hello, world! Enter a substring: cool [...]... program produces the following: $ perl addsizes.pl The total size of all files: 3241 $ There’s More There are many other ways that Perl interfaces to the operating system—I’ve only covered the basics here There are dozens of built-in functions available to do all sorts of system administration stuff (see perldoc perlfunc for a list) Other operating system things that Perl can do include create child... perl ex2.pl ex2.dat To double-check your work, take the standard output from the program and pipe it back into the standard input of the same program: $ perl ex2.pl ex2.dat | perl ex2.pl 214 C H A P T E R 10 ■■■ Interfacing to the Operating System Perl is a popular language for system administrators and programmers who have to work with files and directories due to the fact that there are many built-in... 2 16 CHAPTER 10 ■ INTERFACING TO THE OPERATING SYSTEM if (-x "$dir/$prog") { If so, you print the directory/filename, set $found to true since you found the program, and then last out of the foreach loop Finally, if you did not find the program, the program says so: print "$prog not found in PATH\n" unless $found; Executing this code produces the following: $ perl whereis.pl sort /usr/bin/sort $ perl. .. print(), _ is the default filehandle for Perl s file tests It actually refers to the last file explicitly tested Since you tested $_ previously, you can use _ for as long as you’re referring to the same file ■ Note When Perl does a file test, it actually looks up all the data at once—ownership, readability, writability, and so on; this is called a stat of the file _ tells Perl not to do another stat, but to... is a regular file: print -s _ if -r _ and –f _; Executing this code produces the following: $ perl directory-glob.pl Contents of the current directory: a.dat rwo addsizes.pl rwo b.dat rwo backquote.pl rwo dir1 drwxo directory-dir.pl rwo directory-glob.pl rwo links.pl rwo os.pl rwo 20 242 20 297 460 371 3 16 1049 219 CHAPTER 10 ■ INTERFACING TO THE OPERATING SYSTEM system.pl whereis.pl $ rwo rwo 132... directory handle This program produces the same result as directory-glob.pl: $ perl directory-dir.pl Contents of the current directory: a.dat rwo addsizes.pl rwo b.dat rwo backquote.pl rwo dir1 drwxo directory-dir.pl rwo directory-glob.pl rwo links.pl rwo os.pl rwo system.pl rwo whereis.pl rwo $ 20 242 20 297 460 371 3 16 1049 132 334 ■ Note Well, it produces almost the same results Reading from the... executed on a Unix system: $ perl os.pl please enter a directory name: newdir made the directory newdir ok! changed into newdir ok! enter new file name: new.dat file moved successfully! contents of the new directory: new.dat we are all done goodbye! $ Executing External Programs There are times when you want your Perl program to execute external programs such as another Perl script, shell commands... follows: Most actions that you want to take in Perl are implemented in the language in a way that does not require launching a shell Mentioning every feature of Perl is not the intent of this book, so I will not discuss all the different ways of doing the same thing.1 But a little bit of searching on your part may uncover an efficient, cool way of taking action in Perl without going out to the shell, so get... taking action in Perl without going out to the shell, so get in the habit of looking deeper into this language when you are trying to do something new.2 1 2 26 Remember TMTOWTDI? Divining how many is left as an exercise to the reader 2 www .perl. com, www.perlmonks.org, www.google.com, and perdoc are our friends CHAPTER 10 ■ INTERFACING TO THE OPERATING SYSTEM Here is an example program that executes the... Gilmour” looking for the substring “i” It finds it at position 7 (the “i” in “Gilmour”) This function also has an optional third argument that is the character position from which it begins looking for the substring This invocation: rindex('David Gilmour', 'i', 6) starts at position 6 (the “G” in “Gilmour”) and looks right to left for an “i” and finds it at position 3 The substr() Function When processing . program is invoked as CHAPTER 8 ■ FILES AND DATA 2 06 $ perl ex2.pl file1.dat it displays the contents of file1.dat. If invoked as $ perl ex2.pl file2.dat file3.dat it displays the contents. an example of running this program: $ perl index.pl Enter a string: Perl is cool! Enter a substring: cool the substring was found at index: 8 $ perl index.pl Enter a string: hello, world!. optional third argument that is the character position from which it begins looking for the substring. This invocation: rindex('David Gilmour', 'i', 6) starts at position 6 (the