710 Perl: The Complete Reference find The find command does not really have a language, but it does have a complex array of command line options that can specify, fairly expertly, the definition of the files you want to find. The find2perl script takes the command line options you would normally supply to the find command, and generates a Perl script that will perform the same function. The script produced actually makes use of the File::Find module, which provides a mechanism for parsing a directory tree, following all the subdirectories. For each file or directory found, a user-specified function is called, with the name and location of the current file being available via the $_ variable and via some variables located within the File::Find module. The result is that Perl has the ability not only to locate a file within the current directory structure, but also to do any number of other operations to convert, translate, summarize, and so on, the contents of the files found. The find2perl script does the basics of the file specification process for you, producing a script that you can modify to your own ends. If you know how to use the find command, then using the find2perl script should be easy. The command $ find2perl / -name '*bin*' -type d -print produces #!/usr/local/bin/perl Option Description -Dx Sets debugging, using a value of x. The value affects the output produced by the conversion process, and adds a number of additional statements to the script to output debugging information during the script’s progress. -Fc Specifies that the awk script was always invoked with a -F switch, which changes the default input field separator to c. -nfields Specifies the names of the input fields, rather than automatically using a value of $Fld1, $Fld2, and so on. Fields can be separated by any of the normal separation characters. -number Forces a2p to assume that the input is always made up of the number of fields specified by number. Table 20-11. Command Line Options to the awk Converter TEAMFLY Team-Fly ® Chapter 20: Extending and Embedding Perl 711 DEVELOPING APPLICATIONS eval 'exec /usr/local/bin/perl -S $0 ${1+"$@"}' if $running_under_some_shell; require "find.pl"; # Traverse desired filesystems &find('/'); exit; sub wanted { /^.*bin.*$/ && (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) && -d _ && print("$name\n"); } You can also specify more complex constructs directly to the find2perl script without having to modify the code. There are two options: one to create a tar file and the other to specify a Perl-specific evaluation for the file. The -tar option takes a file name and adds the necessary code to the Perl script to generate a file list to a piped tar command that then generates the tar file. The -eval option takes a string that will be evaluated as a Perl statement; if it returns true, the file will be considered as a match. Converting Perl to Other Languages With Perl 5, the facilities have been put in place to resolve a Perl script to its lowest common denominator—the string of optimized opcodes that are executed by the Perl interpreter proper. At the moment, two modules (B and O) provide a Perl interface to the internals of a Perl script. The result is that the internal opcode tree can be converted and parsed into a number of different formats to provide a range of different pieces of information. At present, this is limited to more extensive debugging features and the cross- referencing abilities that are often available to other languages. The same interface also provides you with the ability to generate a file in binary format called bytecode. This binary code can then be executed directly by a special Perl interpreter. The code has already been parsed and optimized, and much of the typical interpretation process has already taken place. This makes the code execution much faster and also ensures, to a greater or lesser extent, that the Perl source is hidden from casual view. The most interesting feature of the B and O modules, however, is that they can generate raw C code, which can then be compiled into a stand-alone executable. The 712 Perl: The Complete Reference final executable does not require Perl and cannot be reverse engineered. The performance benefits are debatable, but the distribution and security offered by the process are obvious advantages. Because this is a significant part of the entire Perl environment, it’s discussed more fully in Chapter 22. Calling Other Languages from Perl You have seen many times how Perl can call and interact with an external program. In some cases, the level of interaction has been as simple as calling the program with some specified options. There is no reason why, with the use of dual pipes, you couldn’t call and interact with another program, or even another programming language. The most obvious road to cooperating with another language from Perl, however, is to use Perl as a high-level environment that generates an optimized or customized program that can then be executed via a separate language. Perl has many features that make the manipulation of data, particularly strings, significantly easier; and if you want to produce customized or optimized code automatically, it makes sense to use a textual development environment to produce it. When dealing with a CGI or other web-based system, you can generate the JavaScript code that’s embedded into pages—you can even dynamically generate the code to enable different features in the final page. What follows from this is the general ability of Perl to generate the code for any language—it’s perfectly possible to create a Perl application that generates, and even compiles, C source code into a final application. In fact, this is exactly what some parts of the XS language and the Perl compiler (see Chapter 22) actually do. The trick is to make the best use of the Perl environment and, especially, make use of the here document to create customized source code to be passed to the program or language interpreter in question. Although it is possible to make use of pipes, most languages accept an input file as their source. Remember that “compiled” languages— such as C, C++, and Pascal—will require an external compiler, as well as additional processes between code production and the final execution stages; but this should not present too much difficulty. If it truly is interaction with another language that you require, then the obvious method is to set up some form of communication channel over which you can exchange requests and requirements. All modern languages, including Java, Python, Rebol, and Perl—provide the ability to open a network socket and exchange information. Some platforms provide an interface through a Perl module for communicating with other languages. For example, the Win32::OLE module allows you to communicate with Windows objects, which in turn means that you can control the operation of Word, Excel, and other Windows applications using Visual Basic semantics. Under Mac OS, you can communicate directly with AppleScript, which in turn allows you to communicate with the operating system and the applications, and through an application like Word to Visual Basic again. See Appendix B in this book and Web Appendix B on the Web at www.osborne.com for details on some of the platform-specific capabilities and the modules that support them. Part IV Fine-Tuning Applications Copyright 2001 The McGraw-Hill Companies, Inc. Click Here for Terms of Use. This page intentionally left blank. Chapter 21 Debugging and Tuning 715 Copyright 2001 The McGraw-Hill Companies, Inc. Click Here for Terms of Use. O nce you have completed your application, there are a number of things that you might want to do before finally releasing your code to the world. One of the most obvious procedures is to debug your code. Despite your best efforts, there are bound to be bugs and problems in your code that you probably didn’t realize were there, and certainly never intended to introduce. Debugging under Perl can be handled in a number of different ways. There are command line switches to output different levels of information, there’s the output from the Perl compiler, which can be a useful addition to the debugger’s toolkit, and there’s even an interactive debugger that you can use to execute and monitor the progress of your Perl script’s execution. There are also other steps that you need to take before you can release your code. Documentation is always a good idea, not only as pointers for your end-users, but also as a tool for you, so that when you go back to the code a few months later, you know what it does, how, and why. If your script is a general purpose tool, then you can make it appeal to a larger user group by making it cross-platform aware, and if possible, compatible. Knowing which functions, facilities, and systems are available on each platform is a good start, but there are also tricks that you can apply to make the process easier. Finally, Perl includes some tools that make the distribution and installation of modules and applications easier. Learning how to make the best use of these modules can dramatically increase the ease of use and installation by your end-users. The other chapters in this section cover these latter parts of the application development process. In this chapter, we’re going to concentrate purely on the processes and tools available for debugging and optimizing the applications, scripts, and modules that you write. Debugging is a time-consuming and laborious process. When I was at college, I was taught that the proper ratio was 80 percent testing and 20 percent development time, and after many years in the programming business, I can affirm that that ratio is about right. Even in a very simple piece of code, it’s possible to introduce some potential bugs and problems. For every bug you find and eliminate, I can guarantee that two more will be waiting in the wings. Furthermore, the solution to one bug may well introduce new bugs that you aren’t aware of. There is, however, more to debugging than simply finding the bugs and squishing them. A good programmer will take a more pragmatic approach, and try to develop a system that will trap most of the errors before they cause any problems. Remember that the purpose of debugging, and in turn, error trapping, is to ensure that the software does what it is supposed to do, while simultaneously ensuring that it doesn’t do anything it shouldn’t. A simple log-processing script should not delete or corrupt the log in the process, for example. We’ve already seen some examples of basic error trapping in Chapter 9—you should use Chapter 9 as a guide to isolating potential problems before they become real ones. In this chapter, we’ll look at two basic procedures for debugging. The second is the use of more simplified debugging processes, such as using print and caller to 716 Perl: The Complete Reference provide a history of a script’s execution, and therefore to help you to identify where things have gone wrong. The final, but certainly not least important, option that we will look at is the use of the Perl built-in debugger. The final stage of any debugging process is probably the one least followed— optimization. Although many people do not consider it a bug, a badly optimized script is as much a danger as one that overwrites files it shouldn’t have access to. A simple CGI script that processes a form shouldn’t take hours doing so—monitoring the execution time can often give you a good idea of where the problems lie. Within Perl, there are a few solutions to that particular problem, including an extension to the core debugging extensions, the Perl Profiler. This monitors the execution time of each subroutine in your script and gives you an excellent idea of where there could be problems. We’ll also look at some of the manual techniques available for improving performance, and at some of the more obvious resource sinks that can slow execution. It’s impossible in a single chapter to cover everything you need to know about trapping errors and debugging. For a more detailed look at debugging with Perl, see my Debugging Perl: Troubleshooting for Programmers (Osborne/McGraw-Hill, 2000), from which a lot of the material in this chapter comes. See Appendix C for more information. Debugging Techniques There are three basic methods for debugging a script. The first two are modifications on a single theme—the primary tool is just to use print to output the information. The second uses caller, which returns more extended information about the caller of the subroutine, and then uses print to output the information. The last is to use one of the full-blown debuggers. Perl actually comes with its own debugger, which is basically a script and module combination called when you use the -d command line option to Perl. If you use the ActiveState version of Perl under Windows, then you also have a GUI-based debugger. Just to add to the confusion, there is a fourth debugging system built into Perl—the -D command line option—but this doesn’t often give any useful information to the “end-user programmer.” Most of the detail given by -D is intended for those dealing with the internal workings of the Perl compiler and interpreter. See Chapter 15 for more information on the available options supported by the -D option. For a more detailed look at what each of the options does, check out the Debugging Perl title (see Appendix C for more information on this book). Using print To me, print statements have always seemed easy, and, providing you’re careful, they can usually provide enough information for you to trace the bug without having to resort to a full-blown debugger. In fact, the easiest way to use the humble print statement is during Chapter 21: Debugging and Tuning 717 FINE-TUNING APPLICATIONS the development of a script—just inserting a quick “this variable has this value” is an easy way for you to check that your script is doing what you think. You can also use the print statement as a way of reporting debugging information in the final version of the script. You usually use this method in combination with a global variable, perhaps set via the script’s command line, to enable or disable some simple debugging. The benefits of a script that outputs debugging information in this way is that it allows both the user and programmer to perform a post-mortem debug on the script. The only place where they are often useless is within a loop, because they produce a voluminous amount of information that needs to be processed manually after the execution is complete. On occasion, the loop mechanism can prove useful if you want to continually monitor a single variable as it is processed, or when you want to monitor the input or output to a filehandle. Usage is pathetically simple. By using a statement such as this, print "Got here!\n"; you can trace the execution path of a program. You can use the same basic layout for any valid piece of data that you want to output. Because you can print whatever you like, you can be quite explicit about the information: print "Data before processing: $data\n"; #process some data print "Fieldname: $field, data: $data, leftovers: $leftover\n"; More usually, though, you’ll want to be a bit more explicit about the location in which the debug report occurred. Here you can use the __LINE__ and __FILE__ directives, which insert the line number and current file in which the message was printed, respectively. For example, print __FILE__,'(',__LINE__,"): Data before processing $data\n"; might print out like this: process.pl(19): Data before processing Name: Martin Note that the __FILE__ and __LINE__ tokens must be outside of quotes in order for them to be included in the printed line. 718 Perl: The Complete Reference Chapter 21: Debugging and Tuning 719 FINE-TUNING APPLICATIONS Quoting Information When using print to output data, it’s a good idea to delimit the information that you are outputting. This limit helps to make it clear exactly what the actual data was. For example, this report line: process.pl(19): Data before processing Name: Martin doesn’t tell us if there are any trailing tabs or spaces to the information, which may or may not be important. A simple set of braces on either side of the data highlights the full data string: process.pl(19): Data before processing [Name: Martin ] Here it’s obvious that we have some trailing spaces. Don’t use angle brackets, <>, to delimit information, especially when displaying debugging data within a CGI or other HTML script. The HTML parser may either identify the entry as a proper HTML tag or simply ignore the data entirely! You can also go one stage further and quote the special characters. The following script defines two functions—the interface to the whole thing is the mapascii function. This takes a string and then converts the special characters into their equivalent sequence: sub mapasciichar { my ($char) = @_; @map = qw/ \0 [SOH] [STX] [ETX] [EOT] [ENQ] [ACK] \a \b \t \n \v \f \r [SO] [SI] [DCE] [DC1] [DC2] [DC3] [DC4] [SYN] [ETB] [CAN] [EM] [SUB] [ESC] [FS] [GS] [RS] [US] /; return $map[ord($char)] if (exists($map[ord($char)])); return $char; } [...]... equal to 1,000 The loop will parse and execute 1,000 times, and then the debugger will halt to allow you to process each individual line until you trace the problem The Perl Debugger The User Interface To start the debugger, you need to specify the -d option on the command line to the Perl interpreter: perl -d t.pl FINE-TUNING APPLICATIONS The name Perl Debugger is a bit of a misnomer The debugger is... to perform a simple calculation on a value generated by the script The main difference between Perl and many other languages is that you can run the debugger directly—in fact, straight from the command line There isn’t a separate application for doing the debugging, and there’s no reason to make any changes to your code 73 2 Perl: The Complete Reference Alternatively, it can be used with a dummy -e... SUB c Repeats the last n or s command FINE-TUNING APPLICATIONS n EXPR n 73 6 Perl: The Complete Reference Continues execution (all statements) until the next configured breakpoint or the end of the script If LINE or SUB is specified, then a breakpoint, active for one break only, is inserted before LINE or the subroutine SUB l l Lists the next page of lines for the current script from the current line... APPLICATIONS Sets a watch on the variable specified by EXPR A change to the specified variable will be printed before the next line to be executed is printed If EXPR is not specified, then all watchpoints are deleted Perl: The Complete Reference given, then the values of all the current options are printed The option name can be abbreviated to the minimum identifiable name; for example, the pager option can... Now you can just make calls to the writelog function: writelog("Had an error in the line %s from %s", $line, $file); FINE-TUNING APPLICATIONS If you don’t want to redirect the STDOUT and STDERR filehandles, the other solution is to create a function that opens and then writes the information you supply directly to a log file 72 8 Perl: The Complete Reference Debug Logs in the Real World It’s probably... logging the output of your application, you can track problems as the application is used by the end-users Then, when it comes to tracking a reported problem, you have all of the information to hand To get the best use out of a debug log, make sure you also record the time that the error was reported, the process ID, and, if relevant, the machine and/ or user name When the user reports the error, get them... breakpoint on the current line when no arguments are specified If LINE is specified, then the breakpoint is set on the specified line If CONDITION is specified, then each time the breakpoint is reached, it breaks execution only if the condition resolves to true The CONDITION does not use an if statement; it is purely the test If you use /PATTERN/, then the breakpoint breaks only if the statement matches the. .. location zero within the stack You could, however, use a modified form of the callerlog function that returns the stack trace from frame one onward: 72 4 Perl: The Complete Reference Using eval The eval function provides a very simple way of checking certain events without affecting the overall execution of your script In essence, the eval function just initiates a new instance of the Perl interpreter in... within the debugger, so that scripts that use ReadLine can be debugged NonStop Automatically set by noTTY; sets the debugger to non-interactive mode Table 21-1 Internal Options for the Debugger (continued) FINE-TUNING APPLICATIONS Option 74 1 74 2 Perl: The Complete Reference The default values for the options can be obtained by typing O into a new debugger process: perl -de 1 Loading DB routines from perl5 db.pl... pager option can be reduced to p A list of the most commonly used options is shown in Table 21-1 For others, refer to the perldebug man page Description RecallCommand The character(s) used to recall a command ShellBang The character(s) used to spawn a shell Pager The program to use for paging the output using the | command within the debugger The value of the PAGER environment variable will be used . directly to the find 2perl script without having to modify the code. There are two options: one to create a tar file and the other to specify a Perl- specific evaluation for the file. The -tar option. stand-alone executable. The 71 2 Perl: The Complete Reference final executable does not require Perl and cannot be reverse engineered. The performance benefits are debatable, but the distribution and. debugging. The second is the use of more simplified debugging processes, such as using print and caller to 71 6 Perl: The Complete Reference provide a history of a script’s execution, and therefore