187 ■ ■ ■ CHAPTER 29 Automating Interactive Processes A utomating commands that are normally used interactively is something I’ve done many times. The simplest and most common script of this sort that I’ve used automates an ftp session to copy files from system to system. I’ll concede that for security reasons, ftp is not a good method for moving files around, especially when the session requires login/password authentication as a user other than “anonymous.” Usernames and pass- words are then sent across the network unencrypted. Even in a controlled environment, this has its risks, which should be considered carefully. Using this method out in the wild means asking for trouble. In any case, the point of this section is not to weigh the pros and cons of using ftp, but to demonstrate how to script an interactive session, using file copying as an example. This scripting construct is called a here-document. In this case it is gathering the latest black- list from the SquidGuard 1 project to update your Squid web proxy automatically from less-than-desirable Internet locations. The script presented here is just a simple ftp session to get a file. Note the tags for the beginning and the end of the interactive session. In this case EOF is used, but it could just as well have been FLOOBY or CaMeLCaSe (or anything that doesn’t appear at the start of a line within the session itself). Also, the closing tag needs to be present at the beginning of the line that follows the final command of the ftp session, because the shell recognizes it in that position as the closing delimiter. I would prefer to align the closing tag with the last line of the ftp session to make the indentation match up, but if I did this the end of the session wouldn’t be recognized and any following lines of code would be considered part of the session. First we set up the variables to be used for the ftp session, such as the source and des- tination file locations and the Squid server name. #!/bin/sh SGCONFDIR=/opt/apps/squidguard/conf 1. SquidGuard is a plug-in for the Squid proxy server acting as a filter, redirector, and access controller. More information can be found at http://www.squidguard.org . 188 CHAPTER 29 ■ AUTOMATING INTERACTIVE PROCESSES SERVER=ftp.teledanmark.no FILE=blacklists.tar.gz DIR=/pub/www/proxy/squidGuard/contrib Next the script outputs some text about what it is doing and then starts the ftp session. echo "FTPing current squidguard blacklist" ftp -n $SERVER << EOF The -n switch for ftp is used to disable the auto-login feature. You need to add the -n switch so that you can pass the username and password from the script. The first line of the ftp session passes credentials to the ftp server. user anonymous you@yourdomain.com It is generally considered poor security practice to include a username and password in a script. In this case, it isn’t really a problem since it is an anonymous session. Later in this chapter I will explain how you can remove the credentials from the script itself and store them in a separate file for ftp to use. These lines are the ftp commands that make up the session: cd $DIR hash lcd $SGCONFDIR get $FILE bye EOF In this case, the commands change to the appropriate remote and local directories, then they fetch the file and close the session. If you would like to avoid having the username and password inside the script, there are a few changes to make. First you should add an entry to the $HOME/.netrc file of the user who is running this script. The .netrc file contains credentials to use for sessions with specific ftp servers by the owner of the file; it looks something like this: machine ftpserver.your.domain.com login johndoe password Vu1n3rab13 machine ftpserver login johndoe password Vu1n3rab13 machine ftp.teledanmark.no login anonymous password you@yourdomain.com The new .netrc entry corresponds to the ftp server that is being used. After modifying the .netrc file, you have to remove the -n switch that was used in the ftp command at the beginning of the script. You can set up ftp sessions to as many sites as you like in the .netrc file. Remember the security concerns; you’ll want to make sure the file is readable by only the owner. One potential gotcha is that ftp will look for the server name used in the script (or command- line) invocation of ftp verbatim. In other words, if you ftp to the fully qualified domain name (FQDN), that FQDN must be in the .netrc entry. If you use the short name without the domain in the ftp invocation, then the short name must have an entry in the .netrc file. One can’t be substituted for the other. Both types are shown in the preceding example CHAPTER 29 ■ AUTOMATING INTERACTIVE PROCESSES 189 (in the entries for the server with the FQDN ftpserver.your.domain.com). The last entry in the preceding .netrc example file is one of the sites where you can obtain a blacklist file for SquidGuard. This is a very quick and simple example of a scripted session with an interactive program. It would, however, be more secure to use sftp (secure ftp), which uses an encrypted connection; unlike ftp, it doesn’t transfer your username and password across the network in plain text. The here-document technique used in this chapter for an automated ftp session can be expanded into a nested structure for multiple layers of interactive automation, as you may have more-complex needs. su - oracle <<-EOF sqlplus "/ as sysdba" <<-SQL shutdown immediate; exit; SQL lsnrctl stop EOF In the case of the preceding code, the root account is required to run multiple com- mands to stop a database as the oracle user account. The oracle user is required to run the interactive sqlplus command. The first here-document is specified by the EOF tag where root becomes the user oracle. All of the commands in this element are run as oracle. The next here-document nested within the first is specified by the SQL tag. This contains the commands to cleanly shut down the database, which is normally run interactively. Note that the double less-than characters (<<) are followed by a hyphen. This syntax strips any leading tab characters from the beginning of lines within the here-document. This provides the ability to indent your code properly for easier reading. If you prefer, you could use the expect utility for scripting interactive programs. expect implements a whole language designed for this purpose, with many more capabilities. More information on using expect can be found in Chapter 17. . 187 ■ ■ ■ CHAPTER 29 Automating Interactive Processes A utomating commands that are normally used interactively is something I’ve done. information can be found at http://www.squidguard.org . 188 CHAPTER 29 ■ AUTOMATING INTERACTIVE PROCESSES SERVER=ftp.teledanmark.no FILE=blacklists.tar.gz DIR=/pub/www/proxy/squidGuard/contrib