Randal K. Michael Mastering Unix Shell Scripting Free & Share & Open Publisher: Robert Ipsen Executive Editor: Carol Long Developmental Editor: Scott Amerman Managing Editor: Angela Smith Text Design & Composition: Wiley Composition Services This book is printed on acid-free paper. ∞ Copyright © 2003 by Randal K. Michael. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rose- wood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470. Requests to the Pub- lisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4447, E-mail: PERMCOORDINATOR@WILEY.COM. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, inci- dental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of Wiley Publishing, Inc., in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or ven- dor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data: ISBN: 0-471-21821-9 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 This book is dedicated to My Wife Robin, and the girls, Andrea and Ana Free & Share & Open The information that I gathered together in this book is the result of working with some of the most talented UNIX professionals on the topic. I have enjoyed every minute of my association with these UNIX gurus and it has been my pleasure to have the opportunity to gain so much knowledge from the pros. I want to thank every one of these experts for asking and answering questions over the last fifteen years. If my brother, Jim, had not kept telling me, “you should write a book,” after querying me for UNIX details on almost a weekly basis, I doubt this book would have ever been writ- ten. So, thanks Jim! I especially want to thank Jack Renfro at Daimler/Chrysler Corporation for giving me my first shell scripting project so long ago. I had to start with the man pages, but that is how I learned to dig deep to get an answer. Since then I have been on a mission to automate, through shell scripting, everything on every system that I come in contact with. I certainly value the years that I was able to work with Jack. I must also thank the talented people at Wiley Publishing. Margaret Eldridge started me on this project by letting me do my own thing, and Carol Long kept me going. Scott Amerman kept me on schedule, and Angela Smith did the edits that make my writing flow with ease. It has been a valuable experience for me to work with such a fine group of professionals at Wiley. I also want to thank Carole McClendon at Waterside Produc- tions for all of the support on this project. Carole is the best Agent that anyone could ever ask for. She is a true professional with the highest ethics. Of course my family had a lot to do with my success on this and every project. I want to thank Mom, Gene, Jim, Marcia, Rusty, Mallory, and Anica. I want to thank my Wife Robin for her understanding and support. The girls, Andrea and Ana, always keep a smile on my face, and Steve is always on my mind. I could not have written this book without the support of all of these people and the many others that remain unnamed. It has been an honor! Acknowledgments vii Free & Share & Open Acknowledgments vii Introduction xix Chapter 1 Scripting Quick Start and Review 1 Case Sensitivity 1 Unix Special Characters 2 Shells 2 Shell Scripts 2 Functions 2 A Function Has the Form 3 Running a Shell Script 3 Declare the Shell in the Shell Script 3 Comments and Style in Shell Scripts 4 Control Structures 6 Using break, continue, exit, and return 9 Here Document 9 Syntax for a Here Document 9 Shell Script Commands 10 Symbol Commands 13 Variables 13 Command-Line Arguments 13 Shift Command 14 Special Parameters $* and $@ 15 Special Parameter Definitions 15 Double Quotes “, Forward Tics ‘, and Back Tics ` 16 Math in a Shell Script 17 Operators 17 Built-in Mathematical Functions 18 Contents ix Free & Share & Open File Permissions, suid and sgid Programs 18 chmod Command Syntax for Each Purpose 19 Running Commands on a Remote Host 20 Setting Traps 21 User Information Commands 22 who Command 22 w Command 22 last Command 22 ps Command 23 Communicating with Users 23 Uppercase or Lowercase Text for Easy Testing 24 Check the Return Code 25 Time-Based Script Execution 27 cron tables 27 Cron Table Entry Syntax 27 Wildcards 28 at Command 28 Output Control 28 Silent Running 28 Using getopts to Parse Command-Line Arguments 29 Making a Co-Process with Background Function 30 Catching a Delayed Command Output 32 Fastest Ways to Process a File Line -by Line 33 Mail Notification Techniques 34 Using the mail and mailx Commands 34 Using the sendmail Command to Send Outbound Mail 34 Creating a Progress Indicator 35 A Series of Dots 35 A Rotating Line 35 Creating a Psuedo-Random Number 36 Checking for Stale Disk Partitions in AIX 37 Automated Host Pinging 37 Highlighting Specific Text in a File 38 Keeping the Printers Printing 38 AIX “Classic” Printer Subsystem 38 System V Printing 39 Automated FTP File Transfer 39 Capturing a List of Files Larger than $MEG 39 Capturing a User’s Keystrokes 40 Using the bc Utility for Floating-Point Math 40 Number Base Conversions 41 Using the typeset Command 41 Using the printf Command 41 Create a Menu with the select Command 42 Sending Pop-Up Messages to Windows 43 Removing Repeated Lines in a File 43 Removing Blank Lines from a File 44 x Contents Testing for a Null Variable 44 Directly Access the Value of the Last Positional Parameter, $# 45 Remove the Columns Heading in a Command Output 45 Arrays 46 Loading an Array 46 Testing a String 47 Summary 51 Chapter 2 Twelve Ways to Process a File Line by Line 53 Command Syntax 53 Using File Descriptors 54 Creating a Large File to Use in the Timing Test 54 Twelve Methods to Parse a File Line by Line 56 Method 1: cat $FILENAME | while read LINE 57 Method 2: while read $FILENAME from Bottom 58 Method 3: while_line_LINE_Bottom 58 Method 4: cat $FILENAME | while LINE=`line` 59 Method 5: cat $FILENAME | while line LINE 60 Method 6: while LINE=`line` from the Bottom 61 Method 7: cat $FILENAME | while LINE=$(line) 61 Method 8: while LINE=$(line) from the Bottom 62 Method 9: while read LINE Using File Descriptors 63 Method 10: while LINE=’line’ Using File Descriptors 64 Method 11: while LINE=$(line) Using File Descriptors 65 Method 12: while line LINE Using File Descriptors 66 Timing Each Method 66 Timing Script 67 Timing Data for Each Method 73 Timing Command Substitution Methods 77 Summary 78 Chapter 3 Automated Event Notification 79 Basics of Automating Event Notification 79 Using the mail and mailx Commands 80 Problems with Outbound Mail 82 Create a “Bounce” Account with a .forward File 82 Using the sendmail Command to Send Outbound Mail 83 Dial-Out Modem Software 84 SNMP Traps 85 Summary 86 Chapter 4 Progress Indicator Using a Series of Dots, a Rotating Line, or a Countdown to Zero 87 Indicating Progress with a Series of Dots 87 Indicating Progress with a Rotating Line 89 Creating a Countdown Indicator 91 Other Options to Consider 95 Summary 96 Contents xi Free & Share & Open Chapter 5 File System Monitoring 97 Syntax 98 Adding Exceptions Capability to Monitoring 103 The Exceptions File 103 Using the MB of Free Space Method 110 Using MB of Free Space with Exceptions 113 Percentage Used—MB Free and Large Filesystems 118 Running on AIX, Linux, HP-UX, and Solaris 128 Command Syntax and Output Varies between Operating Systems 130 Other Options to Consider 143 Event Notification 143 Automated Execution 143 Modify the egrep Statement 144 Summary 144 Chapter 6 Monitoring Paging and Swap Space 145 Syntax 146 AIX lsps Command 146 HP-UX swapinfo Command 147 Linux free Command 148 Solaris swap Command 148 Creating the Shell Scripts 149 AIX Paging Monitor 149 HP-UX Swap Space Monitor 155 Linux Swap Space Monitor 160 Solaris Swap Space Monitor 164 All-in-One Paging and Swap Space Monitor 169 Other Options to Consider 176 Event Notification 177 Log File 177 Scheduled Monitoring 177 Summary 177 Chapter 7 Monitoring System Load 179 Syntax 180 Syntax for uptime 180 AIX 180 HP-UX 181 Linux 182 Solaris 183 What Is the Common Denominator? 183 Scripting an uptime Field Test Solution 184 Syntax for iostat 186 AIX 186 HP-UX 186 xii Contents Linux 187 Solaris 187 What Is the Common Denominator? 187 Syntax for sar 188 AIX 188 HP-UX 189 Linux 189 Solaris 190 What Is the Common Denominator? 190 Syntax for vmstat 191 AIX 191 HP-UX 191 Linux 192 Solaris 192 What Is the Common Denominator? 192 Scripting the Solutions 193 Using uptime to Measure the System Load 194 Scripting with the uptime Command 194 Using sar to Measure the System Load 197 Scripting with the sar Command 198 Using iostat to Measure the System Load 203 Scripting with the iostat Command 203 Using vmstat to Measure the System Load 208 Scripting with the vmstat Command 208 Other Options to Consider 212 Stop Chasing the Floating uptime Field 212 Try to Detect Any Possible Problems for the User 213 Show the User the Top CPU Hogs 213 Gathering a Large Amount of Data for Plotting 214 Summary 214 Chapter 8 Process Monitoring and Enabling Preprocess, Startup, and Postprocess Events 215 Syntax 216 Monitoring for a Process to Start 216 Monitoring for a Process to End 218 Monitor and Log as a Process Starts and Stops 223 Timed Execution for Process Monitoring, Showing each PID, and Time Stamp with Event and Timing Capability 228 Other Options to Consider 248 Common Uses 248 Modifications to Consider 248 Summary 249 Chapter 9 Monitoring Processes and Applications 251 Monitoring Local Processes 252 Remote Monitoring with Secure Shell 254 Checking for Active Oracle Databases 256 Checking If the HTTP Server/Application Is Working 259 Contents xiii Free & Share & Open Other Things to Consider 260 Application APIs and SNMP Traps 261 Summary 261 Chapter 10 Creating Pseudo-Random Passwords 263 Randomness 263 Creating Pseudo-Random Passwords 264 Syntax 264 Arrays 265 Loading an Array 265 Building the Password Creation Script 266 Order of Appearance 266 Define Variables 266 Define Functions 267 Testing and Parsing Command-Line Arguments 275 Beginning of Main 279 Setting a Trap 280 Checking for the Keyboard File 280 Loading the “KEYS” Array 280 Using the LENGTH Variable to Build a Loop List 281 Building a New Pseudo-Random Password 282 Printing the Manager’s Password Report for Safe Keeping 283 Other Options to Consider 294 Password Reports? 294 Which Password? 295 Other Uses? 295 Summary 295 Chapter 11 Monitor for Stale Disk Partitions 297 AIX Logical Volume Manager (LVM) 298 The Commands and Methods 298 Disk Subsystem Commands 298 Method 1: Monitoring for Stale PPs at the LV Level 299 Method 2: Monitoring for Stale PPs at the PV Level 304 Method 3: VG, LV, and PV Monitoring with a resync 308 Other Options to Consider 315 SSA Disks 315 Log Files 316 Automated Execution 316 Event Notification 316 Summary 317 Chapter 12 Automated Hosts Pinging with Notification 319 Syntax 320 Creating the Shell Script 321 Define the Variables 321 Creating a Trap 323 The Whole Shell Script 324 xiv Contents [...]... Shell Script A shell script can be executed in the following ways: ksh shell_ script_name will create a Korn shell and execute the shell_ script_name in the newly created Korn shell environment shell_ script_name will execute shell_ script_name if the execution bit is set on the file (see the man page on the chmod command) The script will execute in the shell that is declared on the first line of the shell. .. shell script If no shell is declared on the first line of the shell script, it will execute in the default shell, which is the user’s system-defined shell Executing in an unintended shell may result in a failure and give unpredictable results Declare the Shell in the Shell Script Declare the shell! If we want to have complete control over how a shell script is going to run and in which shell it is to execute,... ] ` ‘ “ + Shells A shell is an environment in which we can run our commands, programs, and shell scripts There are different flavors of shells, just as there are different flavors of operating systems Each flavor of shell has its own set of recognized commands and functions This book works entirely with the Korn shell Korn Shell /bin/ksh OR /usr/bin/ksh Shell Scripts The basic concept of a shell script... benefit Every script is written using Korn shell, which is the industry standard for scripting solutions in Unix, although some may argue this point There are several versions of the Korn shell shipped with Unix, depending on the Unix operating system (OS) and the version of the OS release I have found that the shell scripts in this book will run on any of the Korn shell versions without any modification... execute, we MUST declare the shell in the very first line Free & Share & Open 3 4 Chapter 1 of the script If no shell is declared, the script will execute in the default shell, defined by the system for the user executing the shell script If the script was written, for example, to execute in Korn shell ksh, and the default shell for the user executing the shell script is the C shell csh, then the script... of writing a shell script to solve real-world Unix problems and tasks The shell scripts range from using a pseudorandom number generator to create pseudo-random passwords to checking for full filesystems on Unix machines and to sending pop-up messages to Windows desktops The details required to write these shell scripts include using good style and providing good comments throughout the shell script... this book The shell scripts presented in this book are complete shell scripts, which is one of the things that sets this book apart from other shell scripting books on the market The solutions are explained thoroughly, with each part of the shell script explained in minute detail down to the philosophy and mindset of the author How This Book Is Organized Each chapter starts with a typical Unix challenge... build options into the shell script to give the end user more flexibility on the command line When a shell script has to be rewritten for each operating system, a combined shell script is shown at the end of the chapter that joins the Unix flavor differences together into one shell script that will run on all of the OS flavors To do this last step we query the system for the Unix flavor using the uname... good idea to make your default shell environment the Korn shell (ksh); the standard shell on Linux is the Bourne Again shell (bash) shell, and some others use Bourne shell (sh) as the default You can find your default shell by entering echo $SHELL from the command line None of the shell scripts in this book requires a graphical terminal, but it sure does not hurt to have GNOME, CDE, KDE2, or X-Windows... code and say, “Hey, now that’s a groovy way to do that.” Table 1.1 Different Types of Shells to Declare #!/usr/bin/sh OR #!/bin/sh Declares a Bourne shell #!/usr/bin/ksh OR #!/bin/ksh Declares a Korn shell #!/usr/bin/csh OR #!/bin/csh Declares a C shell #!/usr/bin/bash OR #!/bin/bash Declares a Bourne-Again shell Scripting Quick Start and Review Command readability and step-by-step comments are just . your default shell environment the Korn shell (ksh); the standard shell on Linux is the Bourne Again shell (bash) shell, and some others use Bourne shell (sh) as the default. You can find your default shell. Korn shell, which is the industry standard for scripting solutions in Unix, although some may argue this point. There are several versions of the Korn shell shipped with Unix, depending on the Unix. xix Chapter 1 Scripting Quick Start and Review 1 Case Sensitivity 1 Unix Special Characters 2 Shells 2 Shell Scripts 2 Functions 2 A Function Has the Form 3 Running a Shell Script 3 Declare the Shell