Security Log Management I d e n t i f y i n g Pa t t e r n s i n t h e C h a o s Jacob Babbin Dave Kleiman Dr Everett F (Skip) Carter, Jr Jeremy Faircloth Mark Burnett Esteban Gutierrez Technical Editor FOREWORD BY GABRIELE GIUSEPPINI DEVELOPER OF MICROSOFT LOG PARSER Acknowledgments Syngress would like to acknowledge the following people for their kindness and support in making this book possible Syngress books are now distributed in the United States and Canada by O’Reilly Media, Inc.The enthusiasm and work ethic at O’Reilly are incredible, and we would like to thank everyone there for their time and efforts to bring Syngress books to market:Tim O’Reilly, Laura Baldwin, Mark Brokering, Mike Leonard, Donna Selenko, Bonnie Sheehan, Cindy Davis, Grant Kikkert, Opol Matsutaro, Steve Hazelwood, Mark Wilson, Rick Brown,Tim Hinton, Kyle Hart, Sara Winge, Peter Pardo, Leslie Crandell, Regina Aggio Wilkinson, Pascal Honscher, Preston Paull, Susan Thompson, Bruce Stewart, Laura Schmier, Sue Willing, Mark Jacobsen, Betsy Waliszewski, Kathryn Barrett, John Chodacki, Rob Bullington, Kerry Beck, Karen Montgomery, and Patrick Dirden The incredibly hardworking team at Elsevier Science, including Jonathan Bunkell, Ian Seager, Duncan Enright, David Burton, Rosanna Ramacciotti, Robert Fairbrother, Miguel Sanchez, Klaus Beran, Emma Wyatt, Krista Leppiko, Marcel Koppes, Judy Chappell, Radek Janousek, Rosie Moss, David Lockley, Nicola Haden, Bill Kennedy, Martina Morris, Kai Wuerfl-Davidek, Christiane Leipersberger,Yvonne Grueneklee, Nadia Balavoine, and Chris Reinders for making certain that our vision remains worldwide in scope David Buckland, Marie Chieng, Lucy Chong, Leslie Lim, Audrey Gan, Pang Ai Hua, Joseph Chan, June Lim, and Siti Zuraidah Ahmad of Pansing Distributors for the enthusiasm with which they receive our books David Scott, Tricia Wilden, Marilla Burgess, Annette Scott, Andrew Swaffer, Stephen O’Donoghue, Bec Lowe, Mark Langley, and Anyo Geddes of Woodslane for distributing our books throughout Australia, New Zealand, Papua New Guinea, Fiji,Tonga, Solomon Islands, and the Cook Islands v Lead Author Jacob Babbin works as a contractor with a government agency filling the role of Intrusion Detection Team Lead He has worked in both private industry as a security professional and in government space in a variety of IT security roles He is a speaker at several IT security conferences and is a frequent assistant in SANS Security Essentials Bootcamp, Incident Handling, and Forensics courses Jake lives in Virginia Jake is coauthor of Snort 2.1 Intrusion Detection Second Edition (Syngress Publishing, ISBN: 1-931836-04-3), Intrusion Detection and Active Response (Syngress, ISBN: 1-932266-47-X), and Snort Cookbook (O’Reilly, ISBN: 0-596007-91-4) Technical Editor Esteban Gutierrez (CISSP) is currently an information security architect at a Fortune 100 company He works on improving the security architecture of a global computing environment made up of massive amounts of data and tens of thousand of systems In the past he has worked as a senior network security engineer for a “.mil” network as part of a global network operations and security center, where he focused on daily security operations involving IDS and firewall management, incident response and containment, policy guidance, and network architecture He has also done security work in e-commerce environments during the “dot-com” boom and bust (Webvan), provided security for Internet service provider networks, and worked as a consultant Esteban also has experience with Linux, Solaris, BSD, Cisco hardware, routing protocols, DNS, Apache, VPN, and wireless networking His work, however, has focused primarily on network security architecture in large-scale enterprise networks vii He is most interested in being able to point at packet traces and pick out the “bad” traffic Esteban is a graduate of Reed College in Portland, OR He makes his home in the Pacific Northwest with his wife and daughter Contributing Authors Jeremy Faircloth (Security+, CCNA, MCSE, MCP+I, A+) is an IT Manager for EchoStar Satellite, L.L.C., where he and his team architect and maintain enterprise-wide client/server and Web-based technologies He also acts as a technical resource for other IT professionals, using his expertise to help others expand their knowledge As a systems engineer with more than 14 years of real-world IT experience, he has become an expert in many areas, including Web development, database administration, enterprise security, network design, and project management Jeremy has contributed to several popular Syngress technical books, including Snort 2.0 Intrusion Detection (ISBN: 1-931836-74-4), Security+ Study Guide & DVD Training System (ISBN: 1-931836-72-8), Microsoft Log Parser Toolkit (ISBN: 1-932266-52-6), and SSCP Study Guide & DVD Training System (ISBN: 1-931836-80-9) Dr Everett F (Skip) Carter, Jr is President of Taygeta Network Security Services (a division of Taygeta Scientific Inc.).Taygeta Scientific Inc provides contract and consulting services in the areas of scientific computing, smart instrumentation, and specialized data analysis.Taygeta Network Security Services provides security services for real-time firewall and IDS management and monitoring, passive network traffic analysis audits, external security reviews, forensics, and incident investigation Skip holds a Ph.D and an M.S in Applied Physics from Harvard University In addition, he holds two Bachelor of Science degrees viii (Physics and Geophysics) from the Massachusetts Institute of Technology Skip is a member of the American Society for Industrial Security (ASIS) He was contributing author of Syngress Publishing’s book, Hack Proofing XML (ISBN: 1-931836-50-7) He has authored several articles for Dr Dobbs Journal and Computer Language, as well as numerous scientific papers and is a former columnist for Forth Dimensions magazine Skip resides in Monterey, CA, with his wife,Trace, and his son, Rhett Dave Kleiman (CAS, CCE, CIFI, CISM, CISSP, ISSAP, ISSMP, MCSE) has worked in the Information Technology Security sector since 1990 Currently, he is the owner of SecurityBreach Response.com, and is the Chief Information Security Officer for Securit-e-Doc, Inc Before starting this position, he was Vice President of Technical Operations at Intelliswitch, Inc., where he supervised an international telecommunications and Internet service provider network Dave is a recognized security expert A former Florida Certified Law Enforcement Officer, he specializes in computer forensic investigations, incident response, intrusion analysis, security audits, and secure network infrastructures He has written several secure installation and configuration guides about Microsoft technologies that are used by network professionals He has developed a Windows Operating System lockdown tool, S-Lok (www.sdoc.com/products/slok.asp ), which surpasses NSA, NIST, and Microsoft Common Criteria Guidelines Dave was a contributing author to Microsoft Log Parser Toolkit (Syngress Publishing, ISBN: 1932266-52-6) He is frequently a speaker at many national security conferences and is a regular contributor to many security-related newsletters, Web sites, and Internet forums Dave is a member of several organizations, including the International Association of Counter Terrorism and Security Professionals (IACSP), International Society of Forensic Computer Examiners® (ISFCE), Information Systems Audit and Control Association® (ISACA), High Technology Crime Investigation Association (HTCIA), Network and Systems Professionals Association (NaSPA), Association of Certified Fraud ix Examiners (ACFE), Anti Terrorism Accreditation Board (ATAB), and ASIS International® He is also a Secure Member and Sector Chief for Information Technology at The FBI’s InfraGard® and a Member and Director of Education at the International Information Systems Forensics Association (IISFA) Additional Contributors Gabriele Giuseppini is a Software Design Engineer at Microsoft Corporation in the Security Business Unit, where he developed Microsoft Log Parser to analyze log files Originally from Rome, Italy, after working for years in the digital signal processing field, he moved to the United States with his family in 1999, and joined Microsoft Corporation as a Software Design Engineer working on Microsoft Internet Information Services Mark Burnett is an independent researcher, consultant, and writer specializing in Windows security Mark is author of Hacking the Code: ASP.NET Web Application Security (Syngress Publishing, ISBN: 1-932266-65-8), co-author of Microsoft Log Parser Toolkit (Syngress Publishing, ISBN: 1-932266-52-6), co-author of Maximum Windows 2000 Security, and co-author of Stealing The Network: How to Own the Box (Syngress Publishing, ISBN: 1-931836-87-6) He is a contributor and technical editor for Syngress Publishing’s Special Ops: Host and Network Security for Microsoft, UNIX, and Oracle (ISBN: 1931836-69-8) Mark speaks at various security conferences and has published articles in Windows IT Pro Magazine (formerly Windows & NET Magazine), WindowsSecrets.com newsletter, Redmond Magazine, Security Administrator, SecurityFocus.com, and various other print and online publications Mark is a Microsoft Windows Server Most Valued Professional (MVP) for Internet Information Services (IIS) x Contents Foreword xvii Chapter Log Analysis: Overall Issues Introduction IT Budgets and Results: Leveraging OSS Solutions at Little Cost Reporting Security Information to Management Example of an Incident Report: IDS Case No 123, September 2005 Combining Resources for an “Eye-in-the-Sky” View Blended Threats and Reporting 12 Conclusion 16 Code Solutions 16 Bird’s-Eye View for Management: HTML 16 Birds-Eye View for Security Teams: HTML 20 Commercial Solutions: ArcSight and Netforensics 30 Summary 32 Solutions Fast Track 32 Frequently Asked Questions 35 Chapter IDS Reporting 37 Introduction 38 Session Logging with Snort 39 Did That Exploit Work? Did the Attacker Download Any Data? 41 An Example of a Web Connection 43 An Example of a Web Connection with a Backdoor Snort Session 43 Session/Flow Logging with Argus 44 xi xii Contents Database Setup 46 Can You Determine When a DDoS/DoS Attack Is Occurring? 53 Using Snort for Bandwidth Monitoring 57 Using Bro to Log and Capture Application-Level Protocols 65 Tracking Malware and Authorized Software in Web Traffic 67 Determining Which Machines Use a Provided/Supported Browser 71 Tracking Users’ Web Activities with Bro 74 Using Bro to Gather DNS and Web Traffic Data 79 Using Bro for Blackholing Traffic to Malware-Infested Domains 90 Using Bro to Identify Top E-Mail Senders/Receivers 101 Top Mail Server 102 Top E-Mail Address 103 Virus Attachment Du Jour 104 Summary 107 Solutions Fast Track 107 Frequently Asked Questions 111 Chapter Firewall Reporting 113 Firewall Reporting: A Reflection of the Effectiveness of Security Policies 114 The Supporting Infrastructure for Firewall Log Management 116 Parsing the Data 118 Tools for an Overview of Activity 126 Time History Graphics 127 Reporting Statistics 132 Statistics by Country 132 Statistics by Business Partner 135 What Is “Normal” and What Is Threatening 136 Tools and URLs 138 Summary 139 Solutions Fast Track 139 Contents Frequently Asked Questions 141 Chapter Systems and Network Device Reporting 143 Introduction 144 What Should the Logs Log? Everything? 145 The Ws (Who, What, When, Where, and Why) 145 Web Server Logs 147 Recon and Attack Information 148 Identifying User Agent Types 149 Isolating Attacking IP Addresses 151 Correlating Data with the Host System 152 Did They Try to Get In? 152 Did They Get In? 153 What Did They Do While They Were In? 155 Pulling It All Together 156 Awstats Graphical Charting of Web Statistics 156 Top Attacker and Top User for the Web Server 160 Summary 162 Solutions Fast Track 162 Frequently Asked Questions 162 Chapter Creating a Reporting Infrastructure 165 Introduction 166 Creating IDS Reports from Snort Logs—Example Report Queries 166 Prepare Different Report Formats—Text, Web, E-mail 177 Creating IDS Reports from Bro Logs—Application Log Information 178 Prepare Different Report Formats—Text, Web, E-mail 185 Summary 190 Solutions Fast Track 190 Frequently Asked Questions 191 Chapter Scalable Enterprise Solutions (ESM Deployments) 193 Introduction 194 What Is ESM? 196 xiii ... Foreword Logs, logs, logs Ever since I started taking my first steps in the world of security, it has been clear that “the log? ?? plays a crucial—and sometimes undervalued—role in the security management. .. senior network security engineer for a “.mil” network as part of a global network operations and security center, where he focused on daily security operations involving IDS and firewall management, ... analysis.Taygeta Network Security Services provides security services for real-time firewall and IDS management and monitoring, passive network traffic analysis audits, external security reviews, forensics,