“The new software testing methods introduced in this book will show you how to form and apply effective online goal-directed design and testing techniques. You begin with your user’s goals, follow Frank’s recommendations for scalable system design, and end with powerful tests that measure your user’s success in achieving them.” —Alan Cooper, Author of About Face 2.0: The Essentials of Interaction Design and The Inmates Are Running the Asylum. He is also the father of Visual Basic “A must read for Java developers needing to deliver high-quality, scalable, and well performing production Web Services.” —Phil Goodwin, Staff Engineer, JAX Team, Sun Microsystems, Inc. “Testing, scalability, and test-driven design and methodologies are the new ‘battlefield’ of software development. Frank Cohen introduces a new methodology for designing, testing, and maintaining Web-enabled applications. This book is the best battlegear available.” —Andrew Sliwkowski, Performance Analyst, BEA Systems “Frank uses his considerable experience in information system technology to provide a view of the evolution from mainframe to Client/Server to distributed ‘n’ or ‘flapjack’ tier architectures, structured to object-oriented design, and waterfall to Agile/Scrum/XP development approaches. Now that we (as an industry) have moved toward this distributed, open environment—testing, scalability, and interoperability are the next challenges, and Frank’s book is an excellent introduction.” —Ed Hunter, SunOne Performance Team, Sun Microsystems, Inc. “The book is an excellent presentation of metrics and measurement approaches to harnessing and calculating performance and functionality when working with Web-enabled applications, especially Web Services. The book’s emphasis on user archetypes to drive otherwise headless testing agents is a key point.” —Bret Pettichord, Software Testing and Test Automation Guru for IBM/Tivoli, BMC, Segue, Interleaf; and co-author of Lessons Learned in Software Testing “Frank Cohen’s new book takes a theoretical, didactic, and empirical approach to attacking this issue and does a fantastic job at bringing both the novice and expert from the ‘what and why’ to the ‘how’ of software testing design, automation, and scalability/reusability.” —Christian J. Hessler Staff Engineer, Sun Microsystems, Inc. “An unequaled achievement in clarity of Java testing, design, and optimization. To avoid hard-to- trace scalability and performance ‘gotchas’ inherent in early Java development, this book is for you!” —Farooq Khan, Software Development Manager, 2Wire Inc. “What I liked most about this book was the confidence with which Cohen presents a wide range of topics: objects, tests, architecture, frameworks, and design—all the right stuff for a software developer, QA technician, and IT manager.” —Rossana Muriel, Director of Quality, Advise & Services, AMP Global Assets Management PH069-Cohen.book Page i Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. PH069-Cohen.book Page ii Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. P RENTICE H ALL PTR U PPER S ADDLE R IVER , NJ 07458 WWW . PHPTR . COM PRENTICE HALL PROFESSIONAL TECHNICAL REFERENCE UPPER SADDLE RIVER, NJ 07458 WWW.PHPTR.COM Java ™ Testing and Design From Unit Testing to Automated Web Tests Frank Cohen PH069-Cohen.book Page iii Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. To Lorette—truly, deeply, madly ❖ ❖ ❖ A CIP catalog reference for this book can be obtained from the Library of Congress Editorial/Production Supervision: MetroVoice Publishing Services Executive Editor: Greg Doench Marketing Manager: Chris Guzikowski Manufacturing Manager: Alexis Heydt-Long Cover Designer: Nina Scuderi Cover Design Director: Jerry Votta Series Designer: Gail Cocker-Bogusz Full-Service Project Manager: Anne R. Garcia © 2004 by Pearson Education, Inc. Publishing as Prentice Hall Professional Technical Reference Upper Saddle River, New Jersey 07458 Prentice Hall PTR offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact: U.S. Corporate and Government Sales, 1-800-382-3419, corpsales@pearsontechgroup.com. For sales outside of the U.S., please contact: International Sales, 1-317-581-3793, international@pearsontechgroup.com. All company and product names mentioned herein are the trademarks or registered trademarks of their respective owners. All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher. Printed in the United States of America First Printing ISBN 0-13-142189-1 Pearson Education LTD. Pearson Education Australia PTY, Limited Pearson Education Singapore, Pte. Ltd. Pearson Education North Asia Ltd. Pearson Education Canada, Ltd. Pearson Educación de Mexico, S.A. de C.V. Pearson Education—Japan Pearson Education Malaysia, Pte. Ltd . PH069-Cohen.book Page iv Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. v Chapter Contents Foreword xiii Preface xv Acknowledgments xix PART I Gauging Web-Enabled Applications CHAPTER 1 The Forces at Work Affecting Your Web-Enabled Software 1 The First Three Axioms 2 Web-Enabled Applications, the New Frontier 4 Why Writing High-Quality Software Today Is Hard 10 The Myth of Version 2.0 Solidity 11 Management’s Quest for Grail 12 Trying for Homogeneity When Heterogeneity Rules 15 The Language of Bugs 17 The Evil Twin Vice Presidents Problem 18 A Concise History of Software Development 19 Web-Enabled Applications 20 Test Paradigms and Lifecycle Processes 22 Testing Methods 25 Click-Stream Testing 26 Unit Testing 27 Functional System Testing 28 Scalability and Performance Testing 30 Quality of Service Testing 30 PH069-Cohen.book Page v Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. vi Contents Defining Test Agents 30 Scalability and Performance Testing with Test Agents 33 Testing for the Single User 35 Creating Intelligent Test Agents 38 Automated Testing 39 Summary 39 CHAPTER 2 When Application Performance Becomes a Problem 41 Just What Are Criteria? 41 Defining Criteria for Good Web Performance 44 Are the Features Working? 46 Is Performance Acceptable? 46 How Often Does It Fail? 46 Web-Enabled Application Measurement Tools 46 The Web Rubric 47 The Four Tests of Good Performance 49 Components of a Good Test Agent 50 Web-Enabled Application Types 51 The Web-Enabled Application Points System (WAPS) 53 The Web-Enabled Application’s Framework 57 The Flapjacks Architecture 57 Adopting Flapjacks and Intelligent Test Agents 59 Building Intelligent Test Agents in a Flapjacks Environment 64 Script Languages and Test Agents 66 Generating Meaningful Data 75 Summary 76 CHAPTER 3 Modeling Tests 77 Modeling User Behavior for Meaningful Test Results 78 Lifecycles, Projects, and Human Nature 80 The Micromax Lifecycle 83 Categorizing Problems 83 Prioritizing Problems 85 Reporting Problems 86 Criteria for Evaluating Problems 86 Considerations for Web-Enabled Application Tests 87 Functionality and Scalability Testing 87 Functional Testing 90 PH069-Cohen.book Page vi Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Contents vii Scalability Testing 91 Testing Modules for Functionality and Scalability 92 Management Styles 99 Service Level Agreements 101 Grid Computing and Intelligent Agents 104 The Road to Easy Setup, Use, and Maintenance 105 Self-Healing Systems 106 Understanding Performance and Scalability Criteria 108 Defining SPC 108 SPC in Action 111 Modeling a User’s Goals 115 Test States 117 Using UML and Code Comments to Model Tests 118 Putting the Test Together 119 Summary 120 CHAPTER 4 Java Development and Test Automation Tools 121 The Three Waves 122 Desktop Application Development and Test Automation 122 Client/Server Development and Test Automation 124 Web-Enabled Development and Test Automation 127 Achieving the Troika—the Fourth Wave 131 A Test Automation Lifecycle 133 Summary 137 CHAPTER 5 Bridging from Methodology to Design 139 Searching for Tools to Enable the Troika 140 How to Get TestMaker 144 TestMaker and the Open Source Process 145 Spending Five Minutes with TestMaker 146 Installing TestMaker on a Windows or Linux Computer 147 Running TestMaker 148 Getting to Know the TestMaker Graphic Environment 148 Opening and Running Test Agents 150 Building Agents with the New Agent Wizard 153 Why I Like Jython 162 1. Jython Is Quick 163 2. Data Structures and List Iteration Come for Free 163 3. Dynamic Variables with Automatic Typing 164 4. Functions Are First Class 164 PH069-Cohen.book Page vii Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. viii Contents 5. Java Integration 165 6. Bean Property Introspection 165 7. Sun Is Adopting Scripting in Java 166 Using Jython to Incorporate JUnit 166 JUnit for Repeatable Tests 166 A JUnit Example 167 JUnit and TestMaker 169 Summary 172 PART II Components, Interoperability, and Optimization CHAPTER 6 Design and Test in HTTP/HTML Environments 173 The HTTP/HTML Environment 174 What Usually Goes Wrong 177 Compliance, As in Non, and Browser Caching 177 Invalid Data 179 Session Problems 181 Constructing HTTP Test Agents 181 Hands-On HTTP Communication 184 Understanding Cookies, Sessions, and Redirection 193 Validating Response Data 203 Summary 214 CHAPTER 7 Tuning SOAP and XML Web Services 217 The Web Services Vision 218 XML-RPC for Web Interoperability 221 Where XML-RPC Falls Short 227 Universal Interoperability with SOAP 227 Web Service Scalability Techniques 229 Web Service Interoperability Problems 231 Discovery 232 Definition 232 Request/Response 234 On the Horizon 237 Using TestMaker to Understand WSDL 237 Constructing SOAP Calls 242 Different Types of SOAP Calls 245 PH069-Cohen.book Page viii Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Contents ix Validating Response Data 254 Making It Easier to Write Agents 255 Resources 255 Summary 256 CHAPTER 8 State, Design, and Testing 257 A Question of State 257 Lifecycle for Testing Stateful Systems 261 Techniques to Establish State 263 Preparation and Setup Agents 267 Setup_agent 273 Using Databases to Configure Tests 280 Using Lingo to Make Test Content Close to Real 286 Summary 291 CHAPTER 9 Integrating with .NET Web Services 293 Interoperability and Integration 295 How Is .NET Different? 297 Document-Style Scalability 300 SOAP Headers in .NET 302 WSDL .NET Style 304 A Test Agent for .NET Environments 307 Near Term Considerations 315 Summary 316 CHAPTER 10 Building and Testing Intranets and Secure Environments 319 Getting a Head Start 319 Security by Routing 320 Virtual Private Networks 322 Network Segments and Subnets 323 Transport Security 325 SOAP over SSL 330 .NET Passport Authentication 331 HTTP Basic Authentication 332 SOAP and Security 333 PH069-Cohen.book Page ix Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. x Contents Generating Certificates and KeyStores 336 The Java Keytool 339 Summary 341 CHAPTER 11 A Web Application Framework from Construction to Test 343 The Trading Desk and Intelligent Test Agents 343 Scalability Test Goals 344 System Infrastructure 345 User Archetypes 348 Understanding the Test Requirements 350 Constructing the Test 352 Implementing User Archetypes in Code Modules 356 Implementing the Master Component 365 Setup 366 Run 368 Cleanup 371 Property Files for Test Configuration 371 Implementing the Logging Component 372 Avoiding Test Scalability Problems 374 A First Look at the Results 376 Summary 377 CHAPTER 12 Turning Test Agent Results into Actionable Knowledge 379 What to Expect from Results Analysis 380 Goal: Our New Web Site Needs to Handle Peak Loads of 50 Concurrent Users 381 Goal: The Web Site Registration Page Needs to Work Flawlessly 381 Goal: Customer Requests for Month-End Reports Must Not Slow Down the Order-Entry Service 382 Goal Summary 383 The Big Five Problem Patterns 383 Resource Problems 383 Concurrency Problems 384 Component Problems 386 Contention Problems 387 Crash Recovery Problems 388 Key Factors in Results Analysis 389 Scenarios Where Results Data Misleads 391 The Node Problem 391 The Hidden Error 392 PH069-Cohen.book Page x Monday, March 15, 2004 9:00 AM Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. [...]... and Bao Nguyen at 2Wire; Dileep Kumar, Madhu Konda, Rohit Valia, Robin Smith, and Deborah Andrade of the SunONE team at Sun Microsystems; Phil Goodwin and Jeff Sutor on the Java engineering team at Sun Microsystems; Michael Smith, Eric Stahl, Scott Regan, Sam Ramji, Samir Kothari, Michael Bamford, Michael Smith, and Andrew Sliwkowski at BEA Systems; Karen Lee, Deborah Magid, Rod Smith, Rawn Shah, and. .. book puts food on my family table and keeps me going to improve my tools, techniques, and methods I appreciate your interest and hope the experience is rewarding for you personally and professionally You are about to hear from me, I would like to hear from you Please write or email me at fcohen@pushtotest.com and tell me what you thought about this book, and about testing and building scalable Web-enabled... Liberty Avoiding and solving concurrency problems Architecture, code, and test agents for J2EE, Java Web Services, P2P, NET Easy-to-understand test scripts using Python/Jython and Java Extended architectures include email protocols (IMAP, SMTP, POP3) applications The PushToTest Web site supports this book with an active community of users The Web site contains all of the book’s source code and applications,... grid of inexpensive equipment) is everywhere And yet, until this book, there was no guide to show how your choices in design, coding, and testing impact the scalability, performance, and functionality of your Web-enabled applications This book will show you a fast and efficient method to go from basic Java knowledge to building production-worthy, scalable, and high-performing Web-enabled applications... Web-based test techniques, and present TestMaker, my free open-source framework for building intelligent test agents to check Java- based Web software for performance, scalability, and reliability I present case studies and immediately useful code of how Elsevier Science, 2Wire, Sun Microsystems, and BEA successfully use intelligent test agent technology to build scalable Java applications and ensure confidence... scalability, and performance You will then immediately learn useful new processes and techniques to measure and improve scalability and performance The design and architecture choices (server software, protocols, schemas, platforms, hardware, routing) you make impacts functionality, scalability, and performance The second part of this book describes the technology behind interoperating and scalable... security models into Web applications, WS-I standardizes a common set of Web Services programming interfaces and protocols, and BPEL4WS defines a way to express and implement business workflows in Web Service applications And there are many more on the way Part II of this book presents many of these new protocols and gives you a way to approach good design and test strategies for those protocols that... www.PushToTest.com, the Web-based community meeting place where ideas on software design, testing, and automation are exchanged every day In this book, I describe the architectural choices to build Web-enabled applications in Java and show how each choice impacts scalability and reliability I show how to test and optimize these systems in your own environment I describe the need for intelligent test... methods to software test automation and scalable system design Frank’s application of these methods is especially important in today’s environments where J2EE, NET, and the latest integration technologies are used to build information systems The new software testing methods introduced in this book will show you how to form and apply effective online goal-directed design and testing techniques You begin with... (including email address) and I will keep you informed about my current and future work, new products and services, and new books and articles Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark PH069-Cohen.book Page xxi Monday, March 15, 2004 9:00 AM Acknowledgments Also from Frank Cohen • • • Automating Web Tests Using TestMaker, 2003, PushPress, Author Java Web Services Unleashed, . Paradigms and Lifecycle Processes 22 Testing Methods 25 Click-Stream Testing 26 Unit Testing 27 Functional System Testing 28 Scalability and Performance Testing. achievement in clarity of Java testing, design, and optimization. To avoid hard-to- trace scalability and performance ‘gotchas’ inherent in early Java development,