1 Tools 1.1 LoadRunner 6.5 LoadRunner is Mercury Interactive’s tool for testing the performance of client/server systems. LoadRunner enables you to test your system under controlled and peak load conditions. To generate load, LoadRunner runs thousands of Virtual Users that are distributed over a network. Using a minimum of hardware resources, these Virtual users provide consistent. Repeatable and measurable load to execute your client/server system just as real users would. LoadRunner’s in depth reports and graphs provide the information that you need to evaluate the performance of your client/server system. 1.2 WebLoad 4.5 Webload is a testing tool for testing the scalability, functionality and performance of Web-based applications – both Internet and Intranet. It can measure the performance of your application under any load conditions. Use WebLoad to test how well your web site will perform under real- world conditions by combining performance, load and functional tests or by running them individually. Webload supports HTTP1.0 and 1.1, including cookies, proxies, SSL, TSL, client certificates, authentifications, persistent connections and chunked transfer coding. Webload generates load by creating virtual clients that emulate network traffic. You create test scripts (called agendas) using Java Scripts that instruct those virtual clients about what to do. When Webload runs the test, it gathers results at a per-client, per-transaction and per-instance level from the computers that are generating the load. Webload can also gather information server’s performance monitor. You can watch the results as they occur- Webload displays them in graphs and tables in real-time and you can save and export the results when the test is finished. Performance Testing Tools - summary and comparison This table lists several performance testing tools available on the market. For your convenience we compared them based on cost and OS required. Tool Name URL Cost OS Description Web Performance Trainer http://www.webperf center.com/loadtesting. html Price ($) per number of virtual users: 1400-100 2495-200 4995-300 7995-1000 11995-5000 Windows NT, Windows 2000, Linux Solaris Load test tool emphasizing ease- of-use. Supports all browsers and web servers; simulates up to 200 users per playback machine at various connection speeds; records and allows viewing of exact bytes flowing between browser and server. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, and any other parameter to simulate multiple virtual users. Notes: downloadable, will emulate 25 users, and will expire in 2 weeks (may be extend) Astra LoadTest http://www.astratryand Price ($) SunOS, Mercury's buy.com per number of virtual users: 9995-50 17995-100 29995-250 HP-UX, IBM AIX, NCR, Windows NT, WIN2000 load/stress testing tool; includes record/playback capabilities; integrated spreadsheet parameterizes recorded input to exercise application with a wide variety of data. 'Scenario Builder' visually combines virtual users and host machines for tests representing real user traffic. 'Content Check' checks for failures under heavy load; Real-time monitors and analysis Notes: downloadable, evaluation version Benchmark Factory http://www.benchmark factory.com $ Windows NT, Windows2000 E-commerce load testing tool from Client/Server Solutions, Inc. Includes record/playback, web form processing, user sessions, scripting, cookies, SSL. Also includes pre- developed industry standard benchmarks such as AS3AP, Set- Query, Wisconsin, WebStone, and others. Includes optimized database drivers for vendor- neutral comparisons - MS SQL Server, Oracle 7 and 8, Sybase System 11, ODBC, IBM's DB2 CLI, Informix. Notes: downloadable (?), after submitting information A Page with suggestion to apply for next infos to closest dealers appeared Radview's WebLoad http://www.radview.com $ Win95/98 Windows NT, Windows 2000 Solaris, AIX Supports recording of SSL sessions, cookies, proxies, password authentication, dynamic HTML; multiple platforms Notes: downloadable, Evaluation version does not support SSl MS Web Application Stress Test http://homer.rte.microsoft. com Free Windows NT, Windows2000 Microsoft stress test tool created by Microsoft's Internal Tools Group (ITG) and subsequently made available for external use. Includes record/playback, script recording from browser, SSL, adjustable delay between requests Notes: one of the advanced tools in the listing… Rational Suite Performance http://www.rational.com/ products $ Windows NT, Windows2000, Unix Rational's client/server and web performance Studio, Rational SiteLoad testing tool. 'LoadSmart Scheduling' capabilities allow complex usage scenarios and randomized transaction sequences; handles dynamic web pages. Notes: request a cd only. Not downloadable Forecast http://www.facilita.co.uk $ Unix Load testing tool from Facilita Software for web, client-server, network, and database systems Notes: not downloadable Zeus http://webperf.zeus. co.uk/intro.html Free Unix Free web benchmarking/load testing tool available as source code; will compile on any UNIX platform Notes: unsupportable (?), broken download link. E-Load http://www.rswsoftware. com/products/eload_ index.shtml $ Win95/98/ Windows NT Load test tool from RSW geared to testing web applications under load and testing scalability of E- commerce applications. For use in conjunction with test scripts from their e-Tester functional test tool. Allows on-the-fly changes and has real-time reporting capabilities. Notes: downloadable, free cd request, evaluation copy HTTP-Load http://www.acme.com/ software/http_load Free Unix Free load test application to generate web server loads Notes: free and easy. QALoad http://www.compuware. com/products/auto/ releases/QALoad.htm $ Win95/NT - manager; Unix, Windows NT - load test player Compuware's QALoad for load/stress testing of database, web, and char-based systems, works with such middleware as: SQLnet, DBLib or CBLib, SQL Server, ODBC, Telnet, and Web Notes : free cd request SilkPerformer http://www.segue.com/ html/s_solutions/s_perf ormer/s_performer.htm $ Windows NT, Windows 2000 Load and performance testing component of Segue's Silk web testing toolset. Notes: no download. WEBArt http://www.oclc.org/ webart $ Windows 98, Windows NT 4.0, Windows 2000, SunOS/Solaris, AIX, Linux Tool for load testing of up to 100-200 simulated users; also includes functional and regression testing capabilities, and capture/playback and scripting language. Evaluation copy avail. Notes: downloadable Webload http://www.ca.com/ products/platinum/ appdev/fe_iltps.htm $ AIX, Windows NT, Windows 95, Sun Solaris Final Exam WebLoad integration and pre- deployment testing ensures the reliability, performance, and scalability of Web applications. It generates and monitors load stress tests - which can be recorded during a Web session with any browser - and assesses Web application performance under user-defined variable system loads. Load scenarios can include unlimited numbers of virtual users on one or more load servers, as well as single users on multiple client workstations. Notes: downloadable, 15day eval. period Microsoft WCAT load test tool http://msdn.microsoft. com/workshop/server/ toolbox/wcat.asp Free Windows NT, Windows 2000 Web load test tool from Microsoft for load testing of MS IIS on NT Webspray http://www.redhillnet $199 ($99 with Windows 98, Windows NT Load testing tool; includes link testing works.com discount) 4.0, Windows 2000 capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. Notes: not downloadable WebSizr, WebCorder http://www.technova tions.com/home.htm $ Win95(98), Windows NT, Windows 2000 Load testing and capture/playback tools from Technovations. WebSizr load testing tool supports authentication, cookies, redirects Notes: downloadable, 30 eval. period. 1.3 Architecture Benchmarking • Hardware Benchmarking - Hardware benchmarking is performed to size the application with the planned Hardware platform. It is significantly different from capacity planning exercise in that it is done after development and before deployment • Software Benchmarking - Defining the right placement and composition of software instances can help in vertical scalability of the system without addition of hardware resources. This is achieved through software benchmark test. 1.4 General Tests What follows is a list of tests adaptable to assess the performance of most systems. The methodologies below are generic, allowing one to use a wide range of tools to conduct the assessments. Methodology Definitions • Result: provide information about what the test will accomplish. • Purpose: explains the value and focus of the test, along with some simple background information that might be helpful during testing. • Constraints: details any constraints and values that should not be exceeded during testing. • Time estimate: a rough estimate of the amount of time that the test may take to complete. • Type of workload: in order to properly achieve the goals of the test, each test requires a certain type of workload. This methodology specification provides information on the appropriate script of pages or transactions for the user. • Methodology: a list of suggested steps to take in order to assess the system under test. • What to look for: contains information on behaviors, issues and errors to pay attention to during and after the test. [...]... measures However, most available code coverage tools do not provide much beyond basic line coverage In theory, you should have more But in practice, if you achieve 95+% line coverage and still have time and budget to commit to further testing improvements, it is an enviable commitment to quality! 6.6 How Test Coverage Tools Work To monitor execution, test coverage tools generally "instrument" the program... preferred choice for load testing a Web application The testing tools typically use three major components to execute a test: • • • A console, which organizes, drives and manages the load Virtual users, performing a business process on a client application Load servers, which are used to run the virtual users With automated load testing tools, tests can be easily rerun any number of times and the results... memory, disk space and network usage on individual web, application and database servers and make sure those numbers recede as load decreases Cognizant has built custom monitoring tools to collect the statistics Third party monitoring tools are also used based on the requirement 2.1 Client Side Statistics • • • • • Running Vusers Hits per Second Throughput HTTP Status Code HTTP responses per Second • • •... coverage tool is not waiting Like Executable Instrumentation, Runtime Instrumentation is independent of programming language but dependent on operating environment 6.7 Test Coverage Tools at a Glance There are lots of tools available for measuring Test coverage Company Product OS Lang Bullseye BullseyeCoverage Win32, Unix C/C++ CompuWare DevPartner Win32 C/C++, Java, VB Rational (IBM) PurifyPlus Win32,... results, we arrive at the count of Total Test Case Points 7.3 Chapter Summary This chapter covered the basics on      What is Test Coverage Test Coverage measures How does Test coverage tools work List of Test Coverage tools What is TCP and how to calculate the Test Case Points for an application ... Coverage Probably the most basic form of test coverage is to measure what procedures were and were not executed during the test suite This simple statistic is typically available from execution profiling tools, whose job is really to measure performance bottlenecks If the execution time in some procedures is zero, you need to write new tests that hit those procedures But this measure of test coverage is... monitoring :the clients, the network, the webs server, the application server, the database etc., This will result in instantly identifying the performance bottle necks during load testing But if the tools support real time monitoring then testers would be able to view the application performance at any time during the test Thus running the load test scenario and monitoring the performance would accelerate... opportunity lowering transaction costs and strengthening profitability Fortunately, robust and viable solutions exist to help fend off disasters that result from poor performance Automated load testing tools and services are available to meet the critical need of measuring and optimizing complex and dynamic application and infrastructure performance Once these solutions are properly adopted and utilized,... servers, which are used to run the virtual users With automated load testing tools, tests can be easily rerun any number of times and the results can be reported automatically In this way, automated testing tools provide a more cost-effective and efficient solution than their manual counterparts Plus, they minimize the risk of human error during testing 5 Stress Testing 5.1 Introduction to Stress Testing . test is finished. Performance Testing Tools - summary and comparison This table lists several performance testing tools available on the market. For your. decreases. Cognizant has built custom monitoring tools to collect the statistics. Third party monitoring tools are also used based on the requirement. 2.1