MS Web Application Stress Tutorial

Installing WAS

Copy SETUP.EXE to each client machine. Double click on this executable to start the installation. Follow the setup wizard to complete the install. Do not install WAS on your web server, as this may affect the performance of the server being tested.

Using the WAS sample script

After installing and opening WAS for the first time, a script is created called Sample Script. Use this script to get a feel for some of the features in the WAS web stress tool. The actual files used in the Sample Script are stored in the folder where you installed WAS. Copy the 'WASsamples' folder to the root directory of your web site before continuing with this tutorial.

Script view, script items, and script item detail view

The left hand window of WAS is known as the script view. This view displays all of the scripts stored in the current WAS database. If this is a new database then the only items you see in the script view are Defaults and Sample Script.

There are seven script items in the Sample Script. Each script item utilizes a special feature of WAS. For example, notice that one of the script items is a POST. Using your mouse, highlight the row header to the left of the POST script item, then double-click. This opens the script item details view. From this view you can edit the querystring name-value pairs, change POST data, modify the header, and enable secure socket layer.

See full-sized image.

Take a look at the Querystring tab view. Notice that the names User and Password are being passed in the querystring. Also notice that %Username% and %Password% are the values being passed. These are not literal values, they are special variables that tell WAS to pass the next available user and next available password from the list. WAS automatically cycles through the user names and passwords, passing the next set with each POST. See below for more information on WAS users.

Close the script item details view by clicking the OK button.

Page groups

The last two script items in the Sample Script contain the text 'adGrp' under the Group column. This is a page group. A page group is shown as 'Default' unless you change it. Page groups are used to reorganize the order in which the script items are invoked. It is also used to change the number of times that each script item is invoked while a script is running.

You can see a list of all the page groups by selecting the Page Groups node in the script tree view. You may also change the distribution percentages from this view. Notice that keep-alives are enabled for entire page groups at a time. For more in-depth discussion of page groups in Microsoft WAS, see the following knowledge base article: Using page groups.

See full-sized image.

Performance counters

Select the Perf Counters node from the script tree and click on the Add counter button. It may take a moment or so to load the Add Counters dialog when this button is clicked for the first time. Add the following counters:

•	Web Service: Get Requests/sec
•	Web Service: Post Requests/sec
•	System: % Total processor time
•	Active Server Pages: Requests/sec

Change the Collection interval to every 5 seconds. Capturing the correct performance monitor counters is critical to obtaining the correct data to analyze when the test completes. There are several important performance monitor counters to choose from, based on the type of application you are testing. Click here to view a list of the most common web related counters and an explanation of each.

See full-sized image.

Settings

Select the Settings node and change the Test Run Time to 1 minute, down from 15. Leave the other settings as they are for the time being but look over the other options in this view to get an idea of what can be configured for a script. You may also set the default settings for all new scripts by selecting the Defaults node and changing those options. Keep in mind that the settings in the Defaults node will not affect existing scripts, such as the Sample Script.

See full-sized image.

Users

Select the Users node and double-click on the Default user population, indicated by the following icon:

This opens the Users view where you can add and delete users from the default population and create new populations. Each WAS user stores cookie information and authentication data. Notice that there are 20 users in the default population.

WAS users are not the same as Stress level (threads), a setting located in the Concurrent Connections section of the Settings node. The two concepts should be kept separate.

Clients

Select Scripts from the View menu to return to the Script view. Select the Clients node under the Sample Script and double-click on the Default client group, indicated by the following icon:

This opens the Clients view where you can add and delete client machines from the current group or add new groups of client machines. Notice that 'localhost' is the only client and that it has a check box next to it. This means that the current machine is acting as a WAS client. Leave the Client view as is and select Scripts from the View menu to return to the Script view.

For more in-depth discussion of users and clients in Microsoft WAS, see the following knowledge base article: Understanding Threads, Users, and Clients.

Running a test

Once you have created a script and configured all the settings, users, and clients, it is time to start the test. Select the Sample Script and choose Run form the Scripts menu. Allow the test to complete.

Reporting

Select Reports from the View menu to open the Reports view. Expand the Sample Script report to display all of the report nodes. There should be at least one node who's title is the date and time in which your latest test was started. Expand and select the top level of this report node to view a summary of this test.

See full-sized image.

You can select a specific node of the report tree to view more in-depth information. For example, select the Result Codes node to display a sum of the HTTP result codes for every request in the test. Expanding the Perf Counters node displays the counters that have been collected during the test.

Expand the Page data node and select the first script item. The right hand pane displays detailed information regarding this script item. Reports provide the response time for specific pages by determining when the script has finished downloading on the client. This is a good source of performance data.

The Time To First Byte (TTFB) calculates the time from the request for the page until WAS receives the first byte of data, in milliseconds. The Time To Last Byte (TTLB) calculates the total time from the request until the last byte of data has been received on the client, in milliseconds. This number includes the TTFB time and any additional time needed to receive the last byte of data. All of the requests are sorted, then the data is divided into percentiles. For a more in-depth discussion of the p-squared algorithm and percentiles, see the following WAS knowledge base article: Percentiles in WAS.

While in report view, you can select Export to CSV from the File menu. Exporting the report values to a format that Microsoft Excel can read allows you to create charts that show where most of the requests fall. For example, the following chart illustrates Time To Last Byte:

 

See full-sized image.

General guidelines for using WAS

•	As a rule, use between 10 and 100 threads. You will rarely have a need to run a stress test that requires more than 100 threads per client machine. Be sure to monitor the processor utilization on the clients. Anything below 80% should be OK. The client machine(s) may not be capable of sustaining the stress loads where the processor utilization is greater than 80%, at which point the test will become invalid.
•	When adjusting the threads and sockets for a WAS stress test, use just one socket (stress multiplier) unless you are performing a special type of test. See the online help topic "Stress level vs. stress multiplier" if you require more information on this setting.
•	Limit the number of users to less than 1,000 unless there is a specific reason that you require more unique users. Although the number of users allowable is only limited by the amount of memory on the client machine, you may find that it takes too long for a test to initialize when using a large number of users.
•	Avoid creating scripts with more than 1000 script items. The number of scripts is only limited by the amount of memory on the client machine but you may find that it takes too long for a test to initialize when using a large number of script items.

Web testing overview

To use the WAS web stress tool to it's fullest potential, you'll need a general understanding of web testing methodology. The following discussion touches on several of the key areas of this field.

Web testing is divided into three main categories:

•	Performance testing
•	Stability or stress testing
•	Capacity planning

The first task in performance testing is to use a tool to apply stress to the web site and measure the maximum requests per second that the web server can handle. This is a quantitative measurement. The second task is to determine which resource prevents the requests per second from going higher, such as CPU, memory, or backend dependencies. This second task is more of an art than a measurement.

In many situations, the web server processor is the bottleneck. Increase the stress to the point where the requests per second start to decrease, then back the stress off slightly. This is the maximum performance that the web site can achieve. Increasing the stress is accomplished by increasing the Stress level (threads) setting. Increasing the number of WAS client machines will also produce a greater stress level.

On the web server, use Performance Monitor to watch the "System: % Total Processor Time" and "Web Service: Connection Attempts/sec" as well as "Active Server Pages: Requests Queued" counters. If the processors are running at 80 to 85%, then they are most likely the bottleneck. If the Requests Queued fluctuate considerably during the stress and the processor utilization remains relatively low, this is an indication that the script is calling a server COM component that is receiving more calls than it can handle. In this case, the server COM component is the bottleneck.

It is a good idea to have an expected peak load that meets your business needs and then create a test that uses enough threads to reach that capacity. This will help you determine the maximum request rate for the web application and confirm that it is in line with the expected peak load.

The amount of personalization in a web application also plays a significant role in how it performs. WAS contains several features that make testing a personalized web site easy. For example you can create users, which allows WAS to save a cookie with each. You can also use the Querystring editor to help create and store several sets of name-value pairs that are passed with each request.

Common web testing problems

1.	Invalid test platform. Doesn't match production servers.
2.	Invalid script. Doesn't accurately simulate production request behavior.
3.	Thread safety issues with unstable server COM components.
4.	Active Server Page script errors and GLOBAL.ASA issues.
5.	Insufficient processor power and/or scripts are too complex for the hardware.