Many e-business sites get 10 times their normal traffic loads at peak times. Ensuring that your site can handle the traffic while still meeting your customer's speed expectations is critical. To make sure that your site will be there when your customers need it, you need to do hardcore testing. What does this take? Plunder! Spying! Torture! E-business isn't for the meek.
Standard "Nice" Web Load Testing
Most Web site load testing is flawed, even dangerous if the results give you a false sense of your Web site or application's ability to handle a high load. Generally, load testing is done in a lab that simulates your production environment. Software is used to generate a load on the Web servers, and average page load and transaction times are recorded.
This approach doesn't cut it for many reasons:
* The test goals often don't match business goals and customer demands; * Lab environments usually don't match production environments; * Simulated loads usually don't match real loads; * Many potential bottlenecks are not tested in the lab; * Success criteria are often ill defined.
Real world testing requires an understanding of what people do at your site. This requires that you take testing out of the lab environment and include things like Internet bottlenecks, your ISP's limitations, load balancer configurations, and ASP providers. It also requires knowing success metrics based on what your competition is doing.
If you want to make sure your site can handle the stress that the Internet throws at it, then do real world load testing. Yes, it involves plunder, spying, and torture, but at least it's for a good cause.
Real-world Web load testing starts with plunder -- searching for the hidden gold in your Web server logs. Most companies use Web analytics to know how many "hits" they get and not much more. The most valuable information is often overlooked, that's why you need to "plunder" your Web logs.
Your server logs tell you the length of user sessions, the number of pages viewed per session, the most popular pages, common paths through the site, and more. What this represents is a profile of how Internet users use your Web site. This information is a requirement for any load testing to be accurate. This is the swag, the booty, that lesser Web builders will miss.
Plunder your logs and make sure you know these characteristics of your Web site usage:
* When your site is not busy; * How much traffic the site handles at peak load; * Length of user sessions; * Pages per user session; * Distribution of page requests (i.e., what pages are popular).
Using this information, you can plan your load testing for a time when your site is not busy to minimize impact on users. You can determine how much traffic to stress test your site with based upon past peak loads. Finally, past traffic characteristics tell you what sort of traffic your testing needs to mimic.
Finally, you should review what pages are popular against the business value of the pages. Most sites have a few key pages that actually pay the bills. These pages are ones that you will want to be sure to perform well under high levels of traffic.
Spy on Your Competitors
Next, it's time for competitive intelligence, also known as spying on the competition. Your Web logs tell you what pages you need to measure.
Your logs don't tell you how well your site needs to perform, though.
The best way to do that is to compare your pages to similar pages at your competitors' sites.
This can be done easily. First you need to review competitive sites and determine the pages that are the closest matches to yours. Benchmark these pages, using a tool like Sitescope or Keynote, to find out how well these pages perform. You shouldn't need to measure page load times very many times to get a feel for how competitive sites perform.
This gives you a baseline for setting performance goals for your site.
Your goal should be to provide users with better performance on the key pages of your site better than other sites.
Torture Your Site
Now that you know what to test and what your success metrics are, it's time to torture your site. Schedule testing for a time when site traffic is normally lowest.
Develop a test plan that simulates the load that your site gets when it's busy. Load testing for Internet servers should be done from outside your network. Virtual clients can be used from the Internet to simulate traffic to your servers. This is the only way that you can accurately identify real world problems. You may find bottlenecks that are outside your network, such as limitations at your ISP.
Your test should include multiple virtual users that follow various paths throughout the site. Your scenario should create a load that is close to what your site experiences at busy times. Make sure, too, that your virtual users are set up to mimic real world behavior. For example, if a page takes over 10 seconds to load, you should have a portion of the virtual users cancel the transaction. Confirm that your scenario is accurate by doing a short test, and then comparing your Web server logs from the test to logs from busy times. The page distribution from the test should be very similar to that from busy times. If it isn't, you need to tweak your test setup.
Finally, it's time to "drop a load". As the simulated load increases, monitor how key pages are performing. Keep increasing the load until it is well past any traffic load you've previously experienced. How hard you push it depends on how quickly your Web site traffic is growing. At minimum, test with double your current peak, but you may want to push it even harder. It's not uncommon for sites to experience tremendous traffic jumps based on company news or world events. Cranking up your stress test high will let you look into the future and understand how your site will work as your traffic grows.
Check Your Tracks
Once your test is done, check out the Web statistics collected during the testing period. If you did your research right, the distribution of traffic from your test period should be a close match to what your servers get under an actual heavy load. If not, your test is flawed.
Check out the average page response time, and the range of response times for key pages and transactions. Did most of the virtual users get page responses within competitive times? How did the range of response times change when the site was very busy?
Typically, stress testing will show that your infrastructure performs very well up to a certain point. Then, one part of your infrastructure becomes a bottleneck, and you can see dramatic shifts in how well your site performs. In some cases, a few more users will be like the straw that broke the camel's back, and your site may slow dramatically. You may also see the range of response times grow, so that some users get fast response times, while others get very slow times. Finally, you'll see page timeouts, or applications that fail under the load.
Load testing is dirty work. If you do it right, though, you'll be able to sleep at night confident that your site can handle whatever your customers throw at it.
Comments? Tell me what you think at firstname.lastname@example.org.