Product Review: Chariot rides to the rescue

It's three months into your deployment of asymmetric digital subscriber line (ADSL) and your "guaranteed" 1.5Mbit/sec lines are providing, at best, 200Kbit/sec. Senior management is not happy, and your bag of tricks is empty. What do you do?

Judging from our test results, installing Ganymede Software's Chariot 2.2 would be a good start. Chariot is a software tool for stress-testing network hardware and modelling the behaviour of new applications prior to deployment. Chariot 2.2 earns our World Class Award for its ability to zero in on the source of problems, as well as its strong documentation, easy installation and range of application test scripts.

In the interest of full disclosure, Duke University, North Carolina, US, where this test was conducted, has been a beta-test site for Ganymede for more than a year and has used Chariot on a number of occasions to diagnose problems on our enterprise network. That undoubtedly made it easier for us to work with the product, which requires a strong grasp of network essentials. Effective use of Chariot presumes a good understanding of TCP/IP, routers, switches and operating systems.

Pointing the Way

Chariot 2.2 measures performance between networked computers using Network Performance Endpoints (NPE) installed on each machine. An NPE is a small application that resides on a workstation and listens for commands from the Chariot console.

NPEs are used in pairs. Each pair uses an application script that emulates the network behaviour of the application being tested. Ganymede provides a set of application scripts that includes standard performance benchmarks and well-known applications such as SAP/R3, File Transfer Protocol (FTP), Post Office Protocol 3 and Lotus Notes. New in Chariot Version 2.2 are scripts for multicast and multimedia applications.

With a tool as powerful as Chariot, there are some security risks of which prospective users should be aware. The possibility that a poorly planned test could wreak havoc on a production network should not be taken lightly. Fortunately, Ganymede anticipated this risk and took steps to prevent the misuse of the product. Chariot operators can specify which consoles are authorized to run tests by entering the console IP address in the NPE's ENDPOINT.INI file. The installation default is "ALL", which allows any Chariot console to initiate a test.

For sites with fewer than 25 endpoints, the default setting doesn't present a great deal of risk. But for larger installations we recommend configuring endpoints with a list of approved consoles.

We found that Chariot really came into its own as an on-the-fly network testing tool. Case in point: the not-so-hypothetical ADSL throughput problem we mentioned earlier.

Our service provider tried every testing tool it could find: Rick Jones' NetPerf, Maximized Software's TCP-Speed, VitalSign's VitalSuite and plain-vanilla FTP transfers, but it was unable to identify the source of the slowdown. We installed Chariot 2.2 on a server located between our Cisco 7513 router and the service provider's Cisco 4500. After only 15 minutes of testing, we pinned down the source of the congestion: configuration settings on the 4500.

Using Chariot's File Send Long and FTP-Put scripts, we were able to immediately confirm that our link was providing a maximum of 200K bps for a single TCP stream. We then employed one of Chariot's most useful features: the ability to run multiple TCP connections on the same machine while aggregating the results into a single report.

As soon as we kicked off the 10-connection test we saw what all the other attempts at diagnosing the problem had missed: The network was providing adequate throughput. Our 10-connection test gave us 1.3Mbit/sec of aggregated bandwidth. These results indicated that the Cisco router was limiting each stream to around 200Kbit/sec of throughput.

Using the test results we were able to work with Cisco and our service provider to identify the settings on the Cisco 4500 that needed to be changed. After turning off weighted fair queuing and allocating more buffer space, our single-stream throughput shot up to 1.25Mbit/sec. Problem solved.

The tests also demonstrated the strength and clarity of Chariot's reporting capabilities. The Chariot Test screen is divided into three windows: a detailed breakdown of endpoint characteristics; a graph of throughput; and a legend of the endpoints.

Reports can be printed from the Chariot Test window or exported into HTML, Microsoft Excel or text formats. We tried all of the export options and encountered no problems. We found the HTML export to be particularly useful. While diagnosing our ADSL problem, we were able to run tests and export the results to our Web site in near real time, allowing the service provider's engineers to see exactly what effects the changes we made had on performance.

Documentation and Installation

Ganymede's documentation is equally top-notch. Chariot's manuals span three volumes: the User's Guide; Messages and Application Scripts; and Network Performance Endpoints. We found the User's Guide to be especially clear, concise and well written. This volume takes users from installation through the deployment and operation of Chariot in an enterprise environment.

Installation was a snap and didn't require a reboot of the console or endpoint workstation before using - always a nice feature.

While Chariot lets you install an NPE on the workstation being used as the console, it isn't recommended. But we found it to be a handy way of verifying proper operation of the console and test scripts.

Our only concern with Chariot is a Catch-22: The application is so easy to use that you can easily construct a test with disastrous effects. There aren't any warnings before a potentially router-crushing run begins. Chariot isn't an application you want to experiment with on a production network.

Nevertheless, Chariot 2.2 is an invaluable kit for enterprise network managers. Its flexibility and ability to emulate applications and aggregate multiple streams into a single report make this a must-have package. Chariot 2.2 earns our strongest recommendation and a World Class Award.

(Currier is director of data communications at Duke University in Durham, North Carolina, and the 1998 Grand Prize winner in the Excellence in Campus Networking competition sponsored by CAUSE, a user group for computer professionals in higher education. He can be reached at robert.currier@duke.edu.)

Join the newsletter!

Error: Please check your email address.

More about EndPointsGanymede SoftwareMicrosoftNetwork EssentialsSAP AustraliaSECVitalSuite

Show Comments