Extreme traffic loads require extreme measures, as Microsoft found out while building the official Commonwealth Games Web site over the past year.

As the Commonwealth Games takes place once every four years, to say there is considerable pressure on Microsoft to get its IT infrastructure right is an understatement.

While the Games opening ceremony bathed Melbourne in pyrotechnic and piscatorial glory, a team of Microsoft engineers were inhaling sharply as the inevitable crush of Web site visits put 18 months of planning to the test.

Games organizers are anticipating more than 15 million visitors to the site (www.melbourne2006.com.au), generating up to 22 million page views during each of the 12 days of the event. That's the kind of load that makes most corporate information managers cower with fear - and the kind of challenge that has made IT planning for one-off sports events a niche capability restricted to the largest IT companies.

The site features a pantheon of Microsoft server technologies, ranging from Windows 2003 Server and SQL Server 2000 for content management to the newer SQL Server 2005 database for rapid search and analysis of the cumulative Games data. Content Management Server 2002 (CMS) handles online content delivery, BizTalk Server 2004 handles the movement of data between the systems, and Visual Studio 2003 provided the collaborative development environment for the site.

With BizTalk at the centre of the Microsoft ecosystem, data coming from official Omega scoring systems is automatically checked for accuracy and validity. "These rules are essential for ensuring data integrity," says James Simpson, services program manager with Microsoft Australia. "If the system received data for a swimming event while the pool was closed, for example, the system would spot the anomaly and prevent the data from corrupting the official results."

Once it has passed BizTalk's conversion and checking policies, data is simultaneously published to both SQL Server for archiving and analysis, and the CMS for use by the fleet of journalists covering the event. More than 2500 different content areas provide detailed coverage of individual events and countries' performance, as well as player biographies, contextual histories, and other relevant information.

Content creation has been tailored for fast turnaround: use of a range of content templates, intelligently loaded with new data as it comes in from Games venues, means that new results from competitions are highlighted through the Web-based interface. The Games' team of more than 15 content editors will use this interface to quickly pick up leads, then add details and publish completed stories online as they undertake the mammoth effort of keeping site content both fresh and relevant.

Despite the Games' high profile and cost, Simpson says Microsoft's team was facing a tightly constrained budget that made it essential for the system to be developed as quickly, smoothly and accurately as possible.

This brought the site's development within the purview of Microsoft's solution development centre (SDC), which uses dedicated facilities in Sydney and Melbourne to give Microsoft's enterprise customers a sandbox for building and testing massive, complex and highly integrated applications. Entry into the SDC is highly selective, with just three or four projects taken on each year.

Construction of the Games Web site was guided by the Microsoft Solutions Framework and Scrum, an iterative development process that combines extreme programming and agile development philosophies to hasten building of large applications. A key tenet of Scrum is the idea of building and testing frequent test versions of the application throughout its development: in the case of the Games site, the 12-person development team did regular small builds and a complete site build after each of the 12 months of development.

"In the end, we were able to build the site using just 144 person-months of development time," says Simpson. "Using normal methods, a project of this size and scope would have taken more than 250 person-months."

The result of the process was a highly scalable Web environment that is now hosted on a bank of 40 Egenera blade servers in a Telstra-managed data centre in the Melbourne CBD. Multiple gigabits per second of redundant data connections link the data centre with key Games venues both in the city and in regional areas, ensuring fast response times for both internal and external users.

Despite the expected crush of Web site visitors, Simpson is confident the site will be able to keep up. Testing at the SDC hammered the site with as many as 10,000 requests per second, which translates to some 864 million requests per day - providing more than enough headroom for the site's expected load.

By contrast, most companies doing transactional environments would never see more than a few hundred requests per second.

In the lead-up to the Games, the site was happily humming along with some eight to nine million page views per week, but the inevitable spike will test the ADC's development approach to its limits. Once the Games are over, the entire site will be shunted to a lower-volume hosted environment for future reference.

Yet even when the Games finish, the work put into the site won't be for naught: the best-practice lessons learned by the team will be reviewed, documented and donated to the Victorian Government for potential use in its own future projects - and during Microsoft's preparations to do it all over again as it readies for the 2010 Commonwealth Games in New Delhi, India.