Primus outage highlights data centre fragility

Statistically more reliable than server rooms

More mission-critical IP telephony and business computer systems are being hosted in data centres but two recent outages at Equinix and Primus raise new questions about their inherent reliability.

Members of the Australian Network Operators Group began reporting Primus in Melbourne suffered a power loss at around 4pm on Sunday, August 9.

“I just spoke to the datacenter manager who said he was having trouble getting through to them,” one member wrote on the mailing list.

Primus managing director, Ravi Bhatia, cited a fault with a sub-station as the cause of the outage.

According to Primus, the sub-station fault prevented a generator from starting but that’s what generators are there for – to provide power independent of the public grid in the event of an outage.

Primus is working to upgrade its power management infrastructure.

The outage comes one month after Equinix’s Sydney data centre suffered a power outage causing downtime for hosted VoIP services.

“Pretty poor, ~2.5 hrs and counting, this joint has worse uptime than my house... hard to believe we pay for this,” AusNOG member Damien Coxall wrote about the Primus outage.

Carlo Minassian, managing director of Sydney-based managed network security services provider earthwave, said such outages highlight the need for regular testing of power generation equipment.

“It is unfortunate this has happened to tier-1 facilities, but it shows they don’t have the right test procedures and have not validated their infrastructure to work in the event of a disaster,” Minassian said.

One data centre earthwave uses is owned by Fujitsu and located in Sydney. The data centre has multiple generators on site which are kept on standby and tested every quarter.

“Over the past nine years I have been using our facility we have had a number of outages, one lasting about five hours, and the generators worked during that time,” Minassian said.

“I would be definitely asking questions about the business continuity and DR plans and looking for the test results.”

Despite the outages, Minassian said data centres are always going to be more reliable than in-house server rooms as they are designed from the ground up for high-availability.

“Computer rooms in buildings are generally not built to that standard,” he said. “It will be difficult for a corporate or government organisation to invest in excess of $50 million to build a data centre.”

James Braunegg from the Micron21 Melbourne Outer East data centre said the important question will be "is this going to happen again next weekend?" when the power in Primus is upgraded.

More about: Equinix, etwork, Fujitsu, Micron, Primus
References show all

Comments

1

Lily Lewis

Wed 12/08/2009 - 09:58

The amount of small business managers who have no idea what kind of facility their website is stored on astounds me, especially in relation to the potential devastation server downtime can cause to their business in the way of sales and communication.
There are so many "backyard businesses" operating, and with the Primus outage it shows that size obviously doesn't count for reliability!
It is essential when choosing your website host that you have thoroughly checked their security and maintenance policies to ensure that such things as power, local natural disasters or hackers don't affect their uptime or access sensitive secure data.
Lily Lewis www.successfulwebsites.com.au

2

Anonymous

Wed 12/08/2009 - 12:05

Primus should be sued for this outage. When the exact same issue occurs twice in 12 months they clearly arent learning or fixing problems that should never have occurred in the first place. If you have a server in your back office outage is expected but these kinds of centres should be capable of staying online during power outages like black saturday and any other equipment failure events. The Verizon data centre we use in Melbourne has been rock solid for 3 years. Clearly you get what you pay for.

Post new comment

The content of this field is kept private and will not be shown publicly.
Users posting comments agree to the Computerworld comments policy.
Login or register to link comments to your user profile, or you may also post a comment without being logged in.
Related Coverage
Related Whitepapers
Latest Stories
Community Comments
Tags: data centres, power, power management, Primus Telecom
Whitepapers
All whitepapers
Sign up now to get free exclusive access to reports, research and invitation only events.
Featured Download
/downloads/product/58/seamonkey/

Seamonkey

Seamonkey includes an Internet browser, email and newsgroup client with an included web feed reader, HTML editor, IRC chat and web development tools. SeaMonkey will ...

Computerworld newsletter

Join the most dedicated community for IT managers, leaders and professionals in Australia