Power outages in Sydney this week have tested companies' business continuity plans, but at the same time have also shown the size of resources dedicated to system backup.
Fires burning around NSW close to major transmission lines have caused power interruptions since Wednesday afternoon. Many of the state's IT systems have relied on uninterruptible power supplies (UPS), but the power shortages still caused servers and desktops to reboot.
St George Bank group executive for IT, John Loebenstein, said the threat of data loss was minimal due to controls which the bank has in place.
"The main power supply was a bit unsteady for a while, but our UPS switched to battery supply for about four minutes [on Wednesday]," he said.
Servers at some St George branches rebooted, and while "you can't do much about that", Loebenstein said the problems were minor.
With UPSs in place, the only preventative action that needs to be taken when there's a chance of power failure is to have electricians on standby, Loebenstein said.
The bank's main data centre is in suburban Kogarah, with a backup centre in George Street in Sydney's CBD. This site provides backup mirroring in real time and the data would be less than a minute out of date, according to Loebenstein's estimate.
Ernst & Young (E&Y) CIO Stephen Arnold said that, in the event of the Sydney system's failure, the company has a secondary WAN which lets employees around Australia access resources via the Melbourne office.
E&Y's Sydney office holds 200 servers, supporting four terabytes of data.
E&Y experienced several power drops on Wednesday, which its three UPSs managed. UPS activity alerts E&Y IT staff by sending an alarm to their mobiles. However, the IT staff also had the less common advantage of laptops.
"We have laptops because we have a mobile workforce," Arnold said. "But about 18 months ago there was a substation that blew up. We lost power for a few hours. Eventually we ran out of battery power for the UPS, but before we shut down the server we asked people to download information they needed from the network so they could continue working on their laptops."
Insurance firm AAMI uses a decentralised solution for its telephony backup.
"All our phone systems run through a '1-3' number," said Ray Newton, AAMI telecommunications manager.
"If our central system goes down, calls can be rerouted through any state so our call centres etc can keep functioning," he said.
Energy Australia said its major commercial customers should be aware of the need to re-start their internal protection systems after power interruptions.
While Computerworld learned of many approaches to backing up systems, there is no standard metric for determining just how much money IT departments should be spending on them.
"You look at the industry standard, take a worldview," Arnold said. "Our backup strategies have been helped by all the planning we did for Y2K."
Glen Noble, GM of Macquarie Hosting Solutions at Macquarie Corporate Telecommunications, gave this advice:
"Figure the probability of such an event occurring, then the cost and mitigation," he said. "Then determine the cost of disaster recovery or offsite systems.
"Businesses need to ask, if I lost this system for a few hours, what is the real cost? They need to consider redundant telecommunication links, multiple lines and pairing, multiple DNS servers and the like."
As many larger businesses have head offices in Sydney, the fires had national relevance, Noble said.
"In the cases of most national businesses, if their national office in Sydney is affected, so are all the regional offices trying to access the data from the central hub," he said.