Predicting how much energy a household will use in the future -- perhaps a year or five years from now -- is the lifeblood of Direct Energy. The company traditionally forecasts energy consumption at an error rate of less than 4 per cent. That's better than industry standards, but even a small miscalculation could mean millions of dollars in losses.

Predicting long-term energy needs is tricky, says Craig Williamson, practice director at research firm Energy Insights in the US. In a March 2006 survey of 25 utility companies, Williamson found that about 40 per cent had formal targets for determining how closely forecasts matched actual consumption six months to 30 years down the road. The other 60 per cent didn't have goals or just didn't track the accuracy of their forecasts, "because it's difficult to do," he says.

Still, Direct Energy's IT team thought the company could do better. It proposed a way to improve forecasting accuracy that would combine some off-the-shelf technology with specialized forecasting techniques -- all developed by in-house staffers. The new system would allow the company to expand and improve the product without relying on outside consultants or programmers. Significant cost savings would also be realized.

The IT team partnered with key stakeholders, including in-house meteorologists and statisticians, to launch the initiative, known as Project Northstar.

In April 2006, "we started fresh -- nothing from the previous system," says Hugh Scott, vice president of wholesale and risk information systems. The IT team relied heavily on SQL Server 2005, although Scott admits to initially experiencing a "little bit of nervousness" with the product because it was new at the time.

One of the key pieces of functionality is a backcasting feature developed by the IT team. At Direct Energy, "backcasting" refers to looking at what has happened with weather and energy consumption in the past and determining whether their relationship was predicted correctly.

"It allows you to correlate the accuracy of those predictions with the accuracy of the energy forecasting software. That improves your methodology," says Kumud Kalia, CIO and executive vice president of customer operations.

The algorithms in the forecasting system measure weather conditions, water levels and geographic characteristics -- just some of 100 different inputs that go into energy forecasting.

The new system went into full production just six months after the project began. But although Northstar was developed quickly, users weren't ready to accept it right away.

"It took a while to gain acceptance," Kalia says, noting that employees ran the old and new systems side by side for three to five months before switching over exclusively to Northstar. "It's understandable," he adds. "If the new system turns out to be wrong but people accept it, we could lose millions of dollars just as easily."

Kalia says that in retrospect, he would have specifically defined what conditions had to be met in order for the new system to be deemed "acceptable" and the old system cut off.

But so far, Northstar is proving itself. "It accurately forecast our Texas customers at 17 million megawatt-hours of electricity usage within the target error rate of 3 per cent," Scott says. Today, Northstar is being used across multiple business units at Direct Energy, and they're predicting energy consumption rates into 2012.

With all intellectual property maintained in-house, a calculated return on investment of 150 per cent for four years (900 per cent total ROI) and potentially millions of dollars in savings due to decreased risk, Kalia says, "Northstar has surpassed all expectations."

Collett is a Computerworld contributing writer. Contact her at Stcollett@aol.com.