Phil Goodwin, research director for the Meta Group, recently proposed some better ways of making a financial case for new storage systems than the old tried-and-true return on investment (ROI) calculations. He suggests using mean time to data recovery, data availability statistics, and the amount of storage managed per administrator.

He also suggests making better use of chargeback systems, so that IT-ers can tell various business units for sure what they're using for storage and how much that costs. That way, when the sales group says they need an extra Tbyte for all their prospect-tracking data, IT management can tell the sales VP what that will cost the group and/or the company in general.

Goodwin made his remarks at the recent Storage Management 2003 conference, according to published reports. (I wasn't there.) I think he's onto something, even though I'm not sure I agree with all his proposals.

It's clear that the usual ROI and total cost of ownership (TCO) calculations are pretty much the computer industry's dirty little secrets. Like most composite calculations, they can be skewed to show whatever you want them to. One CIO I interviewed calls ROI measurements "liar's poker, where the biggest liar wins." And then, once the ROI number is "cooked," few people actually go back to see if those calculations were correct. (Did the technology investment actually return what was thought? Or did it do better or worse than predicted?)

Instead, once the system is justified on paper and that justification is accepted by whoever is signing the check, the system is bought and paid for and implemented, and that's pretty much it. (Those online "ROI calculators” at some vendors' Web sites really crack me up. As if any single set of calculations could even come close to figuring out actual ROI for different customers' needs. Gimme a break!)

TCO is even a bigger black hole. Except for vendors and consulting groups, nobody does TCO analysis, even though it's a good idea. It just takes too much time and resources to do these kind of financial calculations except for the most expensive or strategic systems, and storage is usually not considered part of that. (Even if it is.) Instead, storage is usually paid for out of the ongoing operations budget.

Which is fine, mostly, if all you're doing is adding capacity onto an existing array, for instance. But even then, someone has to plead the case to spend money, and this is usually an administrator who may not be wise in the ways of the TCO dragon.

So, back to Goodwin's suggestions. I like chargeback, which of course as a concept is as old as the mainframe. (Mainframe systems administrators used chargeback tools to tell various business users what percentage of the Big Iron they needed to pay for out of their pieces of the budget.) As useful a concept as this is, though, I'm not sure how many IT staffs are set up to do this. It involves more than just popping a chargeback tool onto Ye Olde Server. It involves setting up the system and then educating IT staffers and, more important, business unit decision-makers, so they know what you're measuring and how and what it means. And its use and importance will depend on how the company approaches the budget process. Do business units have "real" budgets, or are those budgets on paper only? Executive buy-in - at the board level - is critical for chargeback systems to be really effective.

Some of Goodwin's other metrics - like mean time to failure – are wonderful in the context of the IT department, but I'm wondering how they'll play in User-Land. Similarly, I don't know about using the amount of storage managed per administrator, either. There's a difference between storage that's administered and storage that's actively managed. If we're talking about the latter, and there are some parameters around what that means, that could be useful.

It's wonderful that Goodwin (and others) are raising this issue. I just don't know if we have any meaningful answers that could become (or already are) industry-wide standards. Perhaps the bottom line is that different companies will need to track different metrics that are meaningful to them, and that there's no one "magic number" that will play across all customer shops.