User beware

Vendor-provided Web-based tools for calculating customers' returns on investment in hardware, software and other gear are all the rage in today's show-me-the-value budgeting climate.

Log on to SAP AG's, answer a series of questions about your company's annual revenue, profit margin, cost of capital and business processes and presto! Five or six minutes later, the online calculator "will reveal how much your company can benefit by taking the next step in the CRM revolution."

Spend another five or six minutes with SAP's supply chain management value calculator, and it will tell you "how much your company can benefit when it takes the next step in e-business evolution."

Skeptical? You should be.

Polly Foote's initial reaction to ROI figures served up by Pleasanton, Calif.-based PeopleSoft Inc.'s online calculator was pure astonishment. As a human resources business analyst at Ferguson Enterprises Inc., a US$5 billion plumbing supply distributor in Newport News, Va., Foote last June was trying to make a business case for upgrading to a newer version of the vendor's human resources software.

"The first numbers that we came out with said the ROI was going to be 798 percent and the internal rate of return was going to be 102 percent," Foote recalls. "Obviously, those numbers are pretty astounding."

So Foote reworked the numbers, excluding so-called soft benefits that PeopleSoft said could accrue from using its new software. On the second try, she also plugged in actual data about Ferguson's employees and their salaries and benefits, rather than relying on industry benchmark data provided by PeopleSoft.

The result: a significantly lower ROI figure of 212 percent, a 42 percent internal rate of return and a project payback period of 2.6 years.

The lesson learned, Foote says, is if you do use a vendor's calculator, do so with caution.

"What the calculator does for you is put into hard dollars your argument [for new or enhanced technology]. It gets you to the bottom line that executives want to hear because that's their language," Foote says.

"The caveat [about using the calculators] is that you customize all information to your company," instead of using the benchmark data that the vendor may provide, she says.

One Size Doesn't Fit All

As a wholesale plumbing distributor, "we don't usually fit into benchmarks real well," Foote explains. "Benchmarks are for companies where 100 percent of employees have Internet access and are computer-savvy. Here, we don't sit at desks and use computers all day. Our guys are out in the warehouses and in trucks, and they work at branches with customers. "Our numbers did go down because they were appropriate to our workforce," she adds.

Experts echo Foote's advice.

"If it's not exactly what your environment looks like, it's not an effective tool," notes Shally Bansal Stanley, director of the network economics practice at Greenwich Technology Partners in White Plains, N.Y.

Bansal Stanley adds that it also should be clear to the user exactly how the online ROI calculators come up with results.

"ROI calculations shouldn't be complicated. They're based on rules and criteria which should be transparent. You add or subtract," Bansal Stanley says. "My standard advice is to be very skeptical and not use vendors' ROI tools unless you're able to figure out what exactly the ROI formula is doing."

But many service and equipment vendors' online ROI tools include projected measurements for vaguely defined soft benefits, such as improving customer satisfactionIn these cases, Bansal Stanley recommends stripping out the actual cost information that the tool may provide, such as software licensing fees and upgrade and maintenance costs.

After that, "throw away the rest of the vendor's model because it has so much smoke and mirrors that it is too hard to decipher," she says.

Privacy is another factor to consider. Extract whatever cost information you need, then log off the vendor's Web site and do your own ROI calculations off-line to keep a lid on sensitive company data, says Ian Campbell, president of Nucleus Research Inc., a Wellesley, Mass-based consulting firm that specializes in IT ROI.

Some vendors with online ROI programs are continually gathering and storing the information that users enter, notes Campbell. "Every time you hit the Submit button, that information is going into a [vendor] database," he says.

Keeping It Confidential

Nucleus also develops ROI modeling tools, which it offers for free at its Web site ( But Campbell notes that all of the tools are Excel-based, meaning users can download them to their desktops, without fearing that a vendor at the other end is collecting data. "It's not a black-box approach," Campbell says.

SAP, on the other hand, collects and keeps the information that Web site visitors enter.

"The information is not sold or shared around the company, only with the account executive" who can use the information on a subsequent sales call, says Ed Brice, an SAP vice president of marketing.

For its part, SAP doesn't explicitly inform users upfront that it is collecting information they put into the online value calculators. However, the SAP Web page does contain a link to the company's privacy statement, which indicates the kinds of information the company collects from online users and what it does with the data.

One final note of caution about vendor-provided online ROI tools comes from P.J. Bartlett, a vice president of marketing at Arbortext Inc., an Ann Arbor, Mich.-based software vendor. It has to do with what Bartlett calls "the veneer of objectivity." This happens when a software vendor brings in a so-called neutral third party, such as an analyst firm, to help develop the tool or provide benchmarking data that becomes part of the vendor's ROI offering.

For example, PeopleSoft hired Baltimore-based Cedar Group PLC, a human resources consulting and outsourcing company and a certified PeopleSoft partner, to help build its ROI calculator.

Similarly, SAP promotes the fact that it hired Pittiglio Rabin Todd & McGrath, a Waltham, Mass.-based management consulting company, to help with its online tools.

Bartlett says the challenge facing software marketers today is this: "You're going to provide an ROI calculator, but how do you provide something that users believe?"

The answer? "You get somebody else to say it works," he says.

Not Completely Worthless

You may not want to present the figures they crank out to your board of directors, but online ROI calculators aren't altogether useless.

"For one thing, they acquaint [IT]\ people with some of the language of ROI and economic value, and that's a good thing," says John Berry, a Bend, Ore.-based management consultant who specializes in the economic value of IT. "But these calculations are meant to entice people, and they're not what should be used to make a final decision," he adds.

Noting that most online calculators provided by vendors center on that company's products, he says they're of minimal use to users comparing options.

"In the end, the value of these calculators is to start the discussion and help users understand the value of a particular software investment," Berry says. "That may very well be the limits of their value."

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about Greenwich Technology PartnersPeopleSoftPittiglio Rabin Todd & McGrathSAP Australia

Show Comments