Dirty data plagues CRM deployments

Customer relationship management (CRM) has become a hot-button topic, with companies deploying systems with hopes of providing better service and, in turn, boosting revenues.

But many CRM deployments are thwarted by faulty, inconsistent data sets that prevent user sites from having a clear, unified profile of each customer, users and others monitoring the CRM industry said. Software companies are beginning to tackle the problem, with Evoke Software and Metagenix selling wares intended to profile data and help clean it up.

"It's massive," said Derek Strauss, CEO of Bethesda, Md.-based business intelligence applications host Assurenet, of the problems of data quality in CRM applications.

"One of the biggest issues with CRM is obviously to get to know your customer," Strauss said. "You have to have accurate information, and most of the front end systems which deal with customers do not have accurate information about the customers. There's disjointed [data], there's a lot of blanks in some of the critical fields."

Additionally, many business rules intended for databases have been broken, said Strauss. "Consequently, you have data integrity problems with your data," he said.

Even such seemingly small mistakes such as being one digit off on a customer's street address can plague data sets, said Joe Butt, senior analyst at Forrester Research in Cambridge, Mass. This type of problem can lead, for example, to a company sending out multiple direct mailings to the same customer, because the address is listed in two different ways in company databases, Butt said.

"CRM is the hot topic of the moment, but in terms of consumer data, it's probably the dirtiest data around," Butt said.

Data quality problems are a hindrance to successful CRM deployments, according to analyst firm Gartner.

In a July 2001 report, Gartner wrote that more than 75 percent of enterprises engaged in CRM initiatives cannot combine a comprehensive view of a customer with actionable, personalized advice to customer service and sales agents. A key challenge in implementing a CRM platform is "an information crisis resulting from inconsistent, inaccessible, incorrect, or out-of-synch data sources," according to Gartner. Other factors cited included an inability to prove value, a personnel shortage, and difficulty in tying together a corporate CRM strategy.

At least 80 percent of enterprises underestimate the time and resources needed to cleanse and consolidate data, with many budgets exceeded by 200 percent to 300 percent, Gartner reported.

Integrating customer data sets is challenging, said IBM Corp. official Bryan Foss, a Customer Loyalty Solutions executive in London, who has been working on CRM deployments both internally and at customer sites.

"CRM has a very broad scope," Foss said. Existing systems, such as manufacturing and ordering must be integrated to enable a single view of a customer base, he said.

"In a typical bank or insurance company, for example, there could be 50 to 150 different systems which contain customer data, and if you want to gain a single view of that customer and the value of that customer and their needs, then you need to combine that data," Foss said.

This requires pulling together different data stores, with data of varying ages, on different databases. Foss said. These data stores also use multiple programming languages and data formats, he said.

"Some of the data's [of] poor quality. Maybe it hasn't been used for a long time," Foss said.

"To bring [together] all this data is a big task," Foss said. "It's an enormous task. It can't be done overnight."

To tend to the problem of poor data quality, Foss and Strauss are utilizing third-party data profiling software.

IBM's Foss is using Evoke Software's tool, which bears the same name as the company, to automate the process of data analysis.

"Manually, it would have taken a long time. What people did [formerly] was they tended to tackle the problem during the project and the project would expand both in terms of time and money," Foss said.

"Using Evoke Software, you can automate that assessment process up front," Foss said. "You can understand the structure of the data and understand its quality."

The product finds different relationships in customer data in multiple sources and provides information on restructuring and cleaning the data, Foss said.

IBM is using the Evoke product to build data warehouses and to profile data for a Siebel Systems Inc. CRM application. Once data inconsistencies and issues are uncovered, IBM uses Vality's Integrity to cleanse the data.

Evoke offers an alternative to hiring systems integrators to manually examine data files, said Rick Cortese, president and CEO at San Francisco-based Evoke. "[Manual intervention] is tedious, time-consuming, and expensive, and by definition not very accurate," Cortese said.

"Evoke looks at all the data, irrespective of source code and documentation, and looks at where it resides," said Cortese.

Evoke Software costs from roughly US$400,000 to several million dollars, depending on configuration and services. Version 4.5, due in December, will feature added XML functionality and an enhanced repository for automated profiling.

At Assurenet, the company is using a rival product, Metarecon, from Research Triangle Park, N.C.-based Metagenix. The product analyzes data in source systems, including looking at data values.

"It tells you early in your project where your problems lie," Strauss said.

Metarecon provides data profiling and analysis, and also can perform code generation, including producing online analytical processing specifications, according to Metagenix President Greg Leman.

Metarecon costs $25,000 for an entry-level version, and $250,000 to $300,000 for a standard, perpetual, enterprise license.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about AssurenetEvoke SoftwareForrester ResearchGartnerIBM AustraliaMetagenixSiebel Systems

Show Comments