Wrestling value from all that data

In the X-Files TV show, Fox Mulder steadfastly held to his belief: The truth is out there. The FBI agent hunted for the truth amid a web of deceptions and paranormal phenomenon.

When it comes to your enterprise, Mulder's premise is equally true: The truth is in there. It's in the data that's scattered across your organization's applications, databases, departments and geographic locations. Theoretically, you should be able to put that data together and arrive at a single truth to help guide your decisions, tactics and strategy.

In reality, though, you're probably as frustrated as Mulder. The data is contradictory. There's too much data -- or not enough. There's no easy way to get to the data. With a fragmented information infrastructure, a single version of the truth can be as elusive as the supposed UFOs that Mulder chased.

The result? Companies end up making crucial business decisions not on hard and trusted facts, but guesstimates, gut feel and suspect information.

Sound familiar? Take heart. A potent one-two punch is emerging: the convergence of data integration technology and Web services.

If you think that Web services is something for the IT guys to riddle over, think again. In concert with data integration software, Web services is poised to deliver dividends of enormous value to the business professional -- accurate, timely and complete information on demand.

Let's take a look at the business value that organizations can derive from data integration and Web services.

Web Services + Data Integration = Business Value

There's an abstract quality to Web services that many nontechnical people find frustrating. At a fundamental level, Web services makes it easier for developers to build and maintain software systems, because Web services uses an open set of standards for interoperability among systems.

If you've read about Web services, chances are you've seen these acronyms -- XML, or Extensible Markup Language; SOAP, or Simple Object Access Protocol; UDDI, or Universal Description, Discovery, and Integration; and WSDL, or Web Services Description Language. Because these standards are widely supported, developers can readily exploit them to hook a customer relationship management (CRM) application to call center software, for instance. The alternative is a lot of costly, manual coding to get applications to talk to one another.

Think of traditional banking and online banking. Once upon a time, opening a new money market account required a visit to a bank, a lengthy sit-down with a banker, lots of paper forms and multiple signatures. With online banking, you can set up a new account with a new bank and transfer funds from your current institution -- in 15 or 20 minutes from your PC. Managing and growing your accounts is equally simple.

That's the sort of ease of use that Web services offers to your IT department. Good for the IT guys. But what dividends does the business user get?

By itself, Web services may not make much difference in your day-to-day business. But couple Web services with data integration technology, and the payoff can be enormous:

- Data consolidated and standardized from disparate systems

- Real-time access to up-to-the-minute information

- Key business metrics and alerts delivered to desktops

- Greater visibility into rapidly changing business conditions

- Drill-through to analyze data for details and root causes

- Enhanced data quality, data accountability and compliance monitoring

A Phased Approach to Business Value

None of this happens overnight. If your IT staffers are building Web services (sometimes called a service-oriented architecture), chances are they're focused on application interoperability at a high level. Web services-enabled applications can recognize and communicate with one another. But under the hood, there's a significant issue that Web services does not by itself address -- data-level compatibility.

That's where data integration software comes in. It's a second phase in the Web services evolution that builds on high-level interoperability with the detailed, nuts-and-bolts functionality that business demands -- the data.

By data integration, we mean a specialized enterprise platform (or "engine") fluent in the varying data languages of enterprise applications: Siebel, SAP, PeopleSoft, Oracle, IBM, Microsoft and so forth. The latest versions of leading data integration platform are Web services-ready, in supporting XML, SOAP, UDDI and WSDL.

If your company has a data warehouse, it probably has a data integration platform. Data integration software will extract data from your applications (Siebel, SAP, et al), transform it into a common format and load it into a database (or "warehouse") for analysis by business users.

If you use a data warehouse, you're probably familiar with the value these systems deliver. For more than 10 years, companies in every industry worldwide have used warehouses to analyze data for customer management, target marketing, supply chain optimization, pricing strategies, financial analysis and more.

Particularly for large organizations, data integration and Web services do not obviate the need for a data warehouse. Besides serving as a central repository for data, warehouses excel as a foundation for complex multidimensional analysis with online analytical processing (OLAP) servers, and they are vital to the advanced, computer-intensive statistical analysis of data mining.

Rather, data integration and Web services can complement a data warehouse and address some of the weaknesses of the warehousing approach. Notably, Web services-enabled data integration eliminates the latency inherent in warehousing. A warehouse may be loaded with data once a week or once a night. If you're looking for time-sensitive data in a warehouse, there's a fair chance it may be stale.

A Web services/data integration system functions as something of a "virtual" warehouse. Data is not moved from source applications but is accessed and delivered to a requesting business user or application.

Consider these examples:

Procurement optimization.Your department handles procurement of components for your company's products. Your company is ramping up towards manufacturing of a new product, and your job is to recommend suppliers of these parts.

In the past, simply identifying your five lowest-cost suppliers was a huge challenge. That data resided in applications at individual plants in a dozen countries around the world and was made available to procurement managers only in static quarterly reports. For a companywide view of suppliers, you had to manually merge those reports into a massive Excel spreadsheet. By then, the data was old and pricing had changed.

And now you have a new mandate -- assess suppliers not only by cost, but by component quality and logistical efficiency. Again, that data is scattered across your enterprise. Quality control information can be found at the manufacturing level; customer complaints by e-mail and phone to your tech support center may suggest a defective part. A logistics application tracks delivery, inventory and back orders. A data integration platform is engineered precisely for such a task. Invoked as a Web service, it queries your disparate systems, extracts the relevant information, transforms it to a common format and delivers it to your desktop. With a business intelligence tool, you can drill into that data by cost, quality and logistical efficiency to determine optimal suppliers.

Financial call center. Let's say you manage an inbound call center for a financial services firm. A customer phones in to order liquidation of her account because she's displeased with your interest rate. Your agent plugs in the customer's account number to a call center Siebel application, which invokes a Web service call -- a request for information from linked (or "loosely coupled") applications.

A data integration engine functions as the nucleus for the Web service call. The software will instantly fetch, transform and deliver data to the agent's Siebel interface, a transaction history from a legacy financial application and customer value information from an Oracle CRM application.

With this data, the agent can see that your company regards the caller as "platinum" customer with high lifetime value. The agent is authorized to offer the customer a higher interest rate. At the same time, the Web service call has tapped a marketing application, which returns a summary of current promotions. With this, the agent can capitalize on the opportunity to offer the caller refinancing on her mortgage.

Behind-the-Scenes Value to Business

Behind the scenes, a Web services-enabled data integration system also provides far-reaching benefits:

Multi-application updates: Data integration platforms offer functionally known as read/write, that is they not only fetch data from applications, but they can change the data in those applications, too. For example, the integration engine will respond to a customer transaction by automatically propagating changes across any number of applications, ensuring the business always has accurate, up-to-date information at its disposal.

Data quality: Problems with data quality -- mismatched addresses and differing terminology -- is a widespread and insidious problem that the Data Warehousing Institute has estimated costs U.S. business US$600 billion a year. Data integration platforms standardize and "cleanse" data; functioning as a Web service, they significantly enhance the reliability and value of information.

Data accountability: One byproduct of data integration is metadata, or "data about data." As companies struggle to meet the requirements of Sarbanes-Oxley and other requirements, metadata is emerging as a powerful tool. The metadata repository of a data integration platform provides a complete record of data -- who changed what data where, when and how. The lineage and versioning visibility enabled by metadata management can greatly enhance data accountability, data visibility and compliance monitoring efforts.

Robust security: One of the raps on Web services has concerned security. While Web services security standards continue to evolve, a data integration platform helps address the issue with additional layers of security.

First, a good data integration platform will help govern user authentication -- in others words, who has access to what data. It does this via its support for a standard called Lightweight Data Access Protocol (LDAP) and its support for third-party LDAP servers. Second, a good data integration platform will encrypt data, usually to at least 128 bits -- more than adequate for most domestic business applications.

There's an irony in the way Web services has evolved. Back when it was but a twinkle in a programmer's eye, Web services was viewed principally as a faster, easier way for developers to build and maintain software.

Soon enough, it became clear that Web services was not simply a way to reduce programming complexity but to deliver tangible benefits to business users on a daily basis with instant visibility into accurate, timely and complete enterprise information.

To be sure, Web services is still in its infancy. As these systems are designed and implemented, it's an excellent opportunity for business and IT alike to strategize how Web services can reach its full potential and help organizations achieve their strategic goals. There's no question that the truth is in there -- in your data. Web services and data integration provide a powerful means of flushing it out.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about EvolveFBIIBM AustraliaMicrosoftOraclePeopleSoftSAP AustraliaUDDI

Show Comments