Links that hold: making sense of application integration

Want to integrate all the apps running the company's vital operations? Well, the secret it appears is not in new technology or languages but in getting back to business. To meet the credo of information any time, anywhere, fast, the key is in the interaction between departments, offices, field staff, partners and suppliers. Sue Bushell surveys the market's movesIf e-commerce initiatives are at the forefront of most Executives' thoughts right now, Enterprise Application Integration (EAI) can't be that far behind.

Analysts agree that most organisations will address an EAI agenda in the near future.

Why? Because EAI should allow them to create the ultimate distributed computing environment where every application is an object with known behaviour, the network truly is the computer, and customer demands for integrated information drawn from every business unit of the organisation can be readily met. Or so the theory goes.

To address the application integration nightmare plaguing most organisations since at least the mid-90s, CIOs have previously fiddled with Cobol and legacy access tools, screen scrapers and middleware, largely on an indiscriminate, application by application basis. Sadly that kind of haphazard approach just won't wash under the new imperative to deliver information from anywhere in the organisation to anywhere in the value chain. That's why EAI vendors are racing to not only deliver superior technologies for application integration but also to help organisations integrate the business processes of different business units or of trading partners like supply chain partners.

And now, according to the author of a recent study conducted for the National Electronic Authentication Council (NEAC) on Integration of B2B e-commerce systems, while the technologies allowing EAI are not fully mature, there are at least a range of technological options for integrating applications available.

Sadly it doesn't necessarily make the "how" all that much easier.

With the technologies advancing apace, the bigger impediment to EAI today, particularly for small to medium companies, is their failure to completely understand their own business processes.

"Implementing an EAI strategy represents a considerable and costly challenge for most companies," says Piyush Maheshwari, IT consultant, engineer and architect and author of the study.

"This is especially true for small to medium enterprises which are typically not cash-rich and do not have skilled IT staff and the necessary IT infrastructure. The challenges range from the need to formalise business processes and rules for automation in software, to implementing secure communications links, and selecting appropriate technological solutions from the many alternatives on the market." What Maheshwari's study brought into stark relief was that unless companies first understand both their business and the real needs for integration, their EAI efforts are doomed to failure.

Achieving EAI means changing established business processes and organisational structures. The costs and disruption to the organisation those changes bring means EAI must be justified by improved efficiency or commercial necessity.

"Companies first need to see why EAI is really necessary for them," Maheshwari says.

"The big companies have enough money to let them fully investigate these types of technologies and go for the complete project implementation with respect to EAI initiative. But SMEs don't have such capabilities and resources so they have to really think about why they need to integrate their applications.

"The drive should come from the business side and they should first clearly understand the business processes of the organisation and why EAI is needed."E-efficiencyIf information is the lifeblood of the organisation, then the applications designed to move data around the business and between enterprises should operate as efficiently as arteries. But for most organisations getting data to flow freely remains a huge challenge, and the resulting data clots increase the time it takes to implement new applications, reduce competitive agility and prevent senior management from getting a clear picture of the business.

Enterprises need EAI in the face of increasing competitive pressures created by globalisation, the continuing trend to mergers and the forging of new business partnerships to keep the organisation competitive in an increasingly uncertain world.

In many organisations the new breed e-business applications are typically supported by several legacy systems. Consider a CRM (customer relationship management) application drawing on several customer-facing applications to provide services over the Internet that once would have required a tribe of sales and support reps to deliver. Such Web-based applications can undeniably deliver readily identifiable efficiencies, but the application integration problem remains the biggest barrier to achieving any payback.

With EAI, an application is written to interact in real time with each of these existing systems with a single user interface, most likely a Web interface.

As organisations increasingly look to trading exchanges as a conduit for operating in the new wired world every participant must consider how transactions are going to be pasted onto back-end systems. What links, if any, should be allowed to connect the exchange to the participant's back end? As exchanges gain greater acceptance, "integration is going to become excruciatingly important", said Evan Quinn, an analyst at market researcher Hurwitz Group.

"The more complex the B2B space becomes, the more you need EAI."It is increasingly clear that only the fully integrated enterprise can deliver the kind of consolidated information to customers that's essential to retaining them in an environment where they can be so readily stolen by an enterprise on the other side of the globe. Failure to integrate will severely hamper the ability of bricks-and-mortar businesses to effectively use the data residing in host or legacy systems to support their e-business initiatives. It will also leave them streets behind the dotcoms, which have often had the luxury of building their IT infrastructure from the ground up and in fully integrated fashion and whose very business models are based on the free flow of information.

Smart companies know they must find ways to integrate back-office applications into e-commerce applications, and manufacturing, accounting, and inventory applications into supply chain applications.

As business becomes increasingly networked and interconnected, those who fail to move with the tide are likely to drown.

Further, increasing business opportunities and commercial imperatives are driving the move to B2B e-commerce integration, Maheshwari says. Organisations are reluctant to invest the time and resources needed to integrate their business processes with their business partners without proven needs or benefits.

When connections to business partners are built into the integration plan, a higher order of efficiencies can be reached. Starting simply with electronic procurement, companies can move to disciplines such as collaborative demand forecasting and product design and development.

They can also create more efficient supply chains, B2B exchanges where supplies and parts can be bought and sold and for the purchase of supplies and parts, and Web sites that integrate access to various products and services offered by related groups of companies. Intertwining systems with suppliers and business customers lets companies dramatically improve their business processes.

As a result some analysts even believe that in the next few years more applications will be created through the integration of existing applications than will be developed by creating brand new applications. By the end of this year, more than half of all new front-end applications users develop in-house will leverage pre-existing applications to access corporate data.

Application integration options

EAI can refer to a whole toolbox of products and techniques each suited to a different business or technology need. Worse, many approaches to EAI only solve some of the problems business is grappling with day to day.

In data warehousing, ERP and electronic commerce, businesses tend to face two types of application integration issues: data level - where integration is directly with the physical application data - and business-model level - where integration is with a public interface exposed by the application. Ideally an EAI infrastructure should support both data-level and business-model-level interfaces.

There are plenty of products available capable of supporting the movement of data between applications. Gartner identifies six product types: file transfer, copy management, data propagation, schema-specific data synchronisation, database replication and extraction and transformation.

But the main need is to get data directly into and/or out of an application's data store, changing the format of the source data to fit that target, if needed, along the way. That's where extraction and transformation products come in. These are generally divided into three types of tool: code generators, transformation engines and data warehouse and data mart loaders.

Now, to address the increased demand for EAI, its vendors are increasingly moving to deliver prebuilt integration solutions. Many of these offerings recognise that Web-based applications rely on an entirely different kind of application architecture than do most client/server applications. Experts say the simple fact is that a Web server between client browser and the database server automatically creates a tiered environment for most applications and makes middleware the ideal place to centralise and consolidate application services.

Such middleware should have the capacity to intermediate between legacy data formats and application structures and the new, object-oriented applications and structures.

Middleware in traditional application integration largely consisted of tools that support one-to-one connections. By contrast middleware for Internet applications is directed at supporting both one-to-many and many-to-many connections.

But according to US InfoWorld's test centre the offerings are many and finding a solution to fit your needs can be a lengthy task. Infoworld says the leading vendors in this area are Software Technologies (www.stc.com), Constellar (www.constellar.com), and Viewlocity (www.viewlocity.com). Their solutions can run anywhere from $US50,000 to $US150,000. There are also many less expensive newcomers, such as Enterworks' Enterworks Suite (www.enterworks.com) or Extricity's Extricity Alliance (www.extricity.com). These solutions cost about $US20,000 to $US50,000.

However, EAI means more than just buying customisable, off-the-shelf technology and products. Application integration is a major chore in anyone's book. Any selected solution provider needs to understand business processes comprehensively and have a strong grasp of a broad range of both vertical and horizontal applications and technologies. To this end many vendors, including Ariba, CommerceOne, and Vitria, which all provide solutions based on existing middleware and messaging technologies, are forging alliances designed to provide complete B2B e-commerce solutions from companies like IBM, Microsoft, Tibco, BEA Systems, and Neon.

For instance IBM has added publish-and-subscribe technology to the MQSeries Integrator (MQSI), Version 2.0, product that it developed with NEON. Available since 1998, MQSI combines IBM's MQSeries messaging middleware with Neon's EAI rules and formatting engines.

IBM says adding tooling, a publish-and-subscribe model, a common repository, and XML support underscores MQSI's role as a common messaging link for e-business.

Middleware with muscle

In one sense, EAI can be thought of as middleware on steroids. Like Object Request Brokers (ORBs), EAI tools typically use a message broker as their underlying mechanism for data transport. They can also parse, duplicate or transform the data to present it in a format suited to each application that needs to receive the data.

But EAI tools go further than other forms of middleware by supporting business process rules. That means the user can define correct business processes and ensure data integration bows to those rules.

Hurwitz Group says Vitria's product was the first to incorporate this kind of business process support. Now others are adding the function both inside the organisation and across corporate lines in the supply chain.

Equally, Extensible Markup Language (XML) support is increasingly being added to EAI products, as is support for standards like BizTalk (a Microsoft-sponsored set of business-to-business communication protocols) and RosettaNet, a consortium working to establish electronics industry communication protocols.

And with participants in Internet trading exchanges racing to integrate their applications both EAI and e-business integration providers like Tibco Software, Vitria and ObjectSpace are responding.

The business-to-business exchange phenomenon "is a full employment act for middleware players", says Lisa Williams, an analyst at market researcher The Yankee Group. The complexity of back-end application integration "requires a lot of heavy lifting", she says.

To address that need Tibco has its ActiveExchange, a product set for closing the loop on B2B transactions built on the Tibco ActiveEnterprise suite of EAI components and featuring both the TIB/BusinessConnect and TIB/BusinessPartner offerings.

Likewise Vitria Technology has its Trading Partner Network (TPN), a kind of clearinghouse for exchange users who need to find Internet service providers as well as create connections among exchanges. Vitria has a strategy of adding XML support and providing a combined view of business processes and workflow in its BusinessWare 3.0 offering.

Vitria claims its approach is more tightly integrated than other EAI workflow offerings, thus ensuring greater ease of use and higher productivity.

And although not strictly an EAI player, ObjectSpace has also released its line of OpenBusiness products to compete in the B2B exchange realm. The OpenBusiness Portal and Gateway leverage object middleware and XML to let businesses expose a service or business process over the Internet to another business.

Meanwhile SAP is relying on its mySAP.com Application Server to link its ERP applications to e-business front ends through Internet or third-party technologies.

SAP also provides integration specifications for application and presentation layers.

The mySAP.com Application Server incorporates native support for HTTP and HTTPS, Internet document standards such as HTML and XML, and server-side Java scripting to allow Web developers to update Web front ends without a third-party development toolkit.

For the presentation layer, SAP's architecture includes mySAP.com Workplace and its WebFlow workflow software. The application integration layer encompasses SAP's XML-based Business Connectors, gateways to SAP's Business Application Programming Interfaces (BAPIs), Microsoft BizTalk, and the RosettaNet XML standards.

And JD Edwards has signed licensing and original equipment manufacturing agreements with middleware player Active Software and the XML-based, business-to-business integration vendor NetFish Technologies. Active's EAI architecture will be offered to unite applications internally while the NetFish offerings will be used for external links. EAI products and services are getting better at easing the complexities of integration, at least when it comes to the data and application layers.

But some experts say that as EAI technology merges increasingly with e-commerce solutions and application server products, next-generation versions will need to go much further.It remains to be seen how long it will take vendors to rise to the challenge.

Join the newsletter!

Error: Please check your email address.

More about Active SoftwareAribaBEABEA SystemsCommerceOneEnterworksExtricityGartnerGatewayHurwitz GroupIBM AustraliaJD EdwardsMicrosoftmySAP.comNEACNetfish TechnologiesObjectSpaceRosettaNetSAP AustraliaSTCTibcoTibco Software AustraliaTribeViewlocityVitriaVitria TechnologyYankee Group

Show Comments