OfficeMax had a visibility problem. "Two years ago we started a major, third-party drop-ship program that caused us to lose some visibility of the orders after they left our network," says Brian Walmsley, divisional vice president of EAI technology at OfficeMax. This problem became a customer service issue, particularly for orders that went through many different suppliers. "Not only were call centre staff unable to see where orders were in the process, but there was no way to determine which suppliers were underperforming and which were on target or better. The best we could do was to generate historical reports or have a staff of people carefully track individual vendors manually, which was not a great use of resources."

The company turned to webMethods Optimize, a BAM (business activity monitoring) solution that rides on top of the webMethods EAI platform OfficeMax used to integrate with its suppliers' systems. Walmsley is excited by what he's seen so far. "We'll be able to see in real time the state of each and all our transactions by vendor ... how many transactions are falling out of the process model, and get live, ranked vendor scorecards. The system will also alert us if a vendor hasn't responded to orders in a while, so we can contact them before customers are impacted. If a vendor's scorecard is falling, we'll be able to correct the situation before it becomes a problem."

OfficeMax's experience is typical of today's "real-time" enterprises. "There's a lot of pressure on companies today to respond to business events much more quickly," says Bill Gassman, principle analyst at Gartner. "Business managers don't want to wait for the next day for data warehouse information to be rolled up. They want to know in a couple of minutes if inventory on a product has fallen below a certain amount, if sales didn't make this number of calls, or if they got a purchase order over a certain value from an important customer."

In fast-paced businesses, such as financial services, each hour of nonresponse can cost tens of thousands of dollars. And such regulations as the Sarbanes-Oxley Act -- which requires regulated businesses to report significant business events, such as multimillion dollar orders, within 48 hours -- are also driving the need for real-time information. "How long would it take before such an event percolates up into the reporting world of most large enterprises today?" Gassman asks. "A week? A month? Businesses will have to be able to get to and trust a single version of the truth fast. BAM will be one of the key tools they use."

BAM everywhere

Live or near-live monitoring of business processes and KPIs (key performance indicators) is precisely the mission of BAM, which typically pulls information from a variety of applications and data sources then presents it to the user as live, integrated, "dashboard" displays filled with speedometers, bar graphs, maps, and tables. BAM also filters the information it monitors based on user-defined business rules, sending out alerts when key thresholds are reached, including inventory running below a certain level, or, in the case of OfficeMax, lack of response from a vendor within a certain time period. In some cases alerts may initiate other actions, such as e-mails to staff, customers, or suppliers.

BAM can be used to monitor a single indicator, but in many cases the business processes and indicators it monitors are spread across many enterprise business applications – enterprise resource management, supply chain management, manufacturing resources planning, or customer relationship management -- that don't normally interact. That's why enterprise application integration vendors such as webMethods, Tibco Software, and SeeBeyond Technology have jumped into the BAM market as a way to add value to their business integration offerings. But many other types of vendors also offer BAM. Application and data integration work hand in hand, so business intelligence and real-time data integration vendors such as Informatica, Ascential Software, and Metatomix have also entered the space (the BAM solution offered by webMethods is the result of a partnership with Informatica).

IBM and Microsoft have muscled their way in, too. Two years ago, IBM purchased Holosofx, whose BAM software morphed into IBM's WebSphere Business Integration Monitor. And Microsoft BizTalk Server 2004, released in March, is the first version to offer BAM capability. Meanwhile, pure-play BAM vendors, including iSpheres, FirstRain, and Celequest, cater to organizations that may not need or want to invest in a full-fledged EAI solution. iSpheres provides its own adapters and specializes in correlating multiple asynchronous events in real time. Vendors iSpheres, Informatica, Vitria, and Celequest also offer vertical BAM solutions for financial services companies or health providers and related organizations, particularly for compliance.

ERP vendors PeopleSoft, SAP, and Oracle offer BAM-like monitoring features, but they tend to be simpler and less flexible than those of fully developed BAM solutions and limit themselves to ERP (or even one ERP app). Many of these vendors are working on implementing a more flexible BAM framework within their ERP solutions.

Even the IT infrastructure management vendors Micromuse, Computer Associates, and BMC Software are getting into the BAM act, though they tend to target IT rather than business managers, with the mission of "enabling the alignment of IT with business objectives". And BPM solutions from InterSystems, Lombardi Software, and others offer ways to monitor the business processes they manage.

"BAM is a function, not an application," Gassman says. "Just as we saw browsers become ubiquitous, real-time reporting will become ubiquitous."

BAM finds its footing

So what is or isn't BAM and how does it fit in with BI, BPM, and other enterprise software that feeds on integration? To begin with, BAM is not business intelligence, which is rear-view information, not real-time. BI users tend to be business analysts; BAM users are typically line-of-business managers. And BAM information tends to be focused rather than broad.

"In a data warehouse, you want to collect lots and lots of information that you can slice and dice a hundred ways for future analysis," says David Kelly, president of Upside Research. "BAM alerts the user based on a few carefully chosen business metrics and thresholds that affect the bottom line directly and require quick action."

Debbie Rosen, executive vice president of worldwide marketing at webMethods, agrees. "It's like the difference between looking at a live stock ticker symbol versus looking at the stock's 52 week history and company financial information. You want both."

Whereas BI pulls information from a data warehouse, BAM tends to tap information sources directly, linking to a message broker for example, or using a very low-latency data store. Information gleaned from BI, however, often plays a role in choosing hot spots for BAM to monitor. "Your BI analyst will tell you these are the predictive measures and types of things that lead to problems, so you can start using BAM to catch those things as they happen," says Scott Fingerhut, senior product marketing manager at Tibco. In fact, some BAM solutions continuously compare BAM information to historical data to provide context.

BAM also works hand in hand with business process management, which provides a means of developing end-to-end processes across a range of enterprise business applications. "BAM might send out a notification that a salesperson hasn't called a threshold number of prospects, while BPM might shift some work from one person's work queue to another automatically," Upside Research's Kelly says. On the other hand, BAM typically can expose more business metrics than BPM, because it can consume data outside of a typical BPM process. And BAM can solve problems that BPM can't. "For example, you could automate a process with BPM so that it runs very well technically, but still loses tonnes of money," Tibco's Fingerhut points out.

The BAM market is small and young. "At this point the whole market is at the $US100 million to $200 million level," Gassman says. "It's still in the learning and experimenting stage for most companies." Some of the principal users so far are financial service companies and currency traders, which need to make quick investing decisions based on numerous and varied information sources, including news feeds. Health care organizations and hospitals are using BAM for epidemic outbreak and bioterrorism detection as well as monitoring emergency room visits to determine how many beds they may need.

Many telecommunications companies are using BAM to monitor service levels and speed up provisioning. "We use Tibco's BusinessFactor to monitor each stage of the ADSL provisioning process to make sure our accounts are in line with key performance indicators," says Christian Righi, manager of eFoundation Projects at Telecom Italia. "We can get live information on which orders are bottlenecked at which stage of the process in which region of Italy."

Making the connection

Just about everyone agrees that a successful BAM implementation comes from starting small, using what you have, modelling relevant business processes, focusing tightly on a few important KPIs, and marrying business users with IT.

Integration is a significant part of the BAM implementation process, so if you've already gone through EAI or some other form of business integration, it makes sense to leverage what you have in place and seriously consider what BAM capabilities your integration vendor offers. "Don't ask for all those references, benchmarks, '-ilities', and '-isms'," says the vice president of enterprise services at a large, regional bank that uses webMethods-based BAM. "Just pick the product that's in line with your integration strategy."

If you haven't invested heavily in integration and don't plan to, check out some of the pure-play vendors or a vertical solution. "Pure-play vendors often compete with the vertical vendor that knows everything there is to know about, say, Wal-Mart," Gassman says. The trade-off is generally between flexibility and a deep knowledge of your business.

Whatever the choice of product, deploying useful BAM dashboards requires close collaboration of business and IT. "The business-process owner is the one who knows exactly where in the process she lacks visibility and where things fail on a bad day," Rosen says. IT, on the other hand, can help the business user understand what is possible.

"It's too much to presume that the business user can imagine what he'd like to see," says the bank vice president. "IT needs to work iteratively with him, helping him envision the dashboards by which he can monitor the operational state."

As with most people we spoke with, the bank vice president advises starting small. "Don't apply BAM end to end in your first pilot. Start with a few key processes that run your business."

"Map out the process you want to monitor," Kelly says, "so that you know exactly which KPIs you're looking for. Most organizations haven't done this, and it becomes a big stumbling block." Careful modelling clarifies which dashboards and alerts to deploy and ensures that they provide the context necessary to help the user make decisions. Be sure to choose information that will actually be used.

"If you use BAM to send the sales rep a lead every time someone hits the Web site, pretty soon the rep will ignore those alerts," Gassman says. He also points out that data quality is a big issue. "If you're not doing the right data cleansing, consistency checking, and validation, you'll end up with unreliable data streams and lots of false positives and negatives."

IT should also make sure its BAM solution provides users with easy tools, such as drop-down menus to manipulate KPIs and thresholds as business conditions change, so users don't have to go back to IT each time they want to make a small adjustment.

Once you succeed with your first BAM pilots, your business users will think of many more ways of leveraging BAM to improve business operations. You just may be on your way to realizing the dream of the real-time enterprise.