The Right Way to Scale

FRAMINGHAM (08/15/2000) - Sometimes a problem is hard to solve not because it's actually hard, but because you pick a hard solution to a simple problem. On the other hand, some problems really are hard: With these problems, even if you pick the best solution, they're still hard. If you pick any other solution, you fail. Sometimes it's difficult to tell which of these situations you're in.

Here's one way companies get trapped: They underestimate a problem at the start, and pick a simple solution. Then they discover--typically after hardware and software development tools have been installed and staff has been trained--that the problem was much bigger than they realized, that the chosen solution won't do the job, and that the real solution will be a lot more work and take a lot longer. This makes lots of projects late and busts lots of budgets.

Writing and running enterprise servers and Internet servers that scale is one of those problems that really is hard. I'm talking about the really successful sites that we all know about: stock trading, auction, big retail and industrial sales sites. Sites so visible that when they crash--even if it's only for a little while--they make the next day's Wall Street Journal. Here's what characterizes successful sites:

Scalability: They handle enormous loads with ease, because their load-balancing architecture spreads tasks across a number of machines.

Fault-Tolerance: They're always up, because their fault-tolerant architecture is redundant in hardware and software.

Transactionality: Their transactional architecture makes processing reliable.

Without this, high interaction rates inevitably lead to high error rates that are expensive to correct.

Security: They are secure in both hardware and software. Since all of the sites we're talking about deal with either money or items of substantial value, security of programs and data is critical.

Another thing successful sites consider, often overlooked, is coordination of programming staff. As staff count grows so large that your managers don't know all of their group leaders--never mind all of their programmers--it's comforting to know that your development environment and run-time architecture enforce consistency through type-checking and exception handling, even when calls fly from one machine to another over the network.

B2C sites and B2B sites are different. B2C clients run browsers, communicating over HTTP. This isn't elegant, but it works. Error rates stay low because orders are keyed in by actual people who can check for mistakes (since the client-side browser software doesn't help here, of course). Errors don't matter that much anyhow--there are so many clients out there that, if a single order goes awry and a customer moves to a competitor, your business will survive.

(Just don't let this happen on a regular basis.) But B2B is different. In fact, it presents some of those problems that can look simple at first but then turn ugly as things grow out of control. Here's how:

In the beginning, business clients may be satisfied with placing and checking orders on browsers. But that doesn't last long. Soon, companies want to connect their assembly line just-in-time stocking program to their suppliers' ordering program to place orders and check shipments, and their accounting program to the suppliers' billing program for payment. How many transactions could this involve? How many different parts are there in a 747? A lot! And, suddenly, even small errors matter. Even if it's just a small order, it's part of your business with a big customer. And, if a missing nut or bolt holds up his assembly line for a couple of hours, it's not just a nickel-and-dime matter, and he'll be mad.

We can't perform B2B interactions the same way we do B2C, because the client-browser model doesn't have the reliability that B2B requires. B2B interactions need the characteristics we listed for enterprise and Internet servers, but client and server machines are owned by different companies and don't trust each other, or share an architecture, or even a business or domain model. How can we standardize on a scalable, robust, reliable and secure B2B e-commerce environment but still maintain respect for these infrastructure differences?

Into the breach steps XML--extensible markup language. XML has a lot to offer as a solution to the modeling part of our problem, and this is one reason why it's being received so enthusiastically by Web developers. Unfortunately some people see XML not only as a language for expressing business concepts and other structures, but also as an interoperability solution. This isn't true:

XML is only a data format--there is no single, standard communication method associated with it. Instead, people transmit XML in lots of ways including HTTP, SMTP and over message-oriented-middleware (MOM) systems.

These work, in their fashion, and they're fairly quick to implement because they reuse existing software. But how do they measure up to the characteristics we listed above? They don't: HTTP and SMTP were never intended for this severe use, and MOM implementations lack the uniformity needed for industry-wide adoption.

In order to succeed, a distributed B2B e-commerce environment must couple client and server with reliable, secure, platform-independent network interoperability. At the same time, the extensibility of XML must be preserved.

This must all be built on a robust, scalable, transactional server foundation.

Corba, specified by the Object Management Group (OMG), meets these requirements now.

To Corba, the computing industry's interoperability standard, the OMG has recently added reliable messaging-mode asynchronous network transport and a mapping from XML to OMG IDL. This lets enterprises link their systems for B2B commerce using Corba, without sacrificing either the extensibility of XML or the loose coupling and assurance of MOM. With these additions, Corba meets all of the requirements for B2B e-commerce: scalability, fault-tolerance, transactionality and security.

Jon Siegel is the director of technology transfer at the Object Management Group, a Needham, Mass.-based not-for-profit developer of vendor-independent software specifications.

Join the newsletter!

Error: Please check your email address.

More about ExtensibilityObject Management GroupWall Street

Show Comments