Compute farms bring power to the PC

Christa Schidzik's job as manager of systems administration at NEC Electronics Inc.'s chip design facility in Dusseldorf, Germany, is often more a resource management challenge than a technology challenge. Engineers who design CPUs and application-specific integrated circuits must lay out their complex designs and simulate the chips' program code - compute-intensive tasks that create fluctuating demands for mainframe-class power.

"Before, we had big servers for every department, and normally, it was never enough," Schidzik recalls. Then she heard about compute farms: servers pooled to create a single computing resource optimized for CPU and memory-intensive applications.

After trying to configure a farm on its own, NEC called Blackstone Technology Group (BTG) in Worcester, Mass. Using BTG's ComputeFarm Advantage consulting service and software, Schidzik says, NEC had 50 users accessing a farm within a week.

Server utilization has since doubled to about 25 percent, she says. The need for new hardware has been mitigated, and spikes in demand are more easily accommodated. "I'm quite sure we couldn't have survived without the farm," Schidzik concludes.

Betting the Farm

BTG started in 1996 as a value-added reseller of electronic design automation (EDA) tools before relaunching with its compute-farm services in 1999.

"We started to see a bigger and bigger opportunity" in offering compute farms to customers, says Kanti Purohit, BTG's chief operating officer. Since then, the company has signed on biotech heavyweights such as Biogen Inc. in Cambridge, Mass., and Celera Genomics Group in Rockville, Md.

BTG's offerings range from construction of compute farms from the ground up to reconfiguration and management. It also offers software that lets companies function as application service providers to sell compute power over intranets or the Internet.

Biogen threw the switch on its farm, built on 200 Intel-based Linux servers and a Sun Microsystems Inc. "portal" node, last month. It will use the new capacity mostly for data analysis to look for pharmaceutical "targets" in the human genome.

Biogen selected ComputeFarm Advantage over more generic offerings from Compaq Computer Corp. and Mountain View, Calif.-based Silicon Graphics Inc., according to Rainer Fuchs, director of research informatics.

CEO Ron Ranauro says BTG wants to expand into other compute-intensive markets, starting with financial risk modeling, digital content creation, oil and gas exploration and mechanical design automation. But BTG's industry-specific knowledge is a major selling point, so the company might lose focus, warns Diane Myers, an analyst at San Jose-based Stratecast Partners.

BTG competes with mainframe and supercomputer vendors, as well as other compute-farm vendors such as Compaq and Sun, with whom it also has partnerships. These and other firms, such as Microsoft Corp., also offer related clustering technology for aggregating computing resources. One - Linux Networx Inc. in Sandy, Utah - assembles Linux clusters for some of the same verticals as BTG, including EDA, bioinformatics and oil and gas.

Purohit says the early going was tough because few people understood the compute-farm concept. Making money on load sharing - essentially a commodity - was another problem, as was "coming up with a business model that is going to be sustainable in an environment where the major platform vendors are going to be a player," Ranauro says.

BTG's expertise and technology have won the respect of customers and partners. "They are a powerful emerging player in a space that I don't think has gotten a lot of attention and likely will going forward," says Jilani Zeribi, an analyst at Current Analysis Inc. in Sterling, Va. "Blackstone's competitors, in my opinion, are people doing it on their own. Blackstone's argument - and I think it's really compelling - is that's a lot of money to throw at a problem that you're not dealing with all the time. They've made a compelling business case. Throwing compute power at something is only going to get more and more common, and harder for companies to do."

Essex is a freelance writer in Antrim, N.H.

Sidebar: Computer farms vs. server farmsCompute farms approach the computing power of mainframes by aggregating servers into one large, virtual resource that's accessible by workstations and optimized to run large jobs that may take minutes or hours. They differ from server farms, which process large numbers of short bursts of transactions.

BTG claims that its expertise is in knowing how to configure a company's networked PCs and outfit them with special management, scheduling and performance-monitoring software. Its staff has expertise in compute-intensive scientific specialties like bioinformatics.

BTG says compute farms cost a fraction of what centralized Big Iron systems cost because they harness the computing resources of cheaper hardware. Decentralization across networked PCs produces additional benefits such as improved scalability (since capacity can grow in smaller increments than with mainframes) and reliability (because power failures and other hardware glitches won't bring down the whole system). Compute farms also let users retain the convenience and power of their desktop workstations rather than forcing them to access a mainframe.

Carrying the Load

The key piece of software in BTG's ComputeFarm Advantage isn't made by BTG. San Jose-based Platform Computing Corp.'s LSF Suite is the scheduler and load balancer, scheduling jobs at optimal times and shuttling them to the Unix and Windows NT systems with the most capacity.

BTG also offers Sun Microsystems Inc.'s Grid Engine load-sharing software. BTG's own SmartCache software and an extension, SmartBlast, distribute data across the compute farm, avoiding the bottlenecks that come from squeezing huge databases across low-bandwidth networks. A third BTG tool, SmartWatch, lets administrators monitor and optimize performance and bill for clients for use of the farm.

BTG's security and privacy technology is a necessary component in making each user's system available to others, the company claims. "These resources might be on different workstations in different locations," says Chief Operating Officer Kanti Purohit. "You need a different concept of security." Finding industry-standard encryption tools inadequate, BTG invented what Purohit informally calls a "dynamic firewall" that runs on each system.

- David Essex

Sidebar 2: BlackstoneTechnology Group

Location: 100 Grove St., Worcester, Mass. 01605Telephone: (508) 793-2162Web: www.computefarm.comNiche: Compute farm software and servicesWhy it's worth watching: Technology provides cost-effective access to processing power for compute-intensive applications.

Company officers: - Ron Ranauro, CEO and founder - Rebecca Hyman, vice president of technology - Kanti Purohit, chief operating officerMilestones: - 1996: Company founded - 1999: Relaunched as a compute farm provider - September 1999: ComputeFarm Advantage debutedEmployees: 57; 400 percent growth rate last yearBurn money: US$3.4 million from Molehill Ventures, $15 million round under wayProducts/pricing: ComputeFarm Advantage consulting services and 90 days' use of software, $25,000 to $250,000; software subscription, $12,000 to $80,000 per year thereafter, depending on CPU count; PowerCloud software for ASPs, $250,000Customers: 3Com Corp., Biogen Inc., EMC Corp., Vertex Pharmaceuticals Inc. and othersPartners: Compaq, Sun, Veritas Software Corp. and othersRed flags for IT: - Compute farm hardware vendors may beef up consulting efforts. - Customer experience limited to a few vertical markets.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about 3Com AustraliaBIOGEN IDECBioinformaticsCelera GenomicsCompaqCompaqEMC CorporationIntelLinux NetworxMicrosoftNECPlatform ComputingSilicon GraphicsSun MicrosystemsVeritasVeritas SoftwareVertexVertex Pharmaceuticals

Show Comments