FRAMINGHAM (06/30/2000) - More than 1,000 Web sites were taken off-line or slowed over the past two weeks as a result of problems in the Vienna, Virginia, data center of Internet hosting firm Verio Inc.
Verio officials confirmed Wednesday that about 1,200 Web sites - 2% of the 60,000 hosted at the Vienna facility - were affected to varying degrees by the router problems, which began June 23.
The problem started when Verio did a routine update of configuration tables on its Cisco Systems Inc. routers, said Doug Schneider, Verio's president of Web services.
"It took us a little while to recognize that the configuration tables were causing [the problem]," Schneider said, because the problem was intermittent.
Technicians fixed it by Wednesday, he said.
International Data Corp. in Framingham, Massachusetts, reported in April that Verio had 7.6% of the Web hosting market last year. That figure made it No. 2 in the Web hosting market.
Although many of those affected were small businesses, Verio also sells bandwidth to application service provider Agilera Inc. in Englewood, Colorado, which serves large firms.
Schneider said not all the problems can be blamed on Verio, because the Internet is so complex.
Laura Zung, Verio's vice president of product management, acknowledged that some sites hosted in Vienna were slow, but, she said, "their sites were not dead for five days."
Some customers disagreed.
Charles Rice, CIO at Coatingsmart.com Inc. in Ann Arbor, Michigan, said the outage prevented his start-up from registering companies for an online exchange for industrial and specialty paints and coatings.
Rice, who was previously vice president of technology at bookseller Borders Group Inc., also in Ann Arbor, said a five-day outage was unacceptable.
"If we had a five-day outage [at Borders], I doubt I would have just been fired," Rice said. "I would have been drawn and quartered. This isn't Joe Bob's Web hosting service and auto repair."
This week, customers who bought Web access through a Verio reseller complained of similar outages.
"Some companies' Web sites are dead. Dead and buried. My site has been dead now for six days," wrote Christopher Mott in an e-mail message.
Mott is president of Mott's Miniatures & Doll House Shop Inc. in Buena Park, California. His site is hosted by ABI Hosting Inc. in Durham, California. He said he has lost tens of thousands of dollars in orders since June 23.
Thursday, ABI sent an e-mail to its customers saying, "ABI Hosting was involved in a recent Verio disaster at their Virginia facility."
Faulty Hardware Blamed
Schneider blamed the problem on faulty hardware. Flawed software prevented nightly data backups, he said.
"We've been working again on recovering that data," Schneider said. "We are working with a professional data services recovery company."
Zung said this is the first such problem at Verio. She added that Verio will give all affected customers a free month of services and is installing new software.
An analyst who asked not to be identified said it took as long as 20 minutes to download three pages from her Verio-hosted site as long ago as May 13.
After weeks of Verio technical support blaming the analyst's Internet service provider, Verio acknowledged that the Vienna routers were the problem, she said. At that point, she asked that her site be hosted at Verio's Boca Raton, Florida, site.
Everything worked fine until last Monday. "I cannot save the server," she said.
Customers said a refund is too little, too late.
"The lesson there, I guess, is that I would gladly pay a hundred times what I got refunded not to have the outage," Rice said.