Microsoft suffered an embarrassing setback last month when top benchmark results for its SQL Server 2000 database were cancelled by the Transaction Processing Council (TPC) because they were found to be noncompliant. But the company has now struck back with new numbers that analysts said show it's gaining on rival products.
The four new benchmark numbers, all based on clusters of Compaq ProLiant servers running Windows 2000 and SQL Server, are slightly better than the cancelled numbers. The fastest of the new results was obtained with 12 eight-processor ProLiant 8500 servers, clocking 262,243 transactions per minute (TPM-C) on the TPC-C benchmark.
That's the second-fastest TPC-C result overall, after IBM's recently published 440,879 TPM-C result with its NetFinity servers running Windows 2000 and DB2.
TPC-C is a measure of database performance in transaction processing applications.
But Mark Shainman, an analyst at Meta Group, said the latest TPC-C results are mainly about bragging rights.
"It's not really a great measure of true database performance in real-world applications," Shainman said. One of the problems, he said, is that real-life applications would involve much more data being shipped between the nodes of the Compaq cluster, slowing down performance.
SQL Server 2000 "doesn't quite close the gap with Oracle8i and IBM DB2, but it's getting there," said Teri Palanca, an analyst at Giga Information Group.
Microsoft's benchmarks for SQL Server 2000 were first published in February and figured highly in Microsoft's marketing for Windows 2000. But the numbers were cancelled after being challenged by an undisclosed vendor, widely believed to be Oracle (which refused to comment on confidential TPC procedures).
The results were found "noncompliant" because the tested configuration didn't allow for the primary key of a distributed database to be updated. Microsoft subsequently modified SQL Server 2000 and reran the tests.