Printer Speed Ratings Versus Reality

SAN FRANCISCO (04/21/2000) - Color ink jet printers seem to live in two worlds.

On the one hand, there's the world of printer company test labs, where lightning-quick machines zap out letters and photos. Then there's the world where the rest of us live, where printing out a simple letter or photo can test the patience of a stoic.

What Gives?

An analysis of two years' worth of published PC World tests of the top color ink jets finds that the printer companies are waging a "specsmanship" contest in which consumers rarely get rated speeds. Instead, depending on the vendor, the ink jets average between 40 and 73 percent of rated text speeds in PC World tests.

We arrived at this result by taking the published performance data for the 24 printers that earned a spot on PC World's Top 10 Printers charts between February 1998 and April 2000. The data includes text and graphics speed ratings provided by the printer companies, and speeds for text and graphics printing found in PC World tests. We looked at the four companies whose machines appeared most often on the charts: Canon, Epson, Hewlett-Packard Co., and Lexmark.

In PC World's text tests, HP's printers came closest to rated speeds. HP printers tested, on average, at 73 percent of the company's rated speeds. The Epsons averaged 53 percent of rated speeds in our text tests, and Canon and Lexmark printers checked in at just 40 percent. In graphics tests, results ranged from 22 percent of rated speed for Epson to a mere 12 percent for Lexmark; HP printers averaged 18 percent of rated speed, while Canon printers averaged 14 percent.

Why such wide variation between ratings and PC World results? The industry has no standard for rating color ink jet printer speed, so vendors make their own rules. For example, some vendors test in draft mode. (One reason HP comes closest to PC World text results is HP provides ratings for normal-mode text printing.) Some vendors also test by printing from DOS instead of from everyday applications.

"We try and make our tests as fair and representative as possible," says Carolyn Ticknor, president of HP's imaging and printing systems division. "But what customers, who use a variety of applications, experience may, and I emphasize may, well be different. [PC World's] tests may be more representative of what a user sees," Ticknor says. "In almost every test, we are the most conservative in the industry." HP and the other vendors face the dilemma of the lack of uniform standards, she adds. "We want to be fair to the customer, of course, but also be fair to ourselves."

Speeds Are Subjective, but a Standard Is Unlikely As Lexmark engineer Rhenzi Keys points out, "There would be significant hurdles to cross" in creating a standard. The printer vendors would have to pick the PC platform, the applications, the size and complexity of the job, and the driver settings. HP and Epson told us they'd welcome a standard, but that it would be tough to get the companies to agree on details.

Also, as Tom Miller, senior product delivery manager at Canon, points out, each company might want to capitalize on its own technology.

But given today's situation, "print speeds are completely subjective," says Charles LeCompte, president of the printer market-research firm Lyra Research.

"That doesn't mean there's no test a given printer performed at [a quoted] speed, but the test is completely arbitrary."

Performance Gap

Aggressive ratings aren't a new tactic in the PC industry. For years, monitors carried confusing specs; you could find two "17-inch" monitors with different amounts of screen real estate. But after enough protest from consumers, manufacturers began to report actual viewable area in ads, so shoppers could understand the specs and make comparisons. Color ink jet shoppers have no common, easily understood speed standard.

Without question, color ink jet printers run faster than they did in 1998. They also produce higher-quality output. But the divide between PC World performance results and the vendors' rated speeds hasn't improved much over time, as we found when compared speeds in the April 1998 and April 2000 top printers charts. Why did we only consider the records of printers that made PC World's Top 10 Printers charts? Having already called these the best products on the market, we feel they represent a fair, relevant sample.

Consider the HP DeskJet 812C, which captured first place on our April 2000 Top 10 Printers chart. HP rates the 812C at 5.1 text pages per minute; PC World's tests clocked it at only 3.7 pages per minute. With graphics documents, HP claims 3.1 pages per minute, but in PC World's tests it produces only one-half page per minute.

Lexmark's Z51 Color Jetprinter placed third on that same April 2000 chart. Like the HP, it's a good value among its peers. Lexmark says it prints text at 10 ppm; PC World clocked it at 3.9 ppm. One reason: Lexmark's rated text speeds reflect tests run in draft mode: a lower-resolution setting that prints faster.

It's an approach Canon and Epson also take in the quest for high speed ratings.

We Test, They Test: Understanding Methods To understand any rating, you need to understand the testing process. A printer can be tested in many more ways than you might realize, and many testing scenarios don't mimic everyday work.

PC World's color ink jet testing methodology is designed to approximate real-world tasks. We use a series of 11 documents of different lengths, some containing text, some with text and graphics, and others containing only graphics. We create and print these documents from apps that include Microsoft Excel and Word, Adobe Photoshop, and CorelDraw. We print one Word document in both draft and normal modes; all others we print in normal mode only. From a Pentium III-450 PC, we print each document twice, reboot the system to clear any caching, run another two rounds of tests, and average the four scores. The combined document scores create scores for text and graphics.

We time from the moment you click Print to the moment the paper hits the output tray, a method known familiarly as "click to clunk." But the big four printer vendors measure time from "feed to clunk"--starting when the feed mechanism picks up a sheet of paper. Printers generally delay feeding paper until the driver software is already rasterizing a document (converting it into dots), so the feed-to-clunk method shaves time off recorded speed.

Vendors' Variances

There are technically sound reasons for taking this approach, says Canon's Tom Miller. "A faster processor will rasterize faster, or more-efficient cache management [might affect print speed]," he says. "From an engineer's perspective it's very difficult to reproduce that data." Feed-to-clunk times are reproducible with a range of PCs, he adds.

That's true, but ordinary users have to wait out the whole time.

"If the manufacturers are basing their claims on such [feed-to-clunk] methods, then they should include a caveat," says Wayne Hubbell, president of CompuMetric Labs, an independent company that tests printers for Lyra Research and various printer vendors. "It is not unreasonable to measure performance this way, but they should explain that real-world users will obtain slightly lower performance."

Canon and HP follow another procedure that doesn't mimic routine printing.

Instead of printing from an app such as Word, they prerasterize a document by saving it as a print (.prn) file. Then they issue a DOS command to shoot the file from the system's parallel or USB port to the printer. This procedure bypasses the application, the driver software, and Windows.

Don Okerlund, product manager for HP's DeskJet division, says HP tests this way because it's replicable, even with older PCs.

"This methodology takes some of the variables out of the testing; it doesn't rely on the operating system, the computer, the application."

In contrast, Lexmark and Epson print from applications, as PC World does.

Another Angle: The Draft Card

Print resolution also affects speed, whether you're printing from applications or printing prerasterized files. Canon, Epson, and Lexmark quote speeds recorded in low-resolution, draft mode for text and graphics, which a user might not realize. (One note: Epson rates speed in "fine mode" for photos.) For example, a consumer comparing printers might look up their specs on the respective vendors' Web sites. We checked out the specs for four printers from the April 2000 Top 10 Printers chart: the Lexmark Z51 Color Jetprinter, the Canon BJC-6000, the Epson Stylus Color 760, and the HP DeskJet 932C at the respective companies' Web sites. We didn't see the words "draft mode" anywhere in the Canon, Epson, or Lexmark specifications. We did see phrases like "up to 10 ppm." (Epson notes that it does specify "economy mode" on the customer information "tear pads" you find near printers in retail stores.) Most people feel that draft-quality printing is acceptable for personal documents but generally not for anything you'd share with others. That's why PC World tests almost exclusively in normal mode.

HP, unlike the other three vendors, publishes ratings for draft, normal, and best-quality output (As noted earlier, we base our comparisons on the company's normal-mode ratings). But those numbers appear only on HP's detailed specification sheets and on its Web site. HP's advertising and packaging reflect the draft-mode ratings.

Consumer Catch

Unfortunately, makers of color ink jet printers find themselves in an increasingly difficult spot, says researcher LeCompte. New models must carry faster ratings than old ones, or they won't sell.

"The speeds they quote keep going up, so I think they're getting themselves into a bind," LeCompte says. "The customers definitely care about speed. And it's leading to some preposterous things. Maybe at 12 ppm people will still believe you, but what do you do when you introduce another printer that's faster? Do you say it's 14 ppm? Or say 12 ppm, even though it [really] is faster than the other one you call 12 ppm?"

For now, there's simply no easy way for consumers to decipher ink jet speed ratings. Based on what we've found, you should use rated speeds only to compare color ink jet printers from the same vendor. You should also consider test results from independent sources, including PC World, before making your purchase.

If enough consumers complain, the vendors may change their ways. At the least, they could provide more information for comparison shopping. Above all, when you shop for a color ink jet, don't take rated speeds at face value--or the printer may seem a world apart from what you expected.

Join the newsletter!

Error: Please check your email address.

More about Adobe SystemsCanonEpsonHewlett-Packard AustraliaLexmarkLyra ResearchMicrosoft

Show Comments