The straight story on search engines
- 25 June, 2002 10:53
Let's say you're interested in digital cameras. So you search for exactly that "digital cameras" on MSN (search.msn.com). But when the results come back, you find yourself saying, "What is all this stuff?" One set of matches at the top of the page is called Featured Sites. Below that is a set of links labeled Sponsored Sites. Below those is still another group called Web Directory Sites.
So what's going on? Many of these results are links that advertisers have paid MSN to put in front of you, and many others are links to sites that haven't paid a dime but that provide good information on digital cameras. Which ones are which? Even if you study MSN's explanatory boxes closely, you can't know for sure.
Featured Sites, according to MSN, are a mix of Web pages from three sources: advertisers who have paid to be there; other sites that, like MSN, are owned by Microsoft Corp.; and sites that are just plain useful. MSN doesn't identify which links come from which category. Sponsored Sites consist entirely of spots paid for by advertisers. Once you get to Web Directory Sites you might think, "Finally I'm past the ads and down to the links that will help me." Not so fast. The first link, labeled "10 Most Popular Sites for Digital Cameras," leads you to a pageFrom Ask Jeeves Inc. (www.askjeeves.com). The first ten spots on that results page are also bought and paid for. Talk about an overwhelming response and a lot of confusion.
The fact that search engines have put parts of their results pages up for sale can shake your trust in these gateways to the Internet. But it starts to get even more troubling when you examine the listings themselves. We believe that some sites alter the "real results." (Note: In this story, we use the term real results to refer to search engines' main body of listings that in theory advertisers don't pay for in any way.) Plus, some observers are concerned that small sites and nonprofit groups might get bumped off the listings altogether because they can't afford to pay to be included.
We went behind the scenes to find out how search sites operate. We put the engines through their paces and found that while some sites clearly label paid links, others, like MSN, do a poor job. We also suspect that some sites adjust the results to improve their corporate bottom line the sites deny this, however. And even though it's becoming more common for companies to pay search engines a fee in return for cataloging their Web pages, small sites and nonprofit groups aren't going to be left in the dust anytime soon. But the practice could pose a future threat.
Despite our misgivings, the situation is not completely hopeless. There's always Google Inc. (www.google.com). Not only does Google deliver exceptionally relevant matches, it's also the best of the bunch at identifying ads. In fact, we like Google so much, we gave it two awards this year.
In this article, we've got the best advice on how to decipher the search results at major sites. We also offer some handy search tricks. On top of that, we've discovered some useful sites you may not have stumbled on before.
Avalanche of advertisers
The way that search sites currently handle sponsored links has caught the attention of Commercial Alert, a consumer group founded by Ralph Nader. The organization filed a complaint last July with the Federal Trade Commission alleging that major search sites disguise the links that are paid for. "So many Web searches have been hijacked by advertisers," says Gary Ruskin, Commercial Alert's executive director. "The FTC should require search sites to tell us when ads are ads." At press time, the FTC was still investigating the complaint.
According to recent survey results published by Consumer WebWatch, a group affiliated with the nonprofit organization Consumers Union, 60 percent of Internet users interviewed did not know that search sites take fees to list some Web sites more prominently than others. And 80 percent of these users felt that search engines should disclose advertising deals.
Not too long ago, Web searching was all about cool technology. Search sites competed fiercely for your clicks. Sites like Lycos (www.lycos.com) and AltaVista (www.altavista.com), for instance, vied for your attention by constantly updating their search tools. But the battle among search engines has slowed dramatically at least where innovation is concerned.
The look and feel of each popular search site may still be different, but now a lot of sites are built on the same foundation. Rather than create their own unique search tools, some of the big search sites use technology by Inktomi to conduct searches. The search business is also more focused on making money. Without a profit, search sites won't be around. Banner ads haven't been as lucrative as the sites had hoped, so the search sites are now acting more like the Yellow Pages: They're doing their best to match advertisers with potential shoppers.
Paying to appear at the top
Here's where paid placement links enter the scene. These sponsored links show up along with the real results when you run a query on specific keywords. Lots of search sites have deals with Overture (formerly GoTo.com), a company that specializes in distributing sponsored search results.
Overture-based links currently appear on results pages of nearly all major search sites, except Google. However, at the time of this writing, AOL announced it would not renew its contract with Overture. It will start to display ads purchased through Google this summer. To advertise on Overture's search engine partners, advertisers bid an amount they will pay per click when a searcher opens a sponsored link; the highest bidder wins top placement. Note: PC World participates in Overture's paid placement program and appears as an advertiser on Overture's network of search sites.
Earning money is fine no one wants the search engines to go out of business. But as Nader's group insists, it's up to the sites to clearly identify ads. Some do not, and here lies the problem.
Each search site writes its own rules on how to label paid placement links. How can you tell which links are ads and which are not? On some sites it's easy, and on others it's anybody's guess.
There's no question about Google's ads. They don't look like regular listings. The ads are highlighted in a different color, and each is labeled as a sponsored link. They're always separate on the results page, appearing as shaded boxes to the side of the listings or right at the top. (Full disclosure: PC World is an advertiser on Google and PC World pages appear as sponsored links.)Most of the other search sites show their Overture-powered results before the real results, usually at the top of the page. Yahoo (www.yahoo.com) calls its paid listings Sponsor Matches, and the distinction is clear (even if the results page is a bit cluttered). The site presents you with shopping links first, followed by Category Matches, and then Sponsor Matches. Yahoo provides some details about how its ad system works when you click the link "What are Sponsor Matches?" Lycos calls its ads Sponsored Search Listings, but the label, with its small gray font, doesn't exactly jump out at you, and there's no link to explain the term. Ask Jeeves also offers its "sponsored links" before its real results, but it doesn't spell out what the label means.
MSN, on the other hand, has decided that it's okay to mix paid and unpaid results in its top category, Featured Sites, and not tell you which is which. Click the adjacent About icon and a screen pops up to tell you that Featured Sites are picked by editors who strive to choose the most relevant paid and unpaid results. But you still can't tell which links are paid for and which are not.
At AltaVista, you see paid links in the top spot that are labeled Products and Services not very revealing. When you click the label for an explanation, AltaVista first says that the sites have been reviewed by editors for relevancy to the search term, then identifies the listings as ads. "We feel like we're making it pretty clear," says AltaVista spokesperson Krista Thomas.
Critics like Danny Sullivan, editor of UK-based SearchEngineWatch.com, believe that labels like AltaVista's are misleading.
"I think `sponsored' should become an industry-standard label for paid links," says Sullivan. "People want to know what sponsored links are. It doesn't mean they won't use them."
Thumbs-down on metasearches
Many metasearch sites sites that combine results from several search sources in one results screen do the worst job of labeling paid links, Sullivan says. "People assume metasearch engines are delivering the best editorial goods. The reality is, they query paid listings, so they become meta yellow pages," he says.
For instance, MetaCrawler (www.metacrawler.com) calls upon search engines such as AltaVista and other sites like Sprinks (www.sprinks.com), which are full of paid placements. For most queries at MetaCrawler, the top category in the listings is peppered with paid links, mysteriously labeled Featured Search Results. You also see a sponsored link called MetaCrawler Suggests that often presents completely irrelevant links. For example, when we ran a query on the term "802.11b specifications," the MetaCrawler Suggests ad pointed to "Dating & Romance for over 30s" a link to Udate.com, a matchmaking site. Hello? Relevancy, in this case, had gone out the window.
MetaCrawler's paid links are supposed to be relevant to the search at hand, says Steve Stratz, senior public relations manager at InfoSpace. (InfoSpace's technology powers the searches at Excite, WebCrawler, and MetaCrawler.) However, in recent months, the ad spots have been filled by the company's big network advertisers and they often appear on multiple sites, reports Stratz. InfoSpace recently launched its new metasearch product at Excite and WebCrawler. The company hasn't decided whether these changes will roll out on MetaCrawler.
Unless you really want to pore through tons of ads, we recommend that you kick off your Web searching anywhere but metasearch sites. Try Google or Yahoo. However, if you like the fact that metasearch sites scour numerous engines at once, check out Ixquick (www.ixquick.com), a different kind of metasearch site that forgoes the slew of ads.
paid links that aren't clearly labeled are bad enough, but at some sites, PC World found evidence that you can't always trust the real results. In some cases, results at AOL (www.aol.com) and MSN seem to serve the companies' corporate needs as much as searchers' interests. (Note: PC World regularly provides content for AOL's Computer Center channel.)
In our tests, we found that some portal sites plug their own products and services every chance they get. For example, when we searched for "travel" on AOL and on MSN, the companies ranked their own affiliated e-commerce sites high on the first page. Though you might expect such listings in the sponsored links, we noticed the phenomenon in the real results as well.
When you run a search for "travel" on AOL, four of the first ten results that AOL delivers are affiliated with AOL the number one recommended site is AOL Travel. AOL ranks Expedia, Microsoft's travel site, at number 14. Plug the same keyword into MSN's search field and you'll find no mention of any AOL travel sites within the first 50 results. But Expedia appears in the top spot in two categories, Featured Sites and Sponsored Sites. Expedia is the second-most-visited travel site on the Web, and AOL Travel is number seven, according to recent statistics from the research firm Jupiter Media Metrix.
Why are results so different on the two portals? It's hard to find an innocent explanation. AOL and MSN both use Inktomi's technology for searches. MSN also uses Overture-powered sponsored links. When we carried out our search tests, AOL was still using Overture's technology.
Inktomi and Overture say that their customers can manipulate results and prevent competitors' sites from showing up. But both AOL and MSN deny manipulating Inktomi's search results. Neither company, though, could explain how they produce different results for identical searches.
"Every AOL search [result] features the most relevant content for AOL users, whether it is a partner or nonpartner site," says AOL spokesperson Andrew Weinstein. With the search results that MSN generates, "everything is editorially managed," says Brian Gluth, MSN senior product manager. "It's all relevancy."
Not always commercial-free
AOL and MSN aren't the only sites where commercial interests seem to affect the real results. Try using MetaCrawler to search for a topic like "Abraham Lincoln." Even once we'd looked past the paid links, we were presented with a remarkable list of links to sites selling Lincoln T-shirts, vases, and other memorabilia. Of the first ten real results, only three weren't trying to sell us something. And when we ran a search for "digital cameras" on MetaCrawler, the last handful of links appearing on each page were identical all pointing to shopping sites.
InfoSpace's Steve Stratz says MetaCrawler has no policy to place commercial sites above real results. "The search providers on MetaCrawler currently are heavy on the paid side," says Stratz. "We are working hard to even this out by partnering with pure search providers." Stratz points to recent agreements with Ask Jeeves, FAST, and LookSmart. Search engine AllTheWeb (www.alltheweb.com) is based on FAST's search technology.
Even computer scientists can't prove or disprove whether funny business goes on within relevancy rankings. "Many search engines really don't publish reliable scientific information about the quality of their results," says Amanda Spink, an associate professor of information sciences and technology at Pennsylvania State University who recently completed a five-year study of the search habits of Excite users. "Most search sites don't like to give out this kind of competitive information," she adds.
Search companies provided only basic information about how they rank results. Google employs a complex algorithm, or mathematical formula, to rank pages. Yahoo employs a team of editors to create useful categories of Web sites that they've reviewed for inclusion in the directory. Beyond that, however, search sites refuse to explain the ins and outs of their formulas.
Spink's study showed that about 70 percent of people don't scroll beyond the first or second page of search results. That could prove troublesome if results pages become front-loaded with sponsors or paid inclusion listings links that Web site owners pay a yearly fee to have included in a search engine's results.
Paying to be seen
A site owner who pays a search engine to be included receives a guarantee that the site's pages will be frequently revisited (say, every couple of days) by the search engine's spider_a tool that trolls the Web for new or updated pages. This scheme differs from paid placement, in which advertisers are guaranteed a top position in the results whenever someone searches on certain words. Paid inclusion sites pay to be looked at by the spider, but aren't guaranteed a spot at the top of the results. Sites that buy into Yahoo's plan fork over $299 per year. Lycos customers pay $18 per year, plus $12 for every URL they submit.
Search companies promise that this payment feature won't fiddle with results. "This does not affect the relevancy ranking in any way, shape, or form," says Bryan Burdick, vice president of portal services for TerraLycos, the company that owns the Lycos search site. Lycos is another company that uses the FAST search engine technology. So, according to Lycos, users will see ranked results based on what the FAST formula decides is most relevant, regardless of who has paid to be included.
Google, however, says it firmly opposes paid inclusion. "The best thing for a search engine to do is crawl as many URLs as it can, as often as possible," says Google software engineer Matt Cutts. "Treating a smaller list of sites differently isn't fair."
Search industry veterans like Sullivan foresee possible problems down the road. "There is the potential for abuse," he says. "Pages that consumers want could be ignored by spiders."
Are certain types of nonprofit sites being left behind by search engines as paid inclusion becomes widespread? Not yet, Sullivan contends. "Search engines are probably realizing that they need to do a good job of picking nonprofit and research sites," he says. "They don't want to look bad." LookSmart, for instance, offers a special site, Zeal.com, that lets nonprofit organizations sign up for consideration for free.
Are ads all bad?
Sponsored links and paid inclusion practices aren't all gloom and doom for users. Ads can be useful, depending on what you're searching for. If you're shopping for something specific, sponsored links can often lead you to reputable e-commerce sites. For example, if you search for "Palm M130" on any of the major sites, you'll get links to several stores, and often to ads or sponsored links with the latest prices.
That said, the huge focus on commercialism can leave you feeling like the once cool and democratic Web is becoming too much like the Yellow Pages. But if you don't like the way that search sites are making money through paid placement and paid inclusion programs, then consider the alternatives. You might find them even more distasteful.
AltaVista now lets Web sites pay to add small graphics and tag lines to the search results for a company name or product. When we searched for "oatmeal" on AltaVista, for example, AltaVista ranked the Quaker Oatmeal site number one. We saw the familiar Quaker-man logo next to the listing. In this case, the food company paid to have its logo appear it did not pay to appear at the top. Still, imagine if the whole results page became cluttered with logos and marketing spiels.
Another alternative: Those increasingly aggressive pop-up, pop-under, and "floating" ads. Somill Hwang, a San Francisco public relations account executive, has a strong reaction to such bandwidth-intensive ads. Hwang, whose friends call her the office search engine guru, says she'll take sponsored links over pop-up ads any day. "If you don't want a sponsored link, you just glance past it," she says. Annoying ads are the quickest way for search companies to lose her loyalty.
Pundits like Danny Sullivan agree that paid links will stay around. But searchers should insist that these links be called "sponsored" links. If you see labels on search sites like "special results," "featured links," or other euphemisms, your antennae should go up quickly. The good news is that you can still find results that haven't been paid for. We're impressed with Google and Yahoo, and the ways that they lead you to such listings. You can tune out the ads easily and zoom in on the links you want.
If a search engine doesn't clearly mark its ads, vote with your mouse. You wouldn't watch a TV channel that disguised infomercials as shows, and you wouldn't buy a magazine that allowed merchants to pay to be recommended in its articles. Why put up with confusion at search sites?
Laurianne McLaughlin is a freelance writer based in Massachusetts. Tom Spring, a PC World senior reporter, also contributed to this article.