Enterprise search is set to become a pervasive and demanding force in IT over the next several years, according to Gartner analyst Whit Andrews.

"Information access technology will locate and analyze more than 90 percent of data in more than 50 percent of Global 2000 enterprises" by the end of 2012, according to materials from a presentation Andrews gave at Gartner's Symposium ITxpo conference in Las Vegas this week. Gartner refers to enterprise search by the more general phrase "information access technology."

Some observers saw Microsoft's recent move to acquire enterprise search vendor FAST Search & Transfer as a validation of the market. It will compete with a range of large vendors, such as Autonomy and FAST, along with a series of smaller companies, including Recommind and X1 Technologies.

"All the infrastructure vendors need to respond in some way to the need for effective search technology in their products," Andrews said in an interview on Thursday.

But it's unclear whether the market will see a rush of major consolidation, given the high cost of buying a top independent player, he said. (Autonomy has a market capitalization of US$4 billion, according to its Web site.) "At this point, what I've seen is that this is well short of a gold rush. ... It's not easy to see where this is going to go."

As for smaller players, Andrews said he is "fairly confident they're all looking to be acquired at this point. I think if they don't get bought, then they have to go highly specific."

Whatever path companies choose regarding enterprise search, they will face major challenges, chiefly the high expectations of users, Andrews noted in his presentation: "End-users of information access technology do not recognize, respect and treat as reasonable the divisions that application architecture have forced on information access strategy."

Information itself will need to be organized and augmented to a greater degree, he states. "Critical issues include flexibility of indexing, incorporation of security down to the document level and, in rare cases, the subdocument level, and the flexibility to access APIs in business applications."

The general methodology for collecting search results will change as well, Andrews predicts.

"The classic model for information access technology is the spider, which travels around the threads of a document Web and returns with a picture of its structure ... However, this model demands that the data be somewhat stale, and it is not acceptable for transaction-sensitive business applications or the databases they feed and by which they are fed," he writes.

An emerging model, in Andrews' words, can be described as an ant: "Rather than traverse a document set and store a pattern of what it finds, the ant travels on well-known pathways to discover the 'freshest' morsel of data and return it to the colony where it can be merged with other such morsels for the good of the whole."

Andrews also predicted that:

Through the end of 2012, no Global 2000 enterprise will have standardized "absolutely" on a particular information access platform.

Business intelligence will work in concert with enterprise search at 90 percent of Global 2000 companies in 2012. "Absolute convergence between business intelligence and search applications will, typically, not occur, but business intelligence and search will work in tandem," he wrote. "However, a few exceptional vendors will emerge to provide a combined business intelligence and information access platform."