In a discovery that would probably send the Dr. Evil character of the "Austin Powers" movies into cardiac arrest, Google recently detected more than a trillion unique URLs on the Web.
This milestone awed Google search engineers, who are seeing the Web growing by several billion individual pages every day, company officials wrote in a blog post Friday.
In addition to announcing this finding, Google took the opportunity to promote the scope and magnitude of its index.
"We don't index every one of those trillion pages -- many of them are similar to each other, or represent auto-generated content ... that isn't very useful to searchers. But we're proud to have the most comprehensive index of any search engine, and our goal always has been to index all the world's data," wrote Jesse Alpert and Nissan Hajaj, software engineers in Google's Web Search Infrastructure Team.
It had been a while since Google had made public pronouncements about the size of its index, a topic that routinely generated controversy and counterclaims among the major search engine players years ago.
Those days of index-size envy ended when it became clear that most people rarely scan more than two pages of Web results. In other words, what matters is delivering 10 or 20 really relevant Web links, or, even better, a direct factual answer, because few people will wade through 5,000 results to find the desired information.
It will be interesting to see if this announcement from Google, posted on its main official blog, will trigger a round of reactions from rivals like Yahoo, Microsoft and Ask.com.
In the meantime, Google also disclosed interesting information about how and with what frequency it analyzes these links.
"Today, Google downloads the web continuously, collecting updated page information and re-processing the entire web-link graph several times per day. This graph of one trillion URLs is similar to a map made up of one trillion intersections. So multiple times every day, we do the computational equivalent of fully exploring every intersection of every road in the United States. Except it'd be a map about 50,000 times as big as the U.S., with 50,000 times as many roads and intersections," the officials wrote.
Read up on the latest ideas and technologies from companies that sell hardware, software and services. Business Intelligence and Enterprise Performance Management: Trends for Emerging Businesses
Cutting printer costs
Mimosa™ NearPoint™ for Microsoft® Exchange Server: Email Archiving 101
Gaining Competitive Advantage Through Enterprise Planning
Solve Exchange Mailbox Storage Issues Once and for All
Data grids and service-oriented architecture
Everything you need to know about email and web security (but were afraid to ask)
Enterprise Wireless WLAN Security
Zones provide focussed content from Computerworld and leading technology partners.Discover how SOA can create smarter outcomes for your business.
Attend and learn:
- How SOA is helping leading companies to become more agile
- Where you should be applying SOA processes in your company
- The top SOA implementation mistakes to avoid
Click here for more information.
- +
Computerworld Live Podcast #97: The Future of Enterprise Networking 25/07/2008 09:45:36
This week CW Live chats with Mark Thompson, global sales and marketing manager for HP ProCurve, on the future of the enterprise networking. Mark discusses the trends we can expect to see in the near future and how the right infrastructure can ensure your enterprise network is secure. - +
Computerworld Live Podcast #96: Security at the Edge 11/06/2008 09:22:22
CW Live speaks with Amol Mitra, HP ProCurve Director of Marketing for Asia Pacific and Japan. Today's topic: how enterprises are starting to shift away from simply controlling security via server logins, firewalls and moving to more adaptive security frameworks. - +
Data Management Edition #10: Multi-Petascale Systems 02/05/2008 09:12:33
This week we look at sustainability and the development of multicore technologies to build multi-petascale systems. - +
IT Security Edition #11: How to poison the Storm botnet 01/05/2008 08:51:55
This week CW Live presents a case study on how to poison the notorious Storm botnet . Plus we take a look at Cisco's plans for Ironport. - +
IT Security Edition #10: Cyber-battles fought and won 24/04/2008 11:09:47
Vendors bow to end user pressure to improve product security, and we take a look at the latest concepts shaping the cyber-battlefield of the future.
Vignette Announces 2008 Excellence Awards 2008-11-21 10:50:00+11
PGP and Ponemon Institute Unveil Inaugural Australian Data Breach Study 2008 2008-11-20 17:34:00+11
Symantec Cloud Services Transform Data Centre Operations Through Proactive Management 2008-11-20 12:06:00+11
Verizon Business Offers Tips to Building a Successful Unified Communications and Collaboration Plan 2008-11-20 12:04:00+11
AARNet Brings 4K Digital Cinema to Australia: First 4K HD Video Signal delivered into Australia by AARNet 2008-11-20 12:02:00+11
CRM your salespeople will love
Winning over the sales department and obtaining buy-in at all levels is crucial to the success of any CRM initiative. Discover how you can let salespeople work how they want to and reduce their administrative burden with the latest CRM technology.









