Wikia's project to develop an open-source search engine got another boost with the acquisition of the Grub distributed Web crawler, the company announced Friday.
Wikia acquired Grub from LookSmart and released it under an open-source license, adding a significant component to Search Wikia, scheduled to debut in this year's fourth quarter.
The Search Wikia project seeks to create a search engine based on open-source search protocols and human collaboration, drawing from the concept of the Wikipedia online encyclopedia, which is written and edited by a community of volunteer collaborators.
As such, it will provide a better search experience than the ones offered by commercial, proprietary search engines like those from Google Inc., Yahoo and Microsoft, said Jimmy Wales, Wikia's co-founder and chairman, and Wikipedia founder.
However, people shouldn't expect Search Wikia's first version to deliver on this lofty goal. "When I opened Wikipedia with only three articles in it, it was a pretty bad encyclopedia," Wales said. "That's where we're going to be in December with the search engine: We'll tell people upfront: 'It's not very good yet.' But we'll open it up to get feedback and community involvement to help us make it better."
Wikia, which has about 35 full-time employees, is developing its own relevancy technology to rank search results, he said. At this point, it's not clear whether at launch Search Wikia will offer only general Web searching or if it will have specialty engines for things like images and news.
Visually, Search Wikia will have the standard search-engine interface. "The differences will be that at various points all along that search process, there'll be opportunities for people to engage with the community and join and participate in the construction of the search results," Wales said.
Since Search Wikia's announcement in December, an increasing number of organizations, such as search and online advertising player LookSmart, have expressed interest in the project, Wales said. In the coming months, Wikia will provide more details about third-party partnerships and support.
"A lot of second-tier [search] players understand that competing with Google directly as independent proprietary projects they'll never catch up because they don't have enough individual resources. But by banding together using open-source software, they can effectively compete with Google and improve their services that way," Wales said.
In fact, Wales believes that supporting Search Wikia would be in the best interest not only of second-tier search engines but also of search leaders, including Google. Search isn't a "defensible business" because it's very easy for people to switch among providers, so just like Google unseated AltaVista, another large competitor could emerge, he said. Thus, democratizing search in the way Search Wikia will attempt to do is a good thing for Google.
"Google will be much better off if search becomes ubiquitous and there are lots of players because that doesn't threaten their dominance in the advertising market," Wales said. "Google is much better off with lots of players and being the ad broker for everyone, because ad brokerage is a defensible business."
Allowing users to participate in the design and index of a search engine isn't new. For example, Yahoo's popular Answers search engine lets users pose and answer questions, while Mahalo.com Inc. recently launched a search engine whose index is handcrafted by people, not automated Web crawlers. Social bookmarking sites like Yahoo's Delicious compile users' favorites Web site links and makes the links collection searchable by other users. There is even an open-source search engine project called Nutch.
"There's definitely a place for social search and human editorial input into search results," said industry analyst Greg Sterling from Sterling Market Intelligence.
However, so far the efforts haven't had a significant impact on the overall search market, he said. "If you can do something to really improve the quality of the search results, or if you present results in a radically new interface, or do a mix of those two, you might have shot of getting some attention," Sterling said.
If anyone can pull it off, it's Wales, who proved against all odds that an open, free, volunteer-backed encyclopedia could achieve the level of success Wikipedia has, Sterling said. "Wikipedia does suggest there is a probability of success in this," Sterling said.
Search Wikia and Wikipedia operate under two separate organizations that were both founded by Wales: Wikia is a commercial, for-profit company while The Wikimedia Foundation that oversees Wikipedia is a nonprofit entity. Wikia and Wikimedia operate independently of each other. Search Wikia is one of several projects and services at Wikia.
Read up on the latest ideas and technologies from companies that sell hardware, software and services. Achieving the impossible: Unlimited application scalability
Delivering the Power of Choice with Microsoft Dynamics CRM
Business Intelligence and Enterprise Performance Management: Trends for Emerging Businesses
Best Practice in Building an Integrated Information Management Strategy
Strategies for Eliminating .PST Files
IT Service Management Needs and Adoption Trends: An Analysis of a Global Survey of IT Executives
Mimosa™ NearPoint™ for Microsoft® Exchange Server: Email Archiving 101
The state of Middleware
Zones provide focussed content from Computerworld and leading technology partners.Discover how SOA can create smarter outcomes for your business.
Attend and learn:
- How SOA is helping leading companies to become more agile
- Where you should be applying SOA processes in your company
- The top SOA implementation mistakes to avoid
Click here for more information.
- +
Computerworld Live Podcast #97: The Future of Enterprise Networking 25/07/2008 09:45:36
This week CW Live chats with Mark Thompson, global sales and marketing manager for HP ProCurve, on the future of the enterprise networking. Mark discusses the trends we can expect to see in the near future and how the right infrastructure can ensure your enterprise network is secure. - +
Computerworld Live Podcast #96: Security at the Edge 11/06/2008 09:22:22
CW Live speaks with Amol Mitra, HP ProCurve Director of Marketing for Asia Pacific and Japan. Today's topic: how enterprises are starting to shift away from simply controlling security via server logins, firewalls and moving to more adaptive security frameworks. - +
Data Management Edition #10: Multi-Petascale Systems 02/05/2008 09:12:33
This week we look at sustainability and the development of multicore technologies to build multi-petascale systems. - +
IT Security Edition #11: How to poison the Storm botnet 01/05/2008 08:51:55
This week CW Live presents a case study on how to poison the notorious Storm botnet . Plus we take a look at Cisco's plans for Ironport. - +
IT Security Edition #10: Cyber-battles fought and won 24/04/2008 11:09:47
Vendors bow to end user pressure to improve product security, and we take a look at the latest concepts shaping the cyber-battlefield of the future.
FrontRange Solutions launches HEAT Plus Mobile to reduce help desk costs and improve service management productivity 2008-12-02 15:15:00+11
AARNet Helps to Advance Indigenous Health 2008-12-02 12:44:00+11
Orbis selects Telstra International as its data centre partner for the UK, Europe and Middle East Region 2008-12-02 11:23:00+11
ComOps Deploys Corporate Performance Reporting Solution For Healthcare Test Manufacturer 2008-12-02 10:09:00+11
Mornington Peninsula Shire implements Objective to manage knowledge and deliver service excellence 2008-12-02 09:56:00+11
Everything you need to know about email and web security (but were afraid to ask)
What you don’t know can destroy your business. It’s hard to imagine modern business without the internet but in the last few years it has become fraught with danger. Read on to discover how internet security can give your business a competitive advantage.












