If you're feeling overwhelmed by information overload lately, you may not be alone. The amount of new information stored on various media such as hard drives has doubled in the past three years, to five exabytes of new information produced in 2002, according to a study released Tuesday by the University of California, Berkeley.
That's exabytes, as in one byte with 18 zeros behind it, six zeros more than a terabyte. The amount of information put into storage in 2002, five exabytes, was equal to the contents of a half a million new libraries, each containing a digitized version of the print collection of the entire U.S. Library of Congress, according to the study by professors Peter Lyman and Hal Varian of the UC Berkeley School of Information Management and Systems. The professors estimated that between two and three exabytes of information was generated in 1999.
Most of that data -- 92 percent of it -- was stored on magnetic media, primarily hard drives, the study estimates.
The study, a follow-up to a 2000 study by UC Berkeley, doesn't dwell on how people and companies process these massive amounts of information coming at them, Lyman said, but his next goal is to produce a study examining that very issue. "I'm going to spend the next year on the consumption of information," he said. "How do people make sense of this? How do they cope?"
The current study doesn't address the quality of information and how people choose good information sources, he added. Significant differences exist in the "accessibility and usability and trustworthiness" of information between various sources, Lyman noted. "We treated it all the same, simply to understand how much there was ... but when you get into consumption, the discrimination over the quality of information, and how you make that decision, really becomes important," he added.
With the amount of stored information growing at a rate of about 30 percent a year, a "real change in our human ecology" is taking place, said Lyman, who presented the study at a conference in Florida Tuesday. "Everything is public," he said. "Everything is on the record."
One problem with all this information being stored is that it's not always accurate, he added. As information passes through multiple hands, it can be condensed or mischaracterized. So commentaries or reports on a speech or a paper Lyman gave 20 years ago sometimes contain distortions, he said.
"There are multiple renditions, only one of which I remember," he added.
The study underscores the need for companies to smartly manage their information, said Gil Press, director of corporation information at EMC Corp., an information storage vendor and a sponsor of the study. But IT solutions aren't the only answer, because humans still need to look at information with a critical eye, he added.
"We are getting swamped, and we need better ways to organize and manage information," Press said. "Hopefully, information technology will never replace smart thinking and the human analytical thinking."
The amount of stored information is not all the information that's being produced. Electronic channels -- including TV, radio, the telephone and the Internet -- produced three and a half times as much information as was stored in 2002. Most of that information was exchanged through voice telephone calls and not recorded or stored, Lyman said. The telephone accounts for the largest percentage of information flow -- 17.3 exabytes if stored in digital form -- followed by e-mail, which generates about 400,000 terabytes of new information each year, the study's authors said.
The researchers estimated that the World Wide Web contains 172 terabytes of information on public pages.
The UC Berkeley researchers used various methods to estimate the amount of information generated and stored, including statistics such as hard drive and paper sales, publication statistics and a sampling of the Web. The research team's methods are described in more detail at http://www.sims.berkeley.edu/research/projects/how-much-info-2003/.
One surprise for Lyman was that while digital storage continues to grow, the use of paper to transmit information is not shrinking. His team estimated that the number of terabytes of information put on paper each year increased by 36 percent from 1999 to 2001, while the amount of data stored magnetically each year increased by 80 percent between 1999 and 2002.
North Americans each consume 11,916 sheets of paper each year, while residents of the European Union consume 7,280 sheets, the team estimated. The majority of that paper information is produced by office documents and mail, not in formally published titles such as books or newspapers.
Read up on the latest ideas and technologies from companies that sell hardware, software and services. Email Archiving Implementation: Five Costly Mistakes to Avoid
Gaining Competitive Advantage Through Enterprise Planning
IT Service Management Needs and Adoption Trends: An Analysis of a Global Survey of IT Executives
Strategies for Eliminating .PST Files
Business Intelligence and Enterprise Performance Management: Trends for Emerging Businesses
Taking On Demand CRM Integration to the Next Level
Data grids and service-oriented architecture
Refresh your AUP: Top tips to ensure your acceptable use policy is fit for purpose
Zones provide focussed content from Computerworld and leading technology partners.Discover how SOA can create smarter outcomes for your business.
Attend and learn:
- How SOA is helping leading companies to become more agile
- Where you should be applying SOA processes in your company
- The top SOA implementation mistakes to avoid
Click here for more information.
- +
Computerworld Live Podcast #97: The Future of Enterprise Networking 25/07/2008 09:45:36
This week CW Live chats with Mark Thompson, global sales and marketing manager for HP ProCurve, on the future of the enterprise networking. Mark discusses the trends we can expect to see in the near future and how the right infrastructure can ensure your enterprise network is secure. - +
Computerworld Live Podcast #96: Security at the Edge 11/06/2008 09:22:22
CW Live speaks with Amol Mitra, HP ProCurve Director of Marketing for Asia Pacific and Japan. Today's topic: how enterprises are starting to shift away from simply controlling security via server logins, firewalls and moving to more adaptive security frameworks. - +
Data Management Edition #10: Multi-Petascale Systems 02/05/2008 09:12:33
This week we look at sustainability and the development of multicore technologies to build multi-petascale systems. - +
IT Security Edition #11: How to poison the Storm botnet 01/05/2008 08:51:55
This week CW Live presents a case study on how to poison the notorious Storm botnet . Plus we take a look at Cisco's plans for Ironport. - +
IT Security Edition #10: Cyber-battles fought and won 24/04/2008 11:09:47
Vendors bow to end user pressure to improve product security, and we take a look at the latest concepts shaping the cyber-battlefield of the future.
F-Secure: Growth In Internet Crime Calls For Growth In Punishment 2008-12-05 13:00:00+11
International researchers gather in Sydney to preview the clever web 2008-12-05 09:48:00+11
Borderless corporate networks to shift focus to secure content management in Australia in 2009 2008-12-04 16:06:00+11
IDC Says Asia/Pacific Excluding Japan IT Market Will Remain The Bright Spot... 2008-12-04 15:04:00+11
MySpot SOS "Panic Button" Smartphone Application could save lone worker lives 2008-12-04 13:34:00+11
Email Archiving 101—Customer Case Study
Join Lee Benjamin, a Microsoft Exchange MVP and Ryan Shipkowski, network administrator for Matthews, to discuss the process and ROI of implementing an email archiving solution, with emphasis on a case study from Matthews International.












