Top ten most influential of 2010: Data growth

Exponential data growth is leading to the adoption of new data management technologies such as, data deduplication, tired storage and the cloud
Data growth and management

Data growth and management

Each Friday until the end of the year, Computerworld Australia is revealing one of the top ten most influential people, technologies and trends that shaped 2010 in Australia. The top ten was collated and determined by our editorial team and advisory panel of IT managers, industry experts, consultants and analysts. The list so far:

(See how we chose the top ten)

A Readers’ Choice poll is also open to determine what our readers think should be included, and what shouldn’t.

Coming in at number six for Computerworld Australia's top ten most influential of 2010: Data growth.

Data growth

It may well be referred to as ‘snorage’ but the storage sector is actually one of the more interesting segments of the ICT industry, largely as it is a window into what exactly people and organisations are doing with their IT.

How? Well, think data, the myriad types of data, and the massive, unending, exponential growth in data by both consumers and organisations.

On the consumer side, there’s end-user generated content such as videos on Youtube, tweets on Twitter, photos on Flickr, not to mention email, instant messaging, and SMSes.

Then there’s structured and unstructured organisational data such as, email, attachments, customer data via CRM suites, Excel and Powerpoint files, business data via business analytics, ERP systems, HR and payroll systems, financial and regulatory data necessary for compliance, intellectual property… the list is near endless.

And the biggest part is, the growth only gets bigger and faster. In 2008, IDC and EMC were arguing data requirements were growing at an annual rate of 60 per cent. Back then, that equalled about 281 exabytes in total data. Coming in to 2011, that figure is due to hit 1,800 exabytes. (An exabyte equals one billion gigabytes, or one million terabytes.)

As a result of this, managing data growth has rapidly grown to become a major priority of IT managers and CIOs alike, seeing technologies such as data deduplication, tiered storage and the cloud become project priorities.

Then there was the data centre gold rush which saw the likes of Equinix and Global Switch fire up new data centres. Data growth has also fuelled data centre consolidation, such as that over at the Department of Finance.

Just this month, Gartner declared data growth to be the biggest data centre hardware infrastructure challenge for larger enterprises.

In a survey, nearly half of respondents ranked the issue above system performance (second) and scalability (third) and network congestion and connectivity architecture (fourth) at 36 percent.

As a result, nearly two thirds of respondents will now be investing in data archiving or retirement by the end of 2011 to address the data growth challenge. Half of respondents will have to expand capacity at their existing data centre site by the end of 2011, with 30 per cent having to build new data centres.

"While all the top data centre hardware infrastructure challenges impact cost to some degree, data growth is particularly associated with increased costs relative to hardware, software, associated maintenance, administration and services," said Gartner research director, April Adams, summing up the situation.

"Given that cost containment remains a key focus for most organizations, positioning technologies to show that they are tightly linked to cost containment, in addition to their other benefits, is a promising approach."

The spectre of ISPs being forced into a data retention regime also reared its head in 2010.

And all that’s without even touching on the challenges of how data can really get you into trouble — cue Google and Streetview.

(Do you agree with the data growth ranking in Computerworld’s top ten most influential? Let us know below or vote in the Readers' Choice award to have your say.)

Related articles about data growth

Microsoft: Cloud services hampered by unclear data rules

Better storage tech could trim energy, cooling costs, IDC says

Budget constraints still weigh on IT execs

SMBs overwhelmed with data turn to cloud services