Monday Grok: Reckon you know big data? You know nothing, they know everything
- 03 September, 2012 10:33
The US government is spending $2 billion building a data centre in the middle of nowhere — well actually, it’s in Utah — to eaves drop on communications in and out of the US. It is a huge facility that will cover an area of more than 1,000,000 square feet. The US government is also spending another $2 billion on a listening base in Georgia which will itself, according to ABC Radio’s PM program, employ more than 4000 staff.
While terrorism has proven a boon for the military industrial complex in the last decade, the reality is that the National Security Agency (NSA) in the US and its international partners like Australia’s Defence Signals Directorate were operating these sophisticated snatch and grab operations long before September 11 under the auspices of ECHELON.
What’s changed is that the people they track, or at least the people who know the people they track, are all carrying around iPhones and iPads, or tweeting like bandits, or soaking up T-Bytes worth of bandwidth on Skype.
So much spooking, and there’s still only 24 hours in the day.
The Bluffdale Utah facility will be able to store a yottabyte’s* worth of data. Now, until Grok caught the tail end of the interview between the ABC’s Mark Colvin and security writer James Bamford, he had never heard of a ‘yottabyte’ but apparently it’s the equivalent of 500 quintillion printed pages of information.
Of course, the spook business is not what it used to be — these days it’s awesomely bigger, but not nearly as secret as it once was, as a quick Google search reveals. And Wikipedia even helpfully gives you a map of the spy base.
Bamford’s original piece about the Utah facility was first published in March this year in Wired magazine.
There’s a great quote about half way through the story which gives you a sense of the immense scale of the operation. “It needs that (a yottabyte) capacity because, according to a recent report by Cisco, global Internet traffic will quadruple from 2010 to 2015, reaching 966 exabytes per year. (A million exabytes equal a yottabyte.) In terms of scale, Eric Schmidt, Google’s former CEO, once estimated that the total of all human knowledge created from the dawn of man to 2003 totalled 5 exabytes. And the data flow shows no sign of slowing. In 2011 more than 2 billion of the world’s 6.9 billion people were connected to the Internet. By 2015, market research firm IDC estimates, there will be 2.7 billion users.”
Big data indeed
For those of you reading this outside of the US and who are concerned about your civil liberties...well, a funny thing happened on the way to the data centre… And for those of you in the US, you will be glad to know the NSA doesn’t consider that it has intercepted your communications, even if it has recorded and stored it until one of its operatives actually listens to the phone call or reads the email. Kind of like a burglar who says he or she hasn’t really stolen your money until he or she spends it.
In case you are wondering why this is all turning up in the media now — after all, a million square-foot facility storing 500 quintillion pages worth of data doesn’t just spring up overnight — it’s because once again it seems that maybe you can’t really trust the government after all.
This article mainly concerns the US government but such scepticism applies equally to the governments of Canada, the UK, Australia and New Zealand, who make up the member states of the Five Eyes not to mention the US government’s other international intelligence allies.
A whistle blower named William Binney has starting talking to media outlets like the New York Times about “how the program he created for foreign intelligence gathering was turned inward on this country”.
Laura Poitras, who wrote the story in NYT has herself been on the receiving end of rough handling from the US government. "The United States apparently placed me on a“watch-list” in 2006 after I completed a film about the Iraq war. I have been detained at the border more than 40 times. Once, in 2011, when I was stopped at John F. Kennedy International Airport in New York and asserted my First Amendment right not to answer questions about my work, the border agent replied, ‘If you don’t answer our questions, we’ll find our answers on your electronics’.”
All of this seems a long way removed from the daily troubles of IT leaders and managers in Australia — the principle readers of Grok — but then again, this Computerworld Australia story about the implications of the Patriot Act for Rackspace customers in Australia makes a good jumping off point.
Join the Computerworld Australia group on Linkedin. The group is open to IT Directors, IT Managers, Infrastructure Managers, Network Managers, Security Managers, Communications Managers.
Thanks a million, Drupal
OS upgrades: Cheap is better than pricey, free is better than cheap
Amazon vs. Google vs. Windows Azure: Cloud computing speed showdown
The rise of security-as-a-service in Australia
The rise of security-as-a-service in Australia