AARNet launches CloudStor

Open source powered transfer service will allow for the moving of files in the 100s of terabytes

AARNet is to launch a supercharged file transfer service, CloudStor, capable of transferring files hundreds of gigabits in size via the organisation's high-speed academic network.

The service, due to go live in early 2010, is designed to encourage greater collaboration between research and academic organisations through faster transfer of large data sets such as medical, gene sequencing, and synchrotron imaging and academic presentations.

The CloudStor service, co-developed with Ireland’s HEAnet and Norway’s UniNett, will allow AARNet members and collaborators to upload and download extremely large files to a central storage point hosted by AARNet. Data uploaded is automatically deleted after 30 days.

Guido Aben, director eResearch at AARNet said that while file transfer services such as Rapidshare and Yousendit already existed, the CloudStor service would offer researchers and academics support for file transfers of more than 2GB and the benefit of Australian-based storage.

“The prime benefit is our own storage in Australia for the community, so everybody knows where it is - it’s not like you’re putting it out there with a different company and you don’t quite know what jurisdiction, or deal with legal discovery of the data or whatever legal fears you may have,” he said. “As it’s in Australia it doesn’t have to go somewhere else only to be download again in Australia so there are a number of performance gains there.”

The service will also feature notification of an upload or download via email or instant messaging, a suspend and resume download utility, and a voucher system through which users can invite someone to upload or download a file.

“Only one of the [users] has to have an AARNet affiliation, so you could send something to an overseas collaborator or they could send something to you,” Aben said. “We can’t dictate where the content is so we need to work all over and the system does that.”

Being on AARNet, speed is also another major benefit to the service, however the end-user experience will depend on the tech savvyness of users and university firewalls, Aben said.

“[Speed] is intensely dependent on how well tuned the work station you’re sending it with is,” he said. “Most modern operating systems are still doing a fairly mediocre job of handling really high bandwidth transfers, so you get maybe 10 or 15 megabits per second, yet we have a network with native capacity of a gig to 10 gigs.

“We know server to server we can easily sustain transfers in the multiple hundreds of megabits per second so a 55 gig file could take an hour or so, but I would be surprised if end users got those speeds, because people are using untuned stations behind university firewalls. And they’re people who aren’t necessarily tech savvy.”

The service, Aben said, heavily utilises open source software based on a BSD licence, Google’s Gears library, a BSD backend, Flex front end and MySQL database. The service also runs on Linux or Windows to ensure it is available to the widest range of users. Security is provided via Nowegian partner UniNett through use of its open ID server to authenticate against AARNet’s server.

Email Computerworld or follow @computerworldau on Twitter.

Join the newsletter!

Error: Please check your email address.

Tags open sourcemysqlaarnetUniNettGoogle Gears library

More about AARNetetworkGoogleLinuxMySQLSpeed

Show Comments