If imitation is the sincerest form of flattery, developer of the open source fire transfer tool rsync, Andrew 'Tridge' Tridgell has every reason to smile as Microsoft announced a similar utility will feature in the next Windows server release.
Speaking at this year's TechEd conference on the Gold Coast on Wednesday, Redmond-based Windows Server group director Iain McDonald said the next update of Windows Server 2003, dubbed "R2", will reduce bandwidth for file transfers for organizations with many branch offices.
"There is new functionality we wanted to be able to provide to you in R2," McDonald said. "First of all there is a bunch of stuff for branches. In the US there are new regulations about what [organizations] have to do on branches, what they have to keep around over longer terms. We had a look at that and in discussions we knew one of the big things was 'let's minimize the amount of bandwidth that stuff takes to get into a central location'."
To achieve this, Microsoft will introduce DFSR, or Distributed File System Replication, with R2, which is due for release later this year.
In contrast, Tridge's initial report on the rsync algorithm, dated June 1996, describes how to perform file updates over a network just by sending the changes. Free to download and licensed under the GPL, rsync is now ubiquitous across Linux and Unix systems, including Apple's Mac OS X.
"So we built in ways to reduce your network bandwidth around replicating files down hugely," McDonald said. "That's a thing called DFSR, which basically has the ability to do remote differential compression. Normally, in the past up to now, it will shove the whole file back across. If it is a one-word change in that file the only thing it is going to shove across is that word and pointers to that."
Mc Donald said people doing early deployments have been able to reduce their replication traffic by "somewhere around 85 percent". Tridge, now an Open Source Development Lab (OSDL) Fellow, said although this is the first he's heard of Microsoft implementing an rsync-like utility, he would be delighted to hear about any vendor implementing the rsync algorithm.
"There are many ways [Microsoft] could be achieving this," Tridge said in an e-mail interview. "One of them is the rsync algorithm, but they could also be using a more traditional low bandwidth mirroring system which works by keeping an old copy of the data around (rsync does not need that)."
Tridge said it is "a bit premature" to assume Microsoft is using an rsync-like algorithm, but it could well be because the phrase "remote differential compression", "certainly matches rsync" but it could also easily mean "something quite different."
"Either way, I hope that Microsoft decides to fully document how this works at the bits and bytes level, so that others can interoperate," he said, adding it is unfortunate Microsoft does not have a good track record of providing sufficient documentation for other vendors to be able to create interoperable software.
Not surprisingly, Microsoft's inspiration for developing DFSR may have come from Tridge himself who, back in 1996, presented at the first CIFS (Common Internet File System) conference in Redmond on how the protocol should be extended to include rsync-like features.
"This didn't happen, but I still think it would be a good idea for WAN usage of CIFS and related protocols," he said. Overall, Tridge believes rsync is an important algorithm in the toolbox of software engineers, and its potential has not been exploited "nearly as much as it could be."
"I especially think it has a huge potential in the area of Web acceleration," he said.
What TechEd attendees want
"The reporting services in SQLServer 2005 should greatly reduce the amount of effort for ad hoc reporting." Geoff Clark, development manager, Bluestone Group, Sydney
"Visual Studio Team System is definitely a step up and will make Microsoft development a lot easier to manage." Yatin Prasad, senior analyst, ARIA, Sydney
"Vista looks like it will have a lot of benefits from a server perspective, not just a desktop." Russell Hart, distributed server manager, Queensland Rail, Brisbane
"I would like to see a better reporting structure for IIS 7.0," Russell Wilson, Internet services administrator, University of Southern Queensland, Toowoomba
"I like the GUI for Vista as it seems to be a lot more of an intuitive experience of moving around the operating system." Brad Day, systems analyst, Queensland Health, Brisbane
"The Visual Studio tools look good but the licensing needs to be simpler." Julian Dell, development team manager, News Limited, Sydney
"TechEd is focused on the future, and since I have a job to do, it should be more focused on the here and now," Ian Sampson, solutions director, Glanton Solutions
Rodney Gedda attended the conference as a guest of Microsoft