Sun has agreed to resell Diligent's de-duplication and virtual tape library software despite currently selling rival FalconStor's virtual tape library.

The company has rejected FalconStor's file-level de-duplication, preferring Diligent's sub-file-level technology.

Virtual tape library (VTL) software makes a disk array mimic a physical tape library with backup files being written to and restored from disk at disk speed instead of to slower tape. De-duplication software either removes duplicated files from the backup set (FalconStor's SIR or single instance repository) or, and more efficiently, removes duplicated chunks of characters from files, which is what the Diligent technology does.

Sun's high-end disk array supplier, Hitachi Data Systems, also resells Diligent's de-dupe software.

This software de-duplicates backup files as they come into the VTL rather than after they have been first written to disk, known as post-processing. FalconStor believes post-processing is a better way of doing de-dupe because the ingestion of backup files isn't slowed down while the CPU-intensive de-dupe operation is carried out. However the de-dupe hardware needs more disk storage to hold the incoming and raw backup data. This disk space could hold twenty times as much de-duped data, if not more.

FalconStore CEO Rei-Jane Huai thinks ingest time de-dupe is the wrong approach: "Acquisition time de-dupe is propaganda. It will not be valid."

Clearly Sun disagrees with this as it is taking the ingest-time Diligent software.

Doron Kempel, Diligent's chairman and CEO, said: "We are excited about the partnership with Sun and the opportunity to offer ProtecTIER to its growing customer base."

Sun has recently released its hybrid server+storage product, the X4500 aka 'Thumper,' in VTL form using FalconStor's software. The product is called the StorageTek VTL-V and holds up to 24TB of data with 500GB SATA drives. An upgrade to 750GB drives is planned, meaning a 36TB capacity. With a 20:1 de-dupe ratio its effective capacity would be raised to 720TB.

Sun will also sell the Diligent ProtecTIER de-dupe software running on its X4600 server connected to a StorageTek 6000 array providing up to 168TB raw capacity, and, at a 20:1 ratio, 3.36PB effective capacity.

Diligent and Sun already have joint ProtecTIER customers -- some with over 1PB of physical storage under management

Sun is developing an enterprise-level VTL-E product with upgradability promised from the VTL-V, through a mid-range VTL Plus, also using FalconStor VTL software, to the VTL-E.

FalconStor is developing its own sub-file-level de-duplication software. Current implementations of its SIR and VTL products use two separate server boxes but FalconStor has a one-box implementation.

The Diligent arrangement would imply either a separate VTL product line or the removal of the FalconStor technology to some degree from the VTL-V, VTL Plus and VTL-E products.

From a marketing standpoint, having two incompatible VTL products is confusing. For Sun to take this step underscores the importance of it having a credible de-dupe offering.

A future prospect is the extension of the Diligent de-dupe technology to Sun's network-attached storage (NAS) product line with consequent twenty-fold potential increases in its effective storage capacity. Diligent CTO Neville Yates said: "Diligent will be introducing file system de-duplication with a Network File System interface and leveraging our de-duplication engine to the network-attached storage topology."