If you haven't intuited from my articles yet, I have a strong bias that application data management goes hand in hand with storage management. Unfortunately for most of IT, both seem to be relegated to "afterthought" status, if data and storage management enters the consciousness at all. In the business best practices book, application data and storage management should be highlighted. Why? Proper management of data and storage assets will save companies large amounts of money over time.
OK, so a good percentage of companies are finally backing up their data (when I started as an analyst two and a half years ago, this was not the case). Backup is a good example of the melding of data and storage management. However, companies are using backup as an archival solution as well and, as any company that has had to retrieve data older than two or three years can attest to, backup is better suited for short-term data protection/restoration. Using backup for long-term retention of data is like a carpenter using a drill to saw a piece of wood in half - it can be done but it's not pretty.
But there's a problem with keeping data for long periods of time, isn't there? The problem is retaining the content - the information represented by the data without having to also maintain several, parallel, native environments with which to access the data over time. How many times have you received a tape purported to have the necessary data but have had no idea what format the data was in or how to access it? Industries that traditionally retained data for historical significance have tried to move toward data format standards. An example of this is the RTF (rich text format) standard for textual data but not all data can be represented textually.
You ask then, what should a company do to retain application data over many years? Well, it depends on the type of data and the reason(s) for keeping the data for extended periods of time. In a previous article, I discussed the need to keep all forms of communication data (e-mail, instant messages, voice data) for use in/against litigation. The recovery/access profile for this type of data is that most companies hope they never have to look at it but when they need to access this data, they need to access and sort through large amounts quickly and efficiently, without the need for the native environment. Renew Data addresses this market by transforming all this data to a common format that can support fast text searches for optimal efficiency of both the storage resources containing this data and the companies legal department.
This is fine for data that can be accessed without a native environment but what about databases that provide context through the relationships between data elements within a data object? The other "special" thing about databases is that the larger they become, the slower they become further justifying the need to manage both the data and the storage associated with the database in lock-step.
Many companies understand this and have tried to solve this dilemma using traditional HSM or archival technologies. These technologies will move files from one level of storage, for example, expensive, highly available online storage to tape or optical and more recently cheap disk, dependent upon predefined policies.
These policies often take files that have not been accessed for a while, and move them to other storage. The problem IT has run into is that the database objects, or relationships, are often broken when these files are moved, creating havoc when the database can't find the associated data. This, again, is where application data management - management capabilities that understand and interface with the applications - and storage management must integrate seamlessly. OuterBay, one of the leadership companies in this space, is providing this level of integration.
Creating a business best practice around managing your application data together with your storage has become a necessity in today's business environment. There are many types of data that require archival and the requirements for retaining data are getting longer and more stringent. Talk to your vendors about what level of management integration they can provide as well as what standards they are helping to develop to address this vital issue.