Eight years ago it was already obvious (at least to us) that at some point CIOs were going to morph into CSOs--corporate storage officers. Every year the data river flooded higher and ran faster, with every byte requiring storage methods that were secure, shareable, scalable and monitorable, and could deal nimbly with an ever-wider range of user classes and network types. Sooner or later storage issues had to overwhelm the IS agenda. You can get along with last year's processor, but relying on last year's storage could put a company out of business.
In June 1992 we took a look at the coming era of storage-centric IT, dividing our attention between a new idea in storage management and a laundry list of new hardware technologies, including several flavors of optical media and the then-new concept of molecular storage.
In retrospect, we might have understated the trends. The online revolution has blown storage out along one dimension, by multiplying the number and kinds of users, while storage-intensive applications like data mining and multimedia have increased demands along another. At the Network Storage 2000 conference in June, EMC Corp. CEO Mike Ruettgers predicted that in three years the typical global business will serve more than a petabyte (1,024 terabytes) of online data, under ever-more stringent reliability requirements (a storage failure today can lead to a major stock dump). According to Gartner Group Inc., this year the storage function will take 70 percent of the IT budget--and the curves all point up.
Yet while we probably understated storage needs, we may have overstated the need to move to exotic new technologies. The industry has been able to deal with inflating storage demands without the fancy high-tech media (molecular, optical and others) we examined in our piece. The significant changes in storage have all been in the sphere of management, not in new technologies, a focus that looks to be growing even tighter over the next few years.
One of the first examples of this point was the insight, made by some computer scientists at Berkeley in the late '80s, that a number of disks acting in parallel could read out data at rates unachievable by a single disk. Specifically, if an "intelligent write" operation spread a file across several disks, then an "intelligent read" could pull the various parts of that file from several disks simultaneously, combine them and deliver the file as a whole, thereby emulating a single disk with awesome seek times.
This insight did not require fancy new storage media. On the contrary, in theory the system allowed managers to substitute many cheaper, slower, microcomputer disks for a few large, expensive, high-tech disks. This promise was reflected in its original name: redundant array of inexpensive disks (RAID). (Later, when actual RAID products proved not to be that cheap after all--the individual drives didn't cost much, but users had to buy a lot of them to match the storage of more traditional big-platter drives--the "I" was changed to mean "independent.") RAID was the management idea we were looking at in our 1992 article. Though it was new to the market, we were very enthusiastic, and, in fact, RAID did go on to become one of the great success stories in the history of data storage. (Though according to Peter Doob, vice president for RAID customer liaison at MTI Technology Corp., a data services company in Anaheim, Calif., the major marketing driver was not read performance but security: Writing a database to several drives at once not only enhanced read times, but it also gave a network seamless protection against hard disk crashes.) RAID was only the first example of using better management as opposed to exotic technologies. After our article appeared, a second emerged: storage appliances (SAs), otherwise known as network-attached storage. SAs supply all the storage functions for all the devices attached to a network, including backup, information protection and data compression. They allow a CIO to set a consistent storage policy across an entire network.
According to Howie Wilcox, an evangelist at Network Appliance Inc. in Sunnyvale, Calif., which makes such devices, storage appliances also are defined by sophisticated systems of change detection and handling, which allows them to generate images of databases with far fewer resources than conventional RAID boxes.
SAs have the disadvantage of depending on LAN connections, which in the online era are not always as fast as needed. For such cases, companies like Chatsworth, Calif.-based DataDirect Inc. make storage-area networks, which connect the devices needing and supplying storage with their own very high-speed connections. These two innovations have made it possible for a new industry segment to emerge: the storage service provider (or storage utility), which handles storage on an outsourced basis.
Overall, storage and processing, which once were married in the desktop, have gone in opposite directions. Storage has consolidated, with more of it in fewer places, while processing has differentiated, moving into devices like cell phones and digital cameras. This conceptually simple change has allowed companies to enter and flourish in the storage-centric era. As much fun as new-media technologies are to write about, the lesson of the storage revolution is that, sometimes, good management is really enough.