You don't need me to tell you that there was some odd expenditure on Information Technology during the dot com years. Now that we can look back on it from the relative sanity of 2003, surely there should be an award initiated for the best (worst) examples of IT expenditure during that period? We could call them the 'Technology Turkey Awards', thus following in an established tradition of celebrating the awful.
If I were honored to put forward a nomination for a Technology Turkey Award, I would have no hesitation putting forward the phenomenon of spending money on IT systems that simply stuffed Web content into relational databases.
Here is the scenario I am thinking of. A business in the late Nineties decides it needs a website. It runs its business with a solid relational database and has no track record of handling electronic documents. Historically, all their public facing brochures, forms and so on have been outsourced to a printing company. These are now coming back in house for publication on the new website.
'Why not use the database we have already paid for, and have expertise in, to manage the content for the web site?' asks management. 'Why not indeed', replies a willing IT market, only too happy to help by bludgeoning HTML pages into database tables using BLOB fields. After all, to be managed properly, the HTML must be some sort of database, right?
Not always. In fact, looking back at my own experiences, I would say, not even 'most' of the time. Sure, there are times when this makes total sense but not always. In fact I would go so far as to say that in the engagements I have had with clients since the late Nineties, it has nearly always been difficult to find good technical reasons for the database-centric approach to web content management.
When content gets stuffed into a relational database table, a number of bad things can happen which turn a merely ironic situation into a problematic one. For starters, by funneling all content through a database, you create a single point of failure and a comprehensive Input/Output bottleneck. For sure, these problems can be solved with the aid of some more expenditure. You will find no shortage of solutions that involve expensive database failover trickery and perhaps some liquid helium cooled processing power or application clustering technology.
However, it is often worth asking the question - is all this stuff really necessary? How much of this stuff is required as a direct result of the decision to store the content in a database in the first place? What if the database disappeared in puff of logic? What then?
Well, you could store all the content on the file system (I can hear the shrieks of horror now!) - using the hierarchical directory structure provided for you by the file-system. Why not? You could use sticky HTTP sessions to load balance access to the content across multiple web services. This is trivial stuff these days. You could achieve very high reliability through the use of redundancy of commodity hardware and software such as Linux based Pentium class machines. The result? A website that screams along, is highly scaleable and reliable at probably a fraction of the annual maintenance cost of the database-centric alternative.
Something to think about perhaps.