Managing to learn

There’s a lot to be said about learning management systems (LMS), and not all of it is good. First and foremost, it’s still early days, so it is a technology and an industry sector that is still in transition.

Alright, the hype machine has jumped into top gear over the last few years, with predictions of the “major ERP and technology vendors jumping onto the e-learning gold-rush” and statements that “LMSs are a critical part of an e-learning deployment”.

But statements indicating a headlong rush almost inevitably go hand-in-hand with a major hangover when the rush has gone. Mention dotcom and you know what I mean. Mark Fenna-Roberts, managing director of training company ITC Learning, says the technology “was the subject of a lot of Wall Street investment hype”. But the market has cooled, and the sector, he says, has been “hit hard and a lot of suppliers are struggling, out of business or consolidating”.

LMS is not necessarily the answer to your training dreams, he says, but like all investments, it warrants some cool-headed analysis and assessment, and a hard look at the business requirements. What is LMS?

A learning management system undertakes and automates a range of tasks associated with training operations largely, but not exclusively, e-learning.

An LMS will manage:

  • Aggregation of learning content
  • Distribution of learning content
  • Compliance and skills auditing of staff
  • Skills database and job profiling
  • Reporting on the state of play re training, learning, skill sets, compliance and non-compliance.

Some of these functions can be done by simpler and less expensive applications, but the totality and thus the “value-add” requires an LMS.

As Fenna-Roberts says, “People who want to distribute learning across the network don’t need an LMS.” Those functions can be handled by a content management system (CMS) or, confusingly, a learning content management system (LCMS).

Bryan Chapman and Brandon Hall, e-learning gurus in the US, define the difference: “Learning management systems and learning content management systems really have two very different functions. It’s unfortunate that both have such similar names and a shared acronym, which only serves to confuse e-learning buyers even more.

“The primary objective of an LMS is to manage learners, keeping track of their progress and performance across all types of training activities. By contrast, a learning content management system — LCMS — manages content or learning objects that are served up to the right student at the right time.

“Understanding the difference can be very confusing because most of the LCMS systems also have built-in LMS functionality. In fact, 81 per cent of the systems in [a Brandon-Hall.com report] include LMS functionality as part of their system.”

In other words, LMS manages the students (which is why it’s sometimes called a “learner management system”), LCMS manages what the students study. For instance, you normally can’t create content via an LMS, while you can’t audit student skills via an LCMS. Fenna-Roberts continues: “Once you start mixing the elements [of content, such as live training, workshops and e-learning] it is useful to track the competence of individual staff members, and that’s when an LMS comes into its own.”

Tania Wickman, business development manager of WebRaven, an Australian company which produces the DOTS LMS, says the Australian market is not particularly crowded with suppliers. “There are about 10 players in the CMS/entry level LMS, some of which are Australian-based, and there are about 10 players in the LMS, eight of which are US-based.”

Contrasting with the limited number of suppliers, the “market place is busy”, she says, with particular interest coming from compliance requirements, government policy training, financial services, and occupational health and safety.

Gartner agrees that demand for LMS has increased globally, but adds that that is accompanied by an increased number of players, including some of the “powerhouse” and ERP heavyweights.

Fenna-Roberts agrees that compliance is the strongest driver. “You’re really buying auditability, and getting a lot more than that in the bargain.”

But the nirvana of LMS, as Fenna-Roberts puts it, is the ultimate evaluation and ROI, the effectiveness of training. He suggests following the method developed by Professor Donald Kirkpatrick of the University of Wisconsin, which sees four levels of return on learning activities:

• Level 1 — Reaction: how much did the trainees like the program? This is generally accomplished with the end-of-course smiley-sheet evaluation form (but it could just mean they had a good lunch and didn’t learn that much).

• Level 2 — Learning: basically, skills transfer. What principles, facts, and concepts were learnt in the training program? In the teaching of skills, classroom demonstration by trainees is a fairly objective way to determine how much learning is occurring. Where principles and facts are being taught, paper-and-pencil tests can be used.

• Level 3 — Behaviour: did the job behaviour of the trainees change because of the program? This can also be applied to teams and enterprises. This is often tested three to six months after a course is completed, but it can be very difficult to assess.

• Level 4 — Results: ROI. What were the results of the program in terms of factors such as reduced costs or reduction in turnover? Success here is often dependent on Level 3 results.

Is LMS widely used?

There is a view that LMS is an expensive undertaking, and certainly depending on the system you purchase you can be up for anything from tens of thousands of dollars up into the million-plus. One commentator said that if you can get away with top-end systems for under $500,000 you’re doing well. Consulting costs can be a considerable element in the final cost of ownership.

Brandon Hall adds a word of comfort, though: “A slower economy has diminished capital spending in many organisations, large and small. In addition, workforces have been trimmed, thus reducing the number of registered users within a company’s LMS. For these and other reasons, vendors have increasingly been hearing the call of customers who need lower-cost alternatives.”

Of course, you could always build your own. Hall says “the use of ‘proprietary systems’ is still #1 in market share when compared with even the most popular LMS solutions. Dismayed by the high cost of systems, your organisation may be considering the same strategy. On average, it will take between $US200,000 to $500,000 and about 18 months to build an LMS with moderate functionality.”

Whatever the route, Gartner predicts that by 2005, 70 per cent of large companies will own learning management system applications.

Meta Group recently issued a report on the market, suggesting that “During 2003, organisations will marginally increase LMS investments, driven by business initiatives. By 2005, progressive enterprises will extend learning management frameworks to include content, collaboration and professional services. Through 2007, maturing standards will enable greater interoperability (exchange of learning objects) and more flexible delivery models for internal and external use.”

In many cases, purchase drivers and day-to-day managers are HR departments. Wickman says that while the IT department is responsible technically, particularly for infrastructure compliance and because LMSs are Web-based, largely it’s not a technical decision, and requires little skill to manage on a mundane basis.

But it’s still early days. A survey of manufacturing companies by Peter Boulton of Monash University indicates that for most organisations (76 per cent), less than a quarter of their training is via e-learning mediums, and 58 per cent of respondents said that management support for e-learning was “moderate”. Less than a fifth of e-learning users were also using Web-based LMS.

Decision criteria

And that’s what it comes down to. Not should I use an LMS or an LCMS, or even e-learning as a medium, but what should I be offering in terms of training, and then, and only then, what mediums should I use.

Fenna-Roberts says that “The difficulty a lot of Australian organisations have is, they thought it would be the answer to their prayers regarding monitoring and reporting on performance. A lot of people couldn’t justify the expense; they hadn’t thought through the business requirements. The US has a large-organisation view. It’s different for smaller-sized Australian organisations. Most of the systems available in Australia have been built to supply business requirements that don’t apply here; there are different nuances in the local market.”

Kathleen Norman, Asia Pacific marketing manager for NETg, says, “One of the biggest, most common headaches in e-learning implementation is achieving interoperability between content and the LMS.

“The LMS must function seamlessly with e-learning content or it simply won’t work. Users will become disillusioned and support for the e-learning program will suffer.

“While a variety of standards and guidelines have been established for interoperability, such as AICC and SCORM (sharable content object reference model), these can sometimes restrict content functionality. The best implementations are those that work hard to achieve full interoperability.”

Norman stresses a phased approach to implementation, something which is supported by Meta: “Organisations have moved away from HR-centric enterprise learning projects and are instead focused on smaller, leaner LMS deployments that require less investment and deployment time as a means to support specific business initiatives requiring both training and knowledge. This is due to the cost of technology and change management requirements.”

Meta also recommends keeping a weather eye on the vendor scene. “Due to the immaturity of the market, we recommend that users proceed carefully with LMS platform vendor selection, considering leaders and challengers and vigilantly weighing vendor strengths and weaknesses against objectives in this rapidly evolving market.

“Organisations should select LMS vendors based on their ability to support discrete business initiatives while recognising the long-term need for broad, foundation-level e-learning services.

“IT groups involved in e-learning projects should assess new entrants as well as ongoing merger-and-acquisition activity within the market to maximise investments in technology, improve negotiating position, and reduce technology complexity.”

Finally, it comes down to basic business sense. “You need to start by having a system,” Fenna-Roberts says. “Not a software package-type system, but a systematic approach to assessing performance. You need an approach that works for your enterprise, which means alignment with business plans, vision, and the like.”

Once you have a systematic approach to assessing results, he suggests, then you can look at training.

Sound advice, and something that applies to all investments, learning or otherwise.

Join the newsletter!

Error: Please check your email address.

More about CMSGartnerITCMeta GroupMonash UniversityMonash UniversityNETGNormanTotalityWall StreetWebRaven

Show Comments

Market Place