Learning management system (LMS) solutions are underperforming in the eyes of the organizations that use them, according to an annual LMS Trends Survey conducted by research/analyst firm Brandon Hall Group.
Less than half of the 457 survey respondents (45.5 percent) give their LMS solution high marks in terms of overall value for the price they pay. Additionally, 47.7 percent of respondents are looking to leave their current LMS platform and move to a new provider. That number is virtually unchanged from the previous survey, reported in January 2013, in which 48.2 percent of respondents said they were seeking a new LMS provider.
Among LMS clients, those that deploy an SaaS, or cloud-based, LMS generally are more satisfied with their systems than those that use an installed, or on-premise, solution behind their firewall, according to the survey. Read more detailed results of the deployment portion of the study here.
There are many reasons that drive an organization to make a wholesale change of its LMS. Some of the top reasons identified in this study include:
- Poor reporting features
- Difficult to use
- Poor customer support
- Out-of-date appearance
- Inability to adapt to changing needs
- Lack of mobile features
In a nutshell, far too many organizations feel they are paying too much for systems that are difficult to use, out of date, and do not provide the data and analytics the companies need.
LMS buyers in this survey are clear on what they are looking for in a new system, and priorities have shifted somewhat from the previous study. This year, ease of use is the No. 1 new requirement; in the previous study, robust reporting features topped the list, while ease of use was third. The advent of big data and the growth in importance of analytics also has driven organizations to look for in-depth, full-function reporting features as a top priority.
Companies also are looking for expanded talent management services from their LMS vendors. As an example, 41.5 percent of organizations believe it is highly or very highly important that their next LMS provider offers performance management functions, as well.
Dollars and Sense
All of this desire for change has left many organizations wondering what they are paying for. The study found that organizations that spend more of their learning budget on their LMS and spend more dollars per learner actually report lower satisfaction rates.
Among companies that spend 5 percent or less of their learning budget on an LMS, 67.6 percent give their provider high marks (4 or 5 on a 5-point scale) for overall value for price. For companies that use more than 5 percent of the learning budget for their LMS, 41.8 percent give their provider high marks.
Among those paying less than $5 per learner for their LMS, 59.3 percent rate their providers highly (4 or 5 on a 5-point scale) on overall value for price. That number drops to 41.5 percent for those spending more than $20 per learner.
LMS Satisfaction Levels
In an effort to get a complete picture of how organizations view their LMS solutions, the survey asked several questions around what the companies like and don’t like about their systems, whether or not they want to make a switch, and if they do want to make a switch, what factors prompted them to do so. We also asked organizations to rate their current system across several different metrics:
- Feature set
- Ease of use
- Meeting current needs
- System reliability
- Overall value for price
The only category in which more than half of organizations give their LMS a high score is for system reliability. In a world of cheap storage, redundant servers, and massive bandwidth, it is not surprising that most companies are satisfied with the reliability of solutions boasting 99 percent uptime.
Unfortunately, we no longer live in an environment where simply making something work is enough. Companies are demanding that their solutions be efficient, intuitive, and robust. The lowest satisfaction ratings were given to the available feature set in the systems, with only 39 percent rating giving a high rating.
This doesn’t necessarily mean that organizations want more features—in fact we often hear that systems have too many unnecessary features. Companies instead are looking for the right feature set for their needs. This is clear from the low scores for the systems’ ability to meet the current needs of the client organizations (43 percent). Evaluations for ease of use also remain low, with 34 percent of respondents rating it either a 1 or a 2 on the 5-point scale.
LMS solutions also get an average rating as far as “bang for the buck.” Some 31 percent gave their solution a 3 for overall value for price, and 45 percent gave it a 4 or a 5. Combine this with the low satisfaction scores for ease of use, feature set, and meeting current needs, and it paints a picture of LMS buyers with very high expectations that aren’t quite being met.
Installed vs. SaaS Deployment
However dissatisfied LMS users may be, the survey found that organizations that deploy a SaaS-based LMS generally were more satisfied with their solution than were organizations that used an installed, or on-premise, LMS behind their firewall. SaaS LMS clients also reported having fewer administrators, and needing less technical expertise and IT involvement when it comes to managing their LMS.
Overall, 54 percent of LMS platforms currently are deployed in the cloud, the survey found. Given the ubiquity and popularity of cloud-based software such as Salesforce and the championing of SaaS delivery by LMS providers, this number seems low. In fact, it’s a negligible increase from the previous survey a year earlier, when the number was 50 percent. One of the often-touted advantages of installed solutions is the reliability and stability of having the system behind the firewall. But survey respondents rated system reliability about even between the two: Some 60 percent of SaaS clients are highly or very highly satisfied compared with 58 percent of installed clients.
The separation between the two types of deployment becomes clearer when looking at ease of use, where SaaS clients are 60 percent more likely than installed clients to be highly or very highly satisfied.
When it comes to overall value, 18 percent more SaaS clients than installed clients are highly or very highly satisfied. In a nutshell, SaaS clients think they are getting more bang for their buck than installed clients.
When it comes to challenges with the LMS, both installed and SaaS clients often deal with the same issues, such as ease of use or poor reporting features. However, SaaS clients are far more likely to have customer support complaints, while installed clients are plagued by the inability of their system to adapt to their current needs.
Other Survey Highlights
Company size influences deployment choice. The biggest influencer of deployment choice is company size. Smaller companies may not have the infrastructure or in-house expertise that would allow them to run an LMS internally. In these cases, SaaS solutions become much more attractive by putting those support burdens onto the provider’s shoulders.
LMS implementation. Both SaaS and installed clients use a significant amount of internal resources for LMS implementation. But as you might expect, the primary responsibility differs significantly:
- Organizations utilizing an installed solution are 30 percent more likely than SaaS users to have an internal team primarily responsible for implementation.
- Organizations with SaaS solutions are about three times more likely than installed users to have the LMS vendor primarily responsible for implementation.
LMS administrator headcount. The need for internal resources continues beyond implementation regardless of deployment. Once the system is up and running, it requires administrators to keep things going. How many administrators often depends heavily on company size and the number of learners being served, but the complexity and type of deployment play roles, as well.
About the Research
The online survey was conducted in the fourth quarter of 2013 and garnered a total of 457 responses from a variety of industries and company sizes. Respondents were almost evenly split between small, mid-size, and large organizations. Approximately 63 percent of responses came from the United States. Approximately 18 percent of responses came from Australia, India, Canada, and the United Kingdom, with the remainder spread across 25 countries.
David Wentworth is senior learning analyst for Brandon Hall Group, a research/analyst firm with practices in Learning and Development, Talent Management, Leadership Development, Talent Acquisition, and Human Resources.
