According to Training magazine’s Annual Training Industry Report, organizations spent $1,286 per employee on learning in 2019. Yet many companies are not happy with the return of investment they receive from training expenditures. Organizations aren’t attaining the ROI they are looking for because current courses are not driving the results they’re supposed to be achieving. One of the reason e-learning gurus cited at the Learning Solution Conference 2019 is not being able to identify the gaps and design the content that actually drives the performance.
I am sure there are many delivery methods being used, but let’s focus on one of the most common delivery methods for now: e-learning.
Are your e-learning courses impactful? Trainers know that many e-learning courses are designed for the sake of the learners taking them and so they show up as “completed” in the learning management system (LMS). But is there a way to design courses that can actually improve learners’ skills? It’s imperative to know how performance is measured in corporate sectors and how e-learning courses can help. Also, finding a common denominator between them can help instructional designers to analyze the gap.
The Existing Evaluation System Is Broken
The customer support department is one of those departments where you can see the rapid impact of your e-learning courses. It could be either scenario-based learning or microlearning. Unfortunately, the existing evaluation criteria are broken. Courses are connected with completion rather than skills. Key performance indicators (KPIs) can help you to identify the skills. But e-learning courses that are published via SCORM can only report “completion” as the end-of-course assessment score in the LMS. How do you make a connection between an e-learning course and an agent’s abilities rather than simply reporting course completion?
How to Solve the Mystery
The best solution is to design e-learning courses that follow skill-based learning so each course will help learners classify the skill gaps, along with recommendations on filling them. Depending on the nature of the business, find the best key performance indicators that will measure performance and identify key skills learners likely will need. Once the skills are identified, create a pre-assessment to measure employees’ abilities in those skill areas. Each ability must be tracked separately, which is impossible in SCORM courses. For that, xAPI plays a vital role to report scores as statements. Once gaps are identified, recommend courses you think will fill those gaps.
Scenario
For this article, I have designed a scenario using xAPI to demonstrate what skill-based evaluation looks like.
Let’s meet Bob. Bob is an instructional designer for ABC company. Recently, he found out that too many customer support agents lacked the necessary communication skills to do their jobs effectively. His manager, Sherin, gave him a task to design a pre-assessment module that will help agents measure the ability of each communication skill needed for the job, such as listening, body language, and nonverbal cues or writing skills with course recommendations. Bob’s organization has an in-house learning management system (LMS) integrated with SCORM Cloud.
Challenges
It’s obvious that skill-based learning is the best approach to overcoming the skill gaps of these customer support agents. But Bob ran into several challenges:
1. SCORM limitation:
The biggest limitation of SCORM is, in e-learning developers’ language, that it reports only one result slide per SCORM package. For this scenario, the mode of evaluation must be different. Instead of showing complete or incomplete, the system needed to report the scores of each ability with overall progress of a skill.
2. LMSs that support SCORM only:
Most LMSs support SCORM files only. In this case, Bob needed a reporting tool to track abilities of each skill.
Solution
Bob learned that all the in-house LMS reports were coming from SCORM Cloud, and SCORM Cloud is not only LMS but also a learning record store (LRS) that supports xAPI.
It was important for Bob to understand the fundamentals of xAPI before he moved forward. xAPI is governed by Advanced Distributed Learning (ADL) that gives the authorization to use for data usability. Some authoring tools such as Storyline and Captivate allow Bob to create xAPI statements by using Javascript and publish the e-learning courses as TINCAN xAPI instead of SCORM. It would be easier for Bob to upload the xAPI published e-learning courses in SCORM Cloud and send all the statements to the LRS.
Using the best authoring tool, Bob was able to create pre-assessments that could send results to the LRS via xAPI statements. These statements will help Bob and Sherin analyze the learner’s abilities in each skill and recommend courses. Once a learner takes those courses, Bob and Sherin can compare the skills of e-learning courses with KPIs to track the impact on the learner’s performance.
Results
Using xAPI had a significant impact in ABC company. Manager Sherin was able to analyze the agents’ skill gap with the help of skill-based assessments with course recommendations.
This is just the beginning. Pre-assessments will help to track learners’ existing skills. Courses or modules will be recommended depending on their scores. Once the course has been taken, there will be another round of assessments called post-assessments, which will show the effectiveness of the e-learning course.
Resources
Beer, M., Finnström, M. & Schrader, D. (October 2016). “Why Leadership Training Fails—and What to Do About It,” Harvard Business Review.
Mohammad Hassam has spent more than a decade in instructional design and implementing learning management systems in global organizations, including Citrix, Visa, Higher Colleges of Technologies, and Emirates Global Aluminum. He is skilled in collaborating with subject matter experts (SMEs) and stakeholders to find successful strategies and get desirable outcomes. Hassam has spoken about instructional design at conferences for several years, and recently presented at Harvard Medical University, where he talked about instructional design and gamification.
