It seems simple enough—teach a course in person or maybe present it in self-paced online modules and then give a test to see if employees learned anything. However, gauging how much learners actually took away from a course is more challenging than that. Some Training Top 125 companies are rethinking learning measurements and finding ways to get a more accurate view of their training programs’ success.
GAIN EMPLOYEE PARTICIPATION
Measuring learning traditionally has been a top-down approach, with the teacher or subject matter expert putting together the assessment and delivering it to learners. ESL Federal Credit Union is adjusting that model by working with employees to set learning goals. Learning then is partly measured against the employee’s own expectations.
“ESL has begun using performance contracts for certain training classes. Before attending training, employees meet with their managers to set their expectations and goals for learning,” says Yvette Charleton, CPLP, training and performance specialist II. “Following training, they meet with their managers at the 30-, 60-, and 90-day mark to have conversations about how well their goals have been achieved. Managers forward the results of those discussions to Learning and Development.”
ESL then follows up after the course to find out from learners, as well as trainers, the knowledge and on-the-job help the curriculum provided. “To ensure greater participation, we must set the expectation for follow-up with employees and managers alike. As well, we must make it as convenient as possible,” Charleton says. “We have begun automating the surveys used for measuring application (Kirkpatrick Level 3). Employees and managers receive an electronic link to the survey, making it easy and convenient to complete.”
ENSURE ALIGNMENT WITH COURSE SPONSORS
Along with gaining participant buy-in, measuring learning comes down to meeting the expectations of the course’s sponsors. Western Union keeps in close contact with sponsors right from the start of its curriculum planning process. “At Western Union, our approach is to identify and confirm sponsors for each of our flagship programs/courses, and as a next step, we interview them to develop and confirm metrics for the respective programs that are most important to them for tracking the desired outcomes,” says Joshua Craver, vice president, Global Talent Management and Learning & Development. “We then have monthly meetings with the sponsors to review the progress against the agreed-upon plan.”
In addition, the Learning team at Western Union measures its training programs against overall corporate goals. “Over the last year, we have partnered with David Vance from Center for Talent Reporting to come up with a robust measurement methodology that has program reports that measure outcomes, efficiency, and effectiveness of each of our courses tied to one of our corporate goals,” says Craver. “This creates alignment and demonstrates the business impact of our development programs.”
KEEP IT REAL
Working through real-life business challenges in training courses turns those courses into workshops whose success can be easily measured. Shapiro Negotiations Institute (SNI) has participants do just that. “At SNI, we ask participants to bring real deals to the program with them, so we can apply learning to the deal throughout the program,” says SNI President Todd Lenhart. “Subsequent to the program, we can track progress on that specific deal and measure overall impact of the training.”
SNI then uses mobile technology to allow participants and managers to measure learning back on the job as deals are being worked through. “The use of technology has changed the historical measurement framework,” Lenhart notes. “We offer the SNI On-The-Go App for participants to access tools and habits ‘just in time’ after the training. This app also provides a platform to capture real-time feedback and best practices as participants apply the learning to their work. The precision of the measurement data utilized goes through the roof when we can engage the participant through the app.”
CREATE ACCOUNTABILITY
Companies need to ensure that measurements have taken place, and that the right skills are being assessed. “Rollins University has worked closely with our internal QA and Auditing Department to integrate Level 3 performance evaluations into their annual reviews of each location,” says David Brill, manager of Instructional Design at pest control company Orkin LLC, a subsidiary of Rollins, Inc. “For example, our new hire training program and several internal certifications require the location manager to observe each employee’s performance on the job, using a detailed checklist, which then must be placed in their personnel file. The internal auditors deduct points if these forms are not present. This system enables us to verify that behaviors introduced by Rollins University are being reinforced and transferred to the workplace.”
Orkin has found that learning rolled out globally at the company often can be assessed more meaningfully locally. “In the past, Rollins University would assign a mandatory course in the learning management system (LMS) with an accompanying quiz. However, we realized, for a course such as Respirator Fitness Training, this approach measured knowledge about respirators, but did not guarantee the student actually completed the Respirator Fit Test as required,” Brill explains. “Now, in place of a quiz, employees are assigned a mandatory resource in the LMS where they must download a form and use it to complete their annual Respirator Fit Test conducted by their location manager. The manager then certifies, in the LMS, that the employee completed the process. This allows Rollins University to drive the desired behavior and manage the compliance process.”
Brill says that, along with ensuring that courses have been completed and passed, the impact on learner behavior also must be focused on. “Curricula must be designed with performance objectives validated by executive leadership and tied to observable behaviors by employees already performing a desired skill on the job,” he says. “This empowers the Learning and Development department to demonstrate how it can help create the kind of top-line growth that drives recurring revenue.”
QUICK TIPS
- Get learners to set expectations for the course, which will be used partly to measure their success.
- Meet with program sponsors and plan to meet agreed-on business goals.
- Ask learners to bring real business challenges to the course. Success then is determined by the real-life outcome to those challenges.
- Use mobile technology to get real-time feedback from learners on how helpful the course was— based on questions that arise on the job.
- Globally roll out a program, but locally assess its effe ctiv e ness, allowing local managers or line-of-business leaders to assess and decide whether the global training worked at the local level.
