Does training your employees ever really “work”? Why has it been so hard, for so long, to tell whether putting an employee through a training program will pay off? For decades, business leaders have been trying to connect their expensive training investments to tangible business results, yet few have come close to doing it in a clear, consistent, and convincing way. This means organizations in the U.S. alone are spending nearly $62 billion a year on training and literally end up with nothing to show for it. And even if the training is working, they have no way to show their CEOs and stakeholders how it’s working. In fact, the vast majority of organizations don’t even take the most fundamental steps to measure impact. That is, did it improve the knowledge, skills, and performance it was designed to improve? Are employees even a little more productive than they were before the training? Are they doing anything differently or better back on the job after the training?
So why is there such a huge lack of research and reporting on the impact of employee training? Why do even the most bottom-line-minded businesses (the ones that pinch and squeeze every penny to show profitable growth year over year) seem to be oddly accepting of and complacent about their inability to demonstrate an aggressive return on their training investments? What makes it so hard to measure? Is it the nature of the training itself? Is it the sheer volume and complexity of the training that’s out there? Is it the incredible diversity of the participants who go through the training? Well, the answer is simple and none of the above. After 12 years of conducting impact and ROI studies for some of the top Fortune 100 companies in the world, it became glaringly obvious to me that the biggest factor frustrating and almost always preventing conclusive and consistent results had little to do with the complexity of the training or the diversity of participants, and everything to do with the immediate work environments employees were returning to right after their training was done.
Let me explain. All employees take their new knowledge and skills from the training back to their own unique work environment. And depending on all the environmental influences in that work environment that affect these employees, they are either inspired to or discouraged from applying that training on the job. This enormous variability in post-training environments translates to an even greater variability in potential impact across the organization. That is, based on these critical environmental influences, the same training program can work wonders for some employees, while for others, it becomes a flop. And it’s this endless variety of post-training environments within any organization that has literally paralyzed most of our intentions and attempts to measure and quantify the impact of training.
Welcome to the Transfer Climate
To solve this problem, and put measurement at the forefront of any HR initiative, I began, and have continued for the last 12 years, to research and collect data on all these post-training factors that can “make” or “break” “the impact of training. I call this post-training environment the “transfer climate” because it represents all the factors that can either help or hinder the “transfer” of learning from the training experience back to the job. As with anything you try to seed, plant, or replant somewhere else, no matter how strong the leaves or how pretty the flowers, you still need just the right soil with just the right sun, and just the right amount of rain. So, too, with training. No matter how thoughtful and well-designed the training, you still need to make sure all the right factors are in place to care for and nurture it through that critical and precarious incubation period. If the climate is suitable for learning transfer, the training sticks, thrives on its own, and ultimately blossoms into fruitful returns. If the climate is not suitable for learning transfer, the training impact dies there and all your time and resources wither on the vine.
After almost 100 studies later, I came to one rather conspicuous conclusion: These climate factors were the clearest and most powerful predictors of business impact back on the job. Although things such as the quality of the training, the mode of delivery, and the nature of the audience were significant in most studies, none of these factors weighed as heavily and as directly on the outcomes of training as the post-training climates. Thus, you can’t measure the true effectiveness of any training program until you know what the climate is like. And you can’t really know what the climate is like until you know how to measure training effectiveness. For this reason, I’ve added what I call a Level 6 “transfer climate” to the traditional five levels of evaluation (see diagram below).
In a nutshell, the first 5 levels of training evaluation try to answer everything from “Were the employees satisfied with the experience?” to “Was it worth the investment?” The question I answer with Level 6 is: “What factors will maximize impact and ROI back on the job?” I do this by essentially asking trainees (on the surveys they would be taking anyway for the other levels) some targeted questions about key factors in their post-training environments (e.g., immediate leader support, reward structures, motivators, etc.). I then analyze and compare responses between those who ultimately showed incredible impact via training and those who showed none.
While the first 5 traditional levels are critical to any evaluation strategy, they don’t tell the most important and dynamic part of the story: Why does training work for Joe and not for Mary, and how do we maximize that impact across the organization. By answering these questions, you are not just presenting a report on how some piece of training worked in the past, but describing how you can improve that same piece of training and increase its impact in the future. Once you can identify the “right” climates for training impact, you then can expand and duplicate them for other trainees. That is, by making Mary’s post-training climate just a little closer to Joe’s, you can improve the business results and ROI across the organization.
Another huge benefit of this Level 6 is it allows HR and Training groups to push a lot of the accountability for training effectiveness off on the business. By demonstrating how “warmer” climates can affect impact, you are able to inform your business partners what they can do differently with their employees to drive and maximize their own training dollars. After all, they are the ones who can do the most to create and drive the right “transfer climates” for Mary and Joe back on the job. Now when training doesn’t work as well as they would like, they’ll be less likely to blame HR and those who put out the training, and more likely to ask what they can change internally to increase training impact. By showing them some examples where training did have significant impact in the organization, you are, in essence, taking the spotlight off the training itself and placing it directly on the training climates that they can influence. They’re usually the ones spending all the money—isn’t it only fair that we give them some control over the outcome?
Getting to the Next Level
In all my years of presenting Level 1 to 6 evaluation results, I have yet to encounter a business group that does not find the results about its transfer climate the most interesting and informative piece of the presentation. When I tell my audience I’m giving them Level 1 to 5 results for their training, of course, they’re excited—less than 4 percent of all companies are doing it, and even fewer do it right. But when I tell them I’m also giving them Level 6 results to explain why training succeeds or fails back on the job, that’s what gets them to the edge of their seats. Imagine being able to say to your business partners, “We understand this training was more effective for some employees than others. We know why it was more effective. And this is what you can do to make it more effective in the future.” Now what business leader wouldn’t be happy with that little tidbit?
Dr. Paul Leone is an I/O psychologist with more than 14 years of experience developing measurement strategies and evaluating the impact of employee development initiatives for top Fortune 100 companies around the world. He currently is working with Verizon as its Senior Learning and Development ROI Consultant to define the impact and ROI of its most critical training programs. Dr. Leone is a thought leader and innovator in the training industry and published a new book called Measuring and Maximizing Training Impact.
