Measuring learning’s effectiveness has always been a challenge for organizations, and adding new learning experiences can complicate it further. However, companies can have a relatively direct line of sight to see how effective their learning scenarios or blends are in achieving the goals.
Using the knowledge gained through measurement, learning blends can be adjusted over time to better meet the needs of the business. Across all of this are other influencers, including the audience, subject matter, and required timeframe. These will have an impact on the type of experiences and the blend.
By focusing on performance outcomes from the outset, the model helps organizations solve their measurement problems. So much of learning measurement has been inward-facing because there was no connection to performance. Now companies can map learning initiatives to individual and organizational outcomes.
Companies also struggle with measuring the effectiveness of new and emerging learning technologies and modalities. Within this framework, those new experiences have the same connection to performance as a traditional classroom experience. Because the needs of the business often change rapidly, no two will have the same strategy and approach.
Begin at the Beginning
Learning measurement does not begin when the program is finished. It must begin before the program is ever developed. Start with organizational goals and determine what individual behaviors and performance will help achieve those goals. Designing programs around a set of specific, defined metrics provides a roadmap for measurement. This process is laid out in Brandon Hall Group’s Learning and Performance Convergence Model.
In this model, organizations also are able to measure the specific learning outcomes to identify if the learning programs ultimately will have an impact on the organizational outcomes, without having to wait the months or years it may take to see if those goals are achieved.
Behavior Matters
It is critical that organizations get better at moving beyond Kirkpatrick Levels 1 and 2— even if they don’t ascribe to the Kirkpatrick model. It will always be necessary to measure things such as completion rates and learner satisfaction, but it can’t stop there. A 95 percent completion rate or a high Net Promoter Score tells the organization nothing. But if those data points are combined with outcome-based metrics, a real picture of learning’s impact comes into focus. For instance, are the 95 percent of learners who completed the course exhibiting the behaviors expected to a higher degree than the 5 percent who did not finish?
Measure to Make Improvements
Measurement is not just about demonstrating learning’s ROI or justifying its existence. We’ve seen that 77 percent of companies measure their learning to prove its effectiveness. Again, Learning leaders can’t expect to determine what is working and what is not from completion rates and test scores alone. Referring to the Convergence Model above, we can see that measuring some learning in real-time can help guide the design and development process.
By looking at elements such as how learners interact with the learning elements, where they are struggling, etc., learning can be deployed in a more iterative manner. Software developers have taken this approach for years, rolling out programs that are not 100 percent finished, knowing that they will need to make adjustments based on user feedback and results. Learning should operate the same way if it wants to become agile enough to keep up with the business.
The Role of Stakeholders
A big reason that learning traditionally only receives basic measurements is because the interaction with the learning is the only thing into which the Learning function has insights. They are creating these programs for the business, but they don’t know why it is needed or what the ultimate outcomes are supposed to be, so they default to Level 1 and Level 2 measurement.
By working with leaders and other stakeholders, Learning can get a much better idea of the metrics most important to the business. These partnerships not only contribute greatly to learning measurement efforts, they also give learning a more strategic role within the organization. Learning can leverage itself as an agent of strategic change because it is focused on a set of outcomes determined through collaboration.
Demonstrate Learning’s Role in Achieving Outcomes
To further establish Learning as a strategic partner, a solid measurement strategy gives Learning the ammunition it needs for that seat at the table. Coming into C-level meetings month after month with a series of completion rates and NPS scores tells leadership nothing about what learning is accomplishing. Learning leaders also will claim that there are too many other variables at play for Learning to claim any impact on outcomes. This is what keeps organizations looking at Learning strictly as a cost center.
A strong measurement framework means that Learning leaders can rightfully claim learning’s influence on successfully achieving goals. Of course, there are other variables, but following through on a mature measurement model allows learning to establish its role as one of those variables—if not the most influential one. In Brandon Hall Group’s research, high performers are far more likely to consistently use real outcomes as a measurement of their learning’s effectiveness.
To download a complimentary copy of Brandon Hall Group’s eBook, “Converting COVID-19 Crisis Management to Leading-Edge People Management,” click here.
David Wentworth is principal learning analyst at Brandon Hall Group. The firm’s vision is to inspire a better workplace experience, and its mission is to empower excellence in organizations around the world through its research and tools. Brandon Hall Group has five HCM practices and produces the Brandon Hall Group HCM Excellence Awards and the annual HCM Excellence Conference, in West Palm Beach, FL.
