Organizations put huge amounts of time, money, and energy into learning, yet real insight into the impact it has on the business remains elusive. Most do a good job tracking who completed what training, but that doesn’t tell the whole story or help determine learning ROI.
Brandon Hall Group’s 2018 Learning Measurement Study found that few organizations collect metrics that help link learning to organizational and individual performance. In fact, only about 8 percent of companies truly measure different types of learning with an eye on business results.
This lack of measurement effectiveness is evidenced by the study results, which show companies are not very good at measuring learning that’s not formal and rely far too heavily on traditional metrics such as completion rates and smile sheets. This shortsightedness keeps these companies from making learning a strategic influencer of the business. The study also finds that organizations shouldn’t be overly concerned with just the ROI of learning but rather learning’s impact on business outcomes.
Learning must demonstrate ROI to justify its existence. Companies want to know its business impact. But conventional wisdom suggests organizations demand metrics with results that are difficult to quantify.
Brandon Hall Group’s 2018 Learning Measurement Study found that most of the pressure comes not from the business, but from the Learning function itself.
Among executives and business leaders, internal, non-learning entities drive metrics at fewer than one-third of companies. Another 5 percent say pressure for metrics comes from outside the company, and 11 percent say they have no pressure to gather metrics at all (which is down from 13 percent in 2016). The rest—and the largest group by far—say the Learning function drives the quest for metrics.
This doesn’t mean executives and leaders are disinterested in measurement. But they are either not driving the conversation or not being asked to weigh in. To have an impact on the business, learning leaders must consult with business leaders to understand which areas of development are important and the metrics that will demonstrate learning success.
“Metrics of Success” Apply to All Learning Types and Modalities
Organizations are good at delivering formal learning because that’s where they predominantly focus. As such, they’re much better at measuring formal learning than informal and experiential learning.
More than half of the companies say they aren’t very good at measuring informal or experiential learning. In fact, nearly half (46 percent) say they are not effective at all measuring informal learning. This is one of the major challenges organizations face when deploying a blended learning strategy: They think they can’t measure anything outside of the course or classroom, so they avoid doing it.
Learning measurement doesn’t change with each delivery mode. There may be different data points but what’s measured remains the same: performance.
- Are learners acquiring knowledge better than before?
- Are they achieving optimal productivity more quickly?
- Is the need the learning was designed to remedy adequately addressed?
If the measurement strategy is solid, modality doesn’t matter.
Exacerbating the challenges of measuring informal and experiential learning is that nearly half of organizations do not have a strategy in place for measuring these types of learning.
Once the metrics of success are determined (with input from business leaders), any type of learning can be measured according to those values. Merely applying traditional metrics—participation, completion, and grades—to non-traditional learning doesn’t work. Look at the intended results, then measure accordingly.
As discussed previously, securing proper metrics comes from consulting business leaders. This also can help link learning to outcomes; defining the desired goals before learning begins. As for lack of time and/staff, companies must make hard choices in allocating resources. By dedicating resources to measurement, they ensure the time and effort put into creating and delivering learning is not wasted.
Focus on Impact, Not Investment
We repeatedly hear how learning is expected to determine the ROI of its programs. However, as noted earlier, most of the pressure to measure learning does not come from executives or business leaders, but from the learning function itself. Also, the majority of companies do not use business-based outcomes to measure learning, which would be a key part of determining ROI. More than half (55 percent) of companies say they do not consider revenue growth or profitability when measuring learning. Two-thirds say they do not look at revenue per employee either.
When we look at the overall drivers for measuring learning, ROI ranks low on the list.
Respondents could choose as many drivers as they wish, yet less than half said determining ROI is a main driver for measuring learning. Companies are far more focused on ensuring programs are effective and have an impact on performance.
More organizations should stop thinking in terms of ROI and start thinking of performance. Performance is the return and not something that immediately translates into dollars. If Learning leaders show the impact learning has on individual and organizational performance, it will go a long way to justify investing in learning programs.
This is another area where consultation with business leaders can be critical. Working together to determine what learning can do to help achieve specific outcomes can be translated into bottom-line results. Learning only needs to focus on those outcomes, and ROI inevitably will follow.
Conclusion
Why is there such lackluster measurement from a part of the business so hungry for data? Many reasons can be found in the results of this study, but the reality is that good measurement is difficult. It is why we see such a focus on completion rates and smile sheets. They’re easy to collect and understand, but do not really tell us anything meaningful. What does it matter if 95 percent of the people enrolled in a course completed it? There is no way to extrapolate the impact of that on business goals.
Another harsh truth is that the Learning function traditionally doesn’t have the in-house resources to make sense of the data. They rely on their technology providers for tools to measure learning. But without knowing what questions to ask, all the data and tools in the world won’t help.
This applies to measuring the things companies have been doing for decades: formalized courses and classes. For newer learning methods and modalities, most organizations are at a complete loss. Many find informal, experiential, mobile, or gamified learning more difficult to measure.
To begin to tackle this mountainous data problem, organizations must focus on two things: a measurement strategy based on outcomes regardless of modality and improved data analysis. The first is probably the hardest, but if the learning strategy aligns with the business, it becomes easier to align measurement with it, as well. Without input from the business, it can never align properly.
Organizations must look internally to decide if they need a dedicated data analyst, a shared analyst across the business, or external help. In any case, data cannot be ignored; it only continues to grow.
David Wentworth is principal Learning analyst at Brandon Hall Group, an independent research/analyst firm in the Human Capital Management market. The firm’s vision is to inspire a better workplace experience and its mission is to empower excellence in organizations around the world through its research and tools. Brandon Hall Group has five HCM practices and produces the Brandon Hall Group Excellence Awards; the Women in Leadership Summit, June 6-7 in Fort Lauderdale, FL; and the annual HCM Excellence Conference, held Jan. 22-25, 2019, in West Palm Beach, FL.
