Much has changed since the global recession. Budgets are tight, accountability is everywhere, and business results are expected routinely. All functions in an organization, including Learning & Development, face tremendous challenges to show value. The good news is that the Learning & Development community is responding with changes in their approach to evaluation. Here are seven metrics trends that are occurring globally and particularly in the U.S.
1. There is increased focus on the impact and return on investment (ROI) of major programs.
Although this trend has been evolving for years, the movement has been significant in the last five years—it is being driven by the recession itself. In 2008, many companies found themselves to be bloated, bureaucratic, and inefficient, and they began to trim their organizations during the recession, even if they were still financially strong. Senior executives and the chief financial officer are vowing to make sure their organizations are efficient, responsive, lean, and mean. This is causing major projects to be pushed to the impact and ROI analysis, showing a connection to the business and the financial ROI, the ultimate measure of success. In some cases, an ROI forecast is required before the program is designed, developed, or implemented.
Top executives crave impact and ROI data. A major study, supported by ASTD, showed that the No. 1 measure desired by CEOs from Learning & Development is business impact (“Measuring for Success: What CEOs Really Think About Learning Investments” by Jack J. Phillips and Patricia Pulliam Phillips; ASTD, 2010). With input from 96 Fortune 500 CEOs, this study revealed that the No. 2 measure is ROI. At the same time, these executives indicated that the current level of measurement is far from where they want it. Only 8 percent said that they see the business impact now, while 96 percent wanted to see it. For ROI, 4 percent see it now, and 74 percent want to see it in the future.
An important target for this level of analysis and accountability is soft skills, where it is more difficult for an executive to see the value. When the programs are important and expensive, executives especially want to see the value. In a study of 232 Global Leadership Development directors, 88 percent said there was an emphasis on ROI, and the No. 1 reason was the pressure for cost and efficiency (“Measuring Leadership Development: Quantify Your Program’s Impact and ROI on Organizational Performance” by Jack J. Phillips, Patricia Pulliam Phillips, and Rebecca L. Ray; McGraw Hill, 2012). This same study revealed that for leadership development, 34 percent of programs are measured at Level 3, Application; 21 percent measured at Level 4, Business Impact; and 11 percent at Level 5, ROI. These are the most ambitious numbers we’ve seen from leadership development.
2. The budget for measurement, evaluation, metrics, and analytics (MEMA) is increasing.
The Learning community has underinvested in measurement, evaluation, and metrics. Consequently, during the recession, many Learning leaders were not able to show the value of major programs and projects. Routine accountability at the level of evaluation sought by executives did not exist. Most organizations without a comprehensive approach to MEMA were spending approximately 1 percent of the budget on tools. Proactive Learning leaders are justifying additional expenditures by showing the value of current projects. Learning leaders are using the results to move to a best practice of 5 percent of the Learning budget to be spent on metrics and evaluation.
3. Responsibility for MEMA rests with all the team.
This trend has been shifting for some time, but it accelerated during the recession. Two decades ago, there was a move to centralize evaluation and have a core group of people with that responsibility. Although this appeared to be efficient, it often was ineffective. Every other member of the team— the designers, developers, facilitators, participants, and even managers of participants—would indicate that measurement and evaluation was not their responsibility, claiming the evaluation team should be doing this. Proactive Learning leaders have recognized that evaluation is everyone’s responsibility, and sharing the responsibility makes it much more effective. It reduces the resistance and keeps everyone accountable. Often, it is more efficient in terms of resources, because they all have their full-time job with part-time evaluation. Still, in large organizations, a small core group is available for very technical issues.
4. Finance, Accounting, and the CFO are more involved in L&D.
This trend has both good news and bad news, but is probably obvious to most Learning & Development functions. You don’t have to look far to see increased involvement of the Finance and Accounting departments, not only for Learning & Development but for other functions, as well. The CEO is pressuring the CFO to use the concept of ROI, which originally was developed to show the return on investing in capital expenditures (buildings, tools, and equipment). The concept now has moved to non-capital areas such as Human Resources, Marketing, Technology, and Quality. This has brought the CFO into the process as he or she implements ROI into these areas. According to Gartner research, many chief human resource officers (CHROs) are reporting to the CFO. Since most Learning & Development functions report to the CHRO, this brings the CFO into the reporting chain of command for some Learning functions. Proactive CLOs are stepping up to this challenge, making sure they have CEO- and CFO-friendly data, bringing Finance and Accounting into the process, and pursuing them as a colleague, not as an enemy.
5. Learning leaders are more proactive with impact/ ROI analysis.
Before the recession, many learning leaders would wait for the request to pursue a more rigorous analysis, particularly ROI. Unfortunately, the recession showed that approach to be disastrous. When the Learning & Development function is asked for this ultimate level of accountability and nothing has been put in place, it’s often too late. This places the Learning team on the defensive, with a short time line, and on the top executive agenda—not a good place to be. Proactive leaders learned their lesson and they are not waiting for the request. They are building capacity and experimenting with impact and ROI. They want to be driving the process, not reacting to it. They want to set the agenda, time line, and pace.
6. There are still barriers to impact/ROI use.
Although the concept of connecting learning to impact is all traceable to the early 1950s, and the use of ROI traces to the 1970s, the concept still does not enjoy the widespread use executives prefer. Some significant barriers must be overcome. Proactive Learning leaders are minimizing, diffusing, demystifying, removing, or going around those barriers. Here are the top five:
- Fear of results. As you can imagine, any program owner is nervous when someone is conducting an ROI study on his or her program and it’s negative. How will it affect me or my program, or my performance? Will it be discontinued, diminished, or not respected? While these thoughts are common, proactive leaders are managing the process. They use ROI as a tool for process improvement, not performance evaluation for the team.
- The perceived complexity of ROI use. Some proponents of ROI have created this fear by trying to develop complicated formulas. In reality, ROI is a ratio first encountered in fourth-grade mathematics.
- Perceived cost of an ROI study. Some think ROI costs too much, and they don’t have a budget or the time. In reality, costs are small. For a major program, the total cost of an ROI study is usually less than 1 percent of the program cost. It rarely goes over 5 percent, and that’s when a particular program is inexpensive. Proactive Learning leaders learn to manage this cost by building internal capability.
- They don’t know how to do it. This was a good defense 20 years ago, but not any more. Impact/ROI evaluation is now a part of the preparation for Learning & Development and Human Resource Development degree programs. ROI certification is offered globally, with more than 7,000 individuals having participated in the ROI certification offered by the ROI Institute.
- The client hasn’t asked for it. As mentioned in trend #5, this is a disaster. Proactive Learning leaders are reminding the team that we want to be in control of this issue.
7. Impact/ROI evaluations have many uses. Proactive Learning leaders are pursuing a more comprehensive measurement and evaluation system, linking Learning & Development to the business in credible ways and occasionally developing ROI studies for major programs. This proactive evaluation generates many great uses:
- Increase funding. Since the recession, this is the No. 1 reason for pursuing this issue.
- Satisfy executives in a request for accountability. Before the recession, the No. 1 reason for exploring impact and ROI was to meet a particular request from an executive group.
- Improve programs. This is the preferred reason for using impact and ROI and is the No. 1 reason advocated by the ROI Institute.
- Increase support. A key target for communicating results is participants’ managers. Showing the results at the impact level essentially ties the learning to the key performance indicators (KPIs) of these managers. That’s what’s needed for them to provide the level of support needed to make learning a useful process.
- Build business partnerships. To be effective, partnerships drive programs, funding, and new initiatives and make the process work smoothly. When executives see that L&D is making a contribution, they are more willing to be a viable business partner. Proactive Learning leaders are translating the business contribution of L&D to a successful business partnership.
- Improve client relationships. The ultimate client—the person who funds the program—has a much better image of learning and involvement when he or she sees learning as a valuable business contributor.
- Earn a seat at the table. For more than a decade, we have heard the comment that the Learning & Development leader (i.e., the CLO) should be involved in decision-making at appropriate high-level meetings. When a business contribution is clearly there, it’s much easier to earn and keep a seat at the table.
An expert on accountability, measurement, and evaluation, Dr. Jack J. Phillips provides consulting services for Fortune 500 companies and major global organizations. Dr. Phillips is chairman of the ROI Institute, Inc.; the author or editor of more than 50 books; and creator of the ROI Methodology, a process that provides bottom-line figures and accountability for all types of learning, performance improvement, Human Resources, technology, and public policy programs. For more information, call 205.678.8101 or e-mail jack@roiinstitute.net.
Patti P. Phillips, Ph.D., is president and CEO of the ROI Institute, Inc. She earned her doctoral degree in International Development and her Master’s Degree in Public and Private Management. While working for a large electric utility, she played an integral part in establishing Marketing University, a learning environment that supported the needs of new sales and marketing representatives. An accountability, measurement, and evaluation expert, Dr. Phillips is also the author and co-author of several books, including “The Bottom Line on ROI” (CEP Press, 2002), which won the 2003 ISPI Award of Excellence.
