By Ajay M. Pangarkar, CTDP, CPA-CMA, and Teresa Kirkwood CTDP, Founders, CentralKnowledge.com and LearningSourceonline.com
So, here we go again with another problematic methodology called “return on expectations” (ROE). Just when training professionals are in damage control with the disappointment of “training ROI,” out of the woodwork comes another “quick fix” and repackaged methodology trying to demonstrate training impact on business objectives. ROE, however, is a more elusive and misleading approach compared to others that came before.
For what it’s worth, we are in favor of any (credible) approach that provides training professionals an ability to tangibly demonstrate training effectiveness. What we are against is how many training experts try to repurpose obvious business approaches through disguised methodologies that further undermine training’s credibility. As formally educated business and accounting professionals, those who follow our work know our position on training ROI, but ROE is something that concerns us more.
To be fair, this article assesses the ROE methodology solely through the perspective of senior management and management accounting rules. We take this approach with respect to the ROE methodology since it states that training must meet key business stakeholder expectations. This article intends to determine ROE’s credibility as a tool to evaluate training and not mislead training professionals into believing it is repackaged common business sense (or what we refer to as common BS).
The ROE concept claims it is a foolproof way to show the “value of training” in the terms desired by key stakeholders. Regretfully, the concept fails to recognize that “training” is already considered a business activity and is always expected to demonstrate value like every other business activity. What key stakeholders want is simply common BS.
From a business professional’s perspective, ROE/ROI methodologies have holes like Swiss cheese. They do not build substantial credibility for training professionals and will not gain senior management acceptance. Below are seven critical flaws (there could be more) that undermine ROE’s validity as analyzed from the perspective of a senior-level stakeholder.
Concern #1: Inappropriate Use of Jargon
Like “training ROI,” ROE is just another attempt to throw around what we to refer to as false jargon. At first glance, ROE and training ROI appear valid and possibly credible. It is when you begin to scratch the surface that it becomes apparent both, and especially ROE, lack substance.
Using jargon and believing that ROE is something innovative is incorrect…ROE is actually repackaged common business sense (BS). In short, it is only common sense that every business activity, including training, must satisfy key stakeholders’ expectations all of the time or the activity is irrelevant to achieving the organization’s ultimate goal, its mission.
Another aspect of poor judgment and false jargon are the chosen acronyms and names. ROE is already a well-established financial term…it is “return on equity.” The poor usage of acronyms extends to training “ROI” and one called ROA (return on attendance). Not that we support it, but at least training ROI actually uses an actual return on investment calculation, even though the results do not make sense.
Using the ROE acronym with senior stakeholders, or any formally educated business professional, is more misleading to them than ROI itself (even though training ROI is misleading for many other reasons). The only and universally accepted use of ROE in any business and financial context is “return on equity.”
Some say that these experts create “similar” jargon on purpose in an attempt to sound relevant, but their intentions simply further mislead innocent training professionals. Proponents then will say, “Change the acronym.” The problem is only partly with the acronym; the theory, as you will read, is still weak and common BS. Sufficed to say, improperly using any type of jargon and misleading acronyms in business only further undermines credibility and make you look uninformed about business concerns.
Concern #2: ROE Superseding Needs Assessments
This is not as much a flaw but more an obvious point and common sense for trainers. The ROE concept believes the value of training begins before a program even starts. Trainers recognize the ROE statement as a needs assessment or skills gap analysis (the first step in the ADDIE approach). Every training designer knows that before even trying to suggest a training solution, you must recognize and identify the business gaps to determine if training is the answer.
Training professionals are not stupid. They know their involvement is to respond to, contribute to, or resolve a business need. To do this requires an effective needs assessments to identify the specific skills requirements and ensure that any proposed learning initiative is effective. So naturally, meeting business expectations is part of the training function.
Concern #3: Taking Credit for Established Practices
Proponents claim that ROE is the ultimate indicator of value. ROE fails to disclose that this “miracle” methodology is simply a 25-year-old formal business practice called performance management. ROE simply took credit and repackaged existing performance management business practices. If you don’t believe us, do a quick search for “performance management framework,” “Balanced Scorecard,” or “Kaplan and Norton,” and you quickly will discover ROE’s repackaged claims.
ROE proponents will say that our argument is a matter of semantics. Using “semantics” as a defense is a big stretch to make and somewhat simplistic. Formally business-educated professionals recognize that performance and change management is a highly complex process that requires a holistic approach in executing strategy. Experienced performance strategists view this as a balanced performance approach incorporating lagging (financial) and leading (non-financial) indicators within all business activities to achieve specific strategic goals (its mission and vision).
ROE’s basic claim is that as long you clarify how training contributes to the highest organizational results through leading indicators, then ROE meets stakeholder satisfaction. For performance strategists, skilled employees driving improved performance is only one (very important) component to ensure the organization meets its goals. Senior stakeholders, however, are only satisfied when performance improves all operational activities in the organizational value chain. This is common BS left out of the ROE process.
Concern #4: Believing Training Has Influence on Performance
ROE claims that the development of leading performance measures is a negotiated process implying that training professionals have input. This isn’t true. Simply put, after significant competitive research and close analysis of internal operational activities, senior management develops a mission (the strategic objective) and an accompanying strategy to achieve it. Senior management’s concern is to operationalize strategy to meet expected performance objectives and targets.
From a “training” perspective, there is little, if any, negotiation since management sets forth key performance objectives. The only possible negotiations are for the skills development costs (not investments)…and even then, budgets usually are set in advance for the year. Even the T&D department plans its performance objectives and financial allocations in advance according to organizational requirements.
So, the notion of “negotiation” is also common BS. You don’t agree? Then go to an operational unit manager and suggest negotiating any performance metric with them…but you were warned.
Concern #5: Asking Questions Already Answered
Another ROE claim is that learning professionals often need to convert the typically broad, un-quantified expectations into observable, measurable leading indicators by asking the question, What will success look like to you? The only correct part about this statement is the need for leading indicators.
The question about “what success looks like” closely resembles what the founders of performance management (Dr. Robert Kaplan and Dr. David Norton) state in the granddaddy of frameworks, the Balanced Scorecard. They ask several similar questions for each operational activity including learning.
Yes, it’s true that senior management desires tangible leading performance indicators. This is because there are sufficient lagging indicators (usually financially based) available. Non-financial leading indicators are the only performance measures that provide management with competitive intelligence. These leading measures, however, are already defined and targets set within your organization’s performance management process. Training’s role is to ensure employees within each operational activity possess the skills to attain the targets.
Concern #6: Not Recognizing Partnerships in the Value Chain
This is more of a common BS moment for ROE advocates. Yes, ROE correctly endorses the need to develop “business partnerships.” But again, this is something clearly stated in all management theories. Unlike the ROE theory, this is not a bias but only what is expected from every performance management process.
Business-school-educated managers know that “partnerships” (or the more relevant management term, “strategic alignment”) must exist throughout every organizational process. Harvard economist and strategist Michael E. Porter developed this widely accepted theory in the 1980s. The theory is called the organizational value chain, and it clearly illustrates alignment (partnerships) among business processes.
Until recently, T&D didn’t recognize this concept. The concept of “Learning” gained attention only to the credit of performance management founders Kaplan and Norton and KPMG (1989-1990) in the first performance framework called the Balanced Scorecard.
This performance framework is structured around the three core operational business activities and, to the surprise of many in the business, a fourth activity called “Learning & Growth.” This is when the thought of aligning human and knowledge capital with core operational activities arose…but here’s the issue: It is not to the credit of the ROE movement.
For senior managers, business alignment is all about what we refer to as Kirkpatrick’s Levels 3 and 4. This has always been the case. With respect to T&D, business managers don’t care about whether the training went well (Level 1) or if employees retained any of the learning (Level 2)…they never did. This is largely why T&D suffers from a lack of business credibility.
The one thing ROE gets right is to have training align with existing operational business objectives and targets and not to create independent metrics. But once again, this is common business sense in a performance management process. To build any type of business credibility, training must prove value demonstrating tangible results around core operational activities.
Concern #7: Reinventing the “Wheel”
ROE states that to create value at Level 3 and 4, training should develop a dashboard to monitor effectiveness. If you haven’t discovered it yet, your organization already has a performance dashboard in place…if you haven’t seen or used it, then you are not aligning to core operational needs (see Concern #6). If you think you are “aligned,” then you probably are getting lucky meeting performance needs or not addressing anything relevant at all, even though you may think you are.
Don’t reinvent the wheel creating a T&D dashboard. Use the performance tool in place and discover where “learning” fits. Trust us on this; “learning” is incorporated into your organization’s current performance framework. It may not be obvious since management does not view “training” as a significant business activity. Be sure to align with this performance dashboard.
Final Truth: Be Critical and Analytical, and Question What You Hear
We are passionate about helping T&D gain true credibility. But it troubles us that training professionals buy into the concept of manipulating accepted business principles to build credibility. The issue, however, is the misinformation disseminated by the next “latest and greatest” marketed training evaluation methodology. ROE is another example of repackaged common BS and existing performance management practices.
These types of theories further undermine training’s credibility and benefit only those who preach it. Do yourself a favor. Invest in focusing your professional development on business skills and financial literacy. You don’t have to become a business expert, but you do need to learn the fundamental business concepts to facilitate communication with your business leaders.
Read as many performance management and business books available by proven and qualified authors, including performance experts Kaplan and Norton and Paul Niven, along with business publications by authors Michael Porter, Prahalad and Hamel, and Treacy and Wiersema, to name only a few.
Lastly, be wary of misleading theories. Become a critical learner and question what you read and hear, including this article. Experts are either genuine or opportunistic. You are a learning professional, so be informed and knowledgeable. Challenge what is presented to resolve your training challenges and ensure substantiated results for all of your training initiatives. Beware and be cautious.
Ajay M. Pangarkar and Teresa Kirkwood are founders of CentralKnowledge.com and LearningSourceonline.com. They are performance management experts and three-time authors, most recently publishing “The Trainer’s Balanced Scorecard: A Complete Resource for Linking Learning to Organizational Strategy” (Wiley 2009). Read their blog, “Workplace Revolution” at blog.centralknowledge.com or contact ajayp@centralknowledge.com
