By Nik Kinley and Shlomo Ben-Hur
If the discussion forums on networking sites such as LinkedIn are anything to go by, many businesses find choosing which signs of talent to measure confusing. The forums teem with requests for suggestions of which measures to use, and the answers provided often do not seem to make things any clearer. That there is some confusion is not surprising. There is no one sign of talent that you should always measure, and the ability of most measures to predict success remains frustratingly low. Yet, there are some clear guidelines that all businesses can follow that can not just make the task of knowing what to measure easier, but also can fundamentally improve firms’ ability to spot talent and predict success.
We will begin by presenting some guiding principles for deciding what to measure, and for choosing which combinations of factors will enable you to best predict success. We then will take a brief detour to check whether these guiding principles also apply to something that often is seen as very different from measuring someone’s ability to do a job: namely, measuring their long-term potential. Finally, we will look at what businesses can do to actually boost their ability to predict success, no matter what measures they use. The solution, as we will see, promises to fundamentally change both how we think about measurement and our ability to accurately identify and measure talent.
Three Principles for Deciding What to Measure
A basic challenge with talent measurement, then, is deciding what to measure. And in our experience, there are three guiding principles that can help organizations determine which measures to use:
- Measure what you need.
- Ask about validity.
- Ask about incremental validity.
Measure What You Need
The way companies typically approach deciding what to measure is through analyzing the skills and qualities that specific roles, teams, or business units need. This involves defining role requirements or what the company needs people to do. Sometimes this is achieved through a formal process such as a job analysis, and in some countries, having a structured job description is actually a legal requirement. Other times and in other countries, requirements are defined through more informal or intuitive means. Yet it is always there to some degree, even if it is just an idea in the hiring manager’s mind. There is a picture or list, then, of what you need and are looking for.
What is important is to make sure that this list is explicitly stated and that it clearly distinguishes the two or three things that are most critical for ensuring that people succeed. And then, wherever possible, to ensure that your choice of measures is led by this list of most critical qualities or competencies, by the type of talent the business needs. As principles go, it may sound obvious, but it is too often overlooked.
Ask About Validity
Sometimes, of course, there is only a vague idea of what a role or the business needs, and other times, the list of what is required is just too long. When this happens and there are, thus, no clear requirements for guiding the decision of what to measure, businesses tend to revert to what they know or feel familiar with. Yet this is a bad idea.
As we have seen, there is no one measure that you should always use: Different jobs require different qualities. So if you always use the same set of measures and tests for all roles, the chances are that sometimes they will not help you much, and they may even be misleading.
Instead of reverting to the familiar, then, when choosing what to measure, the one question all businesses always should ask is “How predictive of success is this factor in this particular type of role?” Or, in other words, “How valid is this measure?” As validity figures can tell you whether a measure predicts performance, checking validity is a way of checking that you are genuinely measuring what you need.
We have encountered many leaders who do not seem to be in the slightest bit interested in validity, which we frankly find amazing. If you are going to pay good money for measurement results, then you need to make sure the information they are providing is accurate and relevant. On a purely commercial level, anything else is just bad business.
Of course, validity is a technical subject and can be complicated. So in Chapter 5, we present a checklist of specific questions you can ask about the validity of measures to help you decide whether they really can help you predict success. For the moment, though, we just want to highlight the basic rule that you should always ask about validity. In other words, be led by the facts and science, not by traditions or familiarity.
Ask About Incremental Validity
Asking about validity can help you understand which measures are the most predictive of success. Yet, as we have noted, talent is made up of a mix of multiple qualities and abilities. So you need to use multiple measures and, to work out which combination is best, you need to ask a different question. Just asking about validities will not work. This is because when you ask how valid a particular test is, you are asking how good a measure it is on its own, separate from anything else.
To work out what the best combination is, you need to ask a different question: You need to ask about incremental validity. This is the amount of validity that one measure has over and above another one: how much additional information or validity it provides with whatever other measures you are using. For example, we know that personality-based integrity tests are nowhere near as good as intelligence measures when it comes to predicting success. On this basis, we might decide not to use them. But when we look at the incremental validity they offer over intelligence tests, it is a very different story. We find that they can add around 0.14 validity points to the 0.5 or so validity figure that intelligence measures give us (Ones, D.S., Dilchert, S., Viswesvaran, C., & Judge, T.A. (1993), “In Support of Personality Assessment in Organisational Settings,” Personnel Psychology; 60, 995-1027). So we now have total validities of 0.64, which are able to account for more than 40 percent of the causes of success. Likewise, personality tests appear to offer some decent incremental validity over intelligence with most roles, though precisely how much depends on the role and the tests used (Schmidt, F.L. & Hunter, J.E. (1998), “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Psychological Bulletin; 124(2), 262-274).
Excerpted with permission from the publisher, Wiley, from “Talent Intelligence: What You Need to Know to Identify and Measure Talent”by Nik Kinley and Shlomo Ben-Hur. Copyright © 2013.
Nik Kinley is a London-based independent consultant who has specialized in talent measurement and behavior change for more than 20 years. He was formerly the global head of assessment for the BP Group, head of learning for Barclays GRBF, and a senior consultant with YSC, the leading European assessment firm.
Shlomo Ben-Hur is an organizational psychologist and professor of leadership and organizational behavior at the IMD business school in Switzerland. He has more than years’ experience in senior executive positions, including vice president of leadership development and learning for the BP Group and chief learning officer for DaimlerChrysler Services.
