How WWII Mathematicians Can Improve Your L&D Strategy
In order to continually improve our Learning & Development (L&D) designs and deployments, we frequently look to successful participants, members of our team, or the industry to try to understand what learning strategies we can glean from their success. This is a strategy I often am asked to execute when creating coaching or sales training programs for clients. Certainly, there is a good deal to be learned from the most successful in any organization, but isolating them as the only source of “best practices” is a limited and potentially dangerous perspective. There are critical lessons to be learned from those who fail and those who may have moved on. It is sometimes difficult to recognize situations where Survivorship Bias affects our decision-making negatively.
I find the stories about very smart people using math, technology, and brainpower to defeat the enemy during World War II fascinating. One of the lesser-known stories that has offered valuable business insights for me is that of Abraham Wald.
Abraham Wald was a brilliant Hungarian mathematician, grandson of a Rabbi, and son of a kosher baker, who was forced to immigrate to the United States in the late 1930s as Nazi Germany’s influence made life very difficult. Wald was invited to Columbia University and fought his war as part of the Statistical Research Group (SRG) think tank. SRG was the mathematical equivalent of the Manhattan Project, but using statistics as opposed to explosions. The group of mathematicians solved some of the most important problems facing the Allied war effort: where guns should be placed on planes, fuel mixtures for optimal distances, torpedo barrages, bombing patterns, and numerous other problems that required mathematical expertise to be applied to real-life situations.
One of the most interesting dilemmas was where to place armor in order to protect bombers and at the same time reduce the weight of the plane in order to gain distance. The optimal amount of armor was relatively simple to work out, but the placement of that armor was the dilemma. The Air Force studied all the returning bombers to determine where to place the armor. The results were less than effective. The military turned over the data and the responsibility to the SRG and Wald. It seems the damage wasn’t evenly distributed throughout the planes; there were more bullet holes in the fuselage compared to the engine area. The military had incorrectly assumed that the armor (and its weight) was more effectively used on the fuselage. Wald’s approach was much different: Put the armour where the bullets aren’t. When he considered the bombers that did not make it back from bombing runs, Wald decided the solution was to armor the engines. The military never considered the bullet holes in the engine blocks they did not observe.
We may not have Abraham Wald and the big brains of SRG to help solve our learning and development challenges, but we can learn from Survivorship Bias. Modeling learning programs after your best employees only is a perfect example of how Survivorship Bias can be an issue.
If the planes that didn’t return are key to solving the problem for Abraham Wald, what planes should be considered in a Learning & Development strategy?
When we give feedback during a role-play or coaching session, we often start with what the participant did right. It’s not because we want to sugarcoat constructive feedback with something nice first. The feedback is important because the participant often does not know he or she has done something right. Success also can be a result of luck or good timing. A stupid decision in the moment that turns out well is often a brilliant strategy in hindsight. The more valuable insight from success is what not to do, and that requires a failure to work from. When we analyze why participants do not utilize the training they’ve received and you’ve invested in (the planes that did not return), we find that there are three main reasons:
Category 1: Participants are successful using their own methods and do not use the training.
Category 2: Participants try to use the training, experience failure, give up, and go back to previous methods.
Category 3: Participants use the training persistently, change, and are successful.
Research and industry experts tell us that 10 to 15 percent of learning content delivered in formal learning (classroom, e-learning, etc.) is retained successfully. This becomes a low return on investment and makes Category 3 the least effective of the three. The percentage between Category 1 and 2 may not be relevant and varies from program to program, in my experience. Consider the participants in these categories (the planes that did not return) as the key strategic ingredient in how to create and deliver Learning & Development successfully.
Before we explore how to extract this critical data, let’s look at why 85 percent of participants fall into Categories 1 and 2.
The Knowledge-Skill Gap
There are effectively two parts to learning a new skill. The first step is acquiring knowledge. Reading a book, attending a workshop, or participating in an e-learning program are common knowledge transfer vehicles for Learning & Development organizations. The second step is converting that knowledge into a skill. Most learning organizations subscribe to the Center for Creative Leadership’s (CCL) 70:20:10 ratio. 70 percent of learning takes place on the job through practice or experience. 20 percent takes place in informal learning situations such as observing others, and 10 percent takes place in formal learning (workshops, reading, e-learning), the knowledge transfer step. Interestingly enough, according to the Association for Training Development's 2014 State of the Industry report, 55 percent of L&D budgets are targeted at formal learning programs, the 10 percent.
Corporate Learning & Development departments are very good at delivering knowledge (the 10 percent) to a large number of participants in a variety of ways. The low retention and success rates in Category 1 and 2 are a direct result of a lack of investment in the 70 percent. The 70 percent is left to unfunded managers to “make the training stick.” Most managers are far too busy to place practice and coaching as their highest priority. Most managers are not trained L&D facilitators, and do not understand 100 percent of the content. In fact, many sat through the same workshop everyone else did, and, like everyone else, only retained at best 15 percent. Also consider that it is uncomfortable and unprofitable to practice new skills on clients.
Creating Safer Skies
Participants from Category 1 and 2 are the evidence (the missing planes) we are not considering during the learning and development strategy and design process. Where to place the armor, i.e., where to apply your budget for the greatest return, is CCL’s 70 percent.
The time to address retention is during design; the place to address it is in the field; the method to address it is practice. Creating safe and deliberate practice in the field is the solution to preventing Survivorship Bias from working against your objective.
- Create practice-based scenarios that break down the process into small steps.
- Repeat the process and increase skill and intensity.
- Use professional role-players to simulate reality and increase emotional intensity.
- Create trust by providing feedback and coaching throughout the practice.
- Measure a defined set of skills.
Bring More Learners Back from Bombing Runs
Step 5 of implementing a practice holds the key to extracting the data to where to place the armor. The professional role-player measures each skill during the role-play so you end up with relevant cohort data on each skill. Data from the lowest two or three performing skills then are fed back into the knowledge transfer stage (workshop, e-learning) to be reinforced. This feedback loop helps to avoid Survivorship Bias, and tells you where to put your armor.
The advantage to one-to-one scenario-based practice is that you have the opportunity to talk to every participant.
Abraham Wald and the big brains of SRG helped to win WWII in their own way; they fought with math and logic. Although it is not “life or death,” the battle for budget and resources in corporate learning and development is ours to win. We have a responsibility to our participants to arm them with the best training and practice, so they can achieve their objectives in the field.
Randy Sabourin is the co-president of e-roleplay Inc. and co-founder of Anderson Sabourin Consulting Inc. (ASCI). He helps organizations sustain learning and development investments using a combination of practice and behavioral awareness. His focus is on how individuals and teams perform under pressure. Sabourin has worked with companies such as Aon Hewitt, BMO, Unilever, Allstate, HP, Bank of America, ING Direct, SunTrust, eBay, and Biogen.