Each year, Training magazine requires all Training Top 10 Hall of Famers to submit an Outstanding Training Initiative that is judged by each other and shared with our readers. Aside from ensuring Hall of Famers aren’t “resting on their laurels,” this provides an opportunity for the Learning & Development community to learn from the “best of the best” and see some innovative solutions for challenges many face today.
Each Hall of Fame Outstanding Training Initiative submission could achieve a maximum of 20 points (half-point increments can be awarded) as follows:
- Level of potential business impact (i.e., revenue generation, new product launch, change initiative, new technology launch): 0-3 points
- Level of difficulty of challenges faced: 0-2 points
- Project scope (companywide, individual functions, global vs. national, etc.): 0-3 points
- Instructional design (learning objectives linked to business outcomes; level of leadership involved in design, development, and facilitation; reinforcement): 0-4 points
- Innovation of training: 0-4 points
- Business outcomes achieved/expectations met: 0-4 points
The three initiatives that achieved the highest scores are detailed below (Verizon, Booz Allen Hamilton, and Deloitte). The other 13 submissions will be profiled in the four remaining issues for 2019.
Verizon: Verizon’s Emerging Use of AR and VR Technologies for Learning
Verizon’s Learning and Development (L&D) team was eager to harness the power of augmented reality (AR) and virtual reality (VR) technologies for learning. To get started, in December 2017, the team leveraged a design-thinking ideation hackathon complete with a Shark Tank-like panel of L&D senior leaders who determined what projects would be funded. The competition was a multi-day event held virtually via telepresence with L&D team members in physical office hubs. Some 50-plus L&D staff and business partners participated, resulting in the top four most viable projects to be funded for implementation in 2018. The projects that were greenlighted had the best business impact/ROI forecast and the most compelling use cases for AR and VR modalities.
Two of the four projects are:
- SAFE-VR: Retail Store Robbery training experience using VR for 1,500 retail store managers and assistant store managers via the pilot in 2018. Ultimately, all Retail employees and leaders will be trained in fourth quarter 2018 through 2019.
- Digital Apprentice-AR: Live and online performance support via voice-activated AR glasses for 53 field technicians via the pilot in 2018. Ultimately, Verizon’s field technicians, network engineers, and dispatch managers will be trained from late 2018 through 2019.
Program Details
SAFE-VR: Verizon’s L&D, Security, and HRBP teams partnered with VR vendor STRIVR on the identification of the three most critical robbery scenarios plaguing Verizon stores: store opening, “smash-n-grab,” and store closing. Together, they created robbery scripts, chose a Verizon retail store location for the video shoot, and selected actors. Each scenario was based on real-life Verizon robberies and the choices (good and bad) that Verizon store leaders have made when robbed. For each scenario, learners have to select choices during the VR experience. Each choice has specific feedback created so that no matter which choice a learner selects, he or she will learn from that selection. Each scenario is limited to 10 minutes to minimize motion discomfort.
Pilot participants completed an instructor-led training on the procedures/protocol for handling store robberies in a retail training store (a working replica of a Verizon retail store specifically used for store leader training). Once completed, participants entered a room filled with nothing but wheeled chairs and were given Oculus Go headsets and instructed by their trainer how to wear/use them, what to do if nauseated, etc. (89 percent of participants reported no discomfort during the pilot). The trainer also set up the purpose of the VR scenario, then the participants completed the first VR scenario. Once all completed scenario one, the trainer debriefed the experience, the “right/wrong” choices, and each key learning. Participants then completed the second scenario, debriefed, and then completed the final scenario and debriefed it, as well as the full takeaways having now “experienced” three different robberies.
Digital Apprentice-AR: Existing installation and troubleshooting online performance support content that currently is accessed via a mobile tablet was repurposed and edited for display via voice-activated AR glasses (content displays on the top right-hand side when looking through AR glasses). The AR glasses also were enabled so the technician can make an outbound call to a Tech Support subject matter expert (SME) and use “See What I See” live-streaming video capability by focusing a camera in the glasses on what he or she is seeing so the SME can see it, too, and offer live troubleshooting support and coaching.
During the pilot, a group of existing field technicians and a group of new hire field technicians were given the AR glasses to use for online performance support and “live” technical support assistance when needed. Performance support/technical support is needed when working in environments such as up on telephone poles, down in man holes, up on ladders, etc. The AR glasses provide these employees an augmented overlay of content in front of the equipment with a hands-free ability to call a SME coach, who can give an instant live-streaming, real-time video of what the tech/engineer is seeing. The coach then can provide instant feedback and instructions.
Results
SAFE-VR: Learners reported:
- 99 percent felt “highly engaged” or “totally captivated” during the VR training experience.
- 97 percent felt more prepared to handle a store robbery.
- 95 percent had a better understanding of concerns around robbery situations.
Verizon observed a statistically high positive correlation between those who selected the most correct choices and their observed behaviors throughout the VR training and vice versa— it’s likely that those with higher performance during the VR training are more prepared for robbery situations, while the lower performers are not. This data is enabling L&D to be surgical with reinforcement (i.e., providing more VR scenarios in the store locations that are more prone to robberies).
Digital Apprentice-AR: Learners reported the AR solution:
- Helped reduce errors by 77 percent
- Helped complete tasks 86 percent faster
- Grew confidence on the job by 77 percent
Pilot groups were able to use “See What I See” remote technical support vs. calling managers for support, freeing up additional resources to work on other jobs. Verizon estimates a multi-million-dollar potential value with full implementation.
Booz Allen Hamilton: Booz Allen Launch
Two years ago, Booz Allen embarked on a mission to reimagine its onboarding program with a focus on increasing employee engagement from Day One. The content would connect new hires to the firm’s Employee Value Proposition (EVP), values, and culture in a meaningful way. To do this, the initiative needed a scope that went beyond the Learning and Development (L&D) team’s ownership of the Day One and Day Two classroom experience. Instead, the new initiative would integrate touchpoints within the business—from receipt of the initial offer letter through the end of the new hire’s first year at the firm. This expanded scope would require developing Week One orientation; providing learning opportunities throughout the first year with the firm; and redefining processes, program governance structure, and all roles that interface with new hires.
Booz Allen’s objectives for this reimagined program were to:
- Create a more engaged and connected workforce through meaningful networks
- Deepen employees’ affiliation with the firm
- Reduce attrition rates in Year One and Year Two
- Prepare news hires to assimilate and contribute to their first assignment quickly
Content Creation
L&D conducted a comprehensive learning research study and developed learner personas to inform program design, with a focus on developing an engaging employee experience that empowers personalized support at the time of need. A synthesis of regrettable loss exit interviews helped designers align the program with the firm’s purpose and values and EVP, creating connection activities that begin on Day One during an in-person experience. Focus groups and stakeholder interviews helped clarify roles and responsibilities and streamlined pre-onboarding processes in Workday, the firm’s talent management system. These focus groups and interviews also informed a balanced delivery of cultural and tactical content, including foundational learning in the areas of consulting skills, career development, and mobility.
Finally, Booz Allen conducted a firmwide Innovation Challenge that informed its approach to integrating technology at targeted points during the year-long experience. L&D leveraged technology to create immersive, simulated day-in-the-life experiences; gaming activities to solidify knowledge of the firm’s history; and virtual reality to improve job readiness.
Program Details
After accepting an offer, but prior to orientation, Booz Allen new hires are introduced to the firm’s EVP through a virtual reality experience that is accessible by downloading a mobile app to a personal device. The onboarding journey begins with a two-day, in-person, instructor-led orientation where new hires meet other newcomers at all levels and stages of their careers and participate in day-in-the-life simulated experiences that immerse new hires in Booz Allen’s culture and way of working.
The journey continues with a series of follow-up trainings that help new hires develop foundational consulting skills and build their networks. These trainings are delivered through online courses and Webinars employees can complete at their own pace from virtually anywhere via computer, tablet, or mobile app.
During their first year, new hires also receive a New Hire Buddy, an informal mentor who welcomes them and helps them learn about the culture and identify the resources they need to succeed. In addition, L&D developed job aids for hiring teams and New Hire Buddies to create consistency in post-orientation onboarding activities at local office or client sites. These include print literature, playbooks, and templates accessible from a central onboarding Website on the firm’s intranet. Finally, a condensed version of the orientation program is offered virtually, by exception, for those employees who are unable to attend in person. This version of the program can be taken through the learning management system (LMS) on a personal device.
Results
- Booz Allen’s new hire experience has improved based on an overall increase in Net Promoter Score (NPS) from 51 to 60 in the last quarter, a full 10 points above the industry NPS benchmark.
- New hires are more prepared, with all pre-Day One tasks completed in Workday and I-9 completion rates increasing by 10 percent year over year.
- New hire Help Desk tickets have decreased by 21 percent quarter over quarter.
- In the first three-month, new hire focus group, 100 percent of the participants stated they are “very proud” to work for Booz Allen.
- Onboarding’s Year One follow-up training completions have increased by 33 percent quarter over quarter.
- Booz Allen’s average operating cost per hire is 29 percent below plan.
Deloitte LP: Flash Learning & Information Platform (FLIP) Mobile App
Deloitte’s Federal Practice must meet a number of compliance requirements related to doing business with the Federal government—dictating that its professionals successfully complete multiple one-time and annual compliance training courses. This has required a significant investment of time for Federal practitioners—time otherwise spent serving clients—as they must complete several of these courses through their entirety every year, and pass the end-of-course assessments.
Deloitte’s Federal Talent Development team sought a learning solution that would allow the firm to continue to demonstrate compliance with these requirements, but through a platform that could “meet the learners where they are,” with customizable content they could easily absorb, retain, and continuously reference at the point of need.
The final solution consisted of a mobile-based “flashcard” learning experience called the FLIP (Flash Learning & Information Platform) app. It allows learners to quickly consume content at a pace governed simply and explicitly by gaps in knowledge areas, integrating assessments with each portion of content.
Program Details
Upon completion of the modules, learners are able to apply the knowledge, skills, and behaviors needed to comply with Federal compliance laws and regulations, including policies and procedures around the following:
- Time and expense reporting
- Ethics and proper practices
- Federal security
The Federal Talent Development team led the project, with sponsorship and oversight from both the Global & Industry Chief Learning Officer and the Federal Practice Chief Talent Director. The app’s developer (a third-party contractor) had a team of six to eight professionals supporting the project.
The biggest challenges facing the team were to make people aware that FLIP was an option and to drive them to use the solution over the previous courses on the learning management system (LMS) and the study guides. At the time, there were approximately 10,000 professionals working on Federal projects who had to meet the compliance requirements. Assuming 60 to 70 percent of that population had a Deloitte-issued mobile device, 6,000 to 7,000 users represented the target audience for 100 percent adoption of FLIP.
To market FLIP to this audience, the following tactics were employed:
- Leadership messaging: A launch message from practice leadership, coupled with recurring reminder communications from Federal Talent Development during the pilot period (September 2017-December 2017)
- FLIP Website: A dedicated FLIP page on Deloitte’s intranet site, providing an overview of functionality and benefits, as well as download and troubleshooting instructions
- Compliance e-mails/alerts: Recurring and automated reminders from Deloitte’s “My Compliance Dashboard” tool, which routed learners to the FLIP page (vs. the assessment links in the LMS)
- Office signage: Both digital and hard-copy marketing signage, placed in strategic/high-traffic locations in Deloitte’s Federal Practice home office
- Push notifications and texts: In-app alert messaging capabilities pushed to professionals who already had downloaded the app, encouraging them to spread the word about FLIP to others
- Raffles and contests: Activities during the pilot that offered prizes to those who completed one or more assessments in FLIP, and also encouraged others to download the app
Results
When the pilot concluded in December 2017, more than 3,500 professionals had downloaded the app and completed one or more of the assessments—nearly a 50 percent adoption rate.
Some of the key measurement results from nearly 1,000 survey respondents include:
- 94 percent strongly agree/agree FLIP is preferred over the LMS courses.
- 90 percent strongly agree/agree they would use FLIP for on-the-job performance support (for both compliance and other topics).
- Nearly 500 users gave FLIP an average 4.8 out of 5 rating on the Deloitte app center.
The FLIP solution offers greater user accessibility, an enhanced content structure, and an improved assessment mechanism. FLIP enables Deloitte to better adhere to Federal compliance requirements while minimizing business disruption, giving Federal practitioners more time to directly serve clients. It is estimated that during the pilot deployment, the 3,500-plus practitioner users saved an average of one to two hours of time out-of-market by using FLIP compared to the previous compliance training, yielding additional time to serve clients. Additionally, FLIP offers a scalable solution, and—given how professional services is a highly regulated business in a number of other areas—has since been adopted for other compliance training requirements across Deloitte.
