The Mysteries of Training Measurement

Training measurement continues to be a challenge in our industry.   In fact, it is one of the top three areas of concern on the minds of corporate training managers.  

We have been studying this issue for more than five years now, and much of our findings are now available in a new book The Training Measurement Book (now available from Bersin & Associates  and from Amazon.com).  The book details the principles, methodologies, case studies, and findings of this research.

A major part of this research has been the development of our Impact Measurement Framework® and Impact Measurement Model®, which identifies nine critical measurement areas and an entire process for training measurement. 


A few comments to help people demystify this area:

• Training measurement is business process measurement.  Effective training measurement is very similar to the measurement of any corporate business process, and should focus most importantly on measuring adoption, alignment (management buy-in), utility (how vigorously the learner recommends the course to others), and client satisfaction (did the business leader get what they wanted?).
• Categorize your training program types.  The right level of measurement depends on the program type.  We identified four distinct types of training programs:  (1) informational, (2) skills-development, (3) competency development, and (4) certified skills development.  (Note this is slightly different and more practical than the ADDIE model of “awareness, informational and skills.”)  In the case of type 1 programs, it makes sense to measure adoption and completion.  In the type of type 3 or 4 programs, we need to capture more detailed information like learning results and job impact.
• Focus on building and measuring alignment.  No training program delivers high value unless management and the rest of the organization supports it.  Therefore “alignment” (one of our 9 measurement areas) is perhaps the most important measure of all.  You can measure alignment in many ways – through the use of up-front manager signoff forms, certain types of course assessments, and a sound planning and budgeting process.   An employee who does not attend a course with the right management and organizational support will not gain much value, so you must focus your measurement on these “alignment” areas.
• Consider the type of learner who takes the course.  One of the biggest drivers of impact is the nature of the person who attends the course.  At GM, for example, the measurement team found that in some courses as many as 1/4 of attendees come from areas outside of the functional program area.  For example, engineers take manufacturing training and marketing people take sales training.  If you try to measure “job impact” for everyone, results will be meaningless.  GM developed a powerful solution: they ask attendees to identify their functional area so they can see what percentage of the attendees are going to specifically apply the material toward their job.
• Do not focus heavily on ROI.  Measurement of ROI is not as easy or valuable as it sounds.  In the book we give lots of examples where this fails.  Ultimately if you measure alignment, efficiency, adoption, and utility, no one will ask you to measure ROI.  In fact, the CLOs and Training Directors we talk with who are highly aligned with the business’s most strategic priorities rarely measure ROI.
• Think “process” not “project.”  We find that a consistent and scalable process is far more actionable than a series of measurement “projects.”  As the process rolls out and you receive lots of information, you can tweak and modify it over time.  You want to focus 80% of your time on analysis of information and 20% on capturing information, not the other way around.
• Measure what matters to your business.  Excellent IT organizations measure uptime, service levels, response times to trouble tickets, and number of security exposures.   Excellent marketing organizations measure number of leads, website hits, new sales opportunities, and ultimately conversions.  If you are developing a new hire sales training program, take the time to identify the business problems you want to solve (this is the essence of performance consulting) and engage the business leaders (e.g. the VP of Sales).  At Randstad, for example (a temporary staffing company), the critical measure of the effectiveness of the onboarding program is the client’s retention of the temporary worker.  If they are well trained, they stay on the job.  If not, there are high turnover rates.
• Consider training measurement in the context of talent measurement.  Most organizations today not only want to understand the efficiency and effectiveness of training programs, but the impact on retention, employee satisfaction, engagement, performance, and even recruiting.  The GM University measurement program, for example, not only measures training programs, but also measures leadership effectiveness, employee satisfaction, and many other talent measures.  Remember that in today’s “Talent Management” focused organization, the corporate learning organization must contribute to the talent management strategy.   The days of a “standalone” training measurement group may be numbered, so think about how to integrate this effort with the “talent measurement” team in your organization.

Bottom line:  In today’s tightened budget environment, actionable measurement of training is one of your most valuable tools when the time comes to reduce the budget.   There is a lot more to talk about on this topic, and we welcome your comments.