Mettl’s assessments have been the biggest filter in our recruitment process. Their platform has helped us reach out to a higher volume our applicant numbers. Mettl constantly keeps innovating on their products and tries to introduce a new aspect to everything.
Training investments have witnessed a sizable uptick in recent decades. In 2020, the total training expenditure in the USA was approximately USD82.5billion, which is a considerable sum! To put this into perspective, the annual profits of many large corporations are shy of that number. Of course, in the wake of the COVID-19 pandemic, the numbers may have plummeted temporarily. Still, training and development initiatives are deemed crucial for organizational growth and workforce development. Organizations cannot benefit from the heady notions of workforce development and continuous learning unless there is compelling evidence of the training programs’ effectiveness.
Training is an essential investment and a critical component of human resource development and, if done well, can significantly boost employee satisfaction, desired outcomes and organizational effectiveness. A training program’s efficacy depends on its implementation, and the program would be baseless if its impact is not considered or accounted for. It is not about the sums spent on training; what counts is the training’s relevance and the degree to which employees benefit from it – which is why evaluating training effectiveness establishes the usefulness of a training program and the future course of action for further improvement. One of the methods used to measure training effectiveness is the Kirkpatrick Model. This Training Effectiveness Model is best characterized by its easy application, its scope of measuring an array of variables and high usability.
The training objective of any learning initiative is fulfilled when employees who have undergone training gain new skills and can apply them to their jobs. However, training outcomes may not be readily apparent in a trainee’s performance, and the value that can be attributed to training interventions is a tough nut to crack. There is an underlying complexity in assessing training effectiveness in terms of its monetary value. Therefore, organizations need some proven methodologies to measure and improve the impact of training. Such methodologies are called Training Effectiveness Models.
Dozens of Training Effectiveness Models are in use currently, such as the Kirkpatrick Taxonomy, The Phillips ROI Model, The CIPP Evaluation Model, and Kaufman’s Model of Learning Evaluation, Brinkerhoff’s Success Case Method. However, in this blog, we will examine the Kirkpatrick Evaluation Model, its applications and situations where it should not be employed.
The Kirkpatrick Evaluation Model is a widely recognized approach to analyze and assess a training program’s outcome. It is not only specific to training and educational programs but also includes any program evaluation within its ambit. The model considers both formal and informal training styles to help training practitioners objectively analyze learning outcomes, enabling them to understand the impact of the training, how much the trainees have learned and improve training outcomes.
The Kirkpatrick Model of Training Evaluation is Dr. Donald Kirkpatrick’s brainchild, a professor emeritus at the University of Wisconsin, who first introduced this model in 1959. Dr. Donald Kirkpatrick’s pioneering work on training evaluation contributed immensely to enhancing training evaluation measures and practices. Although the model underwent several revisions following its first release, the model, as it stands today, is considered the gold standard for assessing business training by learning and development professionals worldwide.
The Kirkpatric Model can be implemented before, during, and after the training to understand the degree of improvement concerning the training.
The Kirkpatrick Model comprises four distinct levels through which program managers can design training programs and ascertain their effectiveness: reaction, learning, behavior and results.
According to the Kirkpatrick Evaluation Model, a training assessment should always commence at level one, and consequently, time and resource permitting, move subsequently through the remaining levels. The information assimilated from each prior level provides a basis upon which the next level is evaluated. Each successive level specifies a well-defined measure of training effectiveness but simultaneously requires a meticulous and extensive analysis. The complexity of evaluation techniques increases with each subsequent level and the data obtained becomes invaluable. It is not surprising that many training experts and departments limit their measurement efforts to levels 1 and 2, owing to the complexity evident in levels 3 and 4 in the Kirkpatrick Model. Consequently, the most valuable data remains untapped, which can substantially impact training evaluation efforts.
One must know whether the training delivered to the team has been beneficial. Are employees implementing what they have learned? Does it positively impact business results, such as quality, efficiency, productivity and customer satisfaction ratings? The Kirkpatrick Evaluation Model helps provide answers to the questions mentioned above. The model is a sure-fire way to objectively evaluate the training’s effect, determine how much the trainees have learned, and define the scope for improvement.
Now, let us take an in-depth look at each level:
This level aims to identify the participants’ satisfaction levels vis-a-vis the training. Learning and development (L&D) professionals use this level to ascertain how engrossed the trainees were, how did they contribute and respond to the training, to understand how they reacted to the training they received? Was it beneficial? This step can also help in significantly improving future programs by identifying crucial topics that might have been left out.
The participants’ reaction is also measurable through quantitative data collection tools, such as surveys and questionnaires, using rating scales.
An example of key performance indicators:
The second level involves measuring what the participants gained from the training. It gauges the degree to which the trainees assimilated the intended skills, attributes and knowledge through the training. Instructors and trainers can use this level to ascertain whether they accomplished their goals. Companies can improve upon only by measuring the intensity of learning and determining what participants have and have not learned. Supervisors can test participants in critical areas before and after the training to evaluate the extent of their understanding. This mechanism helps determine what training has notably contributed to enhancing the participants’ knowledge and skills.
Moreover, reviews from trainers and peers can also benefit the assessment at this level.
Quantitative data analysis can be conducted through tests and assessments, after the training, at specific intervals.
An example of key performance indicators:
The next level helps ascertain whether the new learning has catalyzed a behavioral change in the participant. As compared to the previous two levels, this level demands much greater participation and effort from line managers. The participants’ behavioral evaluation is possible by monitoring and interviewing them over time to measure behavior change, how substantial that change is and whether it is sustained. This level involves carefully considering the transfer of learning from the training program to the working environment. It also includes analyzing the work performance of the participants and discerning whether they are making the most of their training on the job.
Measuring the behavior in an effective manner is not a one-off event but a continual process that spans weeks or months following the training session. It includes evaluation through reviews and observations before and after the training.
One can conduct quantitative data analysis through third-party observation, analyzing email conversations of participants and text mining.
In this stage, providing trainees with a conducive work environment can ensure that they can apply their new skills easily. If the management does not support or appreciate the changed behavior, employees would likely revert to their previous state.
An example of key performance indicators:
Level 4 aims to understand how successful the program has been in driving tangible results in the business. It is usually regarded as the crux and the ultimate objective of the program. In this stage, it is common to analyze metrics crucial for establishing training outcomes, such as improved efficiency, better productivity, employee retention, increased sales and higher motivation in employees, reduced expenditure, customer satisfaction, etc. It is to be noted that level 4 is a tricky, time-consuming and costly process because it is not easy to categorize which outcomes, results and benefits can be attributed to training- which is why program managers strive to formulate an effective way to measure training outcomes in the longer run.
Modern trainers now tend to use the Kirkpatrick Evaluation Model backward by focusing on the results they want to achieve and then designing the program to achieve them.
At this level, one can conduct qualitative data analysis through focus groups or interviews. For example, customers may be invited to join a focus group to ascertain whether newly trained employees deliver a superior customer experience, enabling managers to measure the extent to which customer experience has improved over time. Moreover, one can approach the managers to gain insights into whether employees have become more efficient or have improved the quality of their work since the training period.
One can undertake quantitative data analysis through surveys to understand stakeholders’ and customers’ perceptions, compare pre-training data with post-training to assess sales and profits and evaluate the overall employee turnover and retention rate.
An example of key performance indicators:
We have, thus far, deep-dived into each level of the Kirkpatrick Evaluation Model. We can now consider a real-life scenario where the framework would help conduct training evaluation.
The four levels of the training effectiveness model are the pillars on which rests the foundation of any successful training program. But the question is: how to implement the model in real life? One of the most recent approaches to using this model is to employ it in the reverse order, starting with the fourth level and concluding with the first level.
Consider this: A large construction company actively seeks ways to reduce workplace accidents and create a safe working environment for employees. It wishes to reduce occupational hazards by 25%. A novice training practitioner may commence by developing a training program, but a seasoned training evaluator would take a step back and plan the chronology meticulously.
She or he would first have to define the result, break it down to the desired behavior that will yield results, translate them into learning goals, and then gauge the participants’ reaction to the training.
Here is how trainers can prioritize the training objectives and increase the training effectiveness using the model backward:
Training providers can plan in reverse and focus first on achieving the desired result, i.e., reducing workplace accidents by 25%.
The only possible means to reducing occupational accidents is ensuring that the floor staff follows all safety protocols each day before beginning work. Trainers have identified the desired behavior that participants would need to exhibit.
What knowledge and training do the floor staff need to learn? By assessing their current awareness levels using pre-tests and surveys, trainers can specify the learning objectives and create a training program accordingly.
Finally, it boils down to how should one develop and deliver this training program to ensure that the trainees (floor staff) enjoy it, correlate it with their jobs and feel fulfilled and empowered once the training is complete? Ultimately, this level aims at determining the participants’ reaction to the training program.
The example mentioned above clearly states that training should always be tied to a business objective and focused on the desired outcome (Level 4). Usually, the value of training is not easily quantifiable, which somewhat complicates outlining its ROI. Applying the Kirkpatrick Model of Training Effectiveness, particularly in the reverse order, can help managers set their L&D programs for success in the longer run.
Now, let us look at the advantages of the Kirkpatrick Model at different levels in more detail:
The Kirkpatrick Model of Training Evaluation is a widely used tool, but one should use it judiciously. Amid a radically altered world of work, the learning and development ecosystem has undergone dramatic changes. Thus, implementing a six-decades-old model without the proper context can pose significant challenges.
Today, there is no dearth of non-formal workplace training methods that have gained steady ground in the pantheon of the training ecosystem. User-directed and personalized training methods have also come to the fore.
Even though levels 3 and 4 are incredibly crucial metrics for the business, these are equally difficult to implement. So, the Kirkpatrick evaluation framework may not be ideal for organizations lacking a dedicated HR or training department to facilitate the training program.
Most importantly, changing organizational dynamics pose a possible impact on behaviors, results and training. For example, significant improvements in productivity and retention could result from a leadership transition, a new technological intervention, rather than simply training.
The Kirkpatrick Model usually measures the effectiveness of a training program through survey-based outcomes. However, organizations are moving toward assessment-based analysis to make it more robust and bias-free. Assessments provide a means of evaluating training effectiveness using evaluation results and performance data. This evaluation is done via pre-and-post-assessments. Employees are assessed pre-training and post-training, and the evaluation outcomes are compared. The efficacy of the training program is then measured based on the improvement demonstrated by the employee, after the training, in his/her assessment scores. Well-structured assessments enable organizations to assess their workforce’s performance and identify whether the training objectives have been met.
For that reason, Mercer | Mettl’s pre and post-assessment framework is a comprehensive tool that organizations can leverage to measure the performance of online training programs. This framework enables a seamless and easy measure of the ROI of any training program.
Mercer | Mettl’s pre-training assessments are designed based on a specific competency framework. These assessments, administered before training, measure the current proficiency level of candidates in desired competencies.
Mercer | Mettl’s post-training assessment framework is based on Kirkpatrick’s Level-4 training evaluation model, the global standard for evaluating training effectiveness.
There are countless other models on the market, and it is debatable whether the Kirkpatrick Model of training effectiveness is the best means to measure training effectiveness. Every model has its pros and cons. After all, at the end of a training program, what matters is not the model but its execution. The Kirkpatrick Model will mostly work fine. However, it comes down to understanding the outcomes we intend to achieve and then implementing the model correctly.
Well, that is the long and short of it!
We hope this blog has provided you with a comprehensive overview of Kirkpatrick’s four levels of training evaluation and how they can be implemented in designing training interventions.
The Kirkpatrick Model is a training evaluation framework that one can employ to streamline the training process, rectify the loopholes in the training initiatives, and create and implement a viable training plan. It is widely used for evaluating training effectiveness.
Kirkpatrick's four levels of training evaluation are reaction, learning, behavior and results. These four stages are integral to a training process.
The four Kirkpatrick evaluation levels are:The reaction stage measures the participants’ initial and final thoughts about the training regimen,The learning stage evaluates the learning, resulting from training and cognitive enhancement post-training,The behavioral step involves observing skills and behavioral enhancements after applying the acquired knowledge on the job,The results stage focuses on measuring the training effectiveness (or the ROI) by understanding the impact of the training on the business.
The Kirkpatrick Evaluation Model helps impart skill development training by leveraging data from all four levels of measurement. The model is structured to help stakeholders understand how training can be assessed and focus them on critical performance metrics such as on-the-job behaviors, aside from other metrics, such as knowledge acquisition and course satisfaction.
For all evaluation purposes, you can use the model in the reverse order by beginning with the results stage, i.e., level 4, and work all the way down to the reaction stage, which is level 1.
Kirkpatrick's level 4 evaluation questionnaire is a research instrument, such as a post-test survey, comprising a range of questions administered to trainees in an exercise to gather information for the training outcomes of level 4.
The Kirkpatrick Model follows an order in which the evaluation starts from level 1, after which, as time and budget permits, should proceed through levels two, three and four. Data acquired from all the levels can be cumulatively analyzed to assess training effectiveness.
Training evaluation models are proven frameworks to measure and improve the impact of training.
Training effectiveness refers to an objective comparison of employee performance before and after the training to identify the extent to which the overall performance has been enhanced by training interventions.
Originally published May 17 2021, Updated August 23 2021
Abhilash works with the Content Marketing team of Mercer|Mettl. He has been contributing his bit to the world of online business for some years now. Abhilash is experienced in content marketing, along with SEO. He’s fond of writing useful posts, helping people, traveling, and savoring delicacies.
Training effectiveness is a method to measure the effectiveness of an organization’s training initiatives. Training effectiveness can be determined by qualitative assessments that evaluate the improvement in a trainee’s knowledge, skills and behavior. It can also be quantified in terms of return on investment (ROI).