A Beginner’s Guide to the Kirkpatrick Four Levels of Evaluation Model

Training investments have witnessed a sizable uptick in recent decades. In 2020, the total training expenditure in the USA was approximately USD82.5billion, which is a considerable sum! To put this into perspective, the annual profits of many large corporations are shy of that number. Of course, the numbers may have plummeted temporarily in the wake of the COVID-19 pandemic. Still, training and development initiatives are deemed crucial for organizational growth and workforce development. Organizations cannot benefit from the heady notions of workforce development and continuous learning unless there is compelling evidence of the training programs’ effectiveness.

Training is an essential investment and a critical component of human resource development and, if done well, can significantly boost employee satisfaction, desired outcomes, and organizational effectiveness. However, training efficacy depends on its implementation, and a training program would be baseless if its impact is not taken into account. It is not about the amount of money spent on training; what counts is how relevant training is and how much employees will benefit. That is why evaluating training effectiveness establishes how useful a training program is and what can be done to improve it in the future. One of the methods used to measure training effectiveness is the Kirkpatrick model. This training effectiveness model is best characterized by its easy applicability, the scope of measuring an array of variables, and high usability.

The importance of training and development function in an organization is paramount. A training objective will be fulfilled when employees gain new skills and apply them in their jobs. However, it is not an easy task to evaluate training effectiveness because of the underlying complexity in assessing training effectiveness in terms of ROI (returns on investment). That’s why some proven methodologies are needed to measure and improve the impact of training; such methods are called Training Effectiveness Models.

There are dozens of Training Effectiveness Models currently in use, such as the Kirkpatrick Taxonomy, The Phillips ROI Model, The CIPP Evaluation Model, Brinkerhoff’s Success Case Method, and many more besides. However, in this blog, we will delve more deeply into the Kirkpatrick Evaluation Model, its applications and situations where it is better not to use it.

The Kirkpatrick Evaluation Model is a widely recognized approach to analyze and assess the training program outcomes. It is not only specific to training and educational programs but also includes any program evaluation within its ambit. The model considers both formal and informal training styles to help training practitioners objectively analyze learning outcomes, enabling them to understand the impact of training, how much the trainees have learned, and improve training outcomes. The Kirkpatrick model is Dr. Donald Kirkpatrick’s brainchild, a professor emeritus at the University of Wisconsin.

Although the model underwent several revisions following its first release, the model, as it stands today, is considered the gold standard for assessing business training by learning and development professionals worldwide. The model is suitable to be implemented before, in the course of, and after training to understand the amount of improvement pertaining to training.

According to the Kirkpatrick model, a training assessment should always commence at level one, and consequently, time and resource permitting, move subsequently through the remaining levels. The information assimilated from each prior level provides a basis for evaluating the next level. Each successive level specifies a well-defined measure of training effectiveness but simultaneously requires a meticulous and extensive analysis. The complexity of evaluation techniques increases with each level and the data obtained becomes invaluable. It is not surprising that many training experts and departments limit their measurement efforts to levels 1 and 2, owing to the complexity evident in levels 3 and 4 in the Kirkpatrick model. Consequently, the most valuable data remains untapped, which can impact training evaluation efforts.

One must know whether the training delivered to the team has been beneficial. Are employees implementing what they have learned? Does it positively impact business results, such as quality, efficiency, productivity, and customer satisfaction ratings? The Kirkpatrick evaluation model helps provide answers to the questions mentioned above. The model is a sure-fire way to objectively evaluate the training’s effect, determine how much the trainees have learned, and define the scope for improvement.

Now, let us take an in-depth look at each level:

Level 1: Reaction

This level aims to identify the participants’ satisfaction levels vis-a-vis the training. Learning and development (L&D) professionals use this level to ascertain how engrossed the trainees were, how they contributed and responded to the training, and how they reacted to the training they received? Was it beneficial? This step can also help in significantly improving future programs by identifying crucial topics that might have been left out.

The participants’ reaction is also measurable through quantitative data collection tools, such as surveys and questionnaires, using rating scales.

For example:

• On a scale of one to ten, how useful did you find this program?
• On a scale of one to ten, how likely is it that you would recommend this program to your peers/colleagues?

An example of key performance indicators:

• The training completion rate
• The training participation rate

Level 2: Learning

The second level involves measuring what the participants gained from the training. It gauges the degree to which the trainees assimilated the intended skills, attributes, and knowledge through the training. Instructors and trainers can use this level to ascertain whether they accomplished their goals. Companies can improve upon only by measuring the intensity of learning and determining what participants have and have not learned. Supervisors can test participants in critical areas before and after the training to evaluate the extent of their understanding. This mechanism helps determine what training has notably contributed to enhancing the participants’ knowledge and skills.

Moreover, reviews from trainers and peers can also benefit the assessment at this level.

Quantitative data analysis can be conducted through tests and assessments, after the training, at specific intervals.

An example of key performance indicators:

• Newly acquired skills and knowledge
• Training transfer

Level 3: Behavior

The next level helps ascertain whether the new learning has catalyzed a behavioral change in the participant. Compared to the previous two levels, this level demands much greater participation and effort from line managers. The participants’ behavioral evaluation is possible by monitoring and interviewing them over time to measure behavior change, how substantial that change is and whether it is sustained. This level involves carefully considering the transfer of learning from the training program to the working environment. It also includes analyzing the work performance of the participants and discerning whether they are making the most of their training on the job.

Measuring the behavior in an effective manner is not a one-off event but a continual process that spans weeks or months following the training session. It includes evaluation through reviews and observations before and after the training.

One can conduct quantitative data analysis through third-party observation, analyzing email conversations of participants and text mining.

In this stage, providing trainees with a conducive work environment can easily ensure that they can apply their new skills. On the other hand, employees would revert to their previous state if the management did not support or appreciate the changed behavior.

An example of key performance indicators:

• High spirits
• High employee engagement

Level 4: Results

Level 4 aims to understand how successful the program has been in driving tangible results in the business. It is usually regarded as the crux and the program’s ultimate objective. In this stage, it is common to analyze metrics crucial for establishing training outcomes, such as improved efficiency, better productivity, employee retention, increased sales and higher employee motivation, reduced expenditure, customer satisfaction, etc. It is to be noted that level 4 is a tricky, time-consuming, and costly process because it is not easy to categorize which outcomes, results and benefits can be attributed to training- which is why program managers strive to formulate an effective way to measure training outcomes in the longer run.

At this level, one can conduct qualitative data analysis through focus groups or interviews. For example, customers may be invited to join a focus group to ascertain whether newly trained employees deliver a superior customer experience, enabling managers to measure the extent to which customer experience has improved over time. Moreover, one can approach the managers to gain insights into whether employees have become more efficient or have improved the quality of their work since the training period.

One can undertake quantitative data analysis through surveys to understand stakeholders’ and customers’ perceptions, compare pre-training data with post-training to assess sales and profits and evaluate the overall employee turnover and retention rate.

An example of key performance indicators:

• Increased efficiency and improved work performance
• Boost in sales
• Increased customer satisfaction
• Employee retention

We have, thus far, deep-dived into each level of the Kirkpatrick Evaluation Model. We can now consider a real-life scenario where the framework would help conduct training evaluation.

The four levels of the training effectiveness model are the pillars that lay the foundation of any successful training program. But the question is: how to implement the model in real life? One of the most recent approaches to using this model is to employ it in reverse, starting with the fourth level and concluding with the first level.

Consider this: A large construction company actively seeks to reduce workplace accidents and create a safe working environment for employees. It wishes to reduce occupational hazards by 25%. A novice training practitioner may commence by developing a training program, but a seasoned training evaluator would take a step back and plan the chronology meticulously. 

She or he would first have to define the result, break it down to the desired behavior that will yield results, translate them into learning goals, and then gauge the participants’ reaction to the training.

Here’s how trainers can prioritize the training objectives and increase the training effectiveness using the model backward:

The example mentioned above clearly states that training should always be tied to a business objective and focused on the desired outcome (Level 4). Usually, the value of training is not easily quantifiable, which somewhat complicates outlining its ROI. However, applying the Kirkpatrick model of training effectiveness, particularly in the reverse order, can help managers set their L&D programs for success in the longer run.

Now, let us look at the advantages of the Kirkpatrick Model at different levels in more detail:

Benefits of Level 1 Evaluation: Reaction

• It is a straightforward and less resource-intensive approach to evaluation.
• It provides immediate feedback to the trainer about the impact of the training event.
• It is easy to measure how trainees felt about the program.
• It gives insights into specific aspects of the training.
• It offers insights into the aspects that the trainee felt were missing from the training.
• It provides instructors with information that can help create better future training versions.

Benefits of Level 2 Evaluation: Learning

• It provides an excellent opportunity for learners to showcase learning transfer.
• It helps quantify the learning as a direct result of the training.
• The feedback is more objective than level one evaluations.
• It can help identify potential gaps between learning objectives and actual learning.

Benefits of Level 3 Evaluation: Behavior

• It provides a measure of the behavioral change happening at work.
• It can help identify loopholes between training and job performance.
• It provides substantial evidence to uphold the effectiveness of a training program.

Benefits of Level 4 Evaluation: Results

• It provides a broad perspective on the impact the training has had on business results.
• It helps measure the value of training in monetary terms.
• It is pivotal in aligning business objects to the training.

The Kirkpatrick Model of Training Evaluation is a widely used tool, but one should use it judiciously. Amid a radically altered world of work, the learning and development ecosystem has undergone dramatic changes. Thus, implementing a six-decade-old model without the proper context can pose significant challenges.

Today, there is no dearth of non-formal workplace training methods that have gained steady ground in the pantheon of the training ecosystem. User-directed and personalized training methods have also come to the fore.

Even though levels 3 and 4 are incredibly crucial metrics for the business, these are equally difficult to implement. So, the Kirkpatrick evaluation framework may not be ideal for organizations lacking a dedicated HR or training department to facilitate the training program.

Most importantly, changing organizational dynamics poses an impact on behaviors, results, and training. For example, significant improvements in productivity and retention could result from a leadership transition, a new technological intervention, rather than simply training.

The importance of assessments in measuring learning outcomes

The Kirkpatrick Model usually measures the effectiveness of a training program through survey-based outcomes. However, organizations are moving toward assessment-based analysis to make the program more robust and bias-free. Assessments provide a means of evaluating training effectiveness using evaluation results and performance data. This evaluation is done via pre-and-post-assessments. Employees are assessed pre-training and post-training, and the evaluation outcomes are compared. The efficacy of the training program is then measured based on the improvement demonstrated by the employee, after the training, in his/her assessment scores. Well-structured assessments enable organizations to assess their workforce’s performance and identify whether the training objectives have been met.

For that reason, Mercer | Mettl’s pre and post-assessment framework is a comprehensive tool that organizations can leverage to measure the performance of online training programs. In addition, this framework enables a seamless and easy measure of the ROI of any training program.

Pre-training assessment

Post-training Assessment

There are countless other models on the market, and it is debatable whether the Kirkpatrick model is the best means to measure training effectiveness. Every model has its pros and cons. At the end of a training program, what matters is not the model but its execution. The Kirkpatrick Model will mostly work fine. However, it comes down to understanding the outcomes we intend to achieve and then implementing the model correctly.

Well, that is the long and short of it!

We hope this blog has provided you with a comprehensive overview of Kirkpatrick’s four levels of training evaluation and how they can be implemented in designing training interventions.