Mais conteúdo relacionado

Apresentações para você(20)


Training Evaluation Model.pptx

  1. Training Evaluation Model Presented By: Dr Rubee Singh IBM, GLA University
  2. Training Evaluation Model: Training evaluation models are systematic frameworks for investigating and analyzing the effectiveness of training or learning journeys. Donald Kirkpatrick, former Professor Emeritus at the University of Wisconsin, first published his model in 1959. He updated it in 1975, and again in 1993, when he published his best-known work, "Evaluating Training Programs."
  3. Training Evaluation Models • Each successive level of the model represents a more precise measure of the effectiveness of a training program. It was developed further by Donald and his son, James; and then by James and his wife, Wendy Kayser Kirkpatrick. • In 2016, James and Wendy revised and clarified the original theory, and introduced the "New World Kirkpatrick Model" in their book, "Four Levels of Training Evaluation." One of the main additions is an emphasis on the importance of making training relevant to people's everyday jobs. The four levels are Reaction, Learning, Behavior, and Results.
  4. Level 1: Reaction • You want people to feel that training is valuable. Measuring how engaged they were, how actively they contributed, and how they reacted to the training helps you to understand how well they received it. • It also enables you to make improvements to future programs, by identifying important topics that might have been missing. Identify how you want to measure people's reactions and analyze the feedback, and consider the changes that you could make in response Questions to ask trainees include: • Did you feel that the training was worth your time? • Did you think that it was successful? • What were the biggest strengths and weaknesses of the training? • Did you like the venue and presentation style?
  5. Level 2: Learning • It focuses on measuring what your trainees have and haven't learned. In the New World version of the tool, Level 2 also measures what they think they'll be able to do differently as a result, how confident they are that they can do it, and how motivated they are to make changes. • This demonstrates how training has developed their skills, attitudes and knowledge, as well as their confidence and commitment. • To measure how much your trainees have learned, start by identifying what you want to evaluate. • Before the training begins, test your trainees to determine their knowledge, skill levels and attitudes
  6. Level 3: Behavior • This level helps you to understand how well people apply their training. It can also reveal where people might need help. But behavior can only change when conditions are favorable. • Imagine that you're assessing your team members after a training session. You can see little change, and you conclude that they learned nothing, and that the training was ineffective. • It's possible, however, that they actually learned a lot, but that the organizational or team culture obstructs behavioral change. Perhaps existing processes mean that there's little scope to apply new thinking, for example.
  7. • As a result, your people don't feel confident in applying new knowledge, or see few opportunities to do so. Or, they may not have had enough time to put it into practice. • Be sure to develop processes that encourage, reinforce and reward positive changes in behavior. The New World Kirkpatrick Model calls these processes "required drivers." If a team member uses a new skill effectively, highlight this and praise him or her for it
  8. Level 4: Results • At this level, you analyze the final results of your training. This includes outcomes that you or your organization have decided are good for business and good for your team members, and which demonstrate a good return on investment (ROI). (Some adapted versions of the model actually have a Level 5, dedicated to working out ROI.) • It will likely be the most costly and time- consuming. Your biggest challenge will be to identify which outcomes, benefits, or final results are most closely linked to the training, and to come up with an effective way to measure these outcomes in the long term.
  9. Potential Pitfalls of Kirkpatrick's Model • Kirkpatrick's model remains popular, but it should be used with care. The basic structure is now more than 60 years old (despite its many updates), and the ways that people learn and organizations operate has changed radically in this time. Even the term "training" has been largely replaced by "learning and development.“ Today, other, non-formal methods of workplace training are often more popular and effective, with the rise of personalized, user-directed learning, formal training is becoming less prominent. Kirkpatrick's model is not necessarily suited to this new approach to learning.
  10. • Another drawback is that Levels 3 and 4, which arguably yield the most useful information for the business, are time-consuming, resource-intensive, and expensive to implement. So the model may not be practical for all organizations, especially if you don't have a dedicated training or HR department to conduct the analysis. And, it's not ideal for all situations, such as one-off training. • Most importantly, organizations change in many ways, and these changes affect behaviors and results, as well as training. For example, measurable improvements in retention and productivity could result from the arrival of a new boss, or from a new computer system, rather than training. Or it could be a combination of these.
  11. The CIRO Model • In 1970, Peter Warr, Michael Bird, and Neil Rackham and published their book, Evaluation of management training. Their framework for evaluating training became known as the ‘CIRO model’ and offers businesses a way of evaluating training needs and results. • Unlike other models such as Kirkpatrick’s Model that can be applied to a broad range of training and evaluation programs, the CIRO model is specifically aimed at evaluating management training. • CIRO is an acronym that stands for the four levels which make up this approach to learning evaluation.
  12. • The CIRO model is hierarchical, meaning that practitioners must start by studying ‘Context’, before moving through ‘Input’, ‘Reaction’ and ‘Output • Stage 1: Context Evaluation In the CIRO Model, you must first collect performance- deficiency information, i. e. what is the organization lacking? This stage assesses the operational situation that an organization finds itself in. • Stage 2: Input Evaluation During the second stage of the CIRO Model, practitioners must gather information about possible training techniques and methods.
  13. • Stage 3: Reaction Evaluation The third stage of the CIRO Model involves gathering participant views and recording any suggestions they make about the training they received. • Stage 4: Outcome This stage of the CIRO Model involves presenting information about the results of the training. The CIRO Model is a practical way of evaluating management training and has found favor with many organizations the world over
  14. The Phillips ROI Model • As the Kirkpatrick Model grew in popularity during the 1970s, many academics and business practitioners wanted to build and expand on it. Among them was Jack Phillips, who published his own book, Return on Investment in Training and Performance, in 1980. Phillips wanted to build on Don Kirkpatrick’s work and address several of what he perceived to be its shortcomings. Over time, Phillip’s approach to training evaluation became known as the Phillips ROI Model. • The most commonly quoted aspects of the Phillips ROI Model is the addition of a fifth level. This expands upon the Kirkpatrick Model and offers organizations a way of calculating the ROI of their training. The Phillips model has five levels that broadly follow the scope and sequence of the Kirkpatrick model.
  15. • Level 1: Reaction In common with the Kirkpatrick Model, the Phillips ROI Model begins by evaluating the participants’ reaction to the training they received. • Level 2: Learning The second level of the Philips ROI Model evaluates what, if any, learning took place during the training. • Level 3: Application and Implementation Like the Kirkpatrick Model, the Phillips ROI Model looks at whether the participants used what they learned during the training when they returned to the workplace. However, Phillips approach helps an organization determine whether an issue (if there is one) lies with the application of the learning or its implementation.
  16. • Level 4: Impact While the fourth level of the Kirkpatrick taxonomy focuses purely on results, the Phillips ROI model is much broader and looks at the impact of the training. This helps identify whether factors other than training were responsible for delivering the outcomes. • Level 5: Return on investment (ROI) Unlike the Kirkpatrick Model that simply measures training results again stakeholder expectations (ROE), the Phillips ROI model contains a fifth level. This is designed to measure ‘return on investment’, or ROI. This level uses cost-benefit analysis to determine the value of training programs. The Phillips ROI Model found favor with organizations who wanted to assign a monetary value to the results of the training.
  17. The Brinkerhoff model • In 2003, Robert O. Brinkerhoff introduced an entirely new way of evaluating training effectiveness, called the Success Case Method (SCM). The SCM is a methodology that helps an organization understand how a training or coaching program works well, or why it is not working. However, there are two big differences between the SCM and other training evaluation programs. • Firstly, the SCM isn’t simply limited to evaluating training and can be applied to a wide variety of events or activities. For example, it could be applied to a new type of machinery or a new facility in which a company has invested. • Secondly, unlike other training evaluation models (such as the Kirkpatrick Model) SCM isn’t concerned with finding the average performance of training participants. Instead, it deliberately studies the most successful participants and the least successful ones. In other words, it looks at the extreme cases.
  18. The two most important questions asked by the SCM are: • “How well does a program work in a best-case scenario?” • “When a program doesn’t work, what’s the reason for this?” The SCM has five different steps that an organization must go through to evaluate a training course. • Plan a Success Case study. • Write an “Impact Model” that defines what success should look like. • Write a survey that identifies best-case and worst-case scenarios. • Document success cases and conduct interviews. • Draw conclusions, make recommendations and communicate findings to stakeholders
  19. Kaufman's Model of Learning Evaluation • Roger Kaufman and John M. Keller published Levels of evaluation: Beyond Kirkpatrick in the winter 1994 edition of Human Resource Development Quarterly. This work became known as Kaufman’s Five Levels of Evaluation and is commonly referred to as Kaufman’s Model of Learning Evaluation. • Like the Phillips ROI Model, Kaufman’s model was closely based on the Kirkpatrick Model and made a few changes and modifications.
  20. • Kaufman’s Model takes the following approach: • Level 1a: Input • Were the training resources and materials suitable and appropriate? • Level 1b: Process • Was the training well delivered? • Level 2: Acquisition • To what extent did participants acquire new knowledge and skills?
  21. • Level 3: Application • To what degree did participants utilize their learning or training in their on-the-job roles? • Level 4: Organizational Results • In what ways did the organization benefit from the training? • Level 5: Societal/customer consequences How did the training impact on the organization’s customers and society at large?
  22. Kaufman’s model is often praised for separating ‘input’ and ‘process’ as this makes it simpler to assess whether the training materials or the delivery were the cause of the success or failure of a training course. However, the fifth level of Kaufman’s Model – Societal, customer consequence – is often deemed infeasible to implement for the majority of businesses
  23. Anderson Model of Learning Evaluation • The last training evaluation model that we’ll discuss is the Anderson model of learning evaluation. This unique model was first published by the Chartered Institute of Personnel and Development in 2006 as the ‘Anderson’s Value of Learning Model’. This model differs from other training evaluation models in two significant respects. • Firstly, the primary focus of the Value of Learning Model is on aligning an organization’s training programs with its strategic priorities. Whereas other training evaluation models focus closely on the learner and whether they derived benefit from the training, the Anderson Model is designed to be implemented at a management level and cover the whole organization.
  24. • Secondly, this model is unique as it doesn’t focus on the outcome of individual programs. Instead, it looks at an organization's strategic goals and examines whether the training programs are aligned or whether they should shift focus. • Anderson’s model is a three-stage cycle that helps an organization determine the best training strategy for their needs. The three stages are: • Stage 1: Determine the current alignment of training against strategic priorities for the organization. • Stage 2: Use a range of methods to assess and evaluate the contribution of learning. • Stage 3: Establish the most relevant approaches for your organization. The Anderson model of learning evaluation is harder to compare and contrast with other types of training evaluation models as it takes such a unique approach
  25. Measuring Training Effectiveness • Training effectiveness measures the impact of training on the trainee’s knowledge, skills, performance, and the company’s ROI. The training’s goals and objectives should be determined before training occurs, allowing these to be. For example, the trainee’s productivity, sales numbers, and overall mood and happiness might be measured before the training occurs, and then once again after it has taken place. This demonstrates the quality and effectiveness of the training provided and enables businesses to either continue doing more of the same or pivot their approach. • Using scientifically validated techniques to evaluate the effectiveness of training programs is known as training effectiveness management.
  26. • Why measure training effectiveness? • There are many reasons why organizations (large and small) consistently measure training effectiveness. 1. To determine if the training benefits employees. Perhaps the most important reason for evaluating training effectiveness is to see if it benefits your employees’ skills and performance. Additionally, it provides them with a clear idea of what they’ve achieved and the path they need to take to get to the next level. When it comes to learning and development (L&D), feedback and encouragement are crucial.
  27. 2. To see the effect on business performance and determine the training’s ROI. The ultimate goal of all training programs is to boost business performance and see a return on your investment. Changes in productivity, sales, and profits can all be tracked and measured, and you would hope to see an increase in all of the above For example, it’s difficult to determine whether the training in question was responsible for an increase in sales, or if it was the result of something else, like a marketing campaign or a boost in the economy.
  28. 3. To uncover issues in the training process and improve it. When you invest valuable resources like time, money, and energy into your training programs, it’s essential to measure whether they’re working or not. But your intentions for your training will be unique to your business and your long-term goals. This is why you need to define clear objectives at the start. If you fail to do this, then any results you receive will be meaningless because you don’t have a target in sight
  29. • How to measure training effectiveness? Measuring training effectiveness can be conducted through discussions, surveys and questionnaires, post-training, quizzes, assessments, and examinations. Before training commences, it’s essential to decide how you will measure and assess the data you collect. Note: for this you have need to study evaluation models that are most often trusted by companies today: