how to determine return on training ang development
•Transferir como PPT, PDF•
1 gostou•628 visualizações
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a how to determine return on training ang development
Semelhante a how to determine return on training ang development (20)
Mais de university of education
Mais de university of education (11)
how to determine return on training ang development
- 1. Return on Investment: Training and Development Session 1 ROI and Evaluation
- 2. Objectives • At the completion of this unit, students will be able to: > Discuss the importance of determining ROI in training environments. > Identify stakeholders and develop questions to determine their views and needs. > Write training goals and observable objectives based on stakeholders’ needs.
- 3. Objectives > Describe the importance of the assessment of learning for ROI. > Recognize the differences between short-term and long-term assessment. > Identify items that belong on a training budget sheet to determine ROI. > Calculate expenses and ROI. > Write a report on training that includes all relevant data.
- 4. Return on Investment • Rooted in manufacturing. • Advanced to banking, health care, non-profit, public and education sectors. • Part of quality and efficiency methodologies.
- 5. ROI Is Used To: > Quantify the effectiveness of training. > Manage the training budget. > Provide evidence to management and other stakeholders. > Build trust and respect for ourselves and our unit. > Earn the ears of senior management. > Identify areas for improvement. > Provide data requested by senior management. > Keep our jobs.
- 6. Models • Benefit/Cost Ratio BCR = Program Benefits • ROI (%) Program Costs ROI = Benefit - Cost (%) x100 Cost
- 7. Why is ROI important to you?
- 8. Evaluation Levels 1. Reaction and Planned Action 2. Learning 3. Application and Implementation 4. Business Impact 5. Return on Investment
- 9. Stakeholders • Who are they? > Anyone who pays for, participates in, benefits from, or has decision-making responsibility for HRD and training. • What should they know? > The need for training. > The results from training. > The costs of training.
- 10. Needs Assessment and Task Analysis • Needs assessment: identifies gaps between what is and what should be in the organization. • Task analysis: investigates the specific skill(s), knowledge or attitudes and at which point there is a breakdown in use or performance. If the analysis uncovers a lack of knowledge, skills or attitude, then training is required. If it reveals faulty equipment, poor work conditions or lack of incentive, then another solution is needed.
- 11. Return on Investment: Training and Development Session 2 Goals, Objectives, Assessment
- 12. Goals • Organizational Goals These goals are typically targeted toward the organizational level and include such things as productivity, net income, inventory and cash cycles, and customer satisfaction. • Learning Goals These goals typically focus on the individuals in the organization and their performance, knowledge and skills.
- 13. Writing Objectives Easy as A, B, C, D Audience: Who? Behavior: What do “they” do? Condition: What is the setting and method of evaluation? Degree: Measurement to be met.
- 14. Example Objective At the completion of the course the learner will be able to score 85% or better on a written multiple choice 25 question test Audience Behavior Condition Degree
- 15. Example Verbs • Verbs to Use: > Discuss > Explain > Demonstrate > Identify > List > Perform > Compare/Contrast > Score (on an assessment) • Verbs to Avoid > Understand > Comprehend > Know > Do
- 16. Objectives in ROI • Course objectives: What is it that the learners will be able to do upon course completion? • Application objectives: What goals do the HRD/training practitioner have for the application of the skills gained in training? • Impact objectives: What effect will accomplishing the application objectives have on the organization?
- 17. Example Objectives • Course objective: Learners will be able to make 15 entries in a customer database in 15 minutes with no more than 1 error. • Application objective: Learners will be able to reduce the data entry error rate by 50 percent over the next 6 months. • Impact objective: Employee time spent correcting database errors is reduced by 25 percent from last year’s rate. Increasing Scope
- 18. Learning Assessment • Alignment between assessment and objective is paramount. > Easy to do with well-written objectives. • Cognitive, psychomotor and affective domains are assessed in different ways. > Cognitive: written and oral tests; discussion; compare/contrast exercises. > Psychomotor: demonstration; performance. > Affective: longitudinal observation and/or discussion. • Be sure to use the right type of assessment for each objective of the learning event.
- 19. Return on Investment: Training and Development Session 3 Costs, Budgets, Accounting
- 20. Costs, Budgets, Accounting • Quantifying ROI means accounting for all the costs of the program. > Fixed costs: independent of the number of participants. > Variable costs: Dependent on the number of participants. • There are costs at every step – make sure to account for them all.
- 21. Output Measures Outputs are product-driven and include data that are not performance-based but rather outcomes of changes in performance or investment in equipment. Examples include: > Units built. > Time it takes to build the units. > Income from sale of units.
- 22. Benefits and Soft Skills • Change in: > Attitude, work climate, leadership, teamwork. • We desire these changes because they ultimately effect productivity. > Allow time for change in attitude or behavior, then measure these changes and report qualitatively. > Allow time for change in productivity, then measure for data and report quantitatively.
- 23. BCR = Program Benefits Program Costs ROI = Benefit - Cost (%) x100 Cost Karen Kaminski Tobin P. Lopes Models • Benefit/Cost Ratio • ROI (%) • Phillips – See Figure 2-2 in text.
- 24. Benefit/Cost Ratio Example • Data entry clerks’ average wage: $9.50/hr. • Five hours per week were spent correcting errors before training. • 20 percent less time correcting errors saves one hour each week. • 40 clerks. BCR = .076 for one week. What about 13 weeks? 26 weeks? .076 BCR = 9.50x40x1 hr = $5,000
- 25. Now with ROI% • Data entry clerks’ average wage: $9.50/hr. • 20 percent less time correcting errors saves one hour each week. • 40 clerks. • When clerks use their skills for 26 weeks, ROI% is almost 100%!! ROI (%) = 380x1- 5000 x = - x = - 100 92.4% 100 4620 5000 5000 ROI(%) = (380x26) - 5000 x = x = 100 97.6% 100 4880 5000 5000
- 26. Create a Data Collection Plan • What? • New information that needs to be recorded? • Who? • When? • How?
- 27. Additional Thoughts • Access to learners after training: > Do you have access to learners to follow up on application and use of the knowledge and skills developed during the training? • Isolating training effects: > Consider what else may have occurred at the same time as the training that may have made a difference.
- 28. Trend Line Example Trend Line 35 Y axis: 30 Number of Customer Complaints 25 Projected Value 20 Before Training 15 Training Intervention 10 New Projected Value After Training J F M A M J J A S O N D X axis: Months
- 29. Return on Investment: Training and Development Session 4 Reporting Your Findings
- 30. Review of ROI > Justify the training budget. > Quantify effectiveness of training. > Provide evidence to management and other stakeholders. > Build trust and respect for ourselves and our unit. > Earn the ears of senior management. > Identify areas for improvement. > Respond to management directive. > Keep our jobs.
- 31. Stakeholders: Review • Customer • Employee • Supervisor • Subordinate • President • Board of Directors • Stockholders
- 32. Reporting • Needs to be: > Timely, targeted and unbiased. • The purpose is to: > Illustrate success. > Secure approval. > Gain acknowledgement/agreement on solutions. > Build credibility. > Enhance marketing.
- 33. Reporting • Who needs to know? • What do they need to know? > How will you inform them? • Presentation. • Briefing paper. • Detailed report. • Letter. • When and where? • Who will present? • Feedback?
- 34. Items to Include in a Report • Executive summary. • Title of training. > Goals and objectives. > Who participated, when/where, length. • Assessment and outcomes. • Evaluation. • Budget. • Recommendations.
- 35. Summary • Know your audience: > Who, existing and desired skills, knowledge and attitude (audience analysis). • Expectations of/from organization: > Goals and objectives, stakeholders. • Assessment and evaluation: > Budget, data collection. • Reporting.
- 36. Finally Take action on the data collected. Use it to enhance your training and therefore your organization.
Notas do Editor
- Assessment: In the context of training and development, assessment is of the person’s knowledge, skills and abilities related to the training. Evaluation: In the context of training and development, evaluation is the evaluation of the training program for its ability to meet the goals and objectives. These terms, while similar, have distinct meanings in the context of this module.
- Calculating the return on investment, or ROI, began in the manufacturing field where it was easy to measure time to complete a task and the number of widgets produced. ROI then moved to the banking field, where it was used regularly. Today, ROI is becoming a part of every area of business. We calculate ROI for a number of reasons: To justify the training budget (and to make the case to maintain or increase training dollars). To determine the effectiveness of training: The method used, use of time for trainer and employee, usefulness of information, perceived quality of the training. Was there a change in time, cost or behavior as an outcome of the training. Provide evidence to management and other stakeholders. Because you were directed to. A number of process and quality systems include ROI as an important facet. These include Total Quality Management (TQM); Continuous Process Improvement (CPI); Six Sigma; and Baldrige.
- Ask students to brainstorm, “Why do we use ROI?” Then review points on this slide. Quantify the effectiveness of training: showing what effect training can and does have on dollars, time, production, quality and efficiency. Manage the training budget: helps to identify, structure, and budget HRD and training activities. Provide evidence to management/stakeholders: demonstrates results to management and stakeholders that address their concerns. Build trust and respect for ourselves and our unit: provides solid, usually objective, evidence of the value that HRD and training adds to the organization. Earn the ears of senior management: helps HR and HRD professionals “earn a place at the table” with senior and executive management. Identify areas for improvement: with ROI information, you can identify where to focus your energies to have maximum effect on training and development improvements. Provide data to senior management: to help them make more informed decisions. Keep our jobs: by justifying and illustrating our value to the organization.
- Benefit/Cost Ratio: This is a basic definition of ROI. This is a quantification of the relation between the benefits of a program and its costs. When BCR is greater than one, the benefits outweigh the costs and the program is considered a success. When BCR is less than one, the cost exceeds the benefits and demonstrates that improvements or changes probably need to be made to justify continuation of the program. Another useful and often used definition/formula expresses the ROI as the percentage return on the costs incurred. This has the advantage of speaking to many investors and stakeholders in their language. A result greater than 100% means that the program has a net benefit after accounting for the costs involved in running it. For instance, an ROI% = 150% means that the program yields a 150% return on money invested; i.e., the program yields $1.50 for every dollar that the program costs. A result less than 100% means the program has a net cost. This means that the program does not recoup its cost after accounting for the benefit. When this happens, there may be a “hidden” or social benefit that is not quantifiable, such as an increase in employee morale. In these cases, stakeholders and decision makers need to ascertain whether the scale of loss is justifiable given the money spent. A loss of 3 percent of several thousand dollars may be worth it to realize a happier workplace, but 3 percent of several million may not; there may be easier or less expensive ways to create a happier workplace. This is where ROI really becomes useful. Used properly it can be an objective method to compare the benefits, costs and returns for two or more programs. Note – See Figure 2-2 on p. 37 and throughout the Phillips text for the model. This is a widely adopted and complete model for the process of calculating ROI for training and development.
- Ask students, “Why is ROI important to you?” HRD practitioners and their supervisors must be able to determine if learning/change has occurred. Two main reasons are (1) to report to supervisors, and (2) for your own personal knowledge and improvement of your work or the training department’s work. When people can show that what they do has value and how much value it has, organizations can make better choices. Accountability breeds success. Driving improvement with ROI may help strengthen your position.
- Kirkpatrick’s Four Levels of Evaluation; Phillips added ROI as the fifth. Reaction and Planned Action – Frequency: each learning event. Was the customer (trainee) satisfied with the experience? What did they like? What did they learn? Was there anything missing? Consider using a Likert rating scale for feedback. Was the facilitator knowledgeable? Was the subject interesting? Were facilities adequate for the training? Was the training scheduled at a good time? Additional comments? Learning – Frequency: pre- and post-training. Was there a change in attitude, skills and/or knowledge? Assess learning before training, during training and after training to accurately assess learning. Application and Implementation – Frequency: pre- and post-training and particular periods after training is complete (e.g., three months, six months, one year). This speaks to behavior. Are the trainees doing things differently at work by using the trained knowledge and skills? Pre-/post-test, observation, interview. Allow time for the change to occur. Ask employee, supervisor, subordinates for their perception of change in attitude or performance. Business Impact – Frequency: regular intervals over the calendar or fiscal year – monthly or quarterly is typical. This speaks to overall change for the business as a result of the training program—the results. Is there improved quality, improved production, decreased costs, increased job satisfaction, reduced problems or accidents, increased sales? ROI – Frequency: With each new training event or when significant changes are made to existing events. Consider the costs of training versus the benefits of training. How did the bottom line change? Were the benefits greater than the cost?
- Goals and objectives are based on needs assessment, task analysis and desired outcomes. They guide the design and development of training content, assessment and evaluation. Goals are the most general level of a desired state for the organization or the individual. They provide overarching guidelines to measure performance.
- Objectives are very specific measures for an organization or an individual. In a training environment, we need to keep the organizational goals and the learning goals in mind as we write objectives for any single training session. Objectives guide the learning. ABCD of writing objectives, using observable verbs: Audience The group of learners the objective is written for; “the learner…” or “the student…” Behavior The action or observable verb which describes what the learner will be able to do after completing the instruction (e.g., describe, compare/contrast, demonstrate). For a more complete list of verbs, please see the verb list in the readings. Condition The tools, resources, setting the students will have and the assessment method to be used. Degree The standard or degree of accuracy to be considered proficient. This can be based on a normative scale, measured against a standard of performance.
- The ABCD method doesn’t necessarily mean that the elements appear in that order in the objective. The important thing is to make sure that all the elements are included. The most difficult element--perhaps because it is the most obvious--is the condition. The condition typically takes the form of the assessment to be used. For example, suppose you don’t or can’t use multiple-choice tests because the organization thinks there is too much “testing stress” among their employees or you don’t have the tests available, so you decide to use observation to assess skills. You could drop the condition portion and restate the objective as: “At the end of the course, the learner will be able to score 85 percent or better under observation from the instructor.” The scoring (or degree) might not make sense in this case, so you may change that also. Maybe something like: “At the completion of the course, the learner will be able to complete at least 8 out of 10 tasks at a satisfactory level under observation from the instructor.” Of course, this is now a slightly different objective. Remember: A: Audience: Who is it that is going to do this? Typically this is simply the learner or student. B: Behavior : What is A going to do? Achieve a score on a test? Complete a project successfully? Build a miniature bridge that supports 45 lbs? Build a device that prevents an egg from breaking in a 30-foot fall? C: Condition: What is the device or type of assessment? A paper? A test? A creation? D: Degree: What measure or standard needs to be achieved (and if not obvious, who or what will score it)? What is defined as success? A certain score on a test? A complete paper?
- These verbs can be used to help write observable/measurable objectives. Lists of verbs can be found on Bloom’s Taxonomy Web sites.
- In addition to the objectives trainers must create and set for the learners, there are two other types of objectives that HRD and training professionals must set for themselves: application and impact objectives. Course Objectives = Kirkpatrick Level 2 : knowledge, skills, attitudes. Application Objectives = Kirkpatrick Level 3: actual use in the work setting. Impact Objectives = Kirkpatrick Level 4: organization level: customer satisfaction, work atmosphere, increased productivity, etc. (Impact objectives must be achieved to reach high levels of ROI.) See examples on the next slide.
- Content: Learning Assessment How to tie assessment to objectives: If you begin with well-written objectives, creating learning assessments is easy. Your objective will already specify what it is the learner should be able to do and the parameters which will indicate proficiency. Your responsibility then is to set up the opportunities for the learners to show that they have mastered the objective. This might be accomplished during the training session or after the training session via demonstrated/observed changes in the work setting. Cognitive changes cannot be directly observed, but the results of cognitive changes can be. Therefore, they can be evaluated, like with a written or oral test. Example: How do we know if someone knows how to add? Addition is purely cognitive. We know because we ask them what 2+2 equals. If they were unable to do this before training but can do so after training, then they have learned how to add. Another example: Training is held to teach the strategic planning process. How do you know whether trainees learned anything about the strategic planning process? Testing for this might include asking them to describe the strategic planning process in detail. Psychomotor changes are relatively straightforward. Can the learner perform the action to an acceptable level? For example, can they open the customer database and make a complete, correct entry in a timely manner? Affective changes are VERY difficult to measure. The challenge is to somehow assess attitude. The primary way to objectively and effectively assess these types of changes are from observation, interviews, and communication with the person over a period of time. Example: Assessing an employee’s morale or commitment to the organization. Although learners may exhibit changes in the time directly after training, changes like these take time to really measure. Regular interaction, communication and observation of their behavior over a 3-6 month period will tell you if their attitudes changed or not. Looking at their attitude change in the first two weeks after training may yield inaccurate results, as this is known as the “honeymoon” period after training when learners are most likely to keep using the things they learned. This is another reason why the application objectives at the organizational level are so important. If learners learn but don’t apply the learning consistently over time, application objectives may not be met.
- Costs, Budgets, Accounting (PowerPoint slides) - Fixed, variable, hidden, calculated, measured Input measures – see budget sheet – give examples. Fixed Costs vs. Variable Costs Fixed costs are those that remain the same no matter how many individuals participate in the training. Examples include marketing and information distribution about the training, trainer’s time, employees’ time away from job for training, Variable cost are those costs that change based on number of participants. Examples include training manuals and materials, meals (if provided), difference in cost between large room for 50 participants and small room for 15 (if applicable).
- This is the hard data. Examples: Units developed or built, per day, week, month (increased or decreased). Production or process is on-time (reduced late deliveries). Time – length to complete a task (decreased). Equipment utilization (is it sitting idle?). Reduction in mistakes made (therefore saving time that can then be spent on other tasks). Reduced customer complaints. Reduced accidents, waste. Reduced sick leave. Reduced turnover.
- This is the soft data. Typically, soft skills can be tied directly to hard data. For example, a better work climate reduces the amount of sick leave and turnover, and increases productivity; better leadership enhances new ideas, new revenue and output.
- Review Benefit/Cost Ratio: This is a basic definition of ROI. This is a quantification of the relation between the benefits of a program and its costs. When BCR is greater than one, the benefits outweigh the costs and the program is considered a success. When BCR is less than one, then the cost exceeds the benefits and indicates that improvements or changes probably need to be made to justify the continuation of the program. Another useful and often used definition/formula expresses the ROI as the percentage return on the costs incurred. This has the advantage of speaking to many investors and stakeholders in their language. A result greater than 100 percent means that the program has a net benefit after accounting for the costs involved in running it. For instance, an ROI% = 150% means that the program yields a 150 percent return on money invested; i.e., the program yields $1.50 for every dollar that the program costs. A result less than 100 percent means the program has a net cost. This means that the program did not recoup its cost after accounting for the benefit. When this happens, there may be a “hidden” or social benefit that is not quantifiable, such as an increase in employee morale. In these cases, stakeholders and decision makers must ascertain whether the scale of loss is justifiable given the money spent. A loss of 3 percent of several thousand dollars may be worth it to realize a happier workplace, but 3 percent of several million may not; there may be easier or less expensive ways to create a happier workplace. This is where ROI really becomes useful. Used properly it can be an objective method for comparing the benefits, costs, and returns for two or more programs. Note – See Figure 2-2 on p. 37 and throughout the Phillips text for the model. This is a widely adopted and complete model for the process of calculating ROI for training and development.
- #1: Example: We have 40 data entry clerks working full-time. They spend five hours a week correcting errors they make in data entry. To reduce the number of errors and the time spent correcting them, we invest in training. The benefit after the training is that data input errors are reduced by 10 percent. This means that 20 percent less time is spent correcting these errors. So 1 hour is the time they gain (the benefits) of the training. We take that time and the hourly wage for the data entry clerk (in this case, $9.50/hr) and multiply it by 1 hour each week. $9.50 x 1 hour x 40 clerks $380.00 = Benefits for each week after training. Assuming total program costs for data entry training is $5,000 for 40 clerks, the BCR would be 380/5000 = .076, which is VERY low. But this is the BCR for only one week!! The clerks won’t lose the skills they learned after only one week; in fact, they may get even better over time…but let’s not assume that. Assuming we stay with the same improvement over time and assume the skills and abilities the clerks learn are used for 26 weeks (6 months), then we have a BCR of (380*26)/5000 = 1.976, which is VERY GOOD!! For 13 weeks it becomes .988. This means that if the clerks maintain their data entry skills at that level for more than 13 weeks, the BCR for training becomes better than one and the training has a positive ROI.
- Using the same parameters and assumptions from the previous example, we see that after 26 weeks, not only has the training paid for itself but the results have returned almost 98 cents for every dollar we paid. These models say the same thing. Each model simply states its conclusion in a different way.
- Ask students to brainstorm: “What data do you need? What new information needs to be recorded? Who will record it? When will it be recorded? How will it be collected?” Return to the course objectives, application objectives and impact objectives to ensure you know what information you will need to measure the effect of training. Some of it may not be readily available and may require planning or coordination with other people to collect. From whom will you collect the feedback/information? Participants; Participants’ managers; Participants’ co-workers; Participants’ supervisees; Clients/Customers; Other When will you collect the information? During training; Right after training; After time has passed How do you collect the information? Surveys/Questionnaires; Interviews; Test performance (norm or criterion references) ; Simulations; Observations
- Access to learners after training If trainers are consultants, they may not have access to their learners after the training session ends. This makes it difficult to complete Kirkpatrick’s levels 3 and 4 of evaluation. Do they actually use the knowledge and skills gained in the training session, or do they return to their previous habits? If you don’t have access after training, training can become a process of working with the trainee’s management as well. You can encourage follow-up, use of incentives, etc. You might recommend that the organization collect information for levels 3 and 4, offer a plan to the organization so they can complete the final two evaluations and determine change in employee attitude or behavior. Ask if they will share the information with you. Be sure to complete evaluation levels 1 and 2 at the end of the training. Isolating training effects Sometimes we see a change in attitude, behavior or performance that we want to attribute to the training, but we need to determine if something else may have occurred which initiated the change. An example may be the purchase of new equipment or a change in supervision. To determine if any other events occurred, you can ask the trainees directly and ask their supervisors and their supervisees.
- Review When reporting your findings on return on investment or success of the training program, you need to first remind yourself why you are calculating ROI. Let’s review the list.
- Review Then, review who your stakeholders are – who needs or wants the information you have collected.
- Based on the roles and interests of the stakeholders (who needs to know), what they want to know and what you want to tell them (what they need to know), we need to then consider how to inform them. For some, it may be a concise one-page report with the facts clearly stated. For others, it might be a presentation at a meeting or a much more lengthy report. Each report should target a very specific audience. Don’t write a general report and expect people in different roles to extract the information that is important to them. Your report may include an executive summary, background, objectives, evaluation and assessment tools, outcomes and recommendations for future action.
- Once you know the who, what, and how, determine the details. When will the information be shared? The information should be shared in a timely manner. Where will it be shared? Will you have access to presentation technology? Who will present the information? Will you be the presenter or will you need to prepare your supervisor to do the presentation? Allow time and/or opportunity to receive feedback from the stakeholders. If possible, get agreement and support for future training plans.
- The information included in a report depends on who the report is being created for. This slide lists some ideas of what you might include in a report. Early in the project, we obtained information from different groups of stakeholders. Return to this information to remind yourself what key aspects were important to each group, and be sure to highlight that information in the report generated for them.