7. Integrated Systems Framework BUSINESS APPROACH Corporate Direction Executive Quality Philosophy Drive Customer Value Framework Guided Self Assessment People Quality Council Well directed learning and development Improvement Teams Customer Satisfaction Quality Assurance Certification Team Approach
16. Levels of Evaluation 20% 0-20% Level 5 Return on Investment Phillips and Stone 20% 0-20% Level 4 Business Impact 50% 40-70% Level 3 Job Applications 75% 75-90% Level 2 Learning 100% 100% Level 1 Reaction and Planned Action Kirkpatrick BSA Model Research Model Percentage of Programs Evaluated Levels of Evaluation
17.
18.
19. Developing the ROI Formula Adapted from: Phillips, J. J., Stone, R. D., Phillips, P. P. (2001) The Human Resources Scorecard: Measuring the Return on Investment . Boston, MA: Butterworth-Heinemann Develop Evaluation Plans and Baseline Data Stage 1 Evaluation Planning Stage 2 Data Collection Collect Data During Solution Implementation Collect Data After Solution Implementation Develop Objectives Of Solution Level 1 Level 2 Level 3 Level 4
20. Tabulate Costs Of Solution Isolate the Effects of Solution Convert Data to Monetary Value Calculate the Return On Investment Implement Communication Process Stage 3 Data Analysis Identify Intangibles Stage 4 Communicate Results Level 5 Intangible Benefits
21.
22.
23.
24. LDI scorecards Level 1. Reaction & Planned Action 2. Learning 3. Application 4. Impact 5. Return on Investment Measurement Focus Measures participant satisfaction with the program and captures planned actions Measures changes in knowledge, skills, and attitudes Measures changes in on-the-job behavior Measures changes in business measures Compares program benefits to the costs Economic System Individual
25.
26.
Notas do Editor
1. Leadership (Building Organisational Capability) The Leadership Development Program (LDP) addressed the direction, values and enhanced leadership skills through the support, delivery and design by BSA Human Resource Services to address and provide direction for BSA by encouraging and empowering staff. The executive and senior managers and team leaders have participated in a range of learning in interactive action learning based workshops and on-line modular coaching and support. Moreover, as part of their learning, the executive managers have participated in coaching and sponsoring individuals through a shared learning experience based upon emotional intelligence and transformational leadership and it’s application to BSA changing leadership role. 2. Relationship Management (Improving and Responding to Current and Future Relationships). The Creating Business Relationship (CBR) Program is a key component of the learning and development program closely linked to the Customer Continuum Project. The program was designed by Human Resource Services to build upon current and future relationships within and across the building industry to continually improve and respond to customer demands. This seven (7) module organisational learning program was designed, developed and delivered in partnership with universities (Queensland University of Technology and University of Queensland), professional associations (Institute of Public Administration Australia), public sector agencies and leading Consultants (Drew International) to provide a complete program of customer service relationship training 3. People Development (Developing, Valuing and Involving Staff) The strong learning focus within BSA has provided a unique mechanism to set a direction to strengthen our position as a people-oriented employer where people are developed, valued and involved. This is achieved through the Learning and Development Calendar that provides the opportunity to develop a high energy, high spirit and values-based BSA team through action-learning, experiential based learning. The capabilities developed have prepared our people to respond to our customer’s need in the community and a challenging building industry. Programs delivered are strategic linked to the corporate plan and workforce strategic plan and include project management, business writing skills, financial management through to coaching and mentoring.
Level 1 Course Evaluation Level 1 evaluation measured 100% of corporate learning and development activities based upon perceptions about the learning and its effectiveness. Course evaluations were utilised with a special emphasis on relevancy of knowledge and skills transference back to the workplace. Level 2 Pre and Post Assessment Analysis Level 2 evaluations focused on the targeting 90% -100% learning activities offered, to define the extent to which participants changed attitudes, improved knowledge and increased skill. This was achieved by undertaking exhaustive pre and post assessments for all learning activities by internal and external facilitators. Level 3 Transfer of Learning The evaluation process was extended to Level 3 of the Kirkpatrick Model whereby the change in behaviour by participants was sought. It involved testing the participants’ capabilities to perform the learned skills back in the workplace. This level of testing involved supervisors, peers and team leaders. The tools utilised in this level of evaluation include: 6 week follow-up meetings/teleconferences; 6 week evaluation surveys; 3 monthly evaluation surveys; Participation in Business Improvement Projects; and Participation in organisational process reviews. Results from this level of evaluation revealed improvement in skills and knowledge ranging from 10% - 50% with a transference of skill and knowledge still being applied at 95% rate. Level 4 and Level 5 Evaluation Level 4 and level 5 focused on the business impact of critical learning activities that would be the most measurable in terms of building relationships the community and industry stakeholders and thus providing a measure to indicate business outcomes being achieved. At this level, learning activities focused on tracking improvements in strategic courses such leadership development program mediation skills, presentation and facilitation skills and future courses such as managing challenging behaviour and conflict and promotional skills.
reinforce the four-part test; refer to the flow chart on the next page.