2. Results Based Meetings Three Steps of Evaluation That Will Clearly Demonstrate the Value of Your Conference, Sales Meeting or Annual Scientific Meeting Presentation by Jon Wollenhaupt, VP, Excel Meetings & Events www.excelmeetings.com Processes outlined in this presentation are based on the ROI Methodology developed by Jack Philips, PhD, Chairman, The ROI Institute
3.
4. Paradigm Shift in Meetings and Events Results and expectations communicated to participants No effort to prepare meeting participants to achieve results Specific objectives for behavior and business impact No specific measurable objectives Assessment of performance effectiveness No assessment of performance issues Program linked to specific business needs No clearly defined business need for the meeting Results Based Activity Based
5. Paradigm Shift in Meetings and Events Reporting on meetings is output focused Reporting on meetings is input focused Measurement of results and benefit-cost analysis (ROI) No measurement of results or benefit-cost analysis Partnerships established with key managers and clients No efforts to build partnerships with key managers Environment prepared to support meeting No effort to prepare the work environment to support meeting Results Based Activity Based
6. Evaluation Levels Level Measurement Focus 1. Reaction & Measures participant reaction to the Perceived meeting and captures perceived value. Value 2. Learning Measures changes in knowledge, information, skills, and attitudes gained in the meeting. 3. Application & Measures changes in on-the-job Implementation behavior and progress with planned actions.
7.
8.
9.
10.
11. The ROI Methodology Develop Evaluation Plans and Baseline Data Evaluation Planning Data Collection Via Questionnaire Collect Data During Meeting Level 1: REACTION AND PERCIEVED VALUE LEVEL 2: LEARNING Collect Data After Meeting LEVEL 3: APPLICATION AND IMPLEMENTATION OPTIONAL LEVEL 4: IMPACT AND CONSEQUENCES Develop Objectives of Meeting
12.
13.
14. Project Example: Development of Measurable Learning Objectives and Post Event Questionnaire for an Annual Scientific Meeting
15. Overview: Excel Meetings & Events was under contract with a San Francisco Bay Area Pharmaceutical Company to assist in the development of measurable learning objectives for an annual scientific meeting and in the design of a post meeting questionnaire. This meeting brought together research scientist from three separate company facilities located in Silicon Valley, Alabama and India.
16. Overview: General desired outcomes for the meeting were stated as follows: Energized participants - Key scientists and engineers from all research sites will have an opportunity to interact, discuss science, think about the future, get to know each other and establish alliances that will enable success.
17. Stated Measurable Learning Objectives After the annual scientific meeting, research scientist will be able to: • Explain the process for implementing new research • Name three new colleagues you met with whom you can collaborate on your research • List three reasons you agree with the research funding for 2008 • List the three most significant things you learned in the poster session • Identify at least one science or technology the company should acquire or develop
18. Stated Measurable Learning Objectives After the annual scientific meeting, research scientist will be able to: • Explain the process for implementing new research • Name three new colleagues you met with whom you can collaborate on your research • List three reasons you agree with the research funding for 2008 • List the three most significant things you learned in the poster session • Identify at least one science or technology the company should acquire or develop
19.
20. Questionnaire Development Sample questions included: 2) Identify two or three things you learned about future research ideas in the poster session. (open-ended) 3) Identify three major accomplishments attained in 2008 that you learned about in the poster session. (open-ended) 4) List the three reasons why you agree with the research funding for this year as discussed. (open-ended)
21. Questionnaire Development Sample questions included: 5) Name three new colleagues you met at this year’s Research Summit , and list at least one way you will collaborate with each on your research in the coming year. (open-ended) 6) What I learned from the presentations I will be able to use in my work and with all colleagues when I return to work. Please circle a number that best reflects your response. 5 = Strongly Agree, 4 = Agree, 3 = Neutral, 2 = Disagree, 1 = Strongly Disagree
22. “ Excel Meetings & Events worked with our internal meeting planning resources and provided guidance, tools, and ideas. Their contributions enhanced the quality of our annual scientific meeting and set up the foundation for continuous improvement.” – CARLOS SCHULER SENIOR DIRECTOR OF SCIENCE & TECHNOLOGY