Beyond Logic Models: Evaluating Complex Programs
•Transferir como PPT, PDF•
1 gostou•1,012 visualizações
Presentation from National Extension Technology Conference on using developmental evaluation approach for the network literacy community of practice.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (15)
Destaque
Semelhante a Beyond Logic Models: Evaluating Complex Programs
Semelhante a Beyond Logic Models: Evaluating Complex Programs (20)
Mais de Sarah Baughman
Mais de Sarah Baughman (9)
Último
Último (20)
Beyond Logic Models: Evaluating Complex Programs
- 1. Beyond Logic Models: Evaluating Complex Emerging Programs Sarah Baughman John Dorner Karen Jeannette eXtension
- 3. NetLit Steering Committee • Anne Adrian • Connie Hancock • Sarah Baughman, Evaluation Specialist • Amy Hays • Bob Bertsch • Karen Jeannette • Peg Boyles • Stephen Judd, Chair • Kyleen Burgess • Jim Langcuster • Chris Canjar • Kristen Mastel • Thomas G. Coon, Administrative advisor • Rich Phelps • John Dorner, Vice-Chair • Jerry Thomas • Brent Elrod, National Program Leader • Kevin Gamble • Terrence Wolfork • Diana Hagan • Barbara Woods Image Credit: http://www.flickr.com/photos/info_grrl/6862478946/in/set-72157629286123040/
- 4. How do we plan to measure outcomes when we aren't sure where we are going? Image credit: http://www.flickr.com/photos/kelvin255/3518359517/
- 6. Developmental Evaluation adapts to the realities of complex nonlinear dynamics rather than trying to impose order and certainty on a disorderly and uncertain world. – Michael Quinn Patton
- 7. Evaluation Progress Steps along the way internal evaluator with eval committee establishing the framework developing initial evaluation questions surveying steering committee for sensitizing concepts reflective practice
- 8. Reflections to move us forward Participating in online networks is an essential skill for our increasingly inter-connected world. Network Literacy is the ability to leverage technology to create connections with other people and/or organizations in a virtual space. http://www.flickr.com/photos/foxypar4/
- 9. • Networks are powerful • Participating in networks has opened opportunities for creating stronger relationships resulting in professional collaborations • Learning, co-learning, and active learning all happen with participation in networks • Comfort level increases through active participation in networks • Need for recognizing barriers to participating in networks Image credit: http://www.flickr.com/photos/mag3737/
- 10. What role can we play? • Role model good online teaching • Leveraging individual CoP member networks to create impact within the system • Leader in defining, creating best practices for online learning and extension education • Serve as “technology stewards” for CES by showing how tools and approaches can address challenges. • Strategic efforts to leverage our networks around specific events
- 11. Roles we can play • Hold national level online conversations about cooperative extension that are open to the public to raise awareness of cooperative extension programs. • Support the efforts of others within Extension working to build network literacy.
- 12. Next Steps • Discussion results of reflective practice with the steering committee • Engage entire CoP in more specific plans to more forward • Continue to document our work
- 13. What we’ve learned • Time consuming, challenging approach • Stakeholder involvement has been difficult to get • We are making progress in individual learning networks • Increasing engagement and interaction outside of CoP membership
- 14. What we’ve learned • Time consuming, challenging approach • Stakeholder involvement has been difficult to get • We are making progress in individual learning networks • Increasing engagement and interaction outside of CoP membership
- 16. Listening Session • Offer a chance to invest • Opportunity to identify themes (sensitizing concepts to work from) • Create opportunity to request specific contributors/contributions • Provide tighter direction for future educational offerings (webinars, blog posts, etc)
- 17. Themes can help identify contributors and “community indicators” • Can we provide programming that responds to themes and increase: – #new contributors – # contributors from state systems – # contributors from external learning networks – Repeat visitors, contributors
- 18. Contact Information SarahBaughman @programeval sarahbaughman@extension.org John Dorner @jdorner jdorner@extension.org Karen Jeannette @kjeannette Karen.jeannette@extension.org
Notas do Editor
- http://blogs.extension.org/militaryfamilies/military-families/ The purpose of the Military Families Learning Network is to serve military family service professionals through engaged online communities which identify and make use of the highest quality, best practices, research-and evidence-based information, educational and curriculum materials, and programming activities and efforts. The primary concentration areas for the Military Families Learning Network are Personal Finance , Child Care , Family Development , and Network Literacy . The Military Families Learning Network is made up of Cooperative Extension family specialists and military family service professionals from DoD, branch services, and non-governmental organizations. These online communities engage in various places, like Facebook facebook.com/militaryfamilies and Twitter twitter.com/milfamln , and on DoD authorized social sites. This learning network offers professional development opportunities through web conferencing [http://learn.eXtension.org ] and online asynchronous learning modules. eXtension.org/militaryfamilies serves as the hub for the online activities and engagement for the Military Families Learning Network. Military Families Learning Network Leaders Project Leader: Kyle Kostelecky, PhD CFLE Social Media Strategist: Anne Mims Adrian , PhD Evaluation Specialist: Sarah Baughman , PhD
- Steering Committee Members Bob Bertsch - North Dakota State Peg Boyles - New Hampshire Kyleen Burgess - North Carolina State Chris Canjar - Kentucky John Dorner - North Carolina State - Vice-Chair Diana Hagan - Florida Connie Hancock - Nebraska Stephen Judd - New Hampshire - Chair Jim Langcuster - Alabama Coop Ext (Auburn Univ) - communications/marketing leader (added 7-05-2011) Kristen Mastel - University of Minnesota Libraries (liaison to Extension) Rich Phelps - Kentucky Terrence Wolfork , Fort Valley State University, Georgia (added 7-15-2011) Barbara Woods - Iowa State Department of Defense (DoD) representative Supporting members: Military Families Evaluation Specialist - Sarah Baughman Brent Elrod , National Program Leader - Military and Veteran's Programs, Institute of Youth, Family, and Community, National Institute of Food and Agriculture, United States Department of Agriculture - National Program Leader Administrative advisor - Thomas G. Coon , Director - Michigan State University Extension eXtension Representatives: Anne Adrian - Auburn - eXtension Kevin Gamble - North Carolina State - eXtension Amy Hays - Texas A&M - eXtension Karen Jeannette - Minnesota - eXtension Jerry Thomas - Ohio State - eXtension
- http://www.flickr.com/photos/kelvin255/3518359517/ Like Star Trek - looking for the unknown, we couldn't use the logic model to measure what we didn't know we'd be teaching. We needed an evaluation tool that would give us the flexibility to adapt to a very rapidly changing environment. When we started, Google Plus didn't exist.
- This is a traditional logic model for a program
- Developmental evaluation aims to meet the needs of social innovators by applying complexity concepts to enhance innovation and use. Developmental evaluation focuses on what is being developed through innovative engagement. The Network Literacy CoP included an alternative evaluation plan in it’s plan of work to allow for the complexity of it’s innovations to be documented. Hallmarks of a developmental evaluation are including an evaluator on the leadership team who helps facilitate documentation of work, asks evaluative questions and marks decision points.
- Initial evaluation questions: 1.What is developing or emerging as the work of the network literacy CoP expands and grows? 2.How is the Cooperative Extension system responding to the innovations? 3.What adaptations need to be made as the network evolves. 4.What’s missing?
- Patterns in our work
- Perspective from CoP member re this evaluation approach