This document discusses connecting evaluation and design thinking. It argues that evaluation should incorporate principles of design to view evaluation as both a product and service used for learning and adaptation. The document outlines several design thinking principles that are relevant for evaluators, such as embracing whimsy, biasing toward action, and prioritizing people. It then walks through different stages of design thinking - from problem finding and research to prototyping, refining, and evolving - and discusses implications for how evaluations can support and learn from each stage. The goal is for evaluations to help create value and innovation by understanding what is valued for whom and designing accordingly.
1. Connecting Evaluation and Design
An Introduction to Design Thinking, Methods, and Tools for
Evaluators
Cameron D. Norman PhD MDes CE
Cense Ltd.
@cdnorman Spring 2021
3. 3
The programs, products and services are no
longer being created for a world that stays
the same
Organizations are all looking to learn better,
and produce more impactful products and
services and generate greater return on
investment
Connecting evaluation with design is not only
a means to support greater innovation, it’s
generating greater value for everyone
Connecting Evaluation and Design
4. What do we seek
(and what do we
create in response)?
> Design
What value do we get
(and how does it
manifest in the world)?
>Evaluation
4
Connecting Evaluation and Design
5. Linking Imagination with Evidence
1. Imagination creates
possibilities
2. Design brings ideas to
life
3. Evaluation connects
both together through
learning
5
Connecting Evaluation and Design
6. Design-Driven Evaluation
Focuses on incorporating principles of design into the
purpose, process, and products of evaluation
Views evaluation as both a product and service.
It is a stance on creation and utilization
Norman, C.D. (in press). Supporting systems transformation through design-driven
evaluation. New Directions in Evaluation.
6
Connecting Evaluation and Design
9. Going Beyond DE to Designing for Utilization
Developmental Evaluation (DE) is often described as focused
on strategic learning
Design-driven evaluation focuses on the means by which this is
realized in practice
Employs methods for defining and understanding the learner
and how they desire, expect, or could potentially use the
knowledge generated from evaluation
9
Connecting Evaluation and Design
11. Evaluation helps us to understand
● what value is (and what is valued),
● for whom,
● under what conditions (and the conditions it’s created),
● and to what effect
..to facilitate design for innovation - which is creating for learning,
adaptation, growth and evolution
11
Connecting Evaluation and Design
12. Thinking Like a Designer
(Design Thinking)
12
Connecting Evaluation and Design
13. What is Design Thinking?
The way of approaching the world as a designer
Linking our creativity with our perception and craft to creating new
or better ways to address issues and problems
Brings together ways of seeing, thinking, and doing with planning
(intention)
13
Connecting Evaluation and Design
15. Why design thinking for evaluators?
Design thinking is about creating something of value to
someone
Draws on different senses, aptitudes, skill sets, knowledge
Taps into our natural sense of curiosity, creativity, and
generosity
Provides a framework for working and creating together
15
Connecting Evaluation and Design
16. Principles for Design & Evaluation
● Embrace Whimsy. Relax your
assumptions and constraints about the
usual way of working and approach
problems playfully, with imagination.
● Bias Toward Action. Prioritize creating
artifacts over talking about what you’re
doing.
● Focus on Relationships. Apply systems
thinking to envision how ideas, products,
services, and actions connect
● Expand Perspectives. See the world
through the lens of others — how they
think, feel, and what they prefer, desire,
and need.
● Inspire Accessibility. Put what you
make within reach of most people,
most often.
● Prize Possibility. Explore how might
something work – even if seemingly
impossible.
● Share and Steal: No idea is precious or
proprietary -- share what you know, take
what you need.
● Prototype and Test. Build and
evaluate as you go – often and quickly.
● Prioritize People. Design for humans
16
Connecting Evaluation and Design
19. Implications for Evaluation
Time up-front to explore the learning aims of the project to
align data gathering with the goals and intentions
Ask: “How is the problem framed?” “Is this the problem we thought?”
How might an evaluation be of service to understanding the
problem?
Openness to different perspectives on method, timing, and
data
19
Connecting Evaluation and Design
21. Implications for Evaluation
Background research will help illuminate key factors that will
influence the designed product or service AND the evaluation
Provides an understanding of the user(s) and their behaviour,
motivations, and needs (both articulated and unarticulated)
Tests assumptions against hypothetical and real models of
behaviour
21
Connecting Evaluation and Design
23. Cynefin Framework
Evaluation helps us to understand
● what value is (and what is
valued),
● for whom,
● under what conditions (and the
conditions it’s created),
● and to what effect
System relationships determine
how this is achieved
Image by Snowded - Own work
CC BY-SA 3.0, http://bit.ly/38ZvT6p
23
Context Setting
Connecting Evaluation and Design
24. Implications for Evaluation
Sensemaking is a social activity, account for this in time,
process and outcome expectations
Users may have multiple overlapping, contradicting, and
intermined motivations - sensemaking can help reveal these
and make strategic choices based on data
Systems are dynamic and evaluation designs must account for
this
24
Connecting Evaluation and Design
26. Implications for Evaluation
Idea generation allows a chance to surface assumptions and
experiences of the project team about the problems, systems,
and solutions
Creating a safe space to be flexible, creative, and open allows
insights on metrics-in-action
Set the expectation about use ahead of time: expertise may
not be available to inform decisions
26
Connecting Evaluation and Design
29. Implications for Evaluation
Cultural readiness for learning
Refine the language of evaluation into learning, not
judgement
Create short, simple cycles connecting making with evaluation
29
Connecting Evaluation and Design
31. Implications for Evaluation
Where the greatest value for evaluation for innovation lies
Allows for testing evaluation models and designs as well
Builds a self-reflective capacity into the evaluation
Instills feedback loops and risk-assessment early
31
Connecting Evaluation and Design
33. Implications for Evaluation
Feedback loops for evaluation findings translated into design
choices
Close connections between the evaluation team and the
design team make this process work more effectively
Provides a means to assess the adequacy of the design and the
evaluation
Document design choices (show the work)
33
Connecting Evaluation and Design
35. Implications for Evaluation
Evaluations can serve to chronicle the steps taken in each
design stage and the outcomes associated with each
prototyping or development phase
‘Honours the work’ -- gathering administrative, observational,
and other data from the process provides a record of decisions,
actions, and raw material used in the creation of the product
When organized well, allows for tracing back ‘abandoned
paths’
35
Connecting Evaluation and Design
37. 37
Tools for Creation
Visualization tools are critical -- low barrier, high
participation
Whiteboards
Markers
Newsprint
Sticky notes
Play dough, LEGO, Pipe cleaners
Allow for easy-to-use digital collaboration and voting when
at a distance
Miro
Mural
Milanote
Jamboard
LucidSpark
Connecting Evaluation and Design
38. 38
Cameron Norman PhD MDes CE
Principal and President
Cense Ltd
www.cense.ca
linkedin.com/in/cdnorman/