1. 1

2. Motives
More relevance
for pedagogical
decision making
in schools.
(Anne Brown)
Capture the
innovations
that teachers
(not only
researchers)
produce
(Alan Collins)
DBR

3. A definition
 “A systematic but flexible methodology
aimed to improve educational practices
through iterative analysis, design,
development, and implementation, based
on collaboration among researchers and
practitioners in real-world settings, and
leading to contextually-sensitive design
principles and theories.”
Wang & Hannafin, 2005, p. 6

4. Aspiration
“DBR involves the creation of a theoretically-inspired
innovation, usually a learning environment, to directly
address a local problem”.
But credible evidence for local gains is not sufficient. In
addition, the researcher needs to:
“…generate evidence-based claims about learning that
address contemporary theoretical issues and further the
theoretical knowledge of the field”
(Barab, 2008, p. 155)

5. Positioning DBR
Low Control High
R
e
l
e
v
a
n
c
e L
H Action
Research
Correlation
Studies Experiments
DBR
Quasi-
Experiments

6. Variables and context
 Levels: individual, interpersonal, group, resource,
institution
 Types of dependent variables: climate, learning, and
systemic
 Types of contextual variables: setting, nature of learners,
resources for implementation, professional development,
financial requirements, implementation path.
 Time horizon and measurement points.

7. Factorial designs
DVs
IV
IV
IV
CVs
DV = f(IV1, IV2, …Ivk | CV1…m)
with f usually linear

8. Repeated Measurements Design
Time
Design&Refine
Implement&ObserveAnalyze
Design&Refine
Implement&ObserveAnalyze
Design&Refine
Implement&ObserveAnalyze
Design Change
Design Change

9. Repeated Measurements Design
Time
Interventions
Relevant performance
measure
Baseline

10. Not only a research process
 The design artifact is essential:
 it outlasts the study and can be adopted, adapted,
and re-used by others.To the extent that the design
artifact reifies a model of teaching/learning, that
model/theory gets tested (elsewhere) as well.
 Process (models) as artifacts Teaching
experiments
 Software as artifact  “learning environments”

13. What kind of knowledge is
produced by DBR?
1. Designs for process and/or software.
2. Software (where applicable).
3. “Design narratives”:
 How to allow others to re-contextualize (and perhaps
re-implement) a design with respect to local
particulars?
 Rich descriptions how designs are enacted.
4. Theoretical accounts: Propositional systems
reporting “rich models of interaction” .

14. How is a design study (proto-
)typically done?
Phase 1: Preparing the experiment
 Clarifying the instructional goals
 Documenting the instructional starting points
 Delineating an envisioned learning trajectory
 Placing the experiment in a theoretical
context
Phase 2: Experimenting to support learning
 Collecting data in cycles of design and
analysis
 Applying interpretive frameworks
 Formulating and testing domain-specific
instructional theories
Phase 3: Conducting retrospective analyses
 Explicating the argumentative grammar
 Establishing trust in the findings
 Ensuring repeatability
 Ensuring generalizability

15. Delineating a learning trajectory
 Testable conjectures
 about significant shifts in students’ reasoning, and
 the means for supporting and organising these shifts,
including:
 “Tangibles” such as instructional tasks, materials, ICTs,...
 “Intangibles” such as classrooms norms and nature of classroom
discourse, …
 Teacher as co-designer of the ‘means’ and part of them.
 Basis for the conjectures:
 Research literature
 Experience
 Creativity

16. Placing the experiment in a
theoretical context
 The goal of DBR is to produce knowledge that will be
useful in providing guidance to others in their attempts to
support learning processes.
 Hence, the specific experiment needs to be framed as a
paradigmatic instance of broader phenomena.
 This framing is best done in terms of ‘mid-range’ theories
 E.g., meta-representational competence (diSessa, 1992)
rather than ‘constructivism’.

17. Data analysis
 Huge range of methods, ranging from mathematical and
computational modeling over statistical analysis to
content analysis and ethnomethdological or
phenomenological methods.
 Video analysis very prominent
 Log file analysis very prominent in some quarters (CSCL)
 Typically, mixed methods approach applied.
 (Statistical) “Generalisations” often required by policy
makers.

18. DBR’s Argumentative Grammar
 How are claims to knowledge justified? What form does
the warrant take?
 Demarcation: Can the set of methods establish boundaries
between sound and unsound claims about learning and
teaching?
 Meaningfulness: Can the set of methods help to generate
meaningful/powerful (construct-advancing)
hypotheses/models?

19. The full research process
1 Grounded
Models
2
Development
of Artifact
3 Feasability
Study
4 Prototyping
& Trials
(DBR)
5 Field Study
(Quasi-
Experiment,
DBR)
6 Definitive
Test
(Full
Experiment)
7
Dissemination
& Impact

20. Teacher-led design research
 Required for long-term design research.
 Required for continuous design improvements.
 Develops capacity in teachers and in the
organisation “school” to not only ‘implement’, but
t0 generate (sustainable) innovations.

21. Challenges
 Methodological
 Researcher involvement, Hawthorne effect
 Reliability
 Incremental contributions, comparison across studies
 Design methodology
 Data gathering and analysis
 Capturing context and process
 Managing and integrating various, often ‘rich’ data formats
 Data deluge
 Dissemination and scalability
 Publishing design studies
 Going beyond the case
 Researcher “training” (including teacher “training”)

22. Key Readings
 Kelly, A. E., Lesh, R.A., & Baek, J.Y. (Eds.). (2008). Handbook of
design research methods in education. NewYork: Routledge.
 Reimann, P. (2009).Time is precious:Variable- and event-
centered approaches to process analysis in CSCL research.
International Journal of Computer-supported Collaborative
Learning, 4, 239-257.
 Suthers, D. D., Dwyer, N., Medina, R., &Vatrapu, R. (2007). A
framework for eclectic analysis of collaborative interaction. In C.
Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and
society. Proceedings of the 6th International Conference on
Computer-supported Collaborative Learning (CSCL 2007) (pp.
694-703). New Brunswick, NJ: International Society of the
Learning Sciences.