A datateam in higher education: how teams’ conversations contribute to improvement. Erik Bolhuis, Kim Schildkamp & Joke Voogt

1. From decisions based on intuition
to data-informed decision making
Factors hindering the functioning
of a data team® in higher eduction
Erik Bolhuis, Windesheim University of Applied Sciences, The Netherlands Email: e.d.bolhuis@utwente.nl.
Joke Voogt, University of Amsterdam & Windesheim University of Applied Sciecnes, The Netherlands. Email: j.m.voogt@uva.nl
Kim Schildkamp, University of Twente, The Netherlands. Email: k.schildkamp@utwente.nl
Contact details: drs. E.D. Bolhuis, postbus 217, 7500 AE Enschede, The Netherlands. email: e.d.bolhuis@utwente.nl.
http://goo.gl/iXWbzS

2. Program
• Context of the research
• Research questions
• Theoretical framework
• Interactive section
• Results
• Conclusions

3. Context
• Increase data (-use) in education (OECD,
2013)
• Teacher Education Colleges —> data use:
accountability, part of the curriculum
• Knowledge gab: TE —> data use for
school- & instructional improvement

4. Data
Information that is collected and organized to represent some
aspects of the school (Lai & Schildkamp, 2013, p.10).
▪ Input data: e.g. gender, previous school;
▪ Outcome data: e.g. assessments results, written and oral
exams, portfolio’s, classroom observations, student surveys,
parent interviews, assessment results
▪ Process: e.g. the curriculum,
instruction observations
▪ Context data: eg. data
on school culture

5. Ways of data use in education / examples of data:
1 Accountability
Rankings,
drop- out rates
2
School
improvement
Drop-out rates, test results,
questionnaires, results form
intake
3
Instructional
improvement
Test results (formative and
summative), observations

6. The data team® method

7. A data team is:
• Teams 6-8 teacher educators and a school leader
• Educational problem: grade repetition, low
student achievement
• Goals: professional development and school
improvement
• Coach guides them through the eight steps (two
years)
• Data analysis courses

8. Case
• Dropout in the first study year (HE). In the first year drop-out rates from 55% to
62%.
• Question: what causes drop-out? Is this related to previous education? To
gender? To the atmosphere in the class (ambitious study climate)?
• Data: test results, questionnaire (students and supervisors), and the curriculum
• Based on the data, they conclude and develop measurements

9. Depth of inquiry:
More successful teams (i.e. higher student learning gains) —>
more higher level thinking skills (Achinstein 2002; Stokes
2001) —> conversation with a high depth of inquiry.
The depth of inquiry = inquisitive attitude developing new
knowledge and taking action based on data, while reviewing
each step of the procedure critically (Henry 2012).
The conversations —> reasoning, listening, and underpinning
assumptions. Fundamental for making measurements for
improvement, and to the construction of team- and individual
knowledge (Ikemoto & Marsh 2007).

10. Depth (Henry, 2012)
Depth Participating How Results
No depth Individuals
talking
Sending information No shared knowledge
Some
depth
Several
members
involved
Sharing information,
experiences, and sources
No shared knowledge base and/or
assumptions.
Mean
depth
All members are
involved
Actively create a new
knowledge base
No actively test and sharpen this
new knowledge.
Depth All members
involved
The discussion focuses
on exchanging
experiences, information,
and opinions.
The discussions are not shallow and
lead to a shared explicit knowledge
base. Characteristically the
dialogue is based on concrete
research and/or data.

11. From literature we know factors influencing data-use
(Schildkamp & Kuipers, 2010)
http://goo.gl/iXWbzS

12. Research questions
Which factors enable and constrain depth of inquiry
within the data team?
1. Which factors with regard to data and data information systems
enable and prevent depth of inquiry of data team conversations?
2. Which factors on the level of the user enable and prevent depth
of inquiry of data team conversations?
3. Which factors with regard to the assistance of the data team
enable or prevent the depth of inquiry of data team
conversations?

13. 13
1. Which factors are hindering and promoting factors
affecting the depth of the conversations in a data
team?
2. Which factors cause drop-out in first year TE?

14. 1. Go to www.socrative.com

15. 2. Choose the option for student

16. 3. Enter the room number: ERIK-MLI

17. Method
•A single-case study: micro-process study
•The data team procedure: 19 meetings in 2 year
•Respondents: The data team, the management
•Instruments: observations of the meetings (taped on audio,
verbatim transcript), documents of the data team and artefacts
•Analyse: coding according to a codebook, analyze in TamsAnalyzer,
analyzed by a pattern matching-and time series strategy (Yin, 2014)
•A within- and a cross-case analyses
•Quality of the study: Kappa Cohen's of 0.79.

18. Ged.diepgang
Enig diepgang

19. Factors influencing data-use
Factors related to data and data-information systems:
Data-information system which provides timely, accurate, relevant,
reliable and valid data, data which coincides with the needs
Data related to the perception of the data team members
Factors related to the user:
Data literacy, buy-in/belief, ownership and locus of control
Being able to handle cognitive conflicts
Clarify prior knowledge
Avoid affective conflicts.
Factors related to the organization:
Support from the data coach e.g. conversations skills

20. Conclusions
1.Data —> relate to the level of data literacy
2.Stimulating really use data
3.Clarify prior knowledge;
4.Learn from cognitive conflicts —> clarify which knowledge is
conflicting —> manage confusion —> restructure knowledge base;
5.Avoid affective conflicts: but if they do arise, make sure the conflict
can be addressed;
6.Data coach —> get insight level of data literacy —> present the
data that relate to this level —> and intervene in the conversations
to ensure the data team works on a knowledge base together

21. Discussion
• The use of data in the teacher education curriculum,
requires teacher educators, who can improve
education based on data;
• Data-use requires active and explicit knowledge-
building. Integrating Theory and theory. Should PD
pay attention to this process?
• The data coach —> supporting the data team as a
team, but also coach to use data to improve their
instructional practice?

22. Which factors cause drop-out?
• Gender? (Not found)
• Atmosphere of the class (Rejected)
• Academic skills (Confirmed)
• —> They accompanied the hardest module with a study course
• Contrasting test schedule —> management making the schedule
• Modules with different test components —> one component
• Climate in the first year (best [pedagogical] teachers in the first year
• Monitoring student progress based on data

23. Literature
Achinstein, B. (2002). Conflict amid community : The micropolitics of teacher collaboration, 104(3), 421–455.
Bernhardt, V.L. (2004). Continuous improvement: It takes more than test scores. Leadership Magazine, 34(2), 16-19.
Bernhardt, V.L. (2005). Data tools for school improvement. Educational Leadership, 62(5), 66-69.
Carlson, D., Borman, G. D., & Robinson, M. (2011). A multistate district-level cluster randomized trial of the impact of data-
driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378-398.
Earl, L., & Katz, S. (2006). Leading schools in a data-rich world: harnassing data for schoolimprovement. Thousands Oaks, CA:
Corwin Press.
Henry, S. F. (2012). Instructional Conversations : A Qualitative Exploration of Differences in Elementary Teachers’ Team
Discussions. Dissertation at Harvard University.
Ikemoto, G.S., and J.A. Marsh. 2007. “Cutting Through the ‘Data-Driven’ Mantra: Different Conceptions of Data-Driven
Decision Making.” In Yearbook of the National Society for the Study of Education, edited by P.A. Moss.
Lai, M.K., & Schildkamp, K. (2012). Data-based decision making: An overview. In: Schildkamp, K., Lai, M.K., & Earl, L. (Eds.),
Data-based decision making in education: Challenges and opportunities. London: Springer.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and
hindering factors. Teaching and Teacher Education, 26(3), 482–496.
Schildkamp, Poortman, K. C. & Handelzalts, A. (2015). “Data teams for Schoolimprovement.” School effectiveness and School
Improvement. Advanced Online Publication.
Stokes, L. (2001). Lessons from an inquiring school: Forms of inquiry and conditions for teacher learning. Teachers caught in
the action: Professional development that matters, 141-158.
Yin, R.K. (2014). Case study research: Design and methods (5th ed.). London: Sage.