O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Anthony S. Bryk
Master Class, University of Bristol
2
Triple Aims of Educational Improvement
EFFICIENCY	
  
EFFECTIVENESS	
  
ENGAGEMENT	
  
Be0er	
  Use	
  of	
  
Resources	...
How We Work Now: Tower of Babel Problem
3
The Educational R&D Problem
•  Accelerate Improvement Efforts
•  Aim for Quality, Reliably at Scale
4
How We Are Working on This
•  Analogical Scavengers—The Gawande
Inspiration
•  Learning by Doing—Can we actually
make the ...
An Inspiration: Improvement Science in Healthcare
Protecting 5 Million from Harm,
Saving 100,000 Lives
7
We can
accomplish more
together, than
even the best of us
can do alone.
Complex systems
problems that we
now seek to sol...
Networked Improvement Communities:
What are they?
Integrating Two Big Ideas:
•  The discipline of Improvement Science
join...
Six Principles Guide the Work
(plus useful tools to scaffold the activity)
9
Taken Together:
•  Disciplined Inquiry
•  Rud...
I.	
  Problem-­‐	
  &	
  User-­‐Centered	
  
•  What	
  is	
  the	
  specific	
  problem	
  we’re	
  trying	
  
	
  to	
  s...
60-­‐70%	
  
Students	
  assigned	
  to	
  
developmental	
  math	
  
course.	
  
80%	
  
Percent	
  of	
  these	
  
stude...
A	
  Solu<on	
  Framework:	
  
	
  Integrated	
  Pathways	
  
12	
  
Through	
  college-­‐level	
  
sta5s5cs	
  
“To-and-t...
II.	
  Varia5on	
  in	
  Performance	
  is	
  the	
  
problem	
  to	
  solve	
  
•  “What	
  Works”	
  is	
  typically	
  ...
 
	
  
	
  
	
  
	
  
	
  
	
  
	
  
TraditionalSequenceStatway
Effects: Time to Complete a College Level Math Course
1	
 ...
What is Next?
•  Normal Course of Events: “It Works”
– Tout success
– Publish results
– Hope others pick this up
– Go onto...
Varia<on	
  in	
  Pathways	
  Success	
  
Rates	
  by	
  College	
  (n=19)	
  
16	
  
1
23
4
5
6
7
8
9
11
1213
14
15
17
18...
III.	
  See	
  the	
  System	
  to	
  Improve	
  it	
  
	
  
•  Put	
  simply:	
  It	
  is	
  hard	
  to	
  improve	
  wha...
How Do We Heal Medicine? Atul Gawande April, 2012
Gawande’s Closing Observation
Making systems work is the great task of my
generation of physicians and scientists.
But I w...
The	
  Invisible	
  Complexity	
  Schooling	
  
21	
  
The	
  Invisible	
  Complexity	
  of	
  Schooling	
  
60-­‐70%	
  
Students	
  assigned	
  to	
  
developmental	
  math	
  
course.	
  
80%	
  
Percent	
  of	
  these	
  
stude...
The	
  Orien<ng	
  
Problem	
  
	
  	
  	
  Extraordinarily	
  
high	
  failure	
  rates	
  
among	
  students	
  
assigne...
Ins$tu$onal*structures*don’t*
support*student*success*
Students*are*not*engaged*
or*mo$vated*
The*course*material/content*...
The	
  Orien<ng	
  
Problem	
  
Embedded	
  literacy	
  
and	
  language	
  barriers	
  	
  
	
  	
  	
  Extraordinarily	
...
Pathways
Driver
Diagram:
Organizing a
Networked
Improvement
Community
Aim: increase
from 5% to
50%, students
achieving
col...
Pathways
Driver
Diagram:
Organizing a
Networked
Improvement
Community
Aim: increase
from 5% to
50%, students
achieving
col...
IV.	
  You	
  cannot	
  improve	
  at	
  scale	
  what	
  you	
  
cannot	
  measure	
  
•  Measureable	
  targets:	
  “Som...
Produc<ve	
  
Persistence	
  
Suppor<ve	
  social	
  
rela<onships	
  
Target:	
  How	
  do	
  we	
  	
  
	
  	
  	
  	
  ...
V.	
  Accelerate	
  Improvement:	
  	
  	
  
Embrace	
  Disciplined	
  Inquiry	
  
•  Policy	
  Romance	
  of	
  the	
  Si...
A System of Social Learning to Improve
Transla5onal	
  
Research	
  
Interven5ons	
  	
  
(Alpha	
  Labs)	
  
	
  
Will	
 ...
32
Transla5onal	
  
Research	
  
Interven5ons	
  	
  
(Alpha	
  Labs)	
  
•  Will	
  they	
  work	
  for	
  
community	
  ...
Initial Alpha Lab: Mindset Intervention
•  A carefully designed experimental intervention has
changed student mindsets.
• ...
Rapid Iterative DEED cycles
•  Research-Practitioner Team
•  Testing
–  Small double-blind randomized
trial in Algebra cou...
35
Learning	
  from	
  	
  
Network	
  Data	
  
(Hub	
  Analy5cs)	
  
	
  
•  Learning	
  from	
  observed	
  
variability...
36
!
1.Assessing Change: Initial Evidence of Efficacy
of Starting Strong Package
2. Predictive Analytics—targeting support
(a simple at-risk indicator scoring 5 key items/item clusters-day 1)
37
%	
  of	...
Connections to Stereotype Threat
12%	
   13%	
   14%	
  
28%	
  
40%	
  
7%	
  
11%	
  
14%	
  
50%	
  
71%	
  
0%	
  
10%...
39
Expert	
  	
  
Prac55oner	
  
Knowledge	
  
(Subnet)	
  
	
  
Building	
  robust	
  clinical	
  
knowledge	
  about	
  ...
 	
  
PDSA Cycle: Rapid, Small Experimental Trials
PLAN	
   DO	
  
ACT	
   STUDY	
  
The	
  Three	
  Ques5ons:	
  
	
  
• ...
Improving Instructional Routines in Support
of Productive Persistence: PDSA Cycles 
•  Faculty routines and email scripts ...
Sample Run Chart for a PDSA Cycle
(Student Group Noticing Routine)
0%#
20%#
40%#
60%#
80%#
100%#
120%#
1/14/13#
1/21/13#
1...
Developing a Quality Process Reliably at Scale
Develop	
  
A	
  Change	
  
Test	
  under	
  
mul<ple	
  
condi<ons	
  
Tes...
A	
  Developmental	
  
Dynamic	
  
Hunches	
  
Theories	
  
Ideas	
  
Ini<a<ng	
  Resources	
  
P D
SA
P D
SA
P D
SA
P D
S...
VI.	
  Accelerate	
  Improvement:	
  	
  
Tap	
  the	
  Power	
  of	
  Networks	
  
•  A	
  source	
  of	
  innova<on	
  
...
A	
   A	
  
Improvement	
  Networks:	
  Accelerate	
  Learning	
  in	
  
Prac<ce	
  for	
  Improvement	
  
A	
  
B	
  
A	
...
It is all about accelerating how we learn
in and through practice to improve.
Bryk may 2014 using NICs to tackle practical problems in education
Próximos SlideShares
Carregando em…5
×

Bryk may 2014 using NICs to tackle practical problems in education

737 visualizações

Publicada em

Master Class slides from Systems Centre at the University of Bristol 21st May 2014.

Publicada em: Educação, Tecnologia, Negócios
  • Seja o primeiro a comentar

Bryk may 2014 using NICs to tackle practical problems in education

  1. 1. Anthony S. Bryk Master Class, University of Bristol
  2. 2. 2 Triple Aims of Educational Improvement EFFICIENCY   EFFECTIVENESS   ENGAGEMENT   Be0er  Use  of   Resources   Ambi<ous  Learning     For  All  Students   More   Relevance  
  3. 3. How We Work Now: Tower of Babel Problem 3
  4. 4. The Educational R&D Problem •  Accelerate Improvement Efforts •  Aim for Quality, Reliably at Scale 4
  5. 5. How We Are Working on This •  Analogical Scavengers—The Gawande Inspiration •  Learning by Doing—Can we actually make the ideas work? •  Engaging a Larger Community
  6. 6. An Inspiration: Improvement Science in Healthcare Protecting 5 Million from Harm, Saving 100,000 Lives
  7. 7. 7 We can accomplish more together, than even the best of us can do alone. Complex systems problems that we now seek to solve Power of Networks
  8. 8. Networked Improvement Communities: What are they? Integrating Two Big Ideas: •  The discipline of Improvement Science joined to •  The Power of Networks Accelera'ng  Learning  in  and  through  Prac'ce  to  Improve  
  9. 9. Six Principles Guide the Work (plus useful tools to scaffold the activity) 9 Taken Together: •  Disciplined Inquiry •  Rudiments a scientific community •  Aim: systematic practice improvement
  10. 10. I.  Problem-­‐  &  User-­‐Centered   •  What  is  the  specific  problem  we’re  trying    to  solve?     •  What  we  tend  to  do  now:  a  general  issue   comes  into  view  and  we  jump  on  solu<ons      
  11. 11. 60-­‐70%   Students  assigned  to   developmental  math   course.   80%   Percent  of  these   students  that  never   get  past  this  gate.   500,000   students   in  every  cohort  will  never   complete  college  math   requirement.   11 The  Problem  
  12. 12. A  Solu<on  Framework:    Integrated  Pathways   12   Through  college-­‐level   sta5s5cs   “To-and-through” college-level quantitative reasoning Two 1-year pathways “to and through college math” 1   2  
  13. 13. II.  Varia5on  in  Performance  is  the   problem  to  solve   •  “What  Works”  is  typically  the  wrong  ques<on     •  Real  Issue:  Quality  Improvement  Ques<on     “How  to  advance  effec<veness  among  diverse    faculty  engaging  varied    popula<ons    of  students  and  working  in  different    organiza<onal  contexts?”     •  Goal:  Achieve  efficacy  with                      reliability  at  scale  
  14. 14.                 TraditionalSequenceStatway Effects: Time to Complete a College Level Math Course 1  Year   2  Years       Triple the success rate in half the time. 6% 51% 15%
  15. 15. What is Next? •  Normal Course of Events: “It Works” – Tout success – Publish results – Hope others pick this up – Go onto our next project
  16. 16. Varia<on  in  Pathways  Success   Rates  by  College  (n=19)   16   1 23 4 5 6 7 8 9 11 1213 14 15 17 18 19 0% 50% 100% 0% 50% 100% StatwayStudents Non-Statway Matched Comparisons No improvement line We also have a failure, why? What can we learn? Triple success rate line
  17. 17. III.  See  the  System  to  Improve  it     •  Put  simply:  It  is  hard  to  improve  what  we  do  not              fully  understand.  
  18. 18. How Do We Heal Medicine? Atul Gawande April, 2012
  19. 19. Gawande’s Closing Observation Making systems work is the great task of my generation of physicians and scientists. But I would go further and say that making systems work — whether in healthcare, education, climate change, making a pathway out of poverty — is the great task of our generation as a whole.
  20. 20. The  Invisible  Complexity  Schooling   21   The  Invisible  Complexity  of  Schooling  
  21. 21. 60-­‐70%   Students  assigned  to   developmental  math   course.   80%   Percent  of  these   students  that  never   get  past  this  gate.   500,000   students   in  every  cohort  will  never   complete  college  math   requirement.   22 Returning  to  The  Presen<ng  Problem  
  22. 22. The  Orien<ng   Problem        Extraordinarily   high  failure  rates   among  students   assigned  to   developmental   math  instruc<on   Consolidate  the  courses   into  a  1-­‐year  pathway   Real  world  problems  from   sta<s<cs  as  the  organizer   Psycho-­‐social  interven<ons   aimed  at  “produc<ve   persistence”   Rapid  analy<cs  capacity   Faculty  development   Causal  Systems  Analysis:  Why  do  we  con<nue   to  get  the  outcomes  observed?   Primary  Causes    for    High  Failure  Rates     Organizing  Improvement   Hypotheses   event   ???  
  23. 23. Ins$tu$onal*structures*don’t* support*student*success* Students*are*not*engaged* or*mo$vated* The*course*material/content*is* problema$c* Instructors*lack*skills*and*beliefs* that*students*can*succeed* State*policy*does*not* support*student*success* Low*success*rates* in*development* math*func$on*as*a* gatekeeper*to* opportunity* Students*lack*the*skills*to* succeed* Ineffec$ve* advising*system* Ineffec$ve*learning* support*services* Lack*of*social*$es* to*each*other*and* to*their*instructors* Inaccurate* placement* High*dropAout* rates*between* courses* Don’t*believe* they*can*learn* Math*and*tes$ng* anxiety* Poorly*prepared* mathema$cally* They*don’t*know* how*to*study*math* They*don’t*know*how*to* “navigate”*the*college*world* Not*interes$ng*or* relevant** Not*seen*as* useful* Text*is*inaccessible* Does*not*leverage*what* we*know*about*how* students*learn* Too*many*courses*in* the*developmental* sequence* Few*opportuni$es* to*learn*from*others* Lack*knowledge* of*learning*theory* Weak*pedagogy* Don’t*believe* suppor$ng**student* success*is*their*job* Lower*reimbursement*for* developmental*math* Tradi$onal*transfer* requirements*impede* innova$on* Arcane*curricular* topics*create* needless*hurdles* Funding*based*on* enrollment*rather* than*outcomes* OOen*taught*by*adjunct* faculty**with*liPle* professional*support*
  24. 24. The  Orien<ng   Problem   Embedded  literacy   and  language  barriers          Extraordinarily   high  failure  rates   among  students   assigned  to   developmental   math  instruc<on   Lose  large  #  of  students   at  the  transi<ons   Consolidate  the  courses   into  a  1-­‐year  pathway   Students  mindsets   undermine  success   Real  world  problems  from   sta<s<cs  as  the  organizer   Students  “gone”   before  we  know  it   Psycho-­‐social  interven<ons   aimed  at  “produc<ve   persistence”   Rapid  analy<cs  capacity   Course  material  and   instruc<on  are    not   engaging   Faculty  development   Analy<c  Summary  of  Causal  Systems  Analysis   Primary  Causes    for    High  Failure  Rates     Organizing  Improvement   Hypotheses   Eventually  leads  to  a    “Pathways  Strategy”  
  25. 25. Pathways Driver Diagram: Organizing a Networked Improvement Community Aim: increase from 5% to 50%, students achieving college math credit within one year of continuous enrollment Instructional System: Organized around productive struggle, explicit connections, and deliberate practice. Productive Persistence: Students develop skills and maintain positive mindsets Language and Literacy: Students use language in understanding problems, reason mathematically, and communicate results Advancing Teaching: Effective teaching within 2 years of implementation Reduce transitions + assure enrollment across semesters Deliberate focus on “Starting Strong” Promote students’ ties to peers, faculty, pathway Math that matters: students see material interesting, relevant Enhance faculty’s beliefs and relational practices Opening lessons engage interest, assure early success Direct interventions to influence student mindsets Real-time data tracking on student engagement Detail supportive classroom norms and social connections Professional development on “Starting Strong” A  Community  Explicates  its  Causal  Thinking:     A  Community  Explicates    its  Causal  Thinking:     A  Driver  Diagram    to  Organize  Its  Major     Improvement  Hypotheses  
  26. 26. Pathways Driver Diagram: Organizing a Networked Improvement Community Aim: increase from 5% to 50%, students achieving college math credit within one year of continuous enrollment Instructional System: Organized around productive struggle, explicit connections, and deliberate practice. Productive Persistence: Students develop and maintain positive mindsets Language and Literacy: Students use language in understanding problems, reason mathematically, and communicate results Advancing Teaching: Effective teaching within 2 years of implementation Reduce transitions + assure enrollment across semesters Deliberate focus on “Starting Strong” Promote students’ ties to peers, faculty, pathway Math that matters: students see material interesting, relevant Enhance faculty’s beliefs and relational practices Opening lessons engage interest, assure early success Direct interventions to influence student mindsets Real-time data tracking on student engagement Detail supportive classroom norms and social connections Professional development on “Starting Strong” Elabora<ng  Out    The  Driver  Diagram   Produc<ve  Persistence  
  27. 27. IV.  You  cannot  improve  at  scale  what  you   cannot  measure   •  Measureable  targets:  “Some  is  not  a  number;   soon  is  not  a  <me”-­‐-­‐Valued  outcome  measures   – But,  you  just  can  not  stand  at  the  end  of  the  line.     •  We  need  process  measures  <ed  to   intermediate  targets.  
  28. 28. Produc<ve   Persistence   Suppor<ve  social   rela<onships   Target:  How  do  we                  measure  it?   Mindsets  about  the   value  of  math   Mindsets  about   poten<al  to  learn   math   Anxiety  Regula<on   Study  Skills          Conceptual  Task:        reduce  to  5  core  ideas      focus  on  underlying          malleable  causes  +            change  evidence        Prac5cal           Measurement:   reduce  900  items  to  26          “you  have  3  minutes”  
  29. 29. V.  Accelerate  Improvement:       Embrace  Disciplined  Inquiry   •  Policy  Romance  of  the  Silver  Bullet   – Move  quickly  to  large  scale  implementa<on,  but…     •  We  typically  don’t  know  whether:   –   We  can  make  these  ideas  work  at  all;     –   We  have  capacity  and  will  to  execute  with  efficacy  at        scale.     •  Instead,  a  DEED  orienta<on   –  Quick,  minimally  intrusive,  an  empirical  warrant   –   Mantra:  Learn  Fast,  Fail  Fast,  Improve  Fast!  
  30. 30. A System of Social Learning to Improve Transla5onal   Research   Interven5ons     (Alpha  Labs)     Will  they  work  for   community  college   students,  and  if  so,   how?     Expert     Prac55oner   Knowledge   (Subnet)     Building  robust  clinical   knowledge  about     effec<ve  materials  and     instruc<onal  prac<ces.   Learning  from     Network  Data   (Hub  Analy5cs)       Learning  from   observed  variability.   Discerning  the  unseen.  
  31. 31. 32 Transla5onal   Research   Interven5ons     (Alpha  Labs)   •  Will  they  work  for   community  college   students,  and  if  so,   how?    
  32. 32. Initial Alpha Lab: Mindset Intervention •  A carefully designed experimental intervention has changed student mindsets. •  But just because an intervention can work in one setting does not mean it will work in another. •  Need to engineer it to “fit” in instructional contexts. –  Conduct rapid R&D using DEED methodology. –  “Smell testing” –  4 months from small-scale test to larger scale use.
  33. 33. Rapid Iterative DEED cycles •  Research-Practitioner Team •  Testing –  Small double-blind randomized trial in Algebra course (n = 26) –  Larger double-blind experiment (n = 288) •  Introduce to faculty network, carefully study emerging results, continue to revise, refine, and extend. 34 Roberta Carew, Statway faculty Valencia College
  34. 34. 35 Learning  from     Network  Data   (Hub  Analy5cs)     •  Learning  from  observed   variability.  Discerning   the  “unseen.”    
  35. 35. 36 ! 1.Assessing Change: Initial Evidence of Efficacy of Starting Strong Package
  36. 36. 2. Predictive Analytics—targeting support (a simple at-risk indicator scoring 5 key items/item clusters-day 1) 37 %  of  who  failed  the  end-­‐of-­‐term  common  assessment    
  37. 37. Connections to Stereotype Threat 12%   13%   14%   28%   40%   7%   11%   14%   50%   71%   0%   10%   20%   30%   40%   50%   60%   70%   80%   Never   Hardly  Ever   Some<mes   Frequently   Always   Pathways    Dropout   All  students   Black  students   “How  oqen,  if  ever,  do   you  wonder:  ‘Maybe  I   don't  belong  here?’” N  =  714  math  students    
  38. 38. 39 Expert     Prac55oner   Knowledge   (Subnet)     Building  robust  clinical   knowledge  about   effec<ve  instruc<onal   materials  and  prac<ces.  
  39. 39.     PDSA Cycle: Rapid, Small Experimental Trials PLAN   DO   ACT   STUDY   The  Three  Ques5ons:     •  What  specifically  are  we  trying  to  accomplish?     •  What  change  might  we  introduce?     •  How  will  we  know  that  the  changes  are  an  improvement?  
  40. 40. Improving Instructional Routines in Support of Productive Persistence: PDSA Cycles  •  Faculty routines and email scripts re: absent students •  Student group noticing routine •  Effective scaffolding for group roles (rich problems) 41
  41. 41. Sample Run Chart for a PDSA Cycle (Student Group Noticing Routine) 0%# 20%# 40%# 60%# 80%# 100%# 120%# 1/14/13# 1/21/13# 1/28/13# 2/4/13# 2/11/13# 2/18/13# 2/25/13# 3/4/13# 3/11/13# 3/18/13# 3/25/13# 4/1/13# 4/8/13# 4/15/13# 4/22/13# 4/29/13# Percent'of'Student'in'A/endance' A/endance'(By'Day)' Typical#a4endance# Observed#a4endance# n=44# Median:# 0.85# Median#
  42. 42. Developing a Quality Process Reliably at Scale Develop   A  Change   Test  under   mul<ple   condi<ons   Test  under   increasingly  varied   condi<ons   Make  the  change   permanent   Ini5al   Hunches   System   Changes   1  school   1  administrator   5  schools   Many  administrators   En<re  ver<cal  team     A  more  diverse  group   of  administrators   District  Wide   All  administrators   Seeing  Task   Complexity   Seeing   Organiza<onal   Complexity   Learning  to  improve  feedback   conversa<ons  between    principals  and  new  teachers  PLAN   DO   ACT   STUDY  
  43. 43. A  Developmental   Dynamic   Hunches   Theories   Ideas   Ini<a<ng  Resources   P D SA P D SA P D SA P D SA Moving  out  toward   More  diverse  condi<ons:   “factor  of  5  rule  of  thumb”     Aiming  for   Efficacy  with   Reliability  at   Scale  
  44. 44. VI.  Accelerate  Improvement:     Tap  the  Power  of  Networks   •  A  source  of  innova<on   – Dig  into  the  details:  what  worked,  how,  for  whom?   – Can  we  adap<vely  integrate  this  into  other  contexts?     •  Mul<ple  fast  replica<on   – Can  we  make  this  happen  with  efficacy,  reliably  at   scale?     •  Innova<on  diffusion—it  is  largely  about  who  is   connected  to  whom  and  what  they  think  and  do   A  Learning  Educa'onal  System  
  45. 45. A   A   Improvement  Networks:  Accelerate  Learning  in   Prac<ce  for  Improvement   A   B   A   A   A   B   A   A   A   B   A   A   A   B   C   (Englebart,1994)  
  46. 46. It is all about accelerating how we learn in and through practice to improve.

×