ERS System 20/20 Analysis of Denver Public Schools March, 2017
•Transferir como PPTX, PDF•
1 gostou•1,735 visualizações
ERS presented its findings from our School System 20/20 diagnostic of DPS on March 15, 2017 to the district's leadership team, board of education, and members of the Denver education community.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (19)
Destaque
Destaque (20)
Semelhante a ERS System 20/20 Analysis of Denver Public Schools March, 2017
Semelhante a ERS System 20/20 Analysis of Denver Public Schools March, 2017 (20)
Mais de Education Resource Strategies
Mais de Education Resource Strategies (16)
Último
Último (20)
ERS System 20/20 Analysis of Denver Public Schools March, 2017
- 1. © Education Resource Strategies, Inc., 2014 Draft—do not cite or disseminate Denver Public Schools: Leveraging System Transformation to Improve Student Results March, 2017
- 2. © Education Resource Strategies, Inc., 2014 Draft—do not cite or disseminate © 2017 Education Resource Strategies. All rights reserved. Education Resource Strategies (ERS) is a non-profit organization dedicated to transforming how urban school systems organize resources (people, time, technology, and money) so that every school succeeds for every student.
- 3. 3 Saint Paul Chicago Cincinnati Atlanta Philadelphia New York City SyracuseRochester Providence Baltimore Prince George’s County, MD Washington, D.C. Charlotte-Mecklenburg Duval County Boston Georgia Districts Fulton County Hall County Vidalia City Treutlen County Marietta City Cleveland Austin Albuquerque Michigan Tennessee Waterbury, CT Aldine Newark District work State work 2015-16 partners Recent partners Knox County Lake County Indianapolis Nashville Georgia Santa Fe New Haven, CT Tulsa Palm Bach County Los Angeles Oakland Sacramento California Buffalo Over the last 10 years, ERS has worked closely with the nation’s largest school systems to improve resource use
- 4. Why school systems? Every student succeeds SUCCESSFUL STUDENTS EFFECTIVE DISTRICTS Coherent systems that give schools the flexibility, capacity, and support they need Do your district’s structures and policies maximize the enabling conditions for excellent schools? EFFECTIVE SCHOOLS Schools that deliberately manage talent, time, and money around a clear instructional model Are practices and resource use aligned with high performing strategies at every school? EFFECTIVE TEACHING High-quality instruction that is aligned with standards Are the structures in place to improve instructional quality and alignment? 4
- 5. What is the School System 2020 Diagnostic? Every student succeeds SUCCESSFUL STUDENTS EFFECTIVE DISTRICTS Coherent systems that give schools the flexibility, capacity, and support they need Do your district’s structures and policies maximize the enabling conditions for excellent schools? EFFECTIVE SCHOOLS Schools that deliberately manage talent, time, and money around a clear instructional model Are practices and resource use aligned with high performing strategies at every school? EFFECTIVE TEACHING High-quality instruction that is aligned with standards Are the structures in place to improve instructional quality and alignment? District Impact: To help district leaders answer these questions and identify and track changes to System Conditions and Resource Use that lead to improved student achievement Field Building: To build a data set of where districts are along this spectrum, to highlight best practices and ultimately to demonstrate that creating system conditions that promote the strategic use of resources in districts and schools drives improved student outcomes 5
- 6. School System 20/20 identifies key transformational levers across seven areas of district activity Teaching Standards and Instruction LeadershipPartners Funding School Support School Design • Broad, but high level • Research-based • Includes qualitative and quantitative measures • Benchmarks against best practice and other districts 6
- 7. To date we have conducted School System 20/20 in 16 districts, including 3 districts we’ve identified as best practice case studies: Addison, NY (2016) Aldine, TX (2014) Baltimore City, MD (2016) Boston, MA (2015) Charlotte, NC (2016) Cleveland, OH (2016) Denver, CO (2016) El Paso, TX (2017) Indianapolis, IN (2015) Lawrence, MA (2014) Memphis, TN (2017) Norwalk, CT (2016) Oakland, CA (2015) Palm Beach County, FL (2015) Tulsa, OK (2015) Spring Branch, TX (2017) 7 “I’ve been inundated with so much information, and this is by far the most strategic, easiest to understand and most useful.” - Dr. Sonja Santelises, Superintendent, Baltimore City Schoolsen inundated with so “You were successful in doing what is the hardest thing, which is making the link between budget and school design…If we had this information as we were developing our strategic plan, we would have had a better strategic operating plan.” -Steven Adamowski, Norwalk Superintendent
- 8. Looking at this information helps us to understand: How does performance growth in DPS compare to other districts in Colorado and nationwide? What actions over the past five years have driven changes in system conditions, resource use, and student outcomes? How does DPS compare to other leading districts and research-based best practice? Where should DPS target future investments? What can other district leaders and policy makers learn from DPS’ experience? 8
- 9. What is our history with DPS? ERS has partnered with DPS over the past 8 years on a number of paid engagements ERS is currently engaged with DPS in two areas: Teacher compensation workshops New teacher case studies with DPS-specific supplement The work we are reviewing today was funded by the Charles and Lynn Schusterman Foundation as a case study 9
- 11. According to Stanford NCES comparison data, DPS had the second highest growth of any district >25K nationwide between 2009-10 and 2012-13 Denver [CELLRANGE] [CELLRANGE] [CELLRANGE] [CELLRANGE] [CELLRANGE] [CELLRANGE] -1.5 -1 -0.5 0 0.5 1 1.5 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% GrowthinGradeEquivalentUnits,AveragedAcross Grades,2009-2013 District Percent FRL Denver Average Grade Equivalent Unit Growth 2009-2013, Against Large Districts Stanford Educational Data Archive, NCES; comparison includes all districts with more than 25,000 students, data only available 2009-2013 11 Averaged across grades and subjects, DPS performance improved almost an entire grade level, from 1.5 grade levels behind in 09-10 to .5 grade levels behind in 12-13.
- 12. 45% 54% 36% 69% 70% 41% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 DPS vs. Colorado % Proficient & Advanced TCAP Reading DPS Percent Proficient State Percent Proficient (Excluding DPS) 12Source: Colorado Department of Education, Achievement Percentile Rank Report CMAS PARCC Assessment …and within Colorado
- 13. Proficiency increased across all subgroups 54% 44% 42% 46% 0% 10% 20% 30% 40% 50% 60% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 ELAPercentProficient ELA Percent Proficient- all tested grades All Students FRL ELL Non-White Source: CDE website-DPS student enrollment & performance data, 2004-2014 13
- 14. And across both charter and district-run schools 43% 57% 45% 54% 38% 51% 31% 42% 0% 10% 20% 30% 40% 50% 60% 70% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 ELAPercentProficient ELA Percent Proficient by School Type Charter Traditional Charter - FRL Traditional - FRL Source: CDE website-DPS student enrollment & performance data, 2004-2014 14
- 15. The combination of improvements across charter and district schools with a shift of student into charters increased DPS’ % proficient in ELA from 49% to 54% between 2008 and 2014 5% 87% 8% % of Performance Improvement by source 15Source: ERS Analysis Charter school improvement: 8% of students were in charter schools from 08-14; proficiency in those schools improved on average 4 points District-run school improvement: DPS-run schools gained on average 4% pts proficiency since 2008. In 2008 these schools served 92% of students. Shift from district-run to charter: From 2008 to 2014, 7% of students moved from district schools to charter schools. In addition to the 4% they would have gained in district schools (in dark blue), the schools they moved into were performing 4% better than the district-run schools, accounting for 5% of the overall improvement.
- 16. And despite DPS having a significantly higher need population than the rest of the state… 62% 62% 68% 64% 66% 66% 71% 72% 72% 71% 72% 70% 68% 27% 28% 31% 31% 31% 32% 35% 37% 38% 38% 39% 38% 39% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 % of students eligible for free and reduced lunch DPS % FRL State % FRL (Excluding DPS) 16Source: Colorado Department of Education, Achievement Percentile Rank Report
- 17. ..gaps with the state for ELL and FRL students narrowed; and as of 2014 DPS was outperforming the state with white students 0% 20% 40% 60% 80% 100% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 ELL DPS ELL Colorado ELL Source: CDE website-DPS student enrollment & performance data, 2004-2014 17 0% 20% 40% 60% 80% 100% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 x FRL DPS FRL Colorado FRL 0% 20% 40% 60% 80% 100% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Non-Minority Colorado Non-Minority DPS Non-Minority 0% 20% 40% 60% 80% 100% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Minority Colorado Minority DPS Minority ELA Percent Proficient
- 18. Though we don’t have national comparison data beyond 2013, state data indicates that DPS’ improvement trajectory has continued and even steepened 45% 54% 36% 69% 70% 41% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 DPS vs. Colorado Percent Proficient & Advanced TCAP Reading to PARCC ELA DPS Percent Proficient State Percent Proficient (Excluding DPS) 18Source: Colorado Department of Education, Achievement Percentile Rank Report CMAS PARCC Assessment Closed the gap by 2 percentage points from 09-10 to 12-13 Closed the gap by 12 percentage points from 12-13 to 15-16
- 19. More than might be suggested by the narrowing need gap with the rest of the state 62% 62% 68% 64% 66% 66% 71% 72% 72% 71% 72% 70% 68% 27% 28% 31% 31% 31% 32% 35% 37% 38% 38% 39% 38% 39% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 % of students eligible for free and reduced lunch DPS % FRL State % FRL (Excluding DPS) 19Source: Colorado Department of Education, Achievement Percentile Rank Report
- 20. How did DPS do it? 20
- 21. The School System 20/20 framework identifies the key transformations we believe that districts must make to drive significant, sustained improvement 21 System 20/20 Area Key transformations STANDARDS • Rigorous, information-age standards with effective curricula, instructional strategies, and assessments to achieve them. TEACHING • Selective hiring, development, and strategic assignment to schools and teams. Career path and compensation enable growth and reward contribution. SCHOOL DESIGN • Schools with restructured teams and schedules: personalized learning and support that responds to student needs and promotes instructional collaboration. LEADERSHIP • Clear standards and accountability with the support school leaders need to succeed. SCHOOL SUPPORT • A central office that serves as a strategy partner, leveraging data to increase efficiency and identify best practices. FUNDING • A central office that serves as a strategy partner, leveraging data to increase efficiency and identify best practices. • Partnering with families, community institutions, youth service organizations, and online instructors to serve students’ needs.PARTNERS
- 22. For over a decade, DPS has undertaken a broad array of structural changes aimed at many of these key levers 22 System 20/20 Area DPS Actions STANDARDS • Development of CCRS readiness assessment for schools • Creation of curriculum and instructional support resources TEACHING • Implementation of LEAP • Teacher leadership program • Incentives to attract teachers to highest-need subjects and schools • Cross-district year-long plan for professional development SCHOOL DESIGN • Increased school level flexibility • Focus on whole child LEADERSHIP • Expanded principal pipeline and development programs • Implement LEAD • Incentives to attract and retain effective principals in high-need schools SCHOOL SUPPORT • Data systems • Accountability system • Lower instructional superintendent ratios, especially for turnaround schools FUNDING • Shared funding system across charter and district run schools • Incorporation of ELL weights into funding system • Active management of school portfolio across district-run and charter schools
- 23. The result is a measurable improvement in system conditions and practice & resource use 23Source: DPS financial, student, and school data; ERS database. 2015-162009-10 STANDARDS TEACHING SCHOOL DESIGN SYSTEM CONDITIONS PRACTICE & RESOURCE USE LEADERSHIP SCHOOL SUPPORT FUNDING PARTNERS 2015-162009-10
- 24. These efforts have created the most strategic enabling conditions of any district we’ve studied 24 Denver 2015-16 Charlotte 2015-16 Aldine 2012-13 Lawrence 2013-14 Palm Beach County 2014-15 Boston 2014-15 Denver 2009-10 1 1.5 2 2.5 3 Less Strategic More Strategic System CondistionsSystem Conditions
- 25. Practice & resource use in DPS lags system conditions, but is also among the leaders of districts we’ve studied 25 Denver 2015-16 Charlotte 2015-16 Aldine 2012-13 Lawrence 2013-14 Palm Beach County 2014-15 Boston 2014-15 Denver 2009-10 1 1.5 2 2.5 3 Less Strategic More Strategic Practice & Resource Use System CondistionsSystem Conditions
- 26. Key Funding & Portfolio findings DPS is unique in how you have leveraged you chartering authority to create a more level playing field across district-run and charter schools Willingness to put teeth into accountability systems by closing chronically underperforming schools Strategic approach to opening new schools that avoids the unplanned and fragmented enrollment impacts on surrounding schools often seen in other districts Deliberately moving beyond choice to increase equity of access Implementation of single system designed to ensure equitable funding across all schools This approach has resulted in better options for students Increase in high quality seats across all regions; largest in regions with fewest high quality seats More similarity in student needs between district-run and charter schools than in many districts But high quality options are still lagging in some high need regions 26 Funding & Portfolio
- 27. DPS has the most sophisticated portfolio management approach of all districts we’ve studied 27 Metric # Metric Less Strategic More Strategic FundingandPortfolio Portfolio School opening and closing decisions support long-term portfolio goals. The district calculates the cost of different school types and has a clear plan for staffing small and specialty schools to balance access The district has deliberately created a portfolio of school governance types and decision models (e.g., charter, autonomy) to reflect district capacity and to meet the needs of the students in the district. The district has deliberately created a portfolio of school grade levels and sizes to meet the needs of the students in the district. The district has deliberately created a portfolio of school program offerings (e.g., magnet, academy, specialized programming, themed schools) to meet the needs of the students in the district. The district proactively balances the number of seats by deliberately reducing the number of seats at one school when seats are added at another. Boston Charlotte Lawrence Denver Funding & Portfolio
- 28. Across the district, the number of high-quality seats has increased most in areas with the lowest access 27% 28% 34% 57% 68% 31% 41% 44% 60% 67% 0% 10% 20% 30% 40% 50% 60% 70% 80% Far Northeast Northwest Southwest Near Northeast Southeast Percent of Students in High-Quality Seats by Year 09-10 15-16 28Source: DPS school enrollment and performance data 09-10 and 15-16 Funding & Portfolio 15-16 % Poverty 81% 78% 89% 58% 45%
- 29. DPS has more similarity in student needs between charter and district-run schools than we see in many other districts Source: DPS school enrollment 09-10 and 15-16 73% 68% 0% 20% 40% 60% 80% Charter Traditional % FRL 37% 30% 0% 10% 20% 30% 40% Charter Traditional % ELL 10% 12% 0% 2% 4% 6% 8% 10% 12% 14% Charter Traditional % SPED 84% 76% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Charter Traditional % Non-White 30% 34% 0% 5% 10% 15% 20% 25% 30% 35% 40% Charter Traditional % Incoming Proficiency 6th Grade ELA 28% 32% 0% 5% 10% 15% 20% 25% 30% 35% Charter Traditional % Direct Certification Funding & Portfolio 29
- 30. But access for high-need students still lags behind 30 Funding & Portfolio 41% 67% 41% 53% 43% 70% 41% 65% 0% 10% 20% 30% 40% 50% 60% 70% 80% FRL Non FRL ELL Not ELL Non-White White Below- Proficient Proficient and Above Percent of Students in High-Quality Seats, 2015-16 Source: DPS school enrollment and performance data 09-10 and 15-16
- 31. Key Teaching and Leadership Findings DPS investment in LEAP, LEAD and growth supports for teachers and leaders seems to be paying off Increased rigor in teacher evaluation High retention of strongest performers coupled with relatively higher attrition of lowest performers But high need schools are lagging in teacher and leader stability and quality Turnover and % novice teachers and leaders is much higher in higher need schools ProComp incentives for priority schools compete with incentives for high growth/high performing schools Schools are not consistently maximizing the opportunities provided by the district, particularly around the hiring timeline, and teacher leaders 31 LeadershipTeaching
- 32. DPS’ highest performing teachers are also most likely to stay, which is a leading indicator of teaching quality 1% 19% 19% 23% 18% 0% 5% 10% 15% 20% 25% Aldine Boston Charlotte All Teachers Non-Probationary teachers Pct. Point Difference Between Retention of Teachers Below Effective vs Above Effective 32 Retention of Teachers above effective 86% 89% 89% 91% 92% Retention of Teachers below effective 85% 70% 70% 68% 74% Denver
- 33. However, low-performing schools tend to have a higher share of novice teachers and higher attrition rates 33 14% 6% 54% 33% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Lowest-Performing Quartile Highest-Performing Quartile Percent Novice SY17 and Within School Attrition (SY15-SY17) by Performance Quartile Percent Novice Percent Attrition (2 years) Leadership Note: Turnover adjusted to exclude new and closed schools Teaching
- 34. Key School Design/Standards/School Support Findings DPS has implemented strong enabling conditions and supports for school improvement School-level flexibilities Curriculum, assessments, and instructional support Instructional superintendent support targeted to lowest performers But these conditions do not consistently result in strategic school designs that meet the needs of all students and teachers Lack of consistent, adequate, high-quality collaborative planning time is constraining shifts in instructional practice Inconsistent matching of talent and time to student need within schools is a missed opportunity to improve student outcomes 34 School SupportStandardsSchool Design
- 35. DPS schools have more flexibility than comparison districts 35353535 Metric # Metric Less Strategic More Strategic SchoolDesign Flexibility Schools have flexibility over how they spend their budget, including class size and staffing ratios, and can trade staff positions, positions for $, and $ for positions. Schools have the flexibility to hire teachers whose skills and expertise match school and student needs. Schools have the flexibility to make schedule changes without a contract renegotiation or a full faculty vote. Schools have the flexibility to vary special education service and instructional models as long as they meet IEP requirements. Schools have the flexibility to vary teacher teams, assignments, and schedules with data support in order to provide time for collaboration and match resources to student needs. School SupportStandardsSchool Design Boston Charlotte Lawrence Denver
- 37. Denver’s 2020 goal is even more ambitious than its already impressive gains 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 DPS 3rd Grade reading Scores Actual Continued 2010-2014 trend Needed to reach goal 37Source: CDE website-DPS student performance data, 2004-2014; ERS projection Goal #2: A Foundation For Success in School. By 2020 80% of DPS third Grade Students will be at or above grade level in reading and writing And that’s before accounting for the switch to PARCC
- 38. Reaching DPS’ 3rd grade reading goals will require greater gains than those seen in even the most successful districts 38 80% 53% 60% 88% 73% 45% 60% 39% 51% 65% 59% 41% Denver Aspirations RSD New Orleans Denver 09-10 to 13-14 Aldine, TX Norfolk, VA Lawrence, MA Growth in Elementary Reading Proficiency Start End Gain/year (%) # of Years Enrollment (000) +1 4 14 +1.8 8 34 +2.3 10 65 +2.3 4 90 +2.8 5 30 +3.3 6 90 Source: ERS database, DPS performance data, Denver 2020 plan
- 39. And while system-wide efforts have raised performance for all, significant achievement gaps remain 29% 40% 44% 65% 69% 82% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 White,Non-WhitePercentagePointAchievementGap FRL- Non FRL Achievement Gap, ELA Source: 2004-2014 TCAP data for all tested grades from CDE website Note: we limited this view to TCAP data because only one year of PARCC data is publicly available disaggregated by FRL status to date. Non FRL students FRL students 39 pts 38 pts 29 pts 39
- 40. Targeting support to high-need schools and students will be key to continuing progress and closing gaps Focus portfolio management efforts on maintaining and increasing equity as demographics shift Leverage portfolio management and student enrollment processes to maximize opportunities for high-needs students including strategic deployment of proven models and operators and policies that encourage integration in gentrifying neighborhoods. Leverage talent management to support high need schools Continue work to build structures to help lower performing/higher need schools attract, develop and retain talent, including increasing support for the high number of new teachers in these schools. Support schools in implementing high-quality strategic school designs Improve support for school leaders to implement strategic school designs and strong professional learning practices in order to turn district-wide systems into on the ground changes that improve student performance. 40
- 41. DPS’ journey holds valuable lessons for any district working to transform student outcomes The importance of creating effective system supports through a sustained and integrated redesign approach DPS’ systematic creation of the enabling structures and conditions for high-quality schools across all of the School System 20/20 areas has steadily improved performance in district- run schools and should continue to drive improvement into the future. The power of a system-wide approach to human capital management DPS’ has focused on creating the conditions to attract, develop and retain high performing teachers and leaders across all areas of the system including evaluation, compensation and career paths and professional development. The value of deliberate portfolio management DPS has accelerated overall district performance growth through a deliberate approach to portfolio management coupled with its chartering authority. Notably, this has also avoided the unplanned under-enrollment and performance degradation in district schools that too often accompanies charter growth. The challenge of driving from enabling conditions through to practice Translating strong enabling conditions into school-level changes in practice and resource use requires active support to build capacity and change behaviors. 41
Notas do Editor
- I’d like to start by thanking Tom and the entire DPS team both for letting us do this work and for giving us the opportunity to share it with you. There are some really exciting things happening here – and I know that it stretches beyond just the DPS leadership team to support from all of you as well. I work at Education Resource Strategies. We are a non-profit based in Boston and
- Have worked for over a decade with large urban school systems across the country to help them define their strategic priorities and then align their resources – people, time and money – with those priorities
- Before we jump into the content, I want to orient you to our work and our perspective. We believe that in order to have successful students you first need to have effective instruction And that the best way to ensure high quality instruction that is organized and delivered in ways that meet student needs is to deliberately design schools in ways that have a clear instructional model, and then organize in ways that both promote continuous improvement in instruction AND targeted support for students – this includes things like school culture, but also things like varying instructional time and personalized learning based on student needs. So we believe that schools are the units of change – and that schools need to be organized to meet the needs of their students and teachers. But we also don’t believe that you can get there one school at a time. And that if we want to have great schools at scale – high quality schools for ALL kids, not just some – that we need to fix the systems that these schools exist in – and build systems that give schools the flexibility, capacity and support they need.
- To support this work, we developed the School System 20/20 diagnostic to help districts assess where they are along the spectrum – have they created the enabling conditions at the system level for great school? And then are those enabling conditions translating into high quality practice and the school level? We believe that this can help individual districts by helping them answer these questions, understand strengths, and identify the highest priority changes to make next. And we believe we can help the field by building a data set of how districts across the country are doing to provide comparisons, identify best practices, and ultimately show that redesigning systems to create enabling conditions, and then translating those conditions into practice, drives improved student outcomes.
- The 20/20 diagnostic looks across 7 different areas of district activity, from standards, through teaching, school support, funding. It includes both qualitative questions about policies and practices and quantitative metrics. Wherever possible, we’ve grounded our metrics and our standards for those metrics in available research about what works Allows us to benchmark against other districts and research based best practice. Just a note – for those of you who are familiar with ERS, this diagnostic has its roots in, but is different than our deep resource analysis. That deep analysis, which we did in Denver about 7 years ago, takes about 9 months and today would cost about $500-600K. For reference, this diagnostic work took about 2 months and cost about $60K. So it’s broad, but does not go into the deep detail that we do with our more intensive work – great for diagnosing, but not designed to point to the specific actions. I tell you this because I’m pretty sure you’ll have questions as we go through this – some of which we just may not be able to answer because of the scope of this work.
- We introduced the 20/20 diagnostic – in beta – in 2014. Since then we have completed it in 16 districts across the country. In the majority of these districts they have contracted with us to do the work, as a way to help inform strategic planning or other initiatives, or as the first phase of a larger engagement. But in a few cases we have identified districts that we think are leaders nationally where we want to document their story as a case study that others can learn from. Those include Lawrence, MA, Aldine, TX and Denver.
- The diagnostic allows us to answer some key questions that we’ve listed here What is happening with student outcomes in Denver? What actions has DPS taken that have driven these student outcomes through driving more strategic system conditions and changes in practice and resource use? How does DPS compare to other leading districts nationwide? What are the implications of this information for DPS priorities moving forward? And what lessons can the field learn from Denver’s experience?
- Want to be clear about ERS’ relationship with DPS. DPS has contracted with us in the past, as far back as 09-10, on a variety of engagements. In fact we are engaged in two small initiatives currently. These are independent of the 20/20 work. For the 20/2 diagnostic – DPS did not come to us – we approached them last summer, and this work was funded not by DPS but by the Schusterman Foundation to develop a best practice case study.
- Let’s start with performance. Last year, Stanford developed a database that looked at state level high stakes exams and normalized them using state-level NAEP data in order to provide, for the first time, a comparison of district performance nationally. This data exists right now for only 4 years – 09-10 to 12-13. Their plan is to add new years as they become available through NCES. But that means we can look at how Denver compares to every district in the country. For simplicity, we looked just at districts over 25K enrollment
- This chart shows growth between 09-10 and 12-13 for all districts >25K (300) – the horizontal axis is % FRL, as an indicator of student need, the Y access is growth in terms of grade level equivalent units. What this shows is that Denver had the second largest growth in this time period – second only to Lincoln NE – and student need in Denver is clearly much higher. We’ve also labeled the other highest growth districts for reference. In particular, DPS performance on average x grades and subjects, went from 1.5 grade levels behind to .5 levels behind – or growth of almost a full grade level in 4 years. And we don’t have the absolute performance here, but that took Denver from lagging behind the average performance pretty significantly for districts at this level of need to above average. Method/Data: Stanford uses NAEP scale score to adjust state data to give an average score for all districts. They then convert this into grade equivalent units (helped because NAEP sets a standard for each grade level). We then rolled this up to the average across grades. Jordan is likely the best source on further info here.
- That’s the national picture – and it mirrors what you’ve already seen within CO, where DPS has been steadily closing the gap with the state. For now I’m just showing results up to 2014, which is when you made the switch to PARCC. We’ll look at the post PARCC result in a minute, but there just isn’t the detail available yet, particularly for the most recent 15-16 year.
- So the growth is impressive just looking at national and state comparision, but what makes it even more remarkable is how pervasive the improvement was – so we saw improvements across all subgroups – this shows non-white students, FRL students and ELL students…
- And across both charter and district run schools – charters generally outperform district run schools slightly, but you can see both are on an upward trajectory. Dark blue is for all students, light blue is FRL students. This is worth just taking a minute on – because it is different than what we often see. It’s not unusual to see charters outperforming district-run schools, especially in states with tight chartering systems like here, MA. But it is unusual to see charter performance steadily improving AND to see district schools steadily improving at the same time. I feel like a lot of the story in Denver has been about the unique charter environment and approach. And while I don’t want to underestimate both the importance of that for DPS students and for the national conversation about systems and charters – and we’re going to talk more about that – I walso want to emphasize that there is a really important and impressive story around what is happening with district run schools in Denver as well and the steady, sustained improvement we’re seeing there…and much of what we’re going to talk about today is what is driving that. We tried to tease out the dynamics of how district run schools and charter schools have contributed to outcomes to date in DPS.
- We tried to figure out the best way to look at this without investing a huge amount of effort and without access to longitudinal student data. Let me walk you through what we did. I think this gives you a good high level look at what is going on. For the academics in the room I don’t think this would stand up to a rigorous statistical test. As Robin Lake put it – this is being creative about using the data you have to try to understand what is happening. We know that overall proficiency in DPS across all schools increased from 49% to 54% between 2008 and 2014. So what we’ve done here is break that growth into 3 buckets. The top bucked here is charter school students. 8% of students were in charters in 2008, proficiency in those schools started higher than in district run schools and improved about 4 points – so that accounts for about 8% of the total growth The remaining 92% of students were in district run schools in 2008. Those schools also improved about 4 points, representing about 87% of the total growh But in 2014, there were 15% of DPS students in charter schools, so 7% of students (on average) shifted from district run schools to higher performing charter schools. That added about another 5%. I know this feels different perhaps from other data that you’ve seen. I think there are a few reasons for that. First, a lot of the analysis I’ve seen on this in the past is looking only at new school performance, where as this looks at all schools. Second, this only goes through 2014 because we can’t compare pre-2014 to post 2014 because of PARCC. Since 2014 the % of enrollment in charters has increased even more, so both those yellow and the light blue sections will be bigger if we look since 2014 or look out the next 5 years, because the share of students in charter schools has increased. But at the same time, it is a great reminder that in DPS – there is wide improvement across a large number of schools, including district run schools. And that is one of the things that is so exciting for us.
- Last I just want to point out that even though DPS has a much higher need population than the state on average,
- Up until 2014 was closing the gap not just overall but for most subgroups and as of 2014 was outperforming the state average with white students.
- As I mentioned before, we don’t have national data beyond 2013 and we don’t have detailed PARCC data for 2016. But there are indications that DPS has maintained or even increased in’s improvement trajectory since 2013. You’ve probably seen similar data. I know this isn’t exactly the way the state reports data – they report percentile based on scale scores. We didn’t have access to that data – so this was the way we could look at the data over time – going back to 2004 and all the way up to 2016 – this is % proficient. You can see that relative to the state, DPS has been improving faster since 13 than from 09-13. This is think reflects data that A+ and Van have published as well. From 09-13 they closed the gap 2 % points, since 13 they’ve closed the gap 12% points and are now within 5% of state average proficiency. I don’t believe subgroup data is available. There are a couple of caveats – some higher performing students in CO opted out of the exam, which may be pulling the state average down, and
- FRL levels in DPS have started to come down as more affluent families are coming back into the city, while the state level has remaind steady. But it doesn’t feel like these two things could be enough to explain such a massive move.
- As I mentioned up front, we’ve identify key shifts that we believe systems need to make in order to create the enabling conditions for great schools at scale
- And DPS has been working steadily for a long time to systematically tackle many of these big shifts, starting with the introduction of ProComp and WSF, to the rollout of LEAP and LEAD, and the expansion of school flexibilities, to the most recent initiatives around teacher leadership, expanding flexibilities even more, increasing equity of access and integrating the accountability systems across district run and charter schools. I think the big takeaway here is that it isn’t just one thing, or one area, and it isn’t easy or fast. It’s having a vision of what the role of the district is that will best promote equity and excellence and then steadily and systematically putting those things in place. And I see that in DPS as really 3 key things: Giving schools the flexibilities they need to make the best decisions for their students and teachers And then figuring out what else schools need to be successful – Teacher evaluation and compensation Common core curriculum and incentives Teacher leader positions Timely high quality data Sharing best practices Different for different schools Building the mechanisms to ensure equity – great options for ALL students, not just some. That’s things like funding, but also accountability systems and why teacher eval is so importanttrying to develop incentives for great people to work in high need schools.
- This is our System 2020 scorecard and you can see the results of this hard work over the past 8 years. We used 09-10 because that’s when we did our deep work in Denver so we had the data fro then to use as a comparison. But across all areas of system conditions and practice and resource use DPS has seen measurable improvement.
- The result - the most strategic enabling conditions – this looks at the highest scoring districts of the 16
- And even thought P&RU is lagging, still among the highest.
- Want to take you through the highlights of our findings in three areas: Funding and school portfolio management; Human capital – teachers and leaders; and school support and what we can school design I will start by saying that I don’t think we found anything that was materially different from what DPS already knew or you’ve seen. But I think having the national comparison, and being able to look broadly across the whole system, instead of in one specific area, is helpful both for understanding where DPS has been and for helping to think about where they need to go.
- These are the key qualitative metrics that we look at in terms of portfolio management, and we’re comparing DPS here to Charlotte, Boston and Lawrence because of the 16 districts we’ve studied, those are the districts that score highest other than Denver in this area, and Lawrence in particular is often held up as another strong portfolio district. Many similarities. But DPS is – of the districts we’ve studied – gone the farthest in terms of a single funding, enrollment and accountability system. And I think unique in the country.
- If we look at results we see increases in the number of high quality seats across all regions. HQ seats defined as blue or green in the SPF framework. And what’s more, DPS changed the criteria last year and made it harder to get a blue or green rating, and these increases are despite that higher bar – so in fact we’re probably understating the growth. Also worth noting is that the biggest jumps are in the regions with the lowest % of HQ seats. Those regions are still lagging, which is a big issue and we’ll talk bout that more in a minute, but still efforts are focused in the right areas.
- I also mentioned more similar needs between charter and district run schools. Often we see charter schools serving lower need populations. What we see in DPS is that there seems to be less difference. I do want to say that these measures are imperfect. First, every student in a charter school has chosen to be there, while there are still significant #s of students in district run schools who did not choose their school, despite ongoing efforts. Second, these measures don’t necessarily reflect the full depth of student needs – for example, there can be a wide variation in the economic situation of student classified as FRL. And if you look at the % direct cert, which is a more rigorous definition of economic disadvantage/poverty, you can see that district-run schools are serving a higher % But having said that, these numbers do indicate much more similarity between districts and charters than we see in most districts and indicates that much of the work that DPS and charter schools have been doing to try to improve equity of access is paying off
- Having said that, we are still a long way from equity of access in Denver. If you look at the % of students in high quality seats by student subgroup you see across the board that higher need students have less access to HQ seats than lower need students. If you recall the HQ seats by region slide – this is largely driven by differences in school quality by region, and by the fact that even the best access programs aren’t enough to overcome real barriers – both physical and other – to students traveling outside of their local area for school. This is not news – but just reinforces some of our other findings that I’ll review next about the importance of doubling down on these underserved regions.
- Turning to Human Capital, which has been the second important cornerstone of DPS’s system transformation approach. Key findings here: DPS has invested a lot of time and effort in developing systems and structures to attract, develop and retain high quality teacher and leaders into the district. From ProComp through the LEAP and LEAD evaluation systems and investments in professional development and teacher leadership. In addition to these programs being successful in terms of increasing the rigor of evaluations and improving strategic retention, which we’ll talk about in a minute, the Process that DPS used to develop and rollout many of these programs is worth mentioning as well We’ve been impressed with both the commitment of DPS leadership to engage key stakeholders – in the case of LEAP, teachers and school leaders Also procomp – survey and workshops to get teacher feedback. – in the development and rollout process, and the commitment to a cautious and data-driven approach to implementation. Teacher leader – introduced, piloted, good feedback, so rolled out more broadly. GO TO NEXT SLIDE But at the same time, we are finding that, like in every other district, the distribution of teacher quality is not even. And high need, low performing schools struggle to attract and retain high quality teachers and leaders. So, we see turnover is higher at lower performing schools and % novices is much higher. This is currently exacerbated by the ProComp incentive structure – and both DCTA and DPS know that this needs to be fixed and are working on that. But there is a $2500 incentive for working in a “hard to serve” school but a $5K incentive for working in a high growth/high performing school. So right now this incentive structure is working at cross purposes to what we want to happen And while the district has made some changes recently to try to help schools – for example changing the process to allow schools to hire earlier in the year, and providing teacher leadership positions (and I understand this program is being expanded for next year, focused on the highest need schools), there is wide variation in whether and how schools are taking advantage of this. And while we don’t know this for sure, it wouldn’t be surprising that schools with less stability in leadership and teachers are going to be less able to take advantage of some of these more “advanced” strategies
- Here we are showing what TNTP first coined and we’ve adopted as “strategic retention” – this is the difference in the retention of your highest performers and your lowest performers. You want to retain as many of your high performers as possible – DPS is doing that better than other districts we’ve studied (and these are the three highest) over 90% of your best teachers are staying each year – which is fantastic. And you want your lowest performers to either improve or leave, and what we see in DPS is a high % of teachers rated below effective leaving each year. This is a virtuous cycle – as you keep more high performers, and lose more low performers, average quality should improve. Therefore we view this as an important leading indicator of student outcomes. The one thing to keep in mind is that in DPS, like in many districts we see, not a huge number of teachers are rated below effective, and most of those are probationary teachers who are rated “approaching”. So in order to keep seeing the benefits of this retention trend, it’s important to make sure that schools are continuing to be really rigorous with their evaluation ratings. BACK TO PREVIOUS SLIDE
- This is a similar pattern to what we see everywhere – the lowest performing, often highest need schools have difficulty attracting and retaining high quality teacher and leaders. The blue bars here represent the % novice (3 years or fewer experience as a teacher or principal) teachers and leaders in the lowest performing quartile of schools vs. the highest. And you see a much higher percentage of novices in the lower performing schools The green bars represent turnover – so this is % of teachers or principals leaving over a 2 year period – and again, not surprisingly, you see much more stability in higher performing schools. We don’t have school level evaluation data, but I wouldn’t be surprised to see a similar trend here. The district has tried to put in place supports to address this – TLC was launched to better support new teachers, and my understanding is that this is being expanded in the highest need schools next year. There are programs to support struggling teachers; DPS and DCTA are working trying to address the ProComp incentive issues…but this feels like an important area to continue to focus going forward.
- The last area I want to turn to is around school support. The way our 20/20 framework works the different types of schools supports actually fall into 3 different categories, standards and instruction, school support and school design. So the story here is that DPS has again done a very good job in both increasing flexibilities for schools. GO TO NEXT SLIDE They’ve also invested in building supports in some key areas. The process that DPS put in place to develop new Common Core aligned curriculum, assessments and PD is a great example of this. Again, they didn’t try to do this work in a vacuum – they enlisted teachers and school leaders to do this work, but did it in district wide way so that each school didn’t have to figure it out themselves. And if the PARCC results are any indication it worked really well. Finally, the district has invested in differentiated supports that are managed through instructional superintendents to provide more supports to struggling schools, and less to schools that need it less. DPS has among the lowest instructional suprervisor spans of control of any district we’ve studied, and deliberately staffs to have even lower ratios for turnaround schools. This is important because of the second finding -
- On flexibilities, which is a key part of the theory of action – again, Denver gives schools more flexibilities than anyone else we’ve look at, including places like Lawrence that are well known for “school autonomy”. And I know that the team is looking for ways to expand these flexibilities even more GO BACK TO PREVIOUS SLIDE
- Looking forward – DPS has set a goal of 80% of 3rd gade students above grade level (proficient on the state exam) by 20/20. That is admirable, and I wouldn’t necessarily suggest changing it, but I also think it’s important to put it in perspective. DPS had the 2nd highest growth of any district in the country from 09-10 to 12-13. And we think they are continuing along that trajectory or even accelerating…which is good because you would need to accelerate pretty significantly to meet the goal. And that is before the switch to PARCC, which raised the bar and therefore probably indicates the need for a recalibration of the goal.
- We’ve also looked at the best district success stories that we can find anywhere, over the past 20 years, across the country. So these are the largest annual increases in performance – you can see DPS 09-14 trajectory was the second highest we could find – second only to the RSD in Louisiana, which is a bit of a special case given both how low they started off and what happened with Katrina – where they were really starting in a clean slate that it so different from anywhere else. DPS would need to exceed the growth they saw there for the next 6 years. If anyone can do it, DPS can – but at the same time, it would be an amazing accomplishment
- Another important thing to look at going forward….we think a lot at ERS about the tandem goals of excellence AND equity – so we want to close achievement gaps while improving performance for all. It’s not good enough just to get currently lagging student groups up to mediocre. What’s great about DPS is you have raised achievement for ALL GROUPS. The downside is that because lower need students are growing as fast as higher need, achievement gaps haven’t yet begun to close. The key for DPS going forward will be to work to start closing those gaps while continuing to raise the bar for the lower need students – and thereby keeping those students in the district. Because we know that more integrated schools and systems tend to be more equitable and higher performing…and because that maximizes the financial resources available to all of the students of Denver.
- So where do we think the biggest opportunities are for DPS? It is squarely in the place we’ve just talked about: the journey to date has been around creating enabling conditions system wide. DPS needs to continue those efforts but then really focus on the lowest performing schools and the highest need students in three critical areas: Continue to work to increase high quality options in high need areas AND improve access for higher need students in higher performing/gentrifying areas. This will require thinking creatively about perhaps reserving spots in public schools for ED students similarly to how charters do, and looking for other ways to maintain a critical mass of poor and minority students in schools in neighborhoods where demographics are shifting. Really think about how DPS can attract and retain talent in the highest need schools and how to support especially new teachers in these schools ProComp work going on right now to fix the incentive issue Also expanding new teacher support with more TL’s and working together to think creatively about other supports – teacher comp ide of cutting back on preps or sections for those first couple of years in tough schools Working with school leaders to hire earlier Looking at other types of incentives such as moving pods of people to these schools Continue to evolve how the instructional superintendents support school leaders in development and implementing strategic school designs
- And then, stepping back, we think there are some really important lessons for the field from your great work here in Denver. The first and most important lesson is how important it is to take a system-wide approach. There is not just one thing – not choice, not charters, not school autonomy, not teacher evaluation, not tutoring. If we really want to improve outcomes for ALL students, sustainably, we need to build a system that both enables AND supports ALL schools to do what needs to be done for students. Critical to that system work is developing the structures and processes to support attracting, developing, and retaining high quality talent. I want to be clear, we don’t think that HC should be managed centrally – we believe it’s critically important for schools to be able to hire and organize the people they need, the way they need, but there are important things that the system can and I would argue must do to help schools do that including Working with unions to develop compensation systems– particularly for teachers – that professionalize the job and recognize and reward teachers for excellence, continuous improvement and contribution Making it possible for schools to implement strong, team-based, job-embedded professional development including: Ensuring all schools can have CPT Ensuring all schools have expert support Ensuring all schools have the data and training they need Developing a strong pipeline of teachers and leaders both internally and externally, including working with teacher certification bodies – local colleges and universities and alternative certification programs – and leadership development within the district and figuring out ways to make the hiring timeline happen early enough in the year to get the best teachers Equity - Working to figure out how to get great teachers and leaders in ALL schools We’ve talked a lot and others have talked a lot about portfolio mgmt. in Denver and the inclusive approach to accountability and charters. I can’t say enough how devastating the current way this process works in most places is to struggling districts, and hope that Denver can provide a path forward that puts students first Finally, we have been interviewing many of our district partners recently about what tools and information they would find most useful and one person said “I don’t like case studies – they never talk about the messiness”. So two things – the first one is to acknowledge both how hard this is – I’m sure the DPS team can attest to that, and for all of these successes we’ve discussed to day I know there have been other initiatives that didn’t work, or times when even these felt like Second, it’s not just enough to create the enabling conditions – to give schools flexibilities and then set them free. That works in some cases – but not in enough, fast enough – which brings us back to the role of the system to figure out when and how to step in to support schools that need it, while giving the schools that don’t what THEY need.
- And I’d just like to close the way I opened, by thanking you and the DPS team.