Dr. William Allan Kritsonis, Dissertation Committee for Dr. Arthur L. Petterway, PhD Program in Educational Leadership, Prairie View A&M University/Member of the Texas A&M University System.
Dr. Arthur L. Petter, PhD Dissertation Defense, Dr. William Allan Kritsonis, Dissertation Committee Member
1. A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas A Dissertation Defense by Arthur L. Petterway William Allan Kritsonis, PhD Dissertation Committee Member May 04, 2007
2.
3.
4. Theoretical Framework A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas Q U A N T I T A T I V E Q U A L I T A T I V E % ELLs Enrolled In a High School Views/ Opinions Administrators Teachers District Personnel % All 10th Grade Students Passing TAKS ELA/ Math Impact of Statewide Testing On ELLs Student Performance Explanatory Design May 04, 2007
5.
6.
7.
8.
9.
10.
11.
12.
13. Major Findings: Descriptive Statistics English Language Arts 78.05 6.94 2006 59.39 7.62 2005 68.28 8.29 2004 62.87 8.30 2003 Percent of All Students Passing 10 th Grade TAKS Percent of ELLs Enrolled in Schools Year
14. Major Findings: Descriptive Statistics Mathematics 50.13 6.94 2006 47.68 7.62 2005 53.57 8.29 2004 61.85 8.30 2003 Percent of All Students Passing 10 th Grade TAKS Percent of ELLs Enrolled in Schools Year
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42. A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas Q U A N T I T A T I V E Q U A L I T A T I V E % ELLs Enrolled In a High School Views/ Opinions Administrators Teachers District Personnel % All 10th Grade Students Passing TAKS ELA/ Math Impact of Statewide Testing On ELLs Student Performance Explanatory Design May 04, 2007
Notas do Editor
Good afternoon! I am privileged to present my dissertation on: “ A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas”.
This study used the explanatory design of mixed methods analysis to : 1.Display the performance of English Language Learners in the high stakes testing called the Texas Assessment of Knowledge and Skills or TAKS; and, 2.Determine the impact of this same high stakes testing on English Language Learners as viewed by district personnel, school administrators and teachers.
The purpose of this study was to determine the impact of high-stakes testing on English language learners.
The following research question guided the quantitative dimension of the study: Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006?
Two null hypotheses were formulated in line with the quantitative research question; the first null hypothesis dealt with English Language Arts: 1.There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in English Language Arts given in 2003, 2004, 2005 and 2006.
The second null hypothesis dealt with Mathematics : 2. There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in Mathematics given in 2003, 2004, 2005 and 2006.
The subjects of the study included the following: I. Quantitative – 173 Urban High Schools in Texas II. Qualitative - Six principals responded, together with 9 assistant principals, 6 ESL district personnel, 15 ESL certified teachers and 19 non-ESL certified teachers – a total of 55 out of 98 purposive respondents, for a response rate of 56%.
Quantitative data were accessed and retrieved from the TEA website regarding the major urban high schools in Texas. Data were organized for computations utilizing the SPSS software package, Version 14.0. The on-line, open-ended questionnaire provided one of the bases for the qualitative data. The triangulation method included categorizing the responses to the online, open-ended questionnaire into emergent themes, interviewing the focus groups of teachers and assistant principals and one- on-one interviews with the principals and district ESL personnel.
For the quantitative portion of the study: The Descriptive Statistics were used to show percentage of English language learners during the four years under study and the corresponding performance of 10 th grade students (in terms of means or averages) in the TAKS test . Pearson r Correlation Coefficients were then calculated to determine if there was a significant relationship between the percentage of Ells enrolled in a school and the percentage of all 10 th grade students passing the TAKS test in the core areas of English Language Arts and Mathematics. The simple regression analysis yielded the linear equations which can be used to predict the outcome variable of percentage of all 10 th grade students passing in English Language Arts/Mathematics given the predictor variable of percentage of Ells enrolled in a school .
For the regression analysis, the percentage of ELLs enrolled in a school was considered as the predictor variable. The outcome variable was the percentage of all students passing the 10 th grade TAKS test in ELA and Mathematics.
Although the descriptive statistics did not show that when the percentage of ELLs enrolled in a school decreased there was an expected improvement in performance of all students passing the 10 th grade TAKS, it was in the computation of Pearson r correlation that results showed that there was significant relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing in the 10 th grade TAKS in ELA and Mathematics. All of the obtained Pearson r Correlation coefficients were significant at the 0.05 level (two-tailed).
The qualitative research question focused on this main concern: What are the anticipated and observed consequences of the statewide testing specifically TAKS, on ESL curriculum and instruction as viewed by ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
An online open-ended questionnaire was developed from the existing review of literature. Two schools not included in the study were selected for the subjects’ expert opinions while at the same time establishing the trustworthiness of the instrument. Upon completion of the pilot study, the instrument was determined to be trustworthy. (In order to build trustworthiness, four criteria were considered: (1) CREDIBILITY, so that findings are believable and convincing ; (2) TRANSFERABILITY, so that findings can be applied to other settings; (3) DEPENDABILITY, so that findings are consistent with those of other similar investigations; and, (4) CONFIRMABILITY, to ensure that both the processes and the product are auditable (Isaac and Michael, 1995).)
The major research question for the qualitative portion of the study is supported by the following questions. The subsequent frames will provide the views and opinions of the respondents. Themes or categories emerged from these responses. Authors were also cited in concordance with the given probes. 1.Why is TAKS given as a statewide test? 2.What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3.What problems have occurred related to or because of TAKS? 4.What changes were caused by this statewide testing? 5.What are your recommendations to improve this statewide testing? 6.What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test?
The first question asked was: Why is TAKS given as a statewide test? Responses given include the following: TAKS is given as a tool to gauge knowledge in the core areas. TAKS is considered as a means to determine the school’s status (Exemplary, etc.). TAKS is a means to assess the state curriculum or standards. This statewide test is mandated by law and is aligned with NCLB. Abrams and Badaus (2003) and Madaus and O’Dwyer (1999) cited reasons (in italics) why tests are given.
The second question asked was: 2. What are the intended consequences of this statewide testing? Respondents had these views: statewide testing is intended to eventually result in ELLs performing as well as the rest of the students; ELLs can improve academically and eventually join the mainstream; and TAKS is a requirement for graduation. Abedi (2003) and Harlow and Jones (2003) gave reasons why for the reasons why high-stakes tests are given.
3. What problems have occurred related to or because of TAKS? Respondents identified the following as problems encountered by ELLs due to TAKS: (1) higher dropout rate; (2) decrease in graduation rate for ELLs; and, (3) lower self-esteem of ELLs
Rothstein (2002), Jacob (2001) and Anderson (2004) mentioned problems related to high-stakes testing: drop-out rate.
The fourth concern or issue posed was: What changes were caused by this statewide testing? The predominant responses were as follows: Schools experienced the negative reality that: There is a high failure rate among ELLs. Lower self-esteem and exasperation on the part of ELLs. High pressure on the school and ELLs. Too much emphasis is placed on test performance.
Anderson (2004), Lane and Stone (2002) and Flores and Clark (2003) affirmed the answers of the respondents.
When asked the question: What are your recommendations to improve this statewide testing? The respondents gave two major recommendations: (1) deferment of the test, possibly a different but fair test; and, (2) better assistance from the school through the teachers and the curriculum, modifications in teaching and possibly a paced curriculum for ELLs.
Anderson (2004) gave the positive and negative consequences about high stakes testing in his own study about ELLs.
A final question asked in the qualitative portion of the study was: What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? The recommendations seen as more helpful were: Specific interventions for ELLs; Quality instruction; A more intensive English program; and, Help from home and meaningful tutoring in school. Flores and Clark (2003) and Popham (2003) cited proper focus on ELLs regarding statewide testing in order to undertake specific action plans to help ELLs.
Implications or what should be considered in dealing with ELLs are: 1.Performance of schools in high stakes testing is affected by size and proportion of ELLs taking the test. 2.Consider extended deferment of standardized tests administered in English to ELLs. 3.Learning is transmitted through communication. 4.More active role of LPAC in monitoring ELLs in schools. 5.Specific action plans to improve situation of ELLs.
Implications or what should be considered in dealing with ELLs are: 1.Performance of schools in high stakes testing is affected by size and proportion of ELLs taking the test. 2.Consider extended deferment of standardized tests administered in English to ELLs. 3.Learning is transmitted through communication. 4.More active role of LPAC in monitoring ELLs in schools. 5.Specific action plans to improve situation of ELLs.
Based on the results of the study, the researcher recommends further study on the following: 1.Additional supports to ensure ELLs to pass high stakes testing. 2.Identify what data are needed to make fair decisions about ELLs. 3.Determine reasons why ELLs scored lowest among student groups in TAKS. 4.Explore different approaches in campuses in dealing with ELLs in terms of curriculum and instruction.
Based on the results of the study, the researcher recommends further study on the following: 1.Additional supports to ensure ELLs to pass high stakes testing. 2.Identify what data are needed to make fair decisions about ELLs. 3.Determine reasons why ELLs scored lowest among student groups in TAKS. 4.Explore different approaches in campuses in dealing with ELLs in terms of curriculum and instruction.
Additional recommendations for further study include the following: 5.Compare performance of ELLs vs. non-ELLs based on different objectives of TAKS. 6.Impact of high stakes testing as viewed by parents and students. 7.Explore different instruments to measure performance of ELLs. 8.Determine if there is significant difference in performance of different student groups in statewide testing.
Additional recommendations for further study include the following: 5.Compare performance of ELLs vs. non-ELLs based on different objectives of TAKS. 6.Impact of high stakes testing as viewed by parents and students. 7.Explore different instruments to measure performance of ELLs. 8.Determine if there is significant difference in performance of different student groups in statewide testing.
This study affirmed the expected outcome that a significant relationship existed between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in both core areas of English Language Arts and Mathematics. The regression analysis predicted that as the percentage of ELLs in a school increased, the performance in the statewide, high-stakes testing in terms of all students passing the 10th grade TAKS tests decreased. The respondents of the study considered the Texas Assessment of Knowledge and Skills (TAKS) as a tool to gauge knowledge in the different core areas. English language learners were expected to have at least average scores on TAKS. There was a difference in the expected and actual results; respondents observed dismal or failing performance of ELLS in the actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. Higher dropout rate and lower graduation rate of ELLs were problems encountered due to TAKS. Respondents favored a different test for ELLs, possibly given at a later date after ELLs had studied in the country for at least several years. Respondents believed that interventions were needed to help ELLs perform better. Both the school and the home, together with the community have to be involved in preparing ELLs to be better prepared for their present and future roles in the American society.