What We Know

What Do We Know?
SEEKING EFFECTIVE MATH AND SCIENCE INSTRUCTION

THE URBAN INSTITUTE

February 2005 As requested by the GE Foundation, the with relatively credible evaluations to pro-
main goal of this review was to identify vide some choice to districts and schools
mathematics and science curricula as well that wanted guidance in selecting a cur-
as professional development models at riculum and that wished to use effective-
the middle and high school levels that ness as a selection criterion.
had been deemed effective based on their
success in increasing student achievement.
Because of its emphasis on student achieve- Evaluation Study Selection
ment, this approach differs from that Criteria Used
typically used at the local level and that
adopted by other recent studies conducted To be included, studies had to have
to find “effective” curricula and profes- 1. Rigorous methodological design
sional development models. including comparison groups;
Historically, curriculum choice at the 2. Measures of impact on student
local level has often been made by a com- outcomes that included, but were
mittee that decides which curriculum to not limited to, test scores; and
adopt based on considerations only periph- 3. Comparative data with experi-
erally related to student achievement—such mental or quasi-experimental
as state-imposed standards, recommenda- design preferred.
tions of others, cost, and presentations by
publishers’ representatives. Choice of pro-
fessional development models follows a As can be seen from the criteria above,
similar pattern. Indeed, there has been very we chose increases in student achievement
little else available to guide school districts as the measure of effectiveness.2 We limited
in their curriculum selection process, since our review to evaluations that compared
for most curricula and textbooks the only student achievement outcomes elicited by
data at hand are publishers’ figures on the curriculum being studied with student
the number of adoptions. That has been achievement outcomes of another curricu-
changing. There is a growing movement lum. Where data were available, we also
to assess the effectiveness of math and took into account the content validity of the
science curricula through various meth- assessments used as well as the statistical
The Urban Institute
odologies, including content analyses, significance and the effect size of the differ-
Beatriz Chu Clewell
comparative studies, case studies, and syn- ences reported. Our review made a special
Clemencia Cosentino de
thesis studies.1 And while there have been effort to identify eligible studies that dis-
Cohen
several studies of the effectiveness of pro- aggregated results by performance of stu-
Nicole Deterding
Sarah Manes fessional development practices, very few dent subgroups. Our review of evaluation
Lisa Tsui have measured the effects of these practices studies of professional development was
on student achievement. similarly limited to studies that used
Campbell-Kibler The focus of this review, reflecting the student achievement as the measure of
Associates, Inc. GE Foundation’s interests, was to identify effectiveness.
Patricia B. Campbell current math and science curricula and
Lesley Perlman professional development at the middle
Professional Development:
Shay N.S. Rao and high school levels that showed evi-
Becky Branting dence of positive impact on student
What We Found
Lesli Hoey achievement. Our goal was to come up We identified 18 evaluation studies of pro-
Rosa Carson with enough math and science curricula fessional development models that used

1

What Do We Know? Seeking Effective Math and Science Instruction

student achievement outcomes as both change in the teaching mathematics curricula that were
measures of effectiveness. Features of behavior of teachers and change developed as part of whole school
these studies can be found in a full in the classroom environment. reform efforts. A total of 156 studies
report of this review.3 We found that One study found that behavioral of student mathematics achievement
change was only evident after with comparison group data were
Providing professional develop- teachers had received a minimum found for 18 of the curricula (20 per-
ment for teachers of standards- of 80 hours of intensive profes- cent of the total number of curricula
based science curricula is sional development; change in originally identified). A list of the
effective in increasing student the classroom environment was 18 mathematics curricula, together
achievement. There is evidence achieved after 160 hours.4 It is with overviews of the curricula and
from the research that standards- important to note that at the indi- their impact studies, appears in
based science programs that pro- vidual level, teacher preparation in Appendix A.
vide professional development to the content area was an important The following findings relate to
teachers produce higher student factor in influencing effectiveness. methodology:
achievement outcomes than those Some widely held beliefs about
that do not. what constitutes effective profes- Most middle and high school
Professional development that is sional development are not sup- mathematics curricula do not
tied to curriculum, to knowledge ported by research linked to have studies of student achieve-
of subject matter, and/or to how student achievement. ment with comparison groups
students learn the subject is more that can be found through litera-
● Myth: Professional development
effective in terms of improving ture or web searches. Only three
should be distributed over the
student achievement than is of the studies found specified the
school year. While professional
professional development that curriculum to which the target cur-
development in science seems
focuses only on teaching behav- riculum was being compared. The
to benefit from distributed time,
iors. A number of studies have rest compared their curriculum to
the same effect has not been
concluded that the content of pro- some unnamed curriculum, mak-
found for math.
fessional development—what is
● Myth: More is better. There is no ing comparisons across curricula
taught—is more important than its impossible.
evidence to support the belief
format—how it is taught—and that
that more contact time over and If students are going to be judged
content should be linked to subject
above 80 hours and 160 hours on the results of an external test,
matter knowledge, a specific cur-
results in changes in teaching the mathematics curriculum
riculum, or the process of student
behavior or classroom environ- selected should cover the areas
learning. For example, some pro-
ment, respectively. and skills that are included on
fessional development focuses on
● Myth: Classroom visits are neces- that test (i.e., the test and curricu-
training teachers to model specific
sary. Visits to classrooms for lum should be aligned). As would
pedagogical approaches and skills
consultation or coaching as part be expected, students tend to score
using generic lesson formats. The
of professional development do higher on tests focusing on the
goal of these efforts is to get teach-
not necessarily result in greater content and skills covered in their
ers to change the way they teach,
student achievement. curriculum. The content and skills
expecting that this will increase
● Myth: Schoolwide programs are covered in traditional math cur-
student achievement. More effec-
more effective. Providing services ricula tend to reflect the content
tive professional development
to whole schools rather than and skills covered in most stan-
focuses on increasing teacher
individuals may not be the most dardized and statewide tests.
knowledge of the subject matter,
important element of profes- Alternatively, the content and
on a specific curriculum, or on
sional development that leads to skills covered in standards-based
the student learning process. The
increased student achievement. curricula tend to better reflect that
goal of these types of efforts is to
change teacher knowledge, ex- which is covered in standards-
pecting that this will, in turn, Mathematics and Science based tests. (For more on this
both change teacher behavior and Curricula: What We Found issue, see the text box, “Inter-
increase student achievement. preting the Results.”)
The amount of professional Mathematics Curricula
development provided is an Our review netted 89 middle and The following findings relate to
important factor in influencing high school curricula, including eight outcomes and impacts:

2


Studies of six of the curricula ricula and the results of impact stud- more effective than traditional
found that students who use the ies, is provided in Appendix B. science curricula as measured by
curriculum being tested scored The following findings relate to student achievement. The prepon-
higher than comparison students methodology: derance of evidence provided by
on both a majority of standard- meta-analyses and evaluations of
ized and/or state tests used and As with mathematics curricula, individual curricula seems to con-
on a majority of the curriculum- most middle and high school cur- firm that inquiry-based science
based tests used. These curricula ricula do not have evaluation curricula produce larger effects
were Cognitive Tutor, Connected studies of student achievement on student achievement than do
Mathematics (CMP), Interactive with comparison groups that can the more “traditional” science
Mathematics Program (IMP), be found through published liter- curricula.6
Prentice Hall: Tools for Success, ature or web searches. However, It is difficult to determine the
Saxon Math: An Incremental in contrast to the mathematics cur- effect of these science curricula
Development, and University of ricula in Table 1 in Appendix A on different subgroups of stu-
Chicago School Mathematics (many of which had been the sub- dents—such as girls, minority
Project. Because students using jects of multiple studies), most of group members, and urban stu-
these curricula scored higher on the science curricula in Table 2 in dents. Very few of the curricula
both types of tests, it is reasonable Appendix B had only one eval- had studies that met our criteria
to assume that the effectiveness of uation or study, usually unpub- and disaggregated their findings
each curriculum is not an effect of lished works available through the by sex, language minority status,
developer. or urban location. Surprisingly,
the type of test used.
none of the studies reported data
It is difficult to determine any
The following findings relate to disaggregated by race/ethnicity.
differential effect of these math
outcomes and impacts: Most science curricula include a
curricula on girls and boys. Very
professional development compo-
few of the curricula had studies
Science curricula based on the nent. Inclusion of a professional
that met our criteria and disaggre-
inquiry approach are consistently development component as a part
gated their findings by sex. Among
those that did, the results were
inconsistent.
With the exception of Connected
Mathematics, too few results per Interpreting the Results: Validity of Tests Used
curriculum were broken out by To Measure Student Achievement
race/ethnicity to allow us to draw The evaluation studies for mathematics and science curricula on our lists
general conclusions regarding used a variety of assessments—some standardized, some state-mandated,
racial/ethnic effects. Connected some self-developed, and some developed by others specifically for stan-
Mathematics, however, appeared dards-based curricula. In the case of mathematics curriculum studies, the
to reduce racial/ethnic gaps. content and skills covered by most standardized achievement tests tend to
reflect more closely the content and skills covered by traditional mathe-
Science Curricula matics curriculum than those covered by standards-based curriculum,
causing them to have better “content validity” for traditional curriculum.
We identified 80 science curricula at Many researchers of standards-based curriculum are aware of this and
the middle and high school levels.5 develop their own student achievement tests that more accurately test the
Similar to the mathematics cur- skills and content of standards-based curriculum.
riculum, we found seven science If students taking one curriculum score higher than others on both
curricula that had been developed types of test, there is no question of interpretation. Beyond that, however,
specifically as a part of whole school judgments of efficacy must take into account the content validity of the
reform. A total of 45 studies of stu- tests in order to determine which type of curriculum is more effective.
dent achievement in science were Although science curriculum studies face a similar dilemma, several stud-
found that met our criteria, covering ies comparing the results of standardized tests with those of other forms of
21 (or 26 percent) of the total science assessment have found very small differences. The issue of content valid-
curricula identified. The 21 curricula ity, therefore, seems to be less of a concern in judging the effectiveness of
are listed in Table 2, which together science curricula.
with brief descriptions of these cur-

3


of the curriculum is far more com- mines what students will learn. Once an enormous task and an enormous
mon in science than in mathemat- a decision is made, selection should responsibility. We hope that our
ics because science curricula tend be guided by how effective a par- review can provide some guidance
to be more discretionary and vari- ticular curriculum is for the student and assistance in the process.
able than mathematics curricula. population to be taught. The National
Science curriculum developers, Research Council report, On Eval-
therefore, may feel the need to pro-
Endnotes
uating Curricular Effectiveness: Judging
vide more guidance to teachers the Quality of K–12 Mathematics 1. The majority of these efforts have been
undertaken by AAAS, the National Research
about the appropriate instructional Evaluations (2004) describes the value Council (NRC), and the U.S. Department of
approaches to be used for specific of this knowledge to the decision- Education. The AAAS study used content
curricula. making process: analysis, the NRC study did not rate specific
curricular math programs, and the U.S.
Department of Education study reviewed
Clearly, knowing how effective a middle school math programs only.
What Can We Conclude particular curriculum is, and for
2. As determined by quantifiably measurable
whom and under what conditions it
from Our Review? outcomes, such as test scores.
is effective, represents a valuable
The major conclusions of our review and irreplaceable source of informa- 3. The full report, on which this summary
tion to decisionmakers, whether report is based and which contains refer-
that should be most useful to those
they are classroom teachers, par- ences and a bibliography of all studies
wishing to invest in sustainable described in this report, can be found on
ents, district curriculum specialists,
school reform in science and mathe- school boards, state adoption the Urban Institute web site, http://www.
matics can be summarized as follows: boards, curriculum writers and urban.org.
evaluators, or national policymak- 4. “Teacher behavior” in this context refers
Effective mathematics curricula in ers. Evaluation studies can provide to teachers’ use of specific pedagogical
middle and high school may be that information but only if those approaches in instruction, such as inquiry-
either traditional or integrative evaluations meet standards of qual- based teaching practices. “Change in the
ity (p.1).7 classroom environment” refers to teacher
(standards-based). facilitation of an investigative classroom cul-
Effective science curricula in ture through seating arrangements to stimu-
middle and high school should This review of math and science late discussion, use of cooperative learning
be inquiry-based rather than curricula has tried to simplify for groups, encouraging students to explain
schools and districts the complex, concepts to one another, and other such
traditional.
practices.
Effective professional develop- time-consuming process of determin-
ment programs are those that ing curriculum effectiveness by iden- 5. Of the 80 science curricula identified, 59 had
tifying programs that have what we no evaluations or had evaluations that did
focus on content rather than for- not meet our criteria, or evaluations were
mat and that have the following consider to be credible evaluations.
out of print, or we did not receive a re-
features: We have also distilled the findings of sponse from the developers.
achievement-based research on pro-
6. As measured by both standardized tests and
● Content tied to curriculum, fessional development to a handful of tests developed for standards-based curric-
knowledge of subject matter, principles that reflect effective prac- ula in both meta-analyses and evaluations of
and/or how students learn a tice. Once schools and districts have individual curricula.
subject; decided on a curriculum and an 7. Jere Confrey and Vicki Stohl, eds. 2004. On
● A minimum of 80 contact hours appropriate assessment tool, they Evaluating Curricular Effectiveness: Judging the
to effect changes in teachers’ might wish to collect their own Quality of K–12 Mathematics Evaluations.
impact data to evaluate how well the Committee for a Review of the Evaluation
instructional behaviors; and
Data on the Effectiveness of NSF-Supported
● A minimum of 160 contact curriculum they choose is working
and Commercially Generated Mathematics
hours to effect changes in the with their own students.8 It is the Curriculum Materials. National Research
classroom environment. responsibility of the school and dis- Council. Washington, DC: National Acad-
trict community to ensure that the emy Press.
content that they want students to 8. For those schools and districts that already
How to Use This Review
learn is embodied in the curriculum, have programs that they feel are effective
Choice of a curriculum in mathe- that the curriculum is effective for but are not included in Tables 1 and 2, we
matics and science involves deciding suggest that they collect their own effective-
this purpose, and that appropriate
ness data via evaluation studies that adhere
what aspects of these subjects are measures are used to assess whether to the criteria established in this review.
important to address and emphasize students are indeed learning what the They may wish to contract with an evalua-
in schools—this choice thus deter- community wants them to learn. It is tor for this purpose.

4


Appendix A
DETAILED FINDINGS OF THE REVIEW OF MATH CURRICULA
This appendix provides a list of the mathematics curricula included in our review, together with a descrip-
tion of each curriculum and its related evaluation studies. Table 1 lists the mathematics curricula identified
as having evaluation studies that met our criteria.

TABLE 1. Math Curricula with Studies

Grades Covered Curriculum Name Subject Matter

K 12
Edison Schools Mathematics
K 8
Direct Instruction Mathematics
middle school
Cognitive Tutor *[sex, race/ethnicity] Mathematics
middle school
Connected Mathematics (CMP) *[sex, race/ethnicity] Mathematics
middle school
Integrated Mathematics, Science, and Technology (IMaST) Mathematics

middle school
Mathematics in Context (MiC) *[race/ethnicity] Mathematics
6 8
MATHThematics Mathematics
middle school
Prentice Hall: Tools for Success Mathematics

middle school
Saxon Math: An Incremental Development Mathematics

7 12
Mathematics with Meaning *[sex, race/ethnicity] Mathematics
7 12
University of Chicago School Mathematics Project (UCSMP) Mathematics

9 12
College Preparatory Mathematics (CPM) Mathematics

9 12 Contemporary Mathematics in Context: A Unified Approach
(Core-Plus Mathematics Project CPMP) Mathematics
9 12
Interactive Mathematics Program (IMP) *[sex] Mathematics
9 12
Math Connections *[sex, race/ethnicity] Mathematics
9 12
Mathematics: Modeling Our World (MMOW/ARISE) Mathematics
9 12
Systemic Initiative for Montana Mathematics and Science (SIMMS) Mathematics

11-12
Advanced Placement (AP) Calculus Calculus

Note: Shaded curricula are those for which we have found the strongest evidence of effectiveness, that is, quantitative evidence that their use in instruction elicits higher achieve-
ment/performance in students than other curricula to which they are compared on both standardized and/or state tests and on curriculum developed tests. There are several curricula
for which this evidence of effectiveness has not been collected but which might also qualify as effective should appropriate studies be conducted. An omission from this list of many
curricula signifies merely that these curricula have not yet provided quantitative evidence of effectiveness that meets our criteria.
*An asterisk marks curricula for which effectiveness data are provided for subgroups of students, indicated in brackets. (See full report on the web for discussion of research find-
ings by subgroup.)

5


A Description of the Mathematics Curricula ity, and differential and integral calculus, with Calculus BC
and Evaluation Studies Found covering these topics more extensively than Calculus AB.
Prerequisites include knowledge of analytic geometry and
In addition to a description of each curriculum, these
elementary functions in addition to college preparatory
overviews (presented in alphabetical order) include the
algebra, geometry, and trigonometry. Enrolled students are
type of student achievement measure used, the number
expected to take the Advanced Placement examination in
and direction of the results, and, if available, the size of
Calculus AB or BC.
any differences between groups. If the results were broken
out by sex and/or race and ethnicity, this too is indicated. Contact
The types of measures used in the impact studies are College Board Headquarters
broken into five categories: 45 Columbus Avenue
New York, NY 10023-6992
Standardized achievement tests (i.e., PSAT, SAT, SAT-9,
Tel: 212-713-8066
and Iowa Test of Basic Skills)
Web site: http://apcentral.collegeboard.com/article/0,3045,
State-mandated achievement tests (i.e., FCAS, MEAP, 151-165-0-2178,00.html
and MCAS)
Standards-based, curriculum-driven measures (i.e., Results
Balanced Assessment, Problem Situation Test)
Teacher-based measures (i.e., GPA, school tests, and Five results were found from studies on the effect of AP
mathematics courses taken) Calculus courses. These studies compared college mathe-
matics performance of students who took AP Calculus
Percent passing different mathematics courses
with other students who did not.
In the charts that follow within this appendix, results
were tracked rather than the number of studies. Multiple Number
grades and multiple measures were counted as multiple Type of measure of results Results Effect size
results and broken out as such. For example, a study that Teacher-based 3 AP students scored higher No effect
looked at the impact of different subsets of a curriculum measures/GPA in two results; there were size reported
for sixth, seventh, and eighth grades was counted as three no differences in one
results. Cohort studies that tracked students across multi- result.
ple grades were counted as one result. Also counted as one Pass rates/ 2 AP students scored higher No effect
result were findings from different sites within the same courses taken in one result, and there size reported
study using the same measure. If a majority of sites had were no differences in
changes favoring the tested curriculum, the result was one result.
indicated as positive. If the majority of sites did not differ
from the comparison, the result was indicated as no Results were not reported by sex, race, or ethnicity.
change. If a majority of sites had changes favoring the
comparison curriculum, the result was indicated as
negative. Cognitive Tutor
Note: Most impact studies for math curricula reported the statistical sig- The Cognitive Tutor (CT), from Carnegie Learning,
nificance of their results. Only differences that have reached the conserva- includes full curricula in Algebra I, Geometry, Algebra II,
tive minimum acceptable statistical significance level of .05 were included
in the results reported for each study. If differences are statistically signifi-
an Integrated Math Series, and Quantitative Literacy
cant, then there is another measure, called an effect size, which shows Through Algebra. Each curriculum combines software-
how big the difference is. In our description of the study results, we pro- based, individualized computer lessons with collaborative,
vide effect sizes where available, although very few studies reported real-world problem-solving activities. Students spend
these results. Effect sizes greater than .4 are considered large; between .2
about 40 percent of their class time using the software and
and .4 are considered moderate; and less than .2 are considered small.
the remainder of their time engaged in classroom problem-
solving activities.
Contact
Advanced Placement (AP) Calculus
Carnegie Learning, Inc.
The AP Calculus curriculum includes two courses: AP 1200 Penn Avenue
Calculus AB, which is comparable to one semester of col- Suite 150
lege-level calculus, and AP Calculus BC, which is com- Pittsburgh, PA 15222
parable to two semesters of college-level calculus. Both Tel: 888-851-7094
courses include elementary functions, limits and continu- E-mail: info@carnegielearning.com

6


Results Results
Twenty-one results from studies on the effect of CT were Six results from studies on the effect of CPM were found,
found, all of which focused on Algebra I at the middle and one of which focused on algebra and five of which focused
high school level. on grade 9–11 math.

Number Number
Type of measure of results Results Effect size Type of measure of results Results Effect size

Standardized 8 CT students scored higher Moderate Standardized 5 There were no differences No effect
achievement in all eight results. effect size achievement in all five results. size reported
tests in one result tests
Statewide tests 5 CT students scored higher No effect Teacher- 1 There were no differences. No effect
in three results; there size reported developed size reported
were no differences in measure
two results.
Results were not reported by sex, race, or ethnicity.
Curriculum-driven, 7 CT students scored higher Large effect
skill-specific tests in seven results. sizes in two
results
Passing rates 1 CT students had higher No effect Connected Mathematics (CMP)
passing rates. size reported
The Connected Mathematics (CMP) curriculum is com-
posed of eight models, each focusing on one important
Two results looked at sex differences; in one result, CT area of mathematics and emphasizing previously learned
boys’ scores were higher than comparison boys. There content. Connected Mathematics is designed to develop
were no differences between boys in one result, and there students’ knowledge and understanding of mathematics
were no differences for the girls’ scores in either result. through attention to connections between mathematical
Two results reported differences by race and ethnicity. In ideas and their applications in the world outside school;
one result, African-American CT students had higher among the core ideas in mathematics; among the strands
scores than comparison African-American students, and
in a modern mathematics curriculum; and between the
there were no differences for Hispanic students. In the
planned teaching-learning activities and the special apti-
second result, Hispanic CT students scored higher than
tudes and interests of middle school students.
comparison Hispanic students.
Contact
Pearson Education
College Preparatory Mathematics (CPM) P.O. Box 2500
The CPM series offers a four-year integrated curriculum Lebanon, IN 46052-3009
where mathematics topics are revisited and built upon Tel: 800-848-9500
through the years. Problem-solving strategies are empha- Web site: http://www.phschool.com/math/
sized as a vehicle for learning mathematics, and student
study teams are an integral part of the learning process.
Results
Based on the belief that concept mastery requires time, the
curriculum spirals through practice of the main course Thirty-four results from comparison studies looking at
concepts throughout each year and emphasizes students’ the effect of CMP were found focusing on middle grade
supportive group work. The program sees the teacher’s mathematics.
role as a guide.
Number
Contact Type of measure of results Results Effect size

CPM Business Office Standardized 11 CMP students scored No effect
1233 Noonan Drive achievement higher for six results; they size reported
Sacramento, CA 95822 tests scored lower for one result;
Tel: 916-681-3611 and there were no differ-
Web site: http://www.cpm.org ences for four results.
(continued)

7


Number Number

Statewide 16 CMP students scored No effect Standardized 43 In 19 of the results, CPMP No effect
tests higher for 14 results; there size reported achievement students scored higher; size reported
were no differences for tests in 24 results, there were
two results. no differences.
Curriculum-driven, 6 CMP students scored Large effect Statewide 5 Students scored higher in No effect
skill-specific tests higher for all six results. sizes were tests all five results. size reported
found for Curriculum-driven, 2 Students scored higher in No effect
one result. skill-specific tests both results. size reported
Teacher-based 1 CMP students scored No effect Teacher-based 6 CPMP students scored No effect
measure higher in one result. size reported measure/GPA higher in one result, no size reported
differently in another, and
lower in four results.
Seven results were broken out by race/ethnicity and
two by sex. No significant sex differences were found.
African-American CMP students were found to score Results were not reported by sex, race, or ethnicity.
higher than African-American comparison students in six
results, while Hispanic CMP students scored higher than
Hispanic comparison students in four results; in a fifth
result, there were no differences. African-American and
Hispanic CMP students showed greater gains than others Direct Instruction
in two results, while African Americans alone showed
greater gains in one result. In one result, Native-American In Direct Instruction (DI), each program is fully scripted,
CMP student performance decreased. from what the teacher says and anticipated student
responses, to correctional procedures. Each skill is broken
down into its component parts, and then each component
of the skill is taught to mastery. Afterward, the skills are
Contemporary Mathematics in Context: combined within a larger context where they may be uti-
lized across settings, resulting in generalized fluency. The
A Unified Approach
DI mathematics curriculum covers grades K–8, but focuses
(Core PLUS Mathematics CPMP) primarily on grades K–6. The mathematics curriculum
Core-Plus Mathematics consists of a single core sequence covers 19 different topics ranging from addition and sub-
for both college-bound and employment-bound students traction to money, mathematics study skills (graphs,
during the first three years of high school. A flexible fourth charts, maps, and statistics), and geometry.
year course can be used to prepare students for college
mathematics.
Contact
Contact The McGraw-Hill Companies
P.O. Box 182604
Core-Plus Mathematics Project
Columbus, OH 43272
Department of Mathematics
Tel: 888-772-4543
Western Michigan University
Web site: http://www.sraonline.com/index.php/home/
Kalamazoo, MI 49008-5248
curriculumsolutions/di/connectingmath/114
Tel: 866-407-CPMP (2767)
E-mail: cpmp@wmich.edu

Results
Results
Nine results from comparison studies on the effect of DI
Fifty-six results from comparison studies on the effect of were found which focus on seventh and eighth grade
CPMP were found focusing on high school math. math.

8


Number Integrated Mathematics, Science,
Type of measure of results Results Effect size and Technology (IMaST)
Standardized 6 DI students scored higher No effect
The Integrated Mathematics, Science, and Technology pro-
achievement for four results; they size reported
tests scored lower for two gram provides integrated sixth, seventh, and eighth grade
results. curricula that promote hands-on learning for students and
teamwork among teachers from different disciplines.
State test 3 In all three results, DI No effect
students scored higher. size reported IMaST emphasizes learning based on constructivist theory
and active student participation involving a hands-on
Results were not reported by sex, race, or ethnicity. approach comprising a wide variety of activities.

Edison Schools Contact
Founded in 1992, Edison Schools is focused on raising stu- Ronjon Publishing, Inc.
dent achievement through research-based school design, 1001 S. Mayhill Rd.
aligned assessment systems, interactive professional devel- Denton, TX 76208
opment, integrated use of technology, and other program Tel: 800-262-3060
features, including a longer school day and school year. The Web site:
math curriculum for grades 6–8 includes applied arithmetic,
http://www.ilstu.edu/depts/cemast/programs/imast.shtml
prealgebra, and pregeometry, using a spiral curriculum
approach to teach concepts and ideas. The mathematics in
grades 9–10 provides three years of high school math in two Results
years’ time, using an integrated application-based approach
to algebra, geometry, and trigonometry with additional One study was found looking at the effect of IMaST focus-
emphasis on probability, statistics, and discrete mathemat- ing on seventh and eighth grade students.
ics. Grades 11 and 12 incorporate advanced mathematical
modeling, Calculus and Statistics, Algebra 2, and
Number
Precalculus, as well as AP Statistics and Calculus. Type of measure of results Results Effect size
Contact Standardized 1 IMaST students scored No effect
521 Fifth Avenue, 11th Floor achievement higher for the one result. size reported
New York, NY 10175 tests
Tel: 212-419-1600
Web site: http://www.edisonschools.com/
Results
Seventeen results from comparison studies on the effect of
Edison Schools were found; fifteen focus on middle school
math and two focus on high school math.
Interactive Mathematics Program (IMP)
Number
Type of measure of results Results Effect size This four year, problem-based curriculum incorporates tra-
Standardized 8 For four results, Edison No effect ditional branches of mathematics (algebra, geometry, and
achievement students scored higher; size reported trigonometry) with additional topics recommended by the
tests in the remaining four, there NCTM Standards, such as statistics, probability, curve fit-
were no differences.
ting, and matrix algebra. Students are encouraged to
State test 9 For two results, Edison No effect experiment, investigate, ask questions, make and test con-
students scored higher; size reported jectures, reflect, and accurately communicate their ideas
for four results, Edison
students scored lower; and conclusions. Although each unit has a specific mathe-
and for three results, there matical focus, other topics are brought in as needed to
were no differences. solve the central problem. Ideas that are developed in one
unit are usually revisited and deepened in one or more
Results were not reported by sex, race, or ethnicity. later units. Algebra and geometry are distributed through-
out the four years.

9


Contact Results
Key Curriculum Press Fifteen results from comparison studies were found on the
1150 65th Street effect of MC, all of which focused on high school students.
Emeryville, CA 94608
Tel: 800-995-MATH (6824)
Number
Web site: http://www.keypress.com/catalog/products/ Type of measure of results Results Effect size
textbooks/Prod_IMP.html
Standardized 6 MC students scored No effect
Results achievement higher for three results; for size reported
tests three results, there were
Twenty-two results from comparison studies were found
on the effect of IMP, all focusing on high school.
State test 8 MC students scored There was a
Number
higher for all eight results. large effect
Type of measure of results Results Effect size size for three
results.
Standardized 12 IMP students scored No effect
achievement higher for eight results; size reported Curriculum-driven, 1 No difference was found. N/A
tests there were no differences skill-specific test
for four results.
Pass rates or 5 In all five results, IMP No effect One result looked at sex, race, and ethnic differences
courses taken students had higher pass- size reported and found no differences within MC students.
ing rates and/or were more
likely to take more mathe-
matics courses.
Curriculum-driven, 3 IMP students scored Large effect
skill-specific tests higher for all three results. sizes were Mathematics in Context (MiC)
found in the
three results Mathematics in Context is a four-year middle school cur-
Teacher-based 2 In both results, IMP No effect riculum (grades 5–8) that encourages students to discover
measure students scored higher. size reported mathematical concepts and skills through engaging prob-
lems and meaningful contexts. Each year includes lessons
Results were not reported by race or ethnicity. The one in the four strands (numbers, algebra, geometry and statis-
result reported by sex found IMP girls were slightly more tics, and probability) that are interwoven through 10 units.
apt to continue three or more years in math than IMP boys, For example, sample algebra units include Patterns and
while the reverse was the case for comparison students. Symbols (grade 5), Expressions and Formulas (grade 6),
Ups and Downs (grade 7), and Graphing Equations
(grade 8).
MATH Connections
Contact
MATH Connections (MC) is an integrated curriculum that
blends ideas from traditionally separate mathematical Holt, Rinehart and Winston
fields (e.g., algebra, geometry, statistics, and discrete math- Attn: Ms. Web1
ematics) in ways that blur the lines between them. This 10801 N. MoPac Expressway
three year curriculum replaces the traditional Algebra I, Building 3
Geometry, Algebra II sequence and is designed for all stu- Austin, TX 78759
dents in grades 9, 10, and 11, with honors students begin- Tel: 800-HRW-9799 (800-479-9799)
ning the curriculum in grade 8. Web sites: http://www.hrw.com
http://mic.britannica.com/mic/common/home.asp
Contact
750 Old Main Street
Suite 303 Results
Rocky Hill, CT 06067-1567
Tel: 860-721-7010 Twenty results from comparison studies were found on the
Web site: http://www.mathconnections.com effect of MiC, which focused on middle school students.

10


Number Number

Standardized 17 In 15 results, MiC In two Standardized 4 For three results, MMOW No effect
achievement students scored higher; in results, a achievement students scored higher; size reported
tests two results, there were no majority of tests for one result, there were
differences. classes no differences.
showed at
least moder- Results were not reported by sex, race, or ethnicity.
ate effect
sizes.
State test 3 MiC students scored No effect Mathematics with Meaning
higher for all three results. size reported
Mathematics with Meaning (MwM) is not a complete cur-
ricular program; rather, it is a combination of professional
One result was broken out by race. No statistically sig- development, instructional strategies, and carefully
nificant difference was found between African-American planned materials designed to alter the pedagogy and
students using MiC and comparison students. content of middle school and high school mathematics
courses in order to improve student achievement. The pro-
gram consists of instructional units that teachers may use
on a supplementary basis or as their entire instructional
program. MwM takes a student-centered approach based
Mathematics: Modeling Our World on exploratory learning and problem solving, focusing on
(MMOW/ARISE) developing conceptual understanding, connections, and
communication with mathematical concepts through fre-
In Mathematics: Modeling Our World (MMOW), students quent group work and hands-on activities.
are taught to use a variety of resources to solve problems
and to choose resources that meet the needs of a particular Contact
situation. As in real life, MMOW’s problems do not neces- College Board
sarily have perfect solutions. MMOW works to strengthen Dept CBO
the students’ ability to solve problems by setting goals and P.O. Box 869010
thinking strategically about how to achieve these goals, Plano, TX 75074
solving problems through trial and error and/or process Tel: 212-713-8260
of elimination, using technology like calculators and com- Tel: 800-323-7155
puters, and working together to solve semi-structured E-mail: Collegeboardcustomerservice@pfsweb.com
problems and communicating the solutions. Web site: http://www.collegeboard.com

Contact Results
W.H. Freeman and Company Eleven results from comparison studies on the effect of
41 Madison Avenue MwM were found; three focused on middle school math
New York, NY 10010 achievement and eight on high school math achievement.
Tel: 800-446-8923
Web sites: http://www.whfreeman.com/highschool/ Number
Type of measure of results Results Effect size
contact_hs_rep.asp
http://www.comap.com/highschool/projects/mmow/ Statewide tests 11 In six results, there were No effect
introduction.htm no differences; in five reported
results, MwM students
scored higher.
Results
Six results were broken out by sex, four by race. In the
Four results related to MMOW were found, one focusing six results where sex differences were given, boys slightly
on middle school students and three focusing on high outperformed girls. In four results where race/ethnic dif-
school students. ferences were given, achievement scores were significantly

11


lower for African-American students than other students school level. Each lesson includes a Think and Discuss sec-
in both MwM and comparison groups. tion that presents the new material along with questions to
get students actively thinking about and discussing impor-
tant concepts. Textbooks in this curriculum include Work
Together activities that allow students to work in groups,
Middle Grades MATHThematics
often doing hands-on activities to reinforce math topics.
Middle Grades MATHThematics (STEM) is a three-year
curriculum designed for use in grades 6–8. Four unifying Contact
concepts—Proportional Reasoning, Multiple Represen-
Pearson Education
tations, Patterns and Generalizations, and Modeling—are
P.O. Box 2500
used across the three years with seven content strands:
Lebanon, IN 46052-3009
Number, Measurement, Geometry, Statistics, Probability,
Tel: 800-848-9500
Algebra, and Discrete Mathematics.
Web site: http://www.phschool.com/math/
Contact
Results
McDougal Littell Customer Service Center
Eleven results from comparison studies on the effect of
A Houghton Mifflin Company
PHM were found.
1900 S. Batavia
Geneva IL 60134
Number
Tel: 617-351-5326
Tel: 800-462-6595
Web sites: http://www.mcdougallittell.com/ Standardized 8 PHM students scored Moderate
achievement higher in five results; in effect sizes
http://www.classzone.com/math_middle.cfm
tests three results, there in one result
Results
Statewide tests 2 PHM students scored No effect
Six results from comparison studies were found on the higher in both results. size reported
effect of STEM. Curriculum-driven, 1 PHM students scored No effect
skill-specific tests higher in this result. size reported
Number
Standardized 2 In one result, STEM stu- No effect
achievement dents scored higher; in size reported
tests the second, there were no
differences. Saxon Math: An Incremental Development
Statewide tests 2 In one result, STEM stu- No effect Saxon Math (SM) is a K–12 curriculum that systematically
dents scored higher; in size reported distributes instruction, practice, and assessment through-
the second, there were no out the academic year rather than concentrating concepts
differences. in a single unit or chapter. Each increment builds upon the
Curriculum-driven, 2 In both results, STEM stu- No effect foundation of earlier increments, to lead students toward
skill-specific tests dents scored higher. size reported a deeper understanding of mathematical concepts.
Instruction of related concepts is spread throughout the
Results were not reported by sex, race, or ethnicity. grade level, ensuring that students have an opportunity
to master each concept before they are introduced to the
next one.
Prentice Hall: Tools for Success Contact
In Prentice Hall Math (PHM), various mathematical Saxon Publishers
strands (such as number sense, algebra, geometry, mea- 2600 John Saxon Blvd.
surement, data analysis, and problem solving) are inte- Norman, OK 73071
grated throughout the series to ensure that students are Tel: 800-284-7019
prepared for subsequent mathematics courses at the high E-mail: info@saxonpublishers.com

12


Results Number
A total of eight results from comparison studies were
found on the impact of SM, focusing on both middle and Standardized 6 No differences were found No effect
high school. achievement in the six results. size reported
tests
Curriculum-driven, 6 In one result, SIMMS stu- No effect
Number
skill-specific tests dents scored higher; no size reported
differences were found in
Standardized 3 SM students scored No effect the other five.
achievement higher in all three results. size reported
tests Results were not reported by sex, race, or ethnicity.
Statewide tests 3 SM students scored No effect
higher in all three results. size reported
Curriculum-driven, 1 SM students scored No effect University of Chicago School Mathematics
skill-specific tests higher in this result. size reported Project (UCSMP)
Teacher-based 1 SM students scored No effect The UCSMP secondary curriculum consists of six courses:
measure/GPA higher in this result. size reported Transitional Mathematics; Algebra; Geometry; Advanced
Algebra; Functions, Statistics, and Trigonometry; and Pre-
Results were not reported by sex, race, or ethnicity. Calculus and Discrete Mathematics. Transitional Mathe-
matics, which was originally designed for average to above
average seventh graders (but can be started earlier or later),
weaves together three more or less equal strands of major
content: applied arithmetic, prealgebra, and elementary
geometry. Algebra, geometry, and some discrete mathematics
Systemic Initiative for Montana are integrated into all courses, as are statistics and probability.
Mathematics and Science (SIMMS)
Contact
The SIMMS curriculum is divided into six levels, each
UCSMP
consisting of one year of work. Level 1 is typically offered
5835 South Kimbark Avenue
to ninth graders, followed by level 2 in grade 10. After
Chicago, IL 60637
completing level 2, students may choose between levels 3
Tel: 773-702-1130
and 4 and then proceed to either level 5 or 6 in the subse-
Web site: http://socialsciences.uchicago.edu/ucsmp/
quent year. The sequence for potential math and science
majors is 1-2-4-6. Levels 1 and 2 offer basic mathematical Results
literacy.
Fourteen results were found from comparison studies
on the effect of UCSMP. The following UCSMP courses
Contact were covered: Transitional Mathematics (2), Algebra (2),
Kendall/Hunt Publishing Geometry (5), Advanced Algebra (4), and Pre-Calculus and
4050 Westmark Drive Discrete Mathematics (1).
P.O. Box 1840
Number
Dubuque, IA 52004-1840 Type of measure of results Results Effect size
Tel: 800-542-6657
Standardized 5 In all five results, UCSMP No effect
Web sites: http://www.simms-im.com
achievement students scored higher. size reported
http://www.montana.edu/~wwwsimms/
tests
Curriculum-driven, 8 In all eight results, UCSMP No effect
Results skill-specific tests students scored higher. size reported
Teacher-based 1 No differences were found. No effect
Twelve results from comparison studies were found on the measure/GPA size reported
effect of SIMMS. Four results each focused on levels 1 and
2, and two each focused on levels 4 and 6. Results were not reported by sex, race, or ethnicity.

13


Appendix B
DETAILED FINDINGS OF THE REVIEW OF SCIENCE CURRICULA
This appendix provides a list of the science curricula included in our review, together with a description of
each curriculum and its related evaluation studies. Table 2 below lists the science curricula that we identified
as having evaluation studies that met our criteria.

TABLE 2. Science Curricula with Studies

Grades Covered Curriculum Name Subject Matter

K 12
Expeditionary Learning Outward Bound (ELOB) *[LEP] Whole School Reform

K 8
Full Option Science System (FOSS) *[LEP] Multi-Science

PK 8
Great Explorations in Math and Science (GEMS) Math & Multi-Science

K 6
The Science Curriculum Improvement Study (SCIS) Multi-Science

2 8
National Science Curriculum for High Ability Learners Multi-Science

6 8
DESIGNS/DESIGNS II *[sex] Physical Science

6 8
Integrated Math, Science and Technology (IMaST) Math, Science & Technology

6 8
Center for Learning Technologies in Urban Schools (LeTUS) *[sex] Multi-Science

6 8
Learning By Design (LBD) Multi-Science

6 8
Science 2000/Science 2000+ Multi-Science
6 8
Science and Technology Concepts for Middle School (STC/MS) Multi-Science
6 9
Event-Based Science (EBS) Earth Science
78
Constructing Ideas in Physical Science (CIPS) *[sex] Physical Science

7 9
Foundational Approaches in Science Teaching (FAST) Multi-Science
7 9
Global Lab Curriculum (GLC) Multi-Science, Environmental Focus
7 9
Issues, Evidence and You (IEY)/SEPUP Multi-Science

9 11
BSCS: An Inquiry Approach Multi-Science

9 12
High Schools that Work (HSTW) Whole School Reform
9 12
Modeling Instruction in High School Physics *[sex] Physics
9 12
World Watcher/Learning About the Environment (LATE) *[urban] Environmental Science
10 12
Physics Resources and Instructional Strategies for Motivating Students (PRISMS) Physics

Note: Shaded curricula are those for which we have found the strongest evidence of effectiveness, that is, quantitative evidence that (1) their use in instruction elicits higher achieve-
ment/performance in students than other curricula to which they are compared on both standardized and/or state tests and on curriculum developed tests, or (2) they showed large
effect sizes in terms of increasing student achievement. All science curricula listed in Table 2, however, have credible evaluations that show evidence of effectiveness. There are sev-
eral curricula for which this evidence of effectiveness has not been collected but which might also qualify as effective should appropriate studies be conducted. An omission from
this list of many curricula signifies merely that these curricula have not yet provided quantitative evidence of effectiveness that meets our criteria.
*An asterisk marks curricula for which effectiveness data are provided for subgroups of students, indicated in brackets. (See full report on the web for discussion of research find-
ings by subgroup.)
14


A Description of the Science Curricula Number
and Evaluation Studies Found Type of measure of results Results Effect size

The attached descriptions of science curricula, which Content test, 1 Average student gains at No effect
appear in alphabetical order, include the type of student unknown if self- both 9th and 10th grade size reported
achievement measure used, the number and direction of designed or levels were between 20
state-issued and 25 percent.
the results, and, if available, the size of any differences
between groups. When known, effect sizes have been Note: Statistical significance levels were not reported.
listed. Where results have been provided by race/ethnic- The field test was conducted across 10 states and
ity, sex, or other demographic characteristics, we have re- included students in urban, suburban, and rural schools.
ported these. The absence of such notation means that no However, data regarding specific demographic groups
data were reported by subgroup. In the case of three cur- was not provided.
ricula, descriptions of studies do not follow the normal
format because results reported are not best described in
that particular format.
Center for Learning Technologies
Note: Almost all impact studies for science curricula reported the statisti- in Urban Schools (LeTUS)
cal significance of their results. Only differences that have reached the
conservative minimum acceptable statistical significance level of .05 were LeTUS, developed by researchers at the University of
included in the results reported for each study. If differences are statisti- Michigan and Detroit public school teachers, includes pro-
cally significant, then there is another measure, called an effect size, that
ject-based curriculum materials that build from district,
shows how big the difference is. In our description of the study results,
we provide effect sizes where available, although few studies reported state, and national standards to support the development
these results. Effect sizes greater than .4 are considered large, between .2 of integrated science understanding for middle school stu-
and .4 are considered moderate, and less than .2 are considered small. dents. The materials support students’ science learning
through engaging them in inquiry about real world prob-
lems, providing them with multiple opportunities to work
with concepts, and integrating the use of learning tech-
BSCS: An Inquiry Approach nologies in instruction. LeTUS is focused on learning
This program introduces 9th, 10th, and 11th grade stu- about and developing a new machine to construct large
dents to the core concepts in inquiry, the physical sciences, buildings and bridges, an area that has been identified as
the life sciences, and the earth–space sciences as articu- of interest to young urban students.
lated in the National Science Education Standards. In addi-
Contact
tion, the curriculum engages students in integration across
Joseph Krajcik
the disciplines in relevant contexts that explore the stan-
School of Education
dards related to science in a personal and social perspec-
University of Michigan
tive. This program provides high school students with an
610 East University Avenue, Rm. 4109
alternative to the traditional sequence of biology, chem- Ann Arbor, MI 48109-1259
istry, and physics. Included with this program is a profes- Tel: 734-647-0597
sional development component designed to help teachers Fax: 734-615-5245
and school districts implement the materials. E-mail: krajcik@umich.edu

Contact Results
Pamela Van Scotter Two evaluations of the LeTUS program were found, one
Director that utilized skill-specific instruments and one that used
The BSCS Center for Curriculum Development statewide achievement test scores to measure effectiveness.
BSCS 5415, Mark Dabling Blvd.
Colorado Springs, Colorado 80918 Type of Number
Web site: http://www.bscs.org/page.asp?id=curriculum_ measure of results Results Effect size
development|high_school_9-12|An_Inquiry_Approach
Curriculum- 1 Significant content and Large effect size for
driven,skill- process gains that in- content achieve-
Results specific creased with program ment and more
Most of the evaluative work on this was of the materials, tests revision and scale-up moderate effects
for process skills.
not student achievement; however, the results of a nation-
wide field test are summarized below. (continued)

15

What We Know

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (17)

Similar to What We Know

Similar to What We Know (20)

Recently uploaded

Recently uploaded (20)

What We Know