REVIEW OF EXPERIMENTS WITH GRADE RETENTION
December 2000
Review Author: Kristi Kimball
Intro:
Noreen Conell
PART I: INTRODUCTION
1. EPP’S OPPOSITION TO THE NEW YORK
CITY BOARD OF EDUCATION’S 1999 GRADE RETENTION POLICY
The New York City public school district was
one of the earliest and has been one of the most consistent experimenters
with grade retention policies in the post-World War II period. In 1981,
the district launched the "Promotional Gates" system-wide
program, which replaced individual teacher and school decision making
on the students who should be required to repeat a grade. It created
achievement benchmarks that students in grades 4 and 7 had to reach
in order to be promoted to the next grade. The program was discontinued
in 1990 because longitudinal data showed that students retained through
the program performed poorly over time and were more likely to drop
out of school than similar students who were not retained. The district’s
latest system-wide policy, first implemented in the 1999-00 school year,
once again established clear promotion criteria, based on standardized
test scores, attendance, and teacher assessments, for students in every
grade from 3 through 11. Since system-wide grade retention policies
were already in effect for the high schools, based on students’
credit accumulation and test scores, the new policy only affected grades
3 to 8.
"Accountability" and "learning to high
standards" are magic words in public education policy, and, for
this reason, were liberally used when the new initiative to end social
promotion was introduced by the Chancellor and Mayor in 1999. The
Educational Priorities Panel, however, is charged to go beyond idealistic
slogans to evaluate the soundness, the costs, and the effectiveness
of programs to improve public education by monitoring the budget and
administrative practices of the New York City Board of Education.
As a coalition of organizations with a range of positions on social
promotion and grade retention, EPP supported the introduction of the
grade retention policy in 1981 by the Board of Education and opposed
the Board’s re introduction of grade retention policies in 1999.
Our shift in positions stemmed largely from our institutional memory
of how the 1981 Promotion Gates Program played out. As data accumulated
that students who were held over at the 4th and 7th grades needed more
than grade repetition to achieve at acceptable levels, city budget decision
makers invested less in the program as the true costs of providing
student academic support became evident. The city was unwilling to provide
sufficient funds to make a genuine effort to end social promotion.
Since a majority of members of EPP in 1980 remained
core members 20 years later, our coalition’s rejection of the new
policy reflected an unwillingness to re experience the same trajectory
of initial high expectations, followed by sobering data, and ultimately
a refusal by city budget decision makers to back up their rhetoric with
dollars. "Been there, done that, let’s not go through
this again," was the prevailing sentiment.
Despite the opposition of many education reform and
parent organizations, the Board of Education adopted the policy in a
vote of 6 to 1, with only the Manhattan Board Member, Dr. Irving Hammer,
voting in opposition. Its passage was predictable. The intellectual
author of the policy was Mayor Rudoph Giuliani. On the verge of a campaign
for the United States Senate, he was eager to claim that he had "ended
social promotion." Chancellor Rudolf Crew, attempting to patch
up his relationship with the Mayor after an apparent drop in test scores
and his opposition to taxpayer-funded vouchers for private and parochial
schools, rushed through the implementation of the plan starting with
only grades 3, 6 and 8.
There were other reasons why the New York public school
system embarked on yet another attempt at grade retention despite the
dismal outcomes of it first experiment. A powerful political myth had
developed among conservative groups that the Gates Program had been
sabotaged by liberals. Given the turnover of education reporters for
the city’s daily newspapers, the press took scant interest in the
previous evaluations of the Gates Program and tended to paint opponents
of grade retention as fuzzy-minded policy wonks worried about students’
self-esteem. If anything, most education reporters were hard-liners
on grade retention, questioning why students should be exempted from
repeating a grade if they had good attendance or their class work was
acceptable. The implication was that only standardized test scores should
count and that a "real" policy would force as many students
as possible into summer school and prevent as many students as possible
from moving on to the next grade. The tabloids, in particular, were
as vigilant about any signs of "leniency" as they were about
the Chancellor’s administrative missteps. The first evaluation’s
findings that 7th grade students mistakenly promoted and who did not
attend summer school did marginally better on tests than summer school
students received scant attention from reporters.
Once the grade retention policy was in place, the
Education Priorities Panel had two options: A) to monitor the implementation
of a program that we fundamentally opposed or B) to look at a variety
of other school districts’ experiments with grade retention to
gain a perspective as to how New York City’s current program could
be improved. The Panel chose option B on a hunch that future evaluations
would trigger a need for change as it became more evident that summer
school and grade repetition were not "magic bullets." We were
hunting for ways of avoiding the same trajectory followed by the earlier
Gates Program, a steady increase of over-age students and dropout rates
as funding for their academic support evaporated. We did not want
to wait 10 years until a new Chancellor or a new Mayor had the courage
to publicly declare a formal end to a failed grade retention policy.
If the policy could not be repealed within a few years, was there a
way to at least improve the program enough to prevent high dropout rates
and declining student achievement?
2. Lessons
Learned from the Review of School District
Experiments
Part II of this report contains summary of what could
be learned from interviews and documents from 27 school districts across
the nation about their experiments with grade retention of the last
25 years. This information allowed us to view New York City’s current
program in a larger context and gain perspective. Here is a summary
of the key lessons we learned from Kristi Kimball’s review:
The implementation of many school districts’
promotion policies are riddled with problems of timely data collection,
student tracking, and notification. The NYC Board of Education’s
problems in notifying parents and schools of students at risk for
grade retention appear to be common in many school districts. If
standardized tests are administered mid-year, there is not enough
turn around time to score the tests and then collect data on low-scoring
students. Given these constraints, it is unlikely that schools
will be able to provide academic intervention early enough
in the spring semester to help low-scoring students or to notify
parents to change their summer plans. For the 1999-00 school year,
the Board of Education "solved" this problem by over identifying
at-risk students based on their previous year’s test scores.
The early-identification effort was motivated by the fear of law
suits, but did not necessarily trigger meaningful academic support
services for these students in their home schools.
Most summer school programs have lower than average
student attendance. Late or ineffective communication by school
district officials is often cited as the cause. But children
who are achieving at the lowest academic levels also tend to have
higher absentee rates. Programs geared to this student population,
inevitably, will have lower than average attendance. In comparison
to Houston, Cincinnati, Hartford, and Oakland, where only 40 percent
to 52 percent of children at-risk for retention actually showed
up, New York City’s summer school attendance rates of 65 percent
in 1999 and 75 percent in 2000 seem relatively high. These absentee
rates of 25 per cent to 35 percent, however, are an indication that
summer school programs by themselves will probably never
be able to reach all the children who need help.
The NYC Board of Education’s adoption of multiple
criteria for promotion, rather than merely test scores, was an improvement
over the Gates Program. Frustrated that a failed strategy was
again being imposed on students and schools, the Panel failed
to fully recognize that the new policy indicated that there was
some recognition by city education officials that summer school
and grade repetition should not be imposed on every student scoring
poorly on standardized tests. Low-scoring students with records
of regular attendance and/or satisfactory classroom work are indicators
that either they are not good at taking tests or that their home
schools’ low performance levels are contributing factors.
In retrospect, the problems in implementing
the New York City summer school program seem endemic to grade retention
programs in most school districts. By reviewing over 27 grade retention
programs, the NYC Board of Education’s performance and criteria
for promotion seem better than those of many other school districts.
The review, however, also looked at 11 evaluations of grade retention
policies for their impact on student achievement, including the Gates
program and Chicago’s current program. Besides bolstering our hunch
that the current grade retention policy in New York City will be no
more successful than the previous policy, these evaluations also provide
a clue to the reasons why these programs tend to last for only five
years, even though they may survive longer as empty policies:
A sizable proportion of low-achieving students
tend to make only marginal academic gains and remain at risk of
retention year after year. But an even more troubling group of lowest-achieving
children seem not to benefit from summer school, other services,
and grade repetition. First-year evaluations, by their very
nature, can never capture these long-term patterns. In third-year
evaluations, however, the data begin to suggest that many of the
same students are being identified for retention. The premise
of these grade retention programs, that most students will turnaround
their academic performance if given summer school or forced to repeat
a grade, begins to be undercut. As students held back two times
already get socially promoted, a common exemption in most programs,
the negative impact on children becomes more evident and responsible
education officials begin to re asses their policies.
These programs can lead to dramatically higher
dropout rates. Opponents of grade retention policies often refer
to a well-known 1971 study done by the Institute for Social Research
of the University of Michigan that found that once students have
been held back once, their chances of dropping out increase by 40
to 50 percent. Our review found that past the evaluations of the
New York City (1988), Miami-Dade County (1985), and Boston (1986)
confirm this finding. The only current evaluation that contains
longitudinal data on dropout rates is Chicago’s because this
city is now in its fifth year of implementation. Dropout
rates doubled for 8th graders held over once.
Two evaluations found that, paradoxically, students
who were retained had less access to academic help than those
who were promoted. While EPP had focused on the inadequacy of
funding for the previous and current grade retention programs, it
was only when we re read the 1988 Gates Program evaluation and learned
about the 1986 Washington D.C. evaluation that we learned that remediation
and other services for students held over in grade were even
more inadequate than for students not in the program. For example,
in the Gates Program less than 5 percent of students retained in
the 7th grade in 1982-83 had participated in the Attendance Improvement/Dropout
Prevention program (AIDP). In Washington D.C. schools, 75 percent
of low-achieving students who passed their reading and math tests
received assistance from reading and math specialists five days
a week while only 23 to 28 percent of students held in grade received
these services. We infer from this startling information that
education officials either tended to ignore strategies beyond summer
school and grade repetition for this group of students or wanted
to reduce the costs of an already expensive program. Both evaluations
strongly recommended that services for retained students be made
mandatory, but the programs were disbanded within a year
or two after these evaluations were concluded.
Irrespective of the sorry outcome of the Gates Program,
what was the sense of creating a program geared to identifying low-achieving
students that then provided them with inadequate academic services?
It may well be that as it became evident that strategies put in place
to help academically at-risk children were not working, the sheer numbers
of students who remained low achieving began to push up the projected
costs of the Gates Program. As the numbers increased, the services decreased
for this group.
3. Why EPP Remains Opposed to
the Current Grade Retention Policy
The review in Part II of this report provided us with
a better, more balanced perspective on the New York City’s implementation
of the grade retention program. This school district’s program
design and summer school attendance rates were better than many other
school districts. Given the many reports of glitches in test-outcome
data collection, student tracking, and early notification, these problems
may not be easy to solve so long as standardized tests are given mid
year. The review’s summary of evaluations detailing the negative
impact on students, however, buttresses EPP’s major objections
to the current grade retention policy.
Accountability Grade retention essentially shifts
the burden of responsibility for academic achievement from the school
to the student. To a large extent, the quality of curriculum and
instruction at a given school determines the proportion of students
who fail to test at grade level. For example, test data from the NYS
Education Department on the 4th grade English Language Arts exams for
1999 and 2000 show that in 11 elementary schools in Manhattan and 27
elementary schools in the Bronx, 12 elementary schools in Queens,
28 elementary schools in Brooklyn from 80 percent to 90 percent of students
tested below grade level in at least one of these two years and that
at least 40 percent of the students in these schools tested in the bottom
quartile (Level 1), that is, below the level of basic reading comprehension.
Yet this same data show that schools with similar low-income demographics
and situated within the same immigrant neighborhoods were able to bring
a majority of their students up to grade level. Students required
to attend summer school in 1999 and 2000 came disproportionately from
low-performing schools and low-performing districts. While the general
public may assume that children fall behind for lack of individual effort,
the proportion of children at grade level is a key indicator of school
quality.
Demographic factors, such as concentrations of students
struggling to master English and/or students from high poverty families,
can reduce the proportion of students who test at grade level. But there
is no plausible demographic indicator that would account for less than
20 percent of students testing at or above grade level in a community
school district elementary school. School performance, not individual
student effort, accounts for the high numbers of students performing
at Level 1 in New York City. Grade retention, paradoxically, puts the
focus of accountability on individual students.
Learning to High Standards Grade retention policies
tend to confuse cause and effect. When large numbers of students
are promoted who are not able to perform at their new grade levels,
the curriculum inevitably gets watered down. Or worse, teachers mechanistically
go through the motions of covering required subject matter as their
students tune-out of any participation in learning or act out in frustration.
Attempts to "end" social promotion are well intentioned efforts
to re establish the fundamental structure of all educational endeavors,
that students will be able to understand increasingly more complex information
and perform increasingly more difficult tasks. But for large numbers
of students testing far below grade on standardized tests, the source
of the problem is their schools’ instructional programs. In these
schools, students are not progressing to more complex information and
difficult as they should.
The assumption of the 1980 Gates Program was that
students held over to repeat the 4th and 7th grades would perform at
grade level, but grade repetition in schools where the curriculum
and the instructional performance levels were erratic produced only
marginal gains. The assumption of the current effort to end social
promotion is that a highly scripted, 22-day summer school instructional
program geared to test preparation will bring more students up to grade
level, or, at least, above the demarcation line for the bottom quartile
in test performance. But the majority of these students will be returning
to the low-performing schools that contributed significantly to their
lags in learning.
In EPP’s interviews with principals and teachers
in schools that have made substantial academic improvements for our
1996 report, Getting off the List, and for our 1999 report,
Beating the Odds, setting high expectations for all students
was identified as an absolutely essential starting point for creating
a better school. But time after time, the respondents remarked that
"wishing and succeeding" were two distinct phases. What distinguished
these schools from others was not their mission statements, which tend
to be idealistic, such as, "all our children will go to college,"
"all children will meet world-class standards," etc.,. In
schools that succeeded, a whole host of strategies were put in place
to help students achieve academically. Among the strategies were:
- the alignment of curricula between and among grades
(so that children did not ping pong between phonics and whole language
reading instruction);
- follow-up of staff development sessions through teacher
observation to ensure that more effective instructional methods and
lesson plans were executed in the classroom; and
- constant rewards and recognition for students’
good performance.
By shifting the locus of responsibility for academic
performance from the school to the individual student, grade retention
programs deflect time and attention away from strategies that could
potentially improve learning for a majority of students in a school
and place too much of an emphasis on a single improvement strategy targeted
only to one group of students.
Cost Effectiveness School improvement efforts
are more cost effective than grade retention programs. Funds and
effort expended for the summer program provide extra support and learning
time for students, but they are not investments in improving the instructional
environments of schools where students will spend the next nine months
after their five-week summer programs. As cited later in this report,
76 percent of students promoted after the 1999 summer session were at-risk
of retention again in the spring of 2000 and were required to attend
summer school for a second year in a row. What are the real costs of
grade retention? The newspapers have reported the price tag for the
summer school program at $120 million for the first year and $300 million
for the second year, which did not include $150 million dedicated to
install air conditioning in some schools in 2000. However, significantly
higher, but less obvious funding requirements lie in having students
repeat a year of schooling. These additional costs of grade retention
come to $236.4 million for an extra year of schooling. Over half a billion
dollars will be spent for one year’s implementation of the Board’s
grade retention policy.
High costs and a lack of effectiveness doomed New
York city’s previous experiment with ending social promotion. At
its most ambitious period of implementation during the 1982-3 school
year, the Gates Program, which only retained students in the 4th and
7th grades, 16,720 students were held over at an additional cost of
$35,268,499. This was over and above the $23.7 million supporting summer
school, early grade prevention programs, and students who were "double
holdovers." As the hidden costs became evident to city budget decision
makers, state remediation funds were substituted for city funds until
the city’s contribution dwindled to only $3 million for the Gates
services and $15.9 million for grade retention in 1989-90. By that year,
with assessments showing minimal gains from grade retention, few students
were held over. There were only 3,701 students in the Gates Program
despite the fact that 37,000 3rd and 4th graders’ test scores made
them eligible for this program.
In the same year, the State Education Department restructured
its school improvement program by targeting fewer low-performing schools,
but providing them with greater oversight and assistance. Student
academic gains have been far more impressive from this program, which
accounts for why it is still in place after a decade. In the Panel’s
1996 report on this program, Getting off the List, we
looked at the differences in the percentage of students in 6 elementary
schools who tested above the bottom quartile of test takers in grades
3 and 6 when the schools were placed in the Schools Under Registration
Review program (called SURR). On average, there was a 20 percentage
point increase in the proportion of students testing above the bottom
quartile in the 3rd grade Pupil Evaluation Program (PEP) tests in reading
and math. These student achievement gains, however, took three to five
years to secure.
Recent assessments show a continuing pattern of school
improvement among elementary schools in the SURR program, though a more
modest and mixed pattern at the middle schools. These test gains are
significant because the current standardized tests are more demanding
and are no longer normed, that is, based on the average achievement
levels of all students tested. They are now based on a criteria of how
students should perform on tests if they are at or above grade level.
In a January 26, 2000 "Status Report on SURR" to the NYS Board
of Regents, a comparison was made between former SURR schools that had
been identified during 1989-93 and those placed on the list in 1999.
In the newly identified schools just entering the state’s improvement
program, 43 percent of students tested at Level 1 in grade 4 Mathematics
and 45 percent of students were testing at that level in English Language
Arts. Among the schools that had gotten off the list by 1993, the comparable
percentages had dropped to 28 percent and 31 percent, respectively.
For every year in SURR programs, 3 percent to 5 percent more students
test out of Level 1. Even more dramatic improvements are cited by a
September 14, 2000 report of the Board of Education’s Division
of Assessment and Accountability for SURR schools placed in the Chancellor’s
District. In schools where teachers were given extra time and pay incentives,
on average, 43 percent of students had reading scores in grades 3 to
7 that placed them in Level 1 on the 1999 city tests, but on the 2000
city tests 30 percent tested at this level, a one-year improvement of
12.5 points. Mathematics one-year test gains for grades 3, 5, 6 and
7 were similarly encouraging. In 1999, 57 percent of students in these
Extended-Time schools tested at Level 1, but in the next year, 47 percent
tested at this level, a 9.6 percent drop in one year.
Board of Education School Based Budget Reports issued
in June 2000 show that, on average, per-pupil expenditures for Chancellor’s
District schools are $1,200 above those of the highest spending community
school district. The Chancellor’s District superintendent of the
district has estimated that there has been an investment of close to
$1 million for each of the 45 SURR schools that are now under her jurisdiction,
which now includes 5 high schools. This $45 million investment, however,
is showing results in reducing the numbers of students testing at Level
1. As more longitudinal data on students tests become available for
the current grade retention policy, the Board of Education should compare
these results with those of the SURR and Chancellor’s District
school improvement efforts.
Resources for Student Support Remain Insufficient
Grade retention programs appear to provide services to students that
they do not deliver, that every low-achieving child will get help to
succeed. One of the key findings of EPP’s 1989 report, The
Fourth "R": Rethinking Remediation in the Elementary Schools,
is that even though the State Education Department required diagnostic
tests for all students testing in the bottom quartile of test takers
as well as required that remediation services be provided to
all students testing at this level, there was insufficient funding from
state Pupils with Compensatory Education Needs (PCEN). Even when these
dollars were combined with federal Title 1 and the city’s remediation
funding, only a portion of low-achieving students got help. There
were too many low achieving children, so there was insufficient funding
to comply with state mandates.
Ten years ago, when the Gates Program was withering
but still in existence, EPP found that principals targeted their efforts
towards students most likely to improve with limited assistance, those
who were just below grade level. Given the inadequacy of dollars, they
were unappologetic about their need to make this choice. EPP has found
this same pattern has continued to this day. In interviews in 27
schools for two reports on successful turnaround efforts, most principals
admitted that when they had a choice of where to use limited remediation
funds, students just below grade level were targeted for these services.
They had strong motivations for not providing remediation to
students with the greatest academic needs: 1) students just slightly
below grade level tend to respond more readily to remediation, 2) school
performance is evaluated by administrators through the averaging of
all test scores, and 3) newspapers tend to focus more on the percentage
of students at or above grade level. These principals wanted to show
continuing school improvement by pushing up the numbers of students
from Level 2 to Level 3 (grade level). With limited dollars, few principal
would voluntarily choose to use funds to focus on the far more difficult
and uncertain task of trying to get more Level 1 students up to Level
2.
Many of the problems documented in The Fourth
"R" have been solved over the last decade, such as
excessive federal and state restrictions, a focus on compliance with
program requirements rather than effectiveness, and, most importantly,
a lack of attention to whole school improvement strategies which result
in far better outcomes than remediation programs. One of the unheralded
achievements of Chancellor Crew was that he forced all elementary schools
to provide literacy assistance, called Project Read, to students in
the early grades who were most likely to test at Level 1. One of his
objectives was to ensure early academic intervention for the lowest-achieving
students, who had often been bypassed for these services and whom were
most at risk for referrals to special education. But this program is
targeted only to the early grades.
Two of the most important problems, however, have
not been solved: inadequacy of funds for additional academic supports
for students and the incentives for targeting these limited resources
to students just below grade level. When principals, especially those
under the gun, have a choice in where to direct limited remediation
funds, many opt for students testing at Level 2. They are responding
logically to a gut wrenching choice of whom to help when not all students
can be helped. New state regulations tacitly recognize funding limitations.
The State Education Department’s requirements for school districts’
plans for Academic Intervention Services (AIS), issued in January 2000,
are very carefully worded so that students are made "eligible"
for services, but are not "mandated" to receive them. State
Education Department sources confirmed that officials were aware that
their previous PCEN regulations for mandatory services were impossible
to implement in many districts. The state’s Extraordinary Needs
Aid funding to New York City now exceeds $300 million. The Board of
Education has folded these dollars into a large Special Needs/Academic
Intervention Services funding stream that combines city, state, and
federal funds that total almost $1 billion dollars ($985.9 million).
Yet the estimated number of "Special Needs Children" targeted
for these funds is only 149,920 pupils (which includes special education
students), far below the 319,000 students testing at Level 1. In
short, there is money to help only about half of the lowest-achieving
student population.
4. Recommendations
Grade retention policies appeal to public and to the
press. "Accountability" frameworks, however, need to be
consistent, logical, effective, and, above all, honest. The current
grade retention policy in New York City constitutes a step backwards
in holding schools accountable for student performance. In reality,
the equation is reversed and cause and effect are confused when students
are held responsible for school performance.
The members of the Panel fully recognize the cumulative
negative impact of promoting large numbers of students who are performing
below grade on school curriculum and climate.
Grade retention policies are based on deeply held beliefs
about individual responsibility, hard work, and promotion to higher
grades based on merit. But, so far, they have not been effective strategies
to prevent large numbers of children from failing academically. Worse
still, even though these programs are costly, they under fund by as
much as half the amount of funds needed to provide academic support
services to students identified as low-performing.
The Educational Priorities Panel believes that
if funds used for grade retention were targeted to school improvement,
there would be far fewer social promotions and far fewer children testing
at Level 1 in the New York City school system. Our primary recommendation,
one that has little chance of adoption at this time, is to repeal the
current grade retention program. Our more modest and more realistic
recommendations that follow have emerged from the review in Part II
of grade retention policies in the past and in other districts. Our
assumption is that evaluations in year three and four of the current
policy will begin to track of the persistence of student low-achievement,
despite the summer school program and grade repetition. The review
has helped EPP to recognize that the "second wave" of grade
retention policies were improvements over the "first wave,"
a reminder that institutional reform is possible. School officials have
other options beyond slowly defunding and dismantling the program over
a decade, as they did with the Gates Program.
What are these other options? They can restructure
the program so that accountability for student performance is shifted
back to the schools, adopt more effective academic intervention strategies,
and, finally, provide sufficient funds to support students at-risk for
academic failure and dropping out of high school.
School and Government Accountability
1. Make the schools accountable for improving student
academic performance. Students first identified as testing at Level
1 and lacking the other two promotional criteria (attendance and course
work) should be given a "promotion with services" designation,
provided with the opportunity to attend summer school, and receive mandated
academic and social services for a full year in their home school. If
their second year test scores still place them at Level 1 and they still
lack the other two promotional criteria, their attendance in an 8-week
summer school program should be mandatory, including an initial
visual and hearing screening and a diagnosis of academic weaknesses.
Promotion at the end of this summer session would then depend on test
results.
Endemic problems in timely data collection of test assessments
and student tracking will continue to make it difficult for schools
to provide academic intervention in the spring semester. Questions are
still outstanding about whether summer school programs merely improve
test performance, not skills and comprehension. By "provisionally"
promoting students, schools are given the responsibility for efforts
to improve student performance and given a more realistic time frame.
Furthermore, this restructuring of grade retention ensures that students
will continue to get support over a nine-month period in their home
school before they are held back. Boston’s recent evaluation of
their experiment with provisional promotions, which are called "transitional"
in their grade retention program, show that these students do almost
as well on some tests as students who have not been identified as low-achieving.
At a minimum, the New York City Board of Education should pilot a program
to evaluate whether they can duplicate Boston’s experiment. Funds
that are now going towards grade retention could then be redirected
to more academic supports. The policy would still be a grade retention
policy, but students held over would be those who did not improve after
a year of academic intervention
2. Make school accountable for a higher than average
proportion of students testing at Level 1. Both the State Education
Department and the Board of Education should monitor the performance
of schools for their ability to reduce the percentages of "provisional
promotion" students still testing at Level 1, and this performance
factor should be included among the explicit criteria for identification
as a School Under Registration Review and as a Chancellor’s District
school.
As discussed in the earlier part of this introduction,
school administrators are motivated to focus more of their limited remediation
dollars on students who are performing just under grade level. Except
for the early grades where services to the lowest-achieving students
are required, these harder-to-educate students tend to have less
access to services. Longitudinal data on the earlier experiments in
Washington D.C. and New York show that this pattern ultimately undercuts
the grade retention policy. The lowest-achieving children need access
to academic intervention services.
3. Grade retention policies must be accompanied by
sufficient funds to help all low-achieving students.
- The State Legislature, the Mayor, and the City Council
should increase their funding to the New York City school district
so that all students testing at Level 1 have sufficient academic
and social services supports.
- The State Education Department should revise Section
100.2 of the Commissioner’s Regulations to require that school
district plans for Academic Intervention Services outline services
that are mandated for students testing at Level 1 and should
verify compliance.
As strategies geared to summer school and grade retention
failed to reduce the numbers of students achieving at the lowest levels
and as the resulting costs accumulate, the response of education officials
and city budget decision makers was to quietly reduce Gates until it
was shell of a program. Objective bean counters, not liberals, did the
program in. The more effective strategy of shifting accountability back
to the schools for improving student performance should ultimately reduce
the proportion of lowest-achieving students, currently close to one-third
of all enrolled students. Nevertheless, if funding is ultimately not
available to support an ambitious program of ending social promotion,
elected officials should at least be responsible enough to admit that
the grade retention program is only partially funded and that the programs
objective to help at-risk students will for the most part be unattainable.
Effective Academic Intervention
4. LADDER programs should be expanded so that all public
school students from grades K to 3 are in classes no larger than 20
students. This would require an additional investment in ending overcrowding
in elementary schools, ether through leasing or the construction of
new schools.
Careful academic research evaluations have shown that
reducing class size to 15 to 20 students in the early grades can increase
academic achievement levels by half a grade to a full grade, even through
high school, and particularly for low-income, urban children. These
outcomes are superior to those from most remediation programs. Current
federal and state class size reduction funding, even if continued for
the next year, will only bring about two-thirds of K-3 classes to an
average of 20 students. Given the cost effectiveness of this strategy
in preventing low-achievement and the resulting costs of grade retention,
summer programs and remediation, the Mayor and the City Council should
consider finding the funds to cap all early grade class size to no more
than 20 students.
5. The Extended-Day program model should be expanded
to more schools in New York City, and begin in schools where 35 percent
or more of students are testing at Level.
There has been criticism of the Extended-Day pilot that
has been in place for one year in about half [check] of the SURR
schools in the Chancellor’s District. The name of the program implies
that all students are benefiting from a longer school day, but in reality
"Extended-Day" means that teachers get paid for an additional
40 minutes a day so that they can focus on small group instruction,
staff development and school improvement efforts. Asked about the components
of the program, a teachers’ union staff member responded that they
reflect some of the planning that went into a U.F.T. proposal to be
allowed to experiment in turning around low-performing schools, just
as a business community organization, the New York City Partnership,
had been given permission to sponsor model programs in several low-performing
districts. Chancellor Crew ignored U.F.T. President Weingarten’s
proposal, but the union was able to get some of their strategies in
place through negotiations over the Extended-Day program. The key strategy
of the union was to improve the quality and effort that went into staff
development and to ensure on going implementation of new methods of
instruction. "Hit and run" workshops were to be avoided. Time
was set aside for thoughtful and collaborative planning for school improvement
and the alignment of curriculum and standards. These are common aspirations
in most low-performing schools, but less commonly achieved. EPP has
not made school-site visits to fully evaluate this model, but an average
ten-point drop in the numbers of students testing at Level 1 within
one year is a remarkable outcome that merits duplication.
PART II: REVIEW OF SCHOOL DISTRICT EXPERIMENTS
1. Purpose
The Educational Priorities Panel decided to study efforts
to end social promotion in other districts across the nation in order
to evaluate whether the considerable resources allocated to support
New York City’s grade retention policy are being well spent. Our
estimates suggest that the district had to spend approximately $236.4
million to provide an additional year of schooling for retained student
in the 1999-2000 school year alone. An additional $300 million was spent
on the summer school program for about 319,000 students in 2000, not
including the costs of air conditioning more classrooms.
By studying the experiences of other districts, we hoped
to uncover some important lessons to direct implementation and refinement
of the current promotion policy in New York City. Specifically, this
report aggregates available information about past and current promotion
policies in districts across the nation; identifies similarities and
differences between the policies as well as common trends and challenges
in their implementation; and highlights key lessons learned from evaluations
of promotion policies.
The New Wave Over the past five years, a new wave
of efforts to end social promotion’ has begun in states and school
districts across the nation. President Clinton called for an end to
social promotion in our nation’s public schools during his 1999
State of the Union Address, and the U.S. Department of Education released
a guidebook for educators and policy makers later that year. Many states
and districts were already addressing this issue, and others have been
quick to jump on the bandwagon. How extensive is this trend? Fourteen
states have adopted policies designed to end social promotion as students
move from grade to grade, and 28 states have adopted high school exit
exams based on rigorous standards, according to a report released by
the American Federation of Teachers (AFT) in 1999. The Education Commission
of the States (ECS) has identified 29 states with promotion/retention
policies in place. However, the implementation and impact of these policies
varies dramatically. New survey data from researchers at Johns Hopkins
University show that at least 51 of the nation’s 100 largest districts
provide summer school as an alternative to social promotion, and 42
of these districts retain low-performing students who do not successfully
complete the summer session.
The Debate Efforts to end social promotion have
been the subject of great controversy. Supporters of social promotion
argue that it is preferable to retention, which leads to high dropout
rates while providing few educational benefits to retained students.
Opponents say that social promotion robs students of the motivation
to achieve and the skills to participate in modern society. They also
argue that students who are promoted without the necessary mastery of
subject matter and skills "dumb down" the curriculum and instruction
at higher grades, decreasing the quality of high school education.
The balance of scholarly research on grade retention
and social promotion falls heavily in favor of social promotion, and
comparatively little research supports retention. Unfortunately, the
research on both sides of the issue is somewhat biased, as Gregg Jackson
pointed out in 1975, because few studies use a true experimental design
with random assignment of students to promotion or retention.Whole-school
reform initiatives and efforts to identify and restructure low-performing
schools have received more support among researchers because they have
produced clear student achievement gains in some high-poverty schools.
Meanwhile, elected officials, superintendents and
school boards across the nation continue to adopt strict new promotion
policies that will lead to increased retention, at least in the short
run. Why does support for retention continue in the face of extensive
research documenting its negative effects? The concepts of effort and
valid criteria for achievement resonate with the work ethic of American
culture. Even an army of researchers would have a hard time revising
the basic expectation that high school graduates should be well-prepared
for college or employment.
As the movement to end social promotion continues,
it is important to note that ending social promotion is not new. It
has been tried in many districts across the nation over the last 25
years. This report is intended to help policy makers learn from the
successes and failures of past promotion policies and use this experience
to inform future policy making.
The Sample and Data Collection Methods This report
compares policies ending social promotion in 27 school districts across
the nation over the past 25 years, and it presents key lessons learned
from the implementation of these policies, based on evaluations and
outcome data from 11 of these districts. We define "policies ending
social promotion’ as any policies that establish explicit student
achievement criteria for promotion to the next grade or for high school
graduation or any policies that provide extra help to low-achieving
students with the goal of boosting student achievement to a desired
promotion standard. (The matrix at the end of Part II summarizes the
district promotion policies included in this study.)
The districts included in this study were identified
through a review of recent education policy research, education periodicals,
and newspaper articles. We also talked with staff in 18 states and more
than 30 school districts to collect information and documents describing
the implementation of past and present promotion policies. We originally
planned to include state-level policies as well as district policies
in this study, but we narrowed the focus to district policies because
few states have implemented their new promotion policies for more than
a year or two, and none of the states we contacted had released policy
evaluations yet.
In our preliminary research and conversations with
state and district staff, we discovered that state promotion polices
vary dramatically in their implementation and impact at the local level.
For example, some states have a promotion policy in place, but it is
either too vague or too poorly enforced to have a consistent impact
at the local level. Those that establish clear promotion criteria including
standardized test scores seem to have a stronger impact on instructional
practices and promotion decisions at the district level. Lastly, some
states take a "just services" approach to ending social promotion.
They require extra services for low-achieving students to boost their
performance to a desired level, but the policy does not establish strict
promotion criteria. The relationship between state-level promotion policies
and local practices deserves more careful analysis, but is beyond the
scope of this report.
In examining district-level promotion policies, our
key research questions were the following:
- What has been tried? What is the range of policy experimentation?
- What has been learned? How have these policies worked
in practice?
- Do recent promotion policies reflect lessons learned
from past experimentation?
2. SIMILARITIES IN SCHOOL DISTRICT
EXPERIENCES
In comparing the promotion policies implemented in
27 districts over the past 25 years (and the available outcome and evaluation
data), we discovered the following nine similarities in the policies
themselves and also in their implementation and impact over time. Information
collected by the AFT, the ECS, and researchers at Johns Hopkins University,
regarding district and state promotion policies supports some of these
findings. (The matrix at the end of Part II summarizes the district
promotion policies included in this study.)
Large urban districts are taking the lead. The
districts experimenting with promotion policies are predominantly large,
urban districts. Twenty-four of the 27 school districts included in
our study are among the nation’s 100 largest school districts.
It is interesting to note that, in some cases, the adoption of a strict
promotion policy is preceded by severe problems or failures in these
large, urban education systems. For example, students in Baltimore,
Chicago and Washington DC were consistently scoring far below national
norms on standardized tests in reading and math prior to the adoption
of tougher promotion policies in these three districts (in 1978-79,
1985-86, and 1998-99, respectively). The districts that adopt strict
promotion policies also tend to serve high-poverty and high-minority
student populations, and many of these districts enroll a large number
of students who speak English as a second language as well.
Retention rates spike upward initially, when rigorous
promotion policies are first implemented, and then decrease. Rigorous
promotion policies often create dramatic spikes in retention when they
are first implemented, raising rates far above the national average
of about 6 to 8 percent retention a year in grades 1-12. For example:
- In Philadelphia, retention rates for students in grades
1-8 rose dramatically after implementation of a tough new promotion
policy in 1985, from 7 percent in 1982 to 21.1 percent in 1986.
- In Houston, 45 percent of students in grades 1-3 failed
to meet one or more of the three promotion standards during the first
year under the district’s new promotion policy in 1998-99. Only
20 percent of students were actually retained because some students
were promoted after summer school or were promoted to the next grade
even though they did not meet the standards.
- In Chicago, 20 percent of eligible 3rd graders and
10 percent of eligible 6th and 8th graders were retained in 1997 and
1998, the first two years under the district’s new policy, and
almost 1,600 students were retained for a second time in 1998.
- In Boston, the retention rate for middle school students
(grades 6 to 8) rose from 13.5 percent in 1983 to 16.8 percent in
1985, when the district’s new promotion policy was implemented,
and 30 percent of first graders were retained in 1985.
- In Pinellas County, Florida retention rates more than
doubled when a rigorous new promotion policy was implemented for students
in grades 1-8, rising from about 4 percent in 1977 to 10.4 percent
in 1978.
As implementation continues, retention rates tend
to decrease over time.
- In Chicago, data from the first two years of latest
promotion policy show that an increasing percentage of students in
grades 6 and 8 achieved the minimum test-score cutoff for promotion
in 1997 and 1998, as compared to student performance in 1995 before
the policy was instituted.
- In Philadelphia fewer students in grades 1-8 were retained
each year that the district’s policy was in place, dropping from
a peak of 21.2 percent in 1986 to 12.6 percent in 1991.
- In Pinellas County, retention rates also dropped steadily
in the after an initial spike, from 10.4 percent in 1978 to 5.9 percent
in 1981.
These steady declines in retention may indicate that more
students are reaching high standards as a result of rigorous promotion
policies or that students and staff became more familiar with the tests.
However, decreases in the retention rate after an
initial spike may not always reflect gains in student achievement or
test familiarity, but instead may represent "backpeddling."
During the implementation of Washington DC’s 1981 promotion policy,
the retention rate dropped consistently each year, from 19 percent in
1982 to 7 percent in 1986, but a rising proportion of students were
promoted "with deficiencies" each year, representing 18 percent
of students promoted in 1986. Further, the proportion of students promoted
with deficiencies increased in the upper grades, suggesting that these
deficiencies were compounding rather than being addressed after promotion.
Minority students are disproportionately retained.
The impact of tough new promotion policies falls most heavily upon minority
students, who are consistently retained at higher rates than other students
under these policies. This trend likely reflects the fact that minority
students are often concentrated in low-performing, high-poverty schools
that do not have the financial resources, qualified teachers, or effective
administrators needed to prepare students for rigorous promotion standards.
- African-American students in Chicago were 4.5 times
more likely to be retained than White students in 1997 (18 percent
vs. 4 percent retention rates), and Hispanic students were nearly
three times more likely to be retained than White students (11 percent
vs. 4 percent retention rates).
- Pinellas County, Florida also retained minority students
at much higher rates than other students after its promotion policy
was implemented. Non-White students in grades 1 to 5 were retained
at three times the rate of White students (28 percent vs. 9 percent)
in 1978, but the gap closed somewhat in following years (11 percent
vs. 5 percent, respectively in 1982).
- Houston’s new promotion policy produced similar
outcomes in 1999, 24 percent of African-American students in grades
1 to 3 were retained, as compared to 19 percent of Hispanic students
and 16 percent of White students.
- Philadelphia’s 1985 promotion policy also retained
minority students at higher rates than their White counterparts. In
1989, 31 percent of Hispanic students in grades 1 through 8 and 23
percent of African-American students were retained, as compared to
18 percent of White students.
Higher retention is followed by dramatic increases
in dropout rates. Most of the data available come from the "first
wave" of policies, with the exception of Chicago, because more
recent policies have only been in place for two or three years. Districts
that conducted longitudinal policy evaluations reported that dropout
rates increase over time after rigorous promotion policies are implemented.
Perhaps the most often cited example comes from an evaluation of New
York’s 1981 Promotional Gates Program, which showed that approximately
40 percent of students retained in the 8th grade later dropped out of
school, while only 25 percent of students who were not retained dropped
out during that time.
- The evaluation of Chicago’s latest promotion policy
showed that dropout rates for low-achieving 8th graders increased
after the implementation of the district’s new policy, from 8
percent in 1995 to 10 percent in 1997, and the dropout rate among
8th graders retained in 1997 was even higher at 16 percent. An older
study conducted in Chicago, during the time that a rigorous promotion
policy was adopted in the early 1980s, reported that the students
who had already been retained once upon entry to high school were
more likely than other students to drop out before completing high
school.
- Another longitudinal study of students in Miami-Dade
County in the 1980s showed that students who had been retained once
dropped out at twice the rate of students who progressed normally.
- Boston’s drop out rate, which was already high,
also increased during the time that the district was implementing
a tough new promotion policy, from 36 percent in 1982 to 43 percent
in 1985.
We cannot assume that promotion policies cause increased
dropout rates, because a number of different factors influence each
student’s decision to drop out. But there is clearly a correlation
between grade retention and likelihood of dropping out. The well-known
Youth in Transition study found that if students repeat a grade once
during their schooling, their chances of dropping out increase by 40
to 50 percent, and if students are held back twice, their chances of
dropping out increase by 90 percent.
Data problems hinder the implementation of many promotion
policies. Problems with data collection and student tracking riddle
the implementation of many promotion policies and limit the quality
of policy evaluations.
- An evaluation of Houston’s new promotion policy
identified data problems as the main challenge in the first year implementation
of the policy in 1999. Of the 21,000 students eligible for the inclusion
in the policy, 7.2 percent of students had missing data for every
promotion standard, and only 71 percent of students were assessed
on all three required standards. Among the students who were assessed
on all three standards, 3.2 percent passed one or none of the three
required standards but were "incorrectly promoted despite their
lack of academic success." Further problems with missing data
made it impossible for the evaluators to determine whether students
promoted after summer school had actually earned their promotions
or not.
- The Baltimore public school district didn’t officially
evaluate its 1978 promotion policy and changed the standardized tests
used to measure student achievement a few years after implementing
the policy, making it difficult to measure long-term progress in student
achievement.
- The Corpus Christi school district in Texas has reported
difficulties in maintaining course completion records for students
who are promoted during the regular school year, and they have also
reported needing better analysis and more timely dissemination of
student test data to support their promotion policy.
Many districts have difficulty getting student achievement
data early enough in the school year to identify struggling students
and provide the extra instructional services they need to avoid retention.
Very few of the districts in our sample officially identify students
at-risk of retention before the middle of the school year (January),
and many districts wait until spring to notify parents that their children
may not meet the promotional criteria. Often, delays and problems in
identifying students at risk of retention are caused by the schedule
of standardized tests. Because it is difficult to identify students
at-risk of retention early on, many districts focus on remediation and
other "catch up" services to address educational problems
after students have been designated for retention, rather than beforehand.
This may explain why summer school remediation is such a popular support
strategy, because it is not until the late spring that testing data
becomes available.
Attendance is a serious problem for most summer school
programs that support promotion policies. Many districts implementing
tough new promotion policies have problems with non-attendance in summer
school programs intended to boost the performance of low-achievers.
For example:
- Less than half (42 percent) of the students in Houston
who were designated for summer school in 1998-99 actually enrolled.
- In Cincinnati, only 52 percent of students identified
for retention in 1994 attended summer school.
- The school district of Hartford, Connecticut had less
than 40 percent attendance in summer school in 1999 among students
identified for the program in connection with their promotion policy.
- During the same summer, nearly half of the 14,000 students
identified for summer school in Oakland, California chose not to attend.
Some evaluators and district officials point to late or
ineffective communication with parents as a key source of attendance
and participation problems. For example, parents in some districts were
not notified about mandatory summer school until the end of the spring
semester. Earlier and more effective communications with parents to
encourage summer school participation may help resolve attendance problems.
Promotion policies cost big bucks. District spending
on promotion policies and the services to support them varies dramatically
across the districts included in this study.
- For example, New York City spent about $300 million
on summer school for over 319,000 students in 2000.
- Chicago budgeted about $86.5 million on its promotion
policy and related services including after-school and summer programs
in 2000-2001.
- Los Angeles planned to spend $71 million on its new
promotion policy in the 1999-2000 school year but eventually scaled
back the program.
- Washington DC spent $10 million on its summer school
program for 20,000 students in 1998.
Some of the most obvious expenses related to strict promotion
policies are the extra educational services provided to low-achieving
students, such as after-school and extended day programs, Saturday school,
summer school, and reading and math specialists. For example, San Diego’s
new "Blueprint for Student Success," which establishes three
promotional gates, will provide numerous services to students throughout
the elementary and middle school system at a cost of approximately $61
million in the 2000-01 school year alone, more than five percent of
the district’s $1 billion annual budget.
Promotion policies have significant hidden costs as
well. The total cost is not limited to summer school and extra services,
but also includes additional spending for extra years of schooling for
retained students. Lorrie Shepard, one of the nation’s leading
experts on social promotion and grade retention, estimated in 1989 that
our nation spends almost $10 billion annually on retention in the public
schools.
Efforts to end social promotion are often sporadic
and short-lived, as some districts try over and over again to end social
promotion. At least nine out of the 28 districts we studied have
adopted policies to end social promotion two or more times in the last
25 years. Some of the largest districts, Chicago and Philadelphia, have
been the most dogged in their efforts, implementing three or four different
promotion policies during this time. Other districts have tried at least
two different policies, including, New York, Baltimore, Boston, Milwaukee,
San Diego, and Washington DC. Most of the "multiple experimenters"
established their first promotion policy sometime in the 1980s, when
the standards movement was beginning, and discontinued the policy later
(often due to negative student outcomes and/or high costs of retention).
Then, they tried again in the late 1990s as part of an effort to improve
student achievement. These multiple experimenters provide valuable information
about how promotion policies are evolving over time. As discussed in
the next section, we believe recent promotion policies are beginning
to reflect lessons learned from past experiments.
Many districts we studied implemented promotion policies
for just a few years at a time. This short time frame for experimentation
may be a result of the politics surrounding education policy. Elected
officials and school board members largely determine state and local
education policy, so policy can change quickly when new officials sweep
into office and want to take action. For example, Milwaukee implemented
a new district promotion policy during the 1999-00 school year. But
the district is already replacing its policy, in response to a newly
enacted state law, with a different one scheduled to take effect in
2002 and 2003.
The departure of key personnel from schools or districts
can also make it difficult for districts to maintain continuity in their
efforts to end social promotion. For example, Seattle’s recent
effort to end social promotion was described as a pet project of Superintendent
John Stanford. The district implemented a rigorous promotion policy
under his leadership in 1996, establishing a system of "exit profiles"
for students in grades 5, 8, and 11. Unfortunately, Superintendent Stanford
died a few years later, and although many of his reforms have been continued,
the promotion policy was eliminated in 1998.
Some districts’ promotion policies improve over
time. Even when data is collected in a timely and accurate manner,
outcome and evaluation information may not be used effectively to inform
classroom practices or later policy making. In the course of this research,
we talked with staff in 18 state education agencies and more than 30
school districts across the nation, and the lack of institutional memory
about past policy initiatives was striking and consistent across the
board. It was common for staff in areas such as policy planning, evaluation,
assessment, and accountability to have little knowledge of past promotion
policies and no easily accessible record of these initiatives, much
less their evaluations. The Virginia state legislature actually eliminated
the State Education Agency’s Office of Evaluation in the mid-1990s.
This loss of institutional memory may make it difficult for lessons
learned from past experiences to be incorporated into the implementation
of new policies.
Despite the absence of "institutional memory"
and the lack of continuity in many districts, we find that some district
policies are evolving over time in more promising directions. The following
trends seem to be emerging in the evolution of rigorous promotion policies:
- Recent promotion policies often make use of multiple
measures of student achievement, whereas older policies commonly
relied upon a single standardized test score;
- Recent promotion policies include a variety of support
services for low-achieving students, whereas older policies typically
provided few extra services for struggling students. However, there
is still room for improvement in the effectiveness and consistent
delivery of support services;
- Recent promotion policies provide more alternatives
to retention, such as summer school, transition classes, and mid-year
promotions, for students who do not meet the promotion standards,
whereas older policies relied upon repetition of the same courses
through grade retention as the main response to academic failure;
and
- Recent promotion policies require more extensive public
reporting of student outcomes, whereas there was less consistent public
reporting on the impacts of past policies. However, independent evaluations
of promotion policies and their long-term effects are still rare.
These trends, although not universal, suggest that policy
makers can learn from past experiences and use this knowledge to inform
future policy making. New York’s introduction of two additional
standards for promotion, attendance and teachers’ assessment of
the quality of classroom work, are often perceived by the tabloid press
as "loopholes." The use of multiple measurements represent
a better targeting of resources to students who need help and an effort
to eliminate more students whose home schools are low-achieving.
3. DIFFERENCES IN GRADE RETENTION
POLICIES
Below, we discuss four key areas of variation in the
both the characteristics and impacts of different promotion policies.
This broad range of experimentation provides a variety of models, although
still unproven, for educators and policy makers across the nation. (Appendix
Tables 1 and 2 summarize all district- and state-level policies included
in this study.)
Various approaches focus on students in different grades.
Gates: Establishing promotional standards for
students at key "gateway grades" is a popular approach. (We
will refer to these programs as "gates" programs.) About half
of the districts examined in this study (14 of 27) have implemented
promotional gates at various grades. The idea is to measure student
achievement at key developmental stages to ensure that children have
the basic skills needed to benefit from the next level of education.
Third or fourth grade is often chosen as a gateway year to test reading
skills, and later gateway grades are often chosen to test basic math
and higher level literacy skills.
All Grades: Some districts have adopted promotion
policies that apply to all grades or many consecutive grades. This type
of policy is sometimes implemented as part of a larger effort to raise
standards for students in every grade. For example, Corpus Christi (1-12)
and Houston (1-8) are both currently undertaking substantial standards-based
reform efforts. Corpus Christi is also implementing an extensive middle
school reform, which complements the promotion policy, supported by
a special foundation grant. Boston also phased-in a consecutive grade
promotion policy in the mid-80s, after implementing an extensive standards-based
reform initiative throughout the district.
Ungraded: Classroom structures that mix students
of different ages in primary or secondary school are another, less common
approach to ending social promotion. This requires teachers to work
with multi-age groups of students, usually for at least two years, to
ensure that they master certain content and performance standards during
those years. The non-graded approach give students more time to master
different concepts, allowing them to progress at their own pace, while
avoiding the stigma of retention. Kentucky implemented non-graded primary
classes (for 5 to 7 year olds) throughout the state as part of the Kentucky
Education Reform Act of 1990. Philadelphia also briefly experimented
with non-graded elementary and middle schools from 1993 to 1995, and
Washington DC tried multi-age classes as part of its 1981 promotion
policy, which allowed students to be promoted at the end of every semester.
Evaluations of these policies show that teachers have a hard time working
with multi-age classes and request more professional development and
new teaching strategies to support this type of class structure.
Exit Exams: Rigorous high school exit exams
are gaining popularity, especially at the state level. Many states and
districts have had minimum-competency tests for high school graduation
in place for a number of years, but these exams typically test knowledge
at a very basic level and have pass rates above 90 percent.For example,
New York City has required all high school seniors to pass the Regents
Competency Tests (RCTs) since 1985, which tested knowledge and skills
at an 8th grade level. The new exit exams being adopted across the nation
are much more rigorous. Seven of the 27 school districts we studied
are implementing new policies requiring students to pass standardized
tests based on new state or district standards in order to earn a high
school diploma.
Measures of achievement are the source of major controversy.
Standardized test scores are by far the most common student achievement
criterion in the promotion policies we studied. A key distinction between
policies is whether test scores are the only criterion or one of many.
Researchers tend to criticize the use of standardized test scores as
the sole measurement of student achievement because these tests do not
accurately measure the extent of a student’s knowledge and abilities.
Because they are not precise measures, test-score cutoffs can lead to
arbitrary promotion decisions. On any given day, some capable students
may fail to meet the cutoff, while other students with serious academic
deficiencies may get lucky. In addition, it is difficult to determine
what portion of a test-score gain is due to genuine learning as opposed
to increased familiarity with the test. As a result, many districts
(and some states) have established "multiple measures" of
student achievement in their latest promotion policies. Standardized
test scores are usually one of the measures.
Almost every district in our sample assesses student
achievement in reading and math under their promotion policy. Some districts,
such as Corpus Christi and Milwaukee, require student performance data
in as many as four or five different subjects for promotion decisions.
However, the more subjects required, the more assessments students must
undergo. Teachers, parents and school administrators are already balking
at the amount of time students spend in testing.
If the promotional criteria are not well aligned with
the curriculum, two problems may develop. First, teachers may focus
primarily on the material and skills required for promotional tests
and sacrifice other elements of the curriculum or deeper exploration
of topics. Alternatively, teachers may focus on the curriculum without
emphasizing the content and skills required for promotion. In this case,
the promotional test is not a valid measure of what students have learned.
Alignment issues have been raised in a number of evaluations as a major
challenge in implementing promotion policies, because the standardized
tests many districts use to measure student achievement, such as the
Iowa Test of Basic Skills, the Stanford-9, and the Metropolitan Achievement
Test do not reflect the specific curriculum and instructional practices
of individual districts.
Some districts do not use standardized tests at all;
instead, they rely upon teacher evaluations of student performance for
promotion decisions. Cincinnati and Corpus Christi have both developed
extensive guidance and scoring "rubrics" to standardize teacher
evaluations, and Long Beach is also developing clear scoring guidelines
for teachers under its new promotion policy. Washington DC was one of
the pioneers of this approach with its 1981 promotion policy based on
teacher judgments rather than standardized test scores. However, these
programs often have problems ensuring the validity and reliability of
teacher judgments. After four years of implementing and refining a promotion
policy based on teacher evaluations, researchers reported that the Cincinnati
school district’s rubric system was not producing accurate, consistent
measurements of student achievement and was therefore "not suitable
for making individual student decisions." An assessment of Washington
DC’s 1981 policy also seriously questioned the validity and accuracy
of teacher evaluations of achievement.
Districts and states are trying a variety of support
services and alternatives to retention.
Summer School: Summer school is the most common
supplemental service used to boost the performance of low-achieving
students, among districts in our sample. Every one of the 27 districts
we studied provided summer school programs to students who did not meet
promotional standards at the end of the school year. Many of the summer
school programs in districts we studied provide: smaller class sizes
than during the regular school year; a focus on remedial instruction
in math and reading; and more individualized instruction. Some districts
also emphasize after-school and extended day programs during the regular
school year to provide more "time on task" for students who
are struggling with basic skills. For example, Chicago’s latest
promotion policy is supported by the "Lighthouse After-School"
program, and the district has recruited thousands of volunteer tutors,
including retired teachers and community members, to help staff the
program.
Summer and Mid-Year Promotion: Districts are
also developing innovative alternatives to retention for students who
do not meet the promotion standards. The most common alternative among
the district and state policies we studied is promotion after summer
school. Some districts promote all students who "successfully complete"
summer school, which is usually defined as a combination of attendance
and/or classroom performance. However more districts are shifting towards
retesting at the end of the summer and retaining students who do not
reach the promotional standards. Other districts allow retained students
to be retested and promoted at different times during the regular the
school year. For example, the districts of Tacoma, Chicago and Corpus
Christi all allow retainees or transition students to be promoted during
the regular school year.
Transition Classes: Transition classes or transition
centers are another popular an alternative to retention for students
who have been retained before or are significantly overage for their
grade. Transition programs are an effort to find a middle ground between
retention and social promotion. Sometimes these programs allow students
to be promoted along with their peers, but they require students to
continue working on the standards that they have not yet achieved. In
most cases, transition programs have a heavy focus on remediation of
basic skills in math and reading.
Phase-in Period: Additionally, a number of
school districts take a more supportive approach by phasing-in a promotion
policy slowly over time, so that teachers have time to prepare for the
new standards. For example, Durham, NC began its current gates program
at the 8th grade in 1995, added the 5th grade two years later, and then
applied the policy to 3rd grade students in 1999. The Long Beach school
district phased in its three promotional gates at grades 8, 5 and 3,
over four years from 1997-98 to 2000-20001, and each year they added
new supplemental services for struggling students in non-gateway grades
as well.
Districts and states use different combinations of
caps, waivers, exclusion and backpedaling to soften the impact of rigorous
promotion policies.
Caps: Many promotion policies establish a cap
on the number of times a student can be retained in each grade or in
a certain set of grades. Among the 29 districts we studied, 24 districts
had a cap on retention. These caps commonly set a maximum of two retentions
in the grades affected by the promotion policy, or a maximum of one
retention per grade. Caps on retention reflect concerns both that classmates
who are substantially over-age could negatively affect the learning
environment for other students, and that students who have been retained
many times are much more likely to dropout of school.
Waivers: Another method of softening the effects
of strict promotion policies is to "waive" the promotion requirements
for certain students. Waivers are especially popular with districts
that rely on standardized test scores as their only criterion for promotion
decisions. For example, the Chicago and Durham school districts both
use a single test to determine whether students have met the promotion
criteria, and they have both instituted waiver or appeals processes
that allow teachers, principals and parents to argue, based on other
measures of achievement, for the promotion of students who do not meet
the test score cutoffs.
In some districts, waivers have been used to avoid
very high retention rates among minority students who attend low-performing
schools. The rationale is that these students have not had the educational
opportunities and preparation necessary to meet rigorous new promotion
standards, and waivers are considered preferable to massive student
retentions at these schools. However, if the use of waivers holds steady
over time, this could signal that waivers are being used as a safety
valve to reduce retentions without actually improving student achievement
at low-performing schools. (On the other hand, students who received
waivers under Chicago’s most recent promotion policy have performed
similarly to students who were retained. For example, almost 2,000 6th
grade students in Chicago were promoted with waivers in 1997. About
60 percent of those students were able to meet the 8th grade promotional
standards two years later.)
Exclusions: Excluding certain groups of students
from the promotion policy is another popular way to "end social
promotion" without retaining huge proportions of students. The
vast majority of promotion policies we studied exempt some or all special
education students from the promotional requirements, depending on whether
their Individual Education Plans (IEPs) state that they will be tested
with all the other students. Also, districts often exclude students
who are English-language learners (ELL) from promotion policies for
the first two years they are enrolled in the district, and some districts
continue to exclude students for additional years if their English skills
have not reached a certain level. These exclusions can account for a
large percentage of students in some districts. For example, 31 percent
of 3rd grade students and 20 percent of 6th and 8th grade students were
not subject to Chicago’s latest promotion policy during the first
two years of implementation in 1997 and 1998, based primarily on special
education and bilingual exclusions.
Backpedaling: Lastly, "backpedaling"
is another common strategy for districts and states that have second
thoughts after realizing the potential impact of their promotion policies.
For example, Los Angeles dramatically scaled-back the implementation
of its new promotion policy in 1999, from grades 2, 3, 4, 5 and 8 to
only grades 2 and 8, when it became clear that about 40 percent of the
districts students would not meet the promotion standards. Gwinnett
County, Georgia also decided in 1998 to ease off its initial plans of
implementing gateway exit exams for student promotion until it had better
estimates of the students at risk of retention and the extra services
needed for these students. Similarly, many of the first states to adopt
high school exit exams have backed off their original implementation
plans or lowered the minimum passing scores after early testing indicated
that large proportions of students might fail the exams. Maryland pushed
back the implementation date of its high school exit exam from 2005
to 2007;
Virginia eased its new exit exam requirements, allowing
students who fail the state test to graduate anyway if they pass other
standardized tests; and Massachusetts lowered the minimum required score
on its high school exit exam in 1999 to just above the "failing"
category.
4. SUMMARY OF FINDINGS FROM EVALUATION
REPORTS
As part of this research effort, we reviewed the evaluations
of past and present promotion policies implemented in 11 districts over
the past 25 years. (A list of these evaluations is included in the matrix
at the end of Part II.) In this section we present five key findings
from these reports, including common outcomes of promotion policies,
difficulties in implementation, and recommendations of evaluators.
Rigorous promotion policies do not always improve the
achievement of retained students. The reports on achievement of
retained students are mixed and highly disputed. Some evaluations report
that retained students make substantial educational gains during the
year of retention. For example:
- An evaluation of the Pinellas County promotion policy
showed that students retained in 1978, especially those retained in
the early grades, made substantial gains during a year of retention.
Following the year of retention, students showed a median decrease
in grade-equivalent deficit of six months in reading and one year
in math. In fact, students who were retained in 4th grade had a median
score of eight months above the national norm for 5th grade math at
the end of their 5th grade year.
- Under Philadelphia’s 1985 promotion policy, retained
students also made gains, on average, in terms of normal curve equivalents
(NCEs) on standardized tests by the end of the year of retention.
However, a sizable minority of students who were retained in 1987
actually scored lower on standardized tests at the end of a year of
retention in 1988. Overall, 23 percent of students retained in grades
1 to 8 had lower NCE scores in reading and 17 percent had lower NCE
scores in math at the end of a year of retention.
Other districts report that socially promoted students
perform as well or better than students retained under rigorous promotion
policies. For example:
- In an evaluation of New York City’s 1981 Promotional
Gates program, Ernest House reported that students who had been retained
under the policy had similar test scores in later grades to a matched
group of low-performing students who had been socially promoted before
the policy.
- Similar findings are reported in the evaluation of
Chicago’s latest promotion policy, which established control
groups for retained students. The authors report that retained did
not do better than previously socially promoted students. "Only
one-fourth of retained 8th graders and one-third of retained 3rd and
6th graders in 1997 made "normal" progress during the following
school year," meaning they stayed in school and passed the promotional
cutoff. Further, retained third graders performed worse than their
socially promoted counterparts. They gained only 1.2 grade-level equivalents
over 1997 and 1998, while students with similar test scores who had
been promoted prior to the policy made gained 1.5 grade-level equivalents
during the same time period.
Students on the cusp of promotion continue to struggle
in later years. Many school districts provide intensive summer school
programs to boost a number of low-performing students up to the promotion
standard. But students that are promoted in this way tend to fall behind
again in the next grade. For example, in Chicago 13 percent of 6th graders
failed to meet the promotional criteria at the end of the 1996-97 school
year, but achieved the promotion standard after the Summer Bridge program.
However, these students returned to lower level achievement rates after
being promoted and were again at-risk of being retained in 8th grade.
The evaluators concluded, "While Summer Bridge raised students’
performance briefly, there is no evidence that it altered the pattern
of school achievement for these students." They provide two
possible explanations for the poor school year performance of students
who make achievement gains during summer session. First, summer gains
could reflect increased familiarity with promotional tests, rather than
true learning. Second, these students may receive weak school-year instruction
or they may require more sustained, additional help throughout their
schooling, rather than one-shot summer school interventions.
Data from New York City’s summer school program
in 1999 follows the same pattern. About 57 percent of students who took
the promotional test again after summer school passed and were promoted
to the next grade, but these students had trouble keeping up during
the regular school year. In fact, 76 percent of students promoted
after the 1999 summer session were at-risk of retention again in the
spring of 2000.
The lowest achieving students do not seem to be helped
by retention or the standard support services they receive. Rigorous
promotion policies, by their very nature, identify a group of very low-performing
students among retained students, who are unable to boost their performance
after a year of retention and additional services. For example:
- 64 percent of 7th grade students in Philadelphia who
were retained in 1988, still failed to meet the promotion standard
at the end of a year of retention. Overall, almost 40 percent of students
in grades 1 to 8 who were retained in 1988 did not meet the promotion
standards at the end of a year of retention.
- The pattern is similar in Chicago. Only about one-third
of 3rd and 6th graders retained in 1997 passed the promotional standard
in 1998 at the end of a year of retention. After attending summer
school in 1998, only 43 percent of previously retained 3rd graders
and 47 percent of retained 6th graders were able to meet the promotion
criteria. In addition, the majority of students who were retained
at each grade level under the new promotion policy in 1997 failed
to reach the promotion standards in both reading and math, not just
one subject. These students were further behind on average than students
who failed only one subject, and they also benefited less from summer
school.
- A longitudinal evaluation of New York City’s 1981
Gates Program reported that 1982-83 "Gates students," who
had been retained once in reading or math and who were still unable
to meet the promotion criteria after a year of retention, showed "at
best minimal gains" in both reading and mathematics by the 1986-1987
school year. Among students who had been retained more than once,
"extension-eligible" students, less than 25 percent of students
who were retained in reading were still "on-grade" by the
1986-87 school year, and less than 10 percent of extension-eligible
students retained in math were on grade level by that time.
Many of the lowest achieving students are designated
for special education after repeated failure under rigorous promotion
policies, but it is not clear that these students benefit from or belong
in special education classes. For example, approximately one in
ten third graders and sixth graders who were retained during the first
year of Chicago’s new policy in 1997 were excluded from the policy
in 1998 primarily because of special education designation.
The repeated failure of this group of students suggests
that they are not well served by the typical one-size-fits-all, supplemental
services provided to low-achieving students in many district grade retention
programs. Evaluators suggest that these students need more individualized
attention and instruction geared specifically to their learning difficulties.
The continuing failure of this group eventually leads to the cancellation
of rigorous promotion policies over time, because the negative effects
on the lowest-achieving students are so severe.
Transition programs, a popular alternative to retention,
vary greatly in their approach and effects. Evaluation and outcome
data from a variety of "transition" programs show that the
details of these policies make a huge difference. Transition programs
seem to work well when students are not isolated from regular classes
or regular schools, but instead receive supplemental services to help
accelerate their learning.
- One example of this approach is the "Zero Period"
program in Corpus Christi, Texas, where students who have not reached
all of the required standards can be promoted on a probationary basis,
provided that they come to school an hour early each day (during the
zero period) to work on the concepts they have not yet mastered. In
1999-00, about 66 percent of students who participated in this transition
class were eventually promoted during the following year, instead
of being retained.
- The San Diego school district has also developed an
innovative transition program that helps low-achieving students keep
up with their peers by providing these students with a double-dose
of classes in math and reading. The extra "parallel classes,"
which are aligned with the core math and reading classes that all
students attend, focus on basic skills underlying the higher-level
curriculum that are causing problems for low-achieving students.
- The Long Beach school district has also had much success
with its special transition center, the Long Beach Preparatory Academy,
a special school for 8th graders who failed two or more subjects.
Instead of being promoted to high school, low-achieving 8th grade
students spend a year at the Academy, where they receive extra attention
from school staff, small class sizes and extra long class periods.
The Academy opened in 1997-98 and reported a 93 percent promotion
rate (266 students) for the first class that attended the school.
In other school districts, transition classes seem
to create a "slow track" for failing students, reducing the
expectations for these students. In Chicago, where students attend Regional
Transition Centers for over-age 8th graders on separate school campuses,
the district’s lowest passing and promotion rates in 1998 were
those of 8th graders and Transition Center students. Only 27 percent
of retained or Transition Center 8th graders met the promotional criteria
in 1998, and approximately 38 percent of these students reached the
criteria after summer school in 1998, but this is still substantially
lower than the 47 percent pass rate of retained 6th graders in 1998.
In addition, about 15 percent of retained 8th graders and Transition
Center students moved or transferred to another school district between
fall 1997 and fall 1998, and approximately 16 percent of the remaining
retained 8th grade students and Transition Center students dropped out
of school during the same time period. In comparison, only 6 percent
of all 8th graders in the district dropped out of school between 1997
and 1998.
Students who are struggling do not always receive the
supplemental services they need to be successful in school. After
students have been retained, they do not always receive support services
or remediation to address their academic difficulties.
- For example, an evaluation of Washington DC’s
1981 promotion policy found that although remediation and other academic
support services were available, very few of the retained students
received these services. The vast majority of students who were promoted
received assistance from math and reading specialists and teachers
aides five days a week (74 percent of students who passed in reading
and 75 percent of students who passed in math), while few students
who were retained used the services at this high rate (only 28 percent
of students retained in reading and 23 percent of students retained
in math).
- Similarly, an evaluation of New York City’s 1981
Promotional Gates Program showed that less than five percent of students
retained in 7th grade in 1982-83 had participated in the Dropout Prevention
and Attendance Improvement programs targeted to at-risk students over
the following four years. In comparison, students who were not retained
participated in these programs at higher rates. This pattern holds
with summer school services as well, in that the lowest achieving
students are the least likely to participate. In Houston, the lowest
turnout for summer school in 1999 was among the lowest achieving students,
those that failed all three promotion standards. Only 20 percent of
students in this group attended summer school, as compared to 42 percent
attendance among all students designated for summer school. The evaluation
of Chicago’s latest promotion policy also estimates that summer
school non-attendance rates were almost twice as high for the lowest-achieving
students, those who failed in both math and reading, as for students
who failed in only one subject (20 percent vs. 10-13 percent).
Students who make educational gains during a year
of retention tend to fall behind again in later years.
- In Philadelphia, only two years after being retained,
27 percent of students in grades 2 to 9 were scheduled to be retai
