REPORTS

REVIEW OF EXPERIMENTS WITH GRADE RETENTION
December 2000

Review Author: Kristi Kimball
Intro: Noreen Connell

TABLE OF CONTENTS
PART I: INTRODUCTION
1. EPP’s Opposition to BOE’s 1999 Grade Retention Policy
2. Lessons Learned from the Review of Other School District Experiments
3. Why EPP Remains Opposed to the Current Grade Retention Policy
4. Recommendations


PART II: REVIEW OF SCHOOL DISTRICT EXPERIMENTS
1. Purpose
2. Similarities in School District Experiences
3. Differences among Grade Retention Policies
4. Summary of Findings from Evaluation Reports and Review

PART I: INTRODUCTION

1. EPP’S OPPOSITION TO THE NEW YORK CITY BOARD OF EDUCATION’S 1999 GRADE RETENTION POLICY

The New York City public school district was one of the earliest and has been one of the most consistent experimenters with grade retention policies in the post-World War II period. In 1981, the district launched the "Promotional Gates" system-wide program, which replaced individual teacher and school decision making on the students who should be required to repeat a grade. It created achievement benchmarks that students in grades 4 and 7 had to reach in order to be promoted to the next grade. The program was discontinued in 1990 because longitudinal data showed that students retained through the program performed poorly over time and were more likely to drop out of school than similar students who were not retained. [1] The district's latest system-wide policy, first implemented in the 1999-00 school year, once again established clear promotion criteria, based on standardized test scores, attendance, and teacher assessments, for students in every grade from 3 through 11. [2] Since system-wide grade retention policies were already in effect for the high schools, based on studentsŐ credit accumulation and test scores, the new policy only effected grades 3 to 8.

"Accountability" and "learning to high standards" are magic words in public education policy, and, for this reason, were liberally used when the new initiative to end social promotion was introduced by the Chancellor and Mayor in 1999. The Educational Priorities Panel, however, is charged to go beyond idealistic slogans to evaluate the soundness, the costs, and the effectiveness of programs to improve public education by monitoring the budget and administrative practices of the New York City Board of Education. As a coalition of organizations with a range of positions on social promotion and grade retention, EPP supported the introduction of the grade retention policy in 1981 by the Board of Education and opposed the Board’s re introduction of grade retention policies in 1999. Our shift in positions stemmed largely from our institutional memory of how the 1981 Promotion Gates Program played out. As data accumulated that students who were held over at the 4th and 7th grades needed more than grade repetition to achieve at acceptable levels, city budget decision makers invested less in the program as the true costs of providing student academic support became evident. The city was unwilling to provide sufficient funds to make a genuine effort to end social promotion.

Since a majority of members of EPP in 1980 remained core members 20 years later, our coalition’s rejection of the new policy reflected an unwillingness to re experience the same trajectory of initial high expectations, followed by sobering data, and ultimately a refusal by city budget decision makers to back up their rhetoric with dollars. "Been there, done that, let’s not go through this again," was the prevailing sentiment.

Despite the opposition of many education reform and parent organizations, the Board of Education adopted the policy in a vote of 6 to 1, with only the Manhattan Board Member, Dr. Irving Hammer, voting in opposition. Its passage was predictable. The intellectual author of the policy was Mayor Rudoph Giuliani. On the verge of a campaign for the United States Senate, he was eager to claim that he had "ended social promotion." Chancellor Rudolf Crew, attempting to patch up his relationship with the Mayor after an apparent drop in test scores and his opposition to taxpayer-funded vouchers for private and parochial schools, rushed through the implementation of the plan starting with only grades 3, 6 and 8.

There were other reasons why the New York public school system embarked on yet another attempt at grade retention despite the dismal outcomes of it first experiment. A powerful political myth had developed among conservative groups that the Gates Program had been sabotaged by liberals. Given the turnover of education reporters for the city’s daily newspapers, the press took scant interest in the previous evaluations of the Gates Program and tended to paint opponents of grade retention as fuzzy-minded policy wonks worried about students’ self-esteem. If anything, most education reporters were hard-liners on grade retention, questioning why students should be exempted from repeating a grade if they had good attendance or their class work was acceptable. The implication was that only standardized test scores should count and that a "real" policy would force as many students as possible into summer school and prevent as many students as possible from moving on to the next grade. The tabloids, in particular, were as vigilant about any signs of "leniency" as they were about the Chancellor’s administrative missteps. The first evaluation’s findings that 7th grade students mistakenly promoted and who did not attend summer school did marginally better on tests than summer school students received scant attention from reporters. [3]

Once the grade retention policy was in place, the Education Priorities Panel had two options: A) to monitor the implementation of a program that we fundamentally opposed or B) to look at a variety of other school districts’ experiments with grade retention to gain a perspective as to how New York City’s current program could be improved. The Panel chose option B on a hunch that future evaluations would trigger a need for change as it became more evident that summer school and grade repetition were not "magic bullets." We were hunting for ways of avoiding the same trajectory followed by the earlier Gates Program, a steady increase of over-age students and dropout rates as funding for their academic support evaporated. We did not want to wait 10 years until a new Chancellor or a new Mayor had the courage to publicly declare a formal end to a failed grade retention policy. If the policy could not be repealed within a few years, was there a way to at least improve the program enough to prevent high dropout rates and declining student achievement?

2. Lessons Learned from the Review of School District Experiments

Part II of this report contains summary of what could be learned from interviews and documents from 27 school districts across the nation about their experiments with grade retention of the last 25 years. This information allowed us to view New York City’s current program in a larger context and gain perspective. Here is a summary of the key lessons we learned from Kristi Kimball’s review:

In retrospect, the problems in implementing the New York City summer school program seem endemic to grade retention programs in most school districts. By reviewing over 27 grade retention programs, the NYC Board of Education’s performance and criteria for promotion seem better than those of many other school districts. The review, however, also looked at 11 evaluations of grade retention policies for their impact on student achievement, including the Gates program and Chicago’s current program. Besides bolstering our hunch that the current grade retention policy in New York City will be no more successful than the previous policy, these evaluations also provide a clue to the reasons why these programs tend to last for only five years, even though they may survive longer as empty policies:

Irrespective of the sorry outcome of the Gates Program, what was the sense of creating a program geared to identifying low-achieving students that then provided them with inadequate academic services? It may well be that as it became evident that strategies put in place to help academically at-risk children were not working, the sheer numbers of students who remained low achieving began to push up the projected costs of the Gates Program. As the numbers increased, the services decreased for this group.

3. Why EPP Remains Opposed to the Current Grade Retention Policy

The review in Part II of this report provided us with a better, more balanced perspective on the New York City’s implementation of the grade retention program. This school district’s program design and summer school attendance rates were better than many other school districts. Given the many reports of glitches in test-outcome data collection, student tracking, and early notification, these problems may not be easy to solve so long as standardized tests are given mid year. The review’s summary of evaluations detailing the negative impact on students, however, buttresses EPP’s major objections to the current grade retention policy.

Accountability Grade retention essentially shifts the burden of responsibility for academic achievement from the school to the student. To a large extent, the quality of curriculum and instruction at a given school determines the proportion of students who fail to test at grade level. For example, test data from the NYS Education Department on the 4th grade English Language Arts exams for 1999 and 2000 show that in 11 elementary schools in Manhattan and 27 elementary schools in the Bronx, 12 elementary schools in Queens, 28 elementary schools in Brooklyn from 80 percent to 90 percent of students tested below grade level in at least one of these two years and that at least 40 percent of the students in these schools tested in the bottom quartile (Level 1), that is, below the level of basic reading comprehension. Yet this same data show that schools with similar low-income demographics and situated within the same immigrant neighborhoods were able to bring a majority of their students up to grade level. Students required to attend summer school in 1999 and 2000 came disproportionately from low-performing schools and low-performing districts. While the general public may assume that children fall behind for lack of individual effort, the proportion of children at grade level is a key indicator of school quality.

Demographic factors, such as concentrations of students struggling to master English and/or students from high poverty families, can reduce the proportion of students who test at grade level. But there is no plausible demographic indicator that would account for less than 20 percent of students testing at or above grade level in a community school district elementary school. School performance, not individual student effort, accounts for the high numbers of students performing at Level 1 in New York City. Grade retention, paradoxically, puts the focus of accountability on individual students.

Learning to High Standards Grade retention policies tend to confuse cause and effect. When large numbers of students are promoted who are not able to perform at their new grade levels, the curriculum inevitably gets watered down. Or worse, teachers mechanistically go through the motions of covering required subject matter as their students tune-out of any participation in learning or act out in frustration. Attempts to "end" social promotion are well intentioned efforts to re establish the fundamental structure of all educational endeavors, that students will be able to understand increasingly more complex information and perform increasingly more difficult tasks. But for large numbers of students testing far below grade on standardized tests, the source of the problem is their schools’ instructional programs. In these schools, students are not progressing to more complex information and difficult as they should.

The assumption of the 1980 Gates Program was that students held over to repeat the 4th and 7th grades would perform at grade level, but grade repetition in schools where the curriculum and the instructional performance levels were erratic produced only marginal gains. The assumption of the current effort to end social promotion is that a highly scripted, 22-day summer school instructional program geared to test preparation will bring more students up to grade level, or, at least, above the demarcation line for the bottom quartile in test performance. But the majority of these students will be returning to the low-performing schools that contributed significantly to their lags in learning.

In EPP’s interviews with principals and teachers in schools that have made substantial academic improvements for our 1996 report, Getting off the List, and for our 1999 report, Beating the Odds, setting high expectations for all students was identified as an absolutely essential starting point for creating a better school. But time after time, the respondents remarked that "wishing and succeeding" were two distinct phases. What distinguished these schools from others was not their mission statements, which tend to be idealistic, such as, "all our children will go to college," "all children will meet world-class standards," etc.,. In schools that succeeded, a whole host of strategies were put in place to help students achieve academically. Among the strategies were:

• the alignment of curricula between and among grades (so that children did not ping pong between phonics and whole language reading instruction);
• follow-up of staff development sessions through teacher observation to ensure that more effective instructional methods and lesson plans were executed in the classroom; and
• constant rewards and recognition for students’ good performance.

By shifting the locus of responsibility for academic performance from the school to the individual student, grade retention programs deflect time and attention away from strategies that could potentially improve learning for a majority of students in a school and place too much of an emphasis on a single improvement strategy targeted only to one group of students.

Cost Effectiveness School improvement efforts are more cost effective than grade retention programs. Funds and effort expended for the summer program provide extra support and learning time for students, but they are not investments in improving the instructional environments of schools where students will spend the next nine months after their five-week summer programs. As cited later in this report, 76 percent of students promoted after the 1999 summer session were at-risk of retention again in the spring of 2000 and were required to attend summer school for a second year in a row. What are the real costs of grade retention? The newspapers have reported the price tag for the summer school program at $120 million for the first year and $300 million for the second year, which did not include $150 million dedicated to install air conditioning in some schools in 2000. However, significantly higher, but less obvious funding requirements lie in having students repeat a year of schooling. These additional costs of grade retention come to $236.4 million for an extra year of schooling. [4] Over half a billion dollars will be spent for one year’s implementation of the Board’s grade retention policy.

High costs and a lack of effectiveness doomed New York city’s previous experiment with ending social promotion. At its most ambitious period of implementation during the 1982-3 school year, the Gates Program, which only retained students in the 4th and 7th grades, 16,720 students were held over at an additional cost of $35,268,499. [5] This was over and above the $23.7 million supporting summer school, early grade prevention programs, and students who were "double holdovers." As the hidden costs became evident to city budget decision makers, state remediation funds were substituted for city funds until the city’s contribution dwindled to only $3 million for the Gates services and $15.9 million for grade retention in 1989-90. [6] By that year, with assessments showing minimal gains from grade retention, few students were held over. There were only 3,701 students in the Gates Program despite the fact that 37,000 3rd and 4th graders’ test scores made them eligible for this program.

In the same year, the State Education Department restructured its school improvement program by targeting fewer low-performing schools, but providing them with greater oversight and assistance. Student academic gains have been far more impressive from this program, which accounts for why it is still in place after a decade. In the Panel’s 1996 report on this program, Getting off the List, we looked at the differences in the percentage of students in 6 elementary schools who tested above the bottom quartile of test takers in grades 3 and 6 when the schools were placed in the Schools Under Registration Review program (called SURR). On average, there was a 20 percentage point increase in the proportion of students testing above the bottom quartile in the 3rd grade Pupil Evaluation Program (PEP) tests in reading and math. These student achievement gains, however, took three to five years to secure.

Recent assessments show a continuing pattern of school improvement among elementary schools in the SURR program, though a more modest and mixed pattern at the middle schools. These test gains are significant because the current standardized tests are more demanding and are no longer normed, that is, based on the average achievement levels of all students tested. They are now based on a criteria of how students should perform on tests if they are at or above grade level. In a January 26, 2000 "Status Report on SURR" to the NYS Board of Regents, a comparison was made between former SURR schools that had been identified during 1989-93 and those placed on the list in 1999. In the newly identified schools just entering the state’s improvement program, 43 percent of students tested at Level 1 in grade 4 Mathematics and 45 percent of students were testing at that level in English Language Arts. Among the schools that had gotten off the list by 1993, the comparable percentages had dropped to 28 percent and 31 percent, respectively. For every year in SURR programs, 3 percent to 5 percent more students test out of Level 1. Even more dramatic improvements are cited by a September 14, 2000 report of the Board of Education’s Division of Assessment and Accountability for SURR schools placed in the Chancellor’s District. In schools where teachers were given extra time and pay incentives, on average, 43 percent of students had reading scores in grades 3 to 7 that placed them in Level 1 on the 1999 city tests, but on the 2000 city tests 30 percent tested at this level, a one-year improvement of 12.5 points. Mathematics one-year test gains for grades 3, 5, 6 and 7 were similarly encouraging. In 1999, 57 percent of students in these Extended-Time schools tested at Level 1, but in the next year, 47 percent tested at this level, a 9.6 percent drop in one year.

Board of Education School Based Budget Reports issued in June 2000 show that, on average, per-pupil expenditures for Chancellor’s District schools are $1,200 above those of the highest spending community school district. The Chancellor’s District superintendent of the district has estimated that there has been an investment of close to $1 million for each of the 45 SURR schools that are now under her jurisdiction, which now includes 5 high schools. This $45 million investment, however, is showing results in reducing the numbers of students testing at Level 1. As more longitudinal data on students tests become available for the current grade retention policy, the Board of Education should compare these results with those of the SURR and Chancellor’s District school improvement efforts.

Resources for Student Support Remain Insufficient Grade retention programs appear to provide services to students that they do not deliver, that every low-achieving child will get help to succeed. One of the key findings of EPP’s 1989 report, The Fourth "R": Rethinking Remediation in the Elementary Schools, is that even though the State Education Department required diagnostic tests for all students testing in the bottom quartile of test takers as well as required that remediation services be provided to all students testing at this level, there was insufficient funding from state Pupils with Compensatory Education Needs (PCEN). Even when these dollars were combined with federal Title 1 and the city’s remediation funding, only a portion of low-achieving students got help. There were too many low achieving children, so there was insufficient funding to comply with state mandates.

Ten years ago, when the Gates Program was withering but still in existence, EPP found that principals targeted their efforts towards students most likely to improve with limited assistance, those who were just below grade level. Given the inadequacy of dollars, they were unappologetic about their need to make this choice. EPP has found this same pattern has continued to this day. In interviews in 27 schools for two reports on successful turnaround efforts, most principals admitted that when they had a choice of where to use limited remediation funds, students just below grade level were targeted for these services. They had strong motivations for not providing remediation to students with the greatest academic needs: 1) students just slightly below grade level tend to respond more readily to remediation, 2) school performance is evaluated by administrators through the averaging of all test scores, and 3) newspapers tend to focus more on the percentage of students at or above grade level. These principals wanted to show continuing school improvement by pushing up the numbers of students from Level 2 to Level 3 (grade level). With limited dollars, few principal would voluntarily choose to use funds to focus on the far more difficult and uncertain task of trying to get more Level 1 students up to Level 2.

Many of the problems documented in The Fourth "R" have been solved over the last decade, such as excessive federal and state restrictions, a focus on compliance with program requirements rather than effectiveness, and, most importantly, a lack of attention to whole school improvement strategies which result in far better outcomes than remediation programs. One of the unheralded achievements of Chancellor Crew was that he forced all elementary schools to provide literacy assistance, called Project Read, to students in the early grades who were most likely to test at Level 1. One of his objectives was to ensure early academic intervention for the lowest-achieving students, who had often been bypassed for these services and whom were most at risk for referrals to special education. But this program is targeted only to the early grades.

Two of the most important problems, however, have not been solved: inadequacy of funds for additional academic supports for students and the incentives for targeting these limited resources to students just below grade level. When principals, especially those under the gun, have a choice in where to direct limited remediation funds, many opt for students testing at Level 2. They are responding logically to a gut wrenching choice of whom to help when not all students can be helped. New state regulations tacitly recognize funding limitations. The State Education Department’s requirements for school districts’ plans for Academic Intervention Services (AIS), issued in January 2000, are very carefully worded so that students are made "eligible" for services, but are not "mandated" to receive them. State Education Department sources confirmed that officials were aware that their previous PCEN regulations for mandatory services were impossible to implement in many districts. The state’s Extraordinary Needs Aid funding to New York City now exceeds $300 million. The Board of Education has folded these dollars into a large Special Needs/Academic Intervention Services funding stream that combines city, state, and federal funds that total almost $1 billion dollars ($985.9 million). Yet the estimated number of "Special Needs Children" targeted for these funds is only 149,920 pupils (which includes special education students), far below the 319,000 students testing at Level 1. In short, there is money to help only about half of the lowest-achieving student population.

4. Recommendations

Grade retention policies appeal to public and to the press. "Accountability" frameworks, however, need to be consistent, logical, effective, and, above all, honest. The current grade retention policy in New York City constitutes a step backwards in holding schools accountable for student performance. In reality, the equation is reversed and cause and effect are confused when students are held responsible for school performance.

The members of the Panel fully recognize the cumulative negative impact of promoting large numbers of students who are performing below grade on school curriculum and climate.

Grade retention policies are based on deeply held beliefs about individual responsibility, hard work, and promotion to higher grades based on merit. But, so far, they have not been effective strategies to prevent large numbers of children from failing academically. Worse still, even though these programs are costly, they under fund by as much as half the amount of funds needed to provide academic support services to students identified as low-performing.

The Educational Priorities Panel believes that if funds used for grade retention were targeted to school improvement, there would be far fewer social promotions and far fewer children testing at Level 1 in the New York City school system. Our primary recommendation, one that has little chance of adoption at this time, is to repeal the current grade retention program. Our more modest and more realistic recommendations that follow have emerged from the review in Part II of grade retention policies in the past and in other districts. Our assumption is that evaluations in year three and four of the current policy will begin to track of the persistence of student low-achievement, despite the summer school program and grade repetition. The review has helped EPP to recognize that the "second wave" of grade retention policies were improvements over the "first wave," a reminder that institutional reform is possible. School officials have other options beyond slowly defunding and dismantling the program over a decade, as they did with the Gates Program.

What are these other options? They can restructure the program so that accountability for student performance is shifted back to the schools, adopt more effective academic intervention strategies, and, finally, provide sufficient funds to support students at-risk for academic failure and dropping out of high school.

School and Government Accountability

1. Make the schools accountable for improving student academic performance. Students first identified as testing at Level 1 and lacking the other two promotional criteria (attendance and course work) should be given a "promotion with services" designation, provided with the opportunity to attend summer school, and receive mandated academic and social services for a full year in their home school. If their second year test scores still place them at Level 1 and they still lack the other two promotional criteria, their attendance in an 8-week summer school program should be mandatory, including an initial visual and hearing screening and a diagnosis of academic weaknesses. Promotion at the end of this summer session would then depend on test results.

Endemic problems in timely data collection of test assessments and student tracking will continue to make it difficult for schools to provide academic intervention in the spring semester. Questions are still outstanding about whether summer school programs merely improve test performance, not skills and comprehension. By "provisionally" promoting students, schools are given the responsibility for efforts to improve student performance and given a more realistic time frame. Furthermore, this restructuring of grade retention ensures that students will continue to get support over a nine-month period in their home school before they are held back. Boston’s recent evaluation of their experiment with provisional promotions, which are called "transitional" in their grade retention program, show that these students do almost as well on some tests as students who have not been identified as low-achieving. At a minimum, the New York City Board of Education should pilot a program to evaluate whether they can duplicate Boston’s experiment. Funds that are now going towards grade retention could then be redirected to more academic supports. The policy would still be a grade retention policy, but students held over would be those who did not improve after a year of academic intervention

2. Make school accountable for a higher than average proportion of students testing at Level 1. Both the State Education Department and the Board of Education should monitor the performance of schools for their ability to reduce the percentages of "provisional promotion" students still testing at Level 1, and this performance factor should be included among the explicit criteria for identification as a School Under Registration Review and as a Chancellor’s District school.

As discussed in the earlier part of this introduction, school administrators are motivated to focus more of their limited remediation dollars on students who are performing just under grade level. Except for the early grades where services to the lowest-achieving students are required, these harder-to-educate students tend to have less access to services. Longitudinal data on the earlier experiments in Washington D.C. and New York show that this pattern ultimately undercuts the grade retention policy. The lowest-achieving children need access to academic intervention services.

3. Grade retention policies must be accompanied by sufficient funds to help all low-achieving students.

• The State Legislature, the Mayor, and the City Council should increase their funding to the New York City school district so that all students testing at Level 1 have sufficient academic and social services supports.
• The State Education Department should revise Section 100.2 of the Commissioner’s Regulations to require that school district plans for Academic Intervention Services outline services that are mandated for students testing at Level 1 and should verify compliance.

As strategies geared to summer school and grade retention failed to reduce the numbers of students achieving at the lowest levels and as the resulting costs accumulate, the response of education officials and city budget decision makers was to quietly reduce Gates until it was shell of a program. Objective bean counters, not liberals, did the program in. The more effective strategy of shifting accountability back to the schools for improving student performance should ultimately reduce the proportion of lowest-achieving students, currently close to one-third of all enrolled students. Nevertheless, if funding is ultimately not available to support an ambitious program of ending social promotion, elected officials should at least be responsible enough to admit that the grade retention program is only partially funded and that the programs objective to help at-risk students will for the most part be unattainable.

Effective Academic Intervention

4. LADDER programs should be expanded so that all public school students from grades K to 3 are in classes no larger than 20 students. This would require an additional investment in ending overcrowding in elementary schools, ether through leasing or the construction of new schools.

Careful academic research evaluations have shown that reducing class size to 15 to 20 students in the early grades can increase academic achievement levels by half a grade to a full grade, even through high school, and particularly for low-income, urban children. [7] These outcomes are superior to those from most remediation programs. Current federal and state class size reduction funding, even if continued for the next year, will only bring about two-thirds of K-3 classes to an average of 20 students. Given the cost effectiveness of this strategy in preventing low-achievement and the resulting costs of grade retention, summer programs and remediation, the Mayor and the City Council should consider finding the funds to cap all early grade class size to no more than 20 students.

5. The Extended-Day program model should be expanded to more schools in New York City, and begin in schools where 35 percent or more of students are testing at Level.

There has been criticism of the Extended-Day pilot that has been in place for one year in about half [check] of the SURR schools in the Chancellor’s District. The name of the program implies that all students are benefiting from a longer school day, but in reality "Extended-Day" means that teachers get paid for an additional 40 minutes a day so that they can focus on small group instruction, staff development and school improvement efforts. Asked about the components of the program, a teachers’ union staff member responded that they reflect some of the planning that went into a U.F.T. proposal to be allowed to experiment in turning around low-performing schools, just as a business community organization, the New York City Partnership, had been given permission to sponsor model programs in several low-performing districts. Chancellor Crew ignored U.F.T. President Weingarten’s proposal, but the union was able to get some of their strategies in place through negotiations over the Extended-Day program. The key strategy of the union was to improve the quality and effort that went into staff development and to ensure on going implementation of new methods of instruction. "Hit and run" workshops were to be avoided. Time was set aside for thoughtful and collaborative planning for school improvement and the alignment of curriculum and standards. These are common aspirations in most low-performing schools, but less commonly achieved. EPP has not made school-site visits to fully evaluate this model, but an average ten-point drop in the numbers of students testing at Level 1 within one year is a remarkable outcome that merits duplication.

PART II: REVIEW OF SCHOOL DISTRICT EXPERIMENTS

1. Purpose

The Educational Priorities Panel decided to study efforts to end social promotion in other districts across the nation in order to evaluate whether the considerable resources allocated to support New York City’s grade retention policy are being well spent. Our estimates suggest that the district had to spend approximately $236.4 million to provide an additional year of schooling for retained student in the 1999-2000 school year alone. . [8] An additional $300 million was spent on the summer school program for about 319,000 students in 2000, not including the costs of air conditioning more classrooms.

By studying the experiences of other districts, we hoped to uncover some important lessons to direct implementation and refinement of the current promotion policy in New York City. Specifically, this report aggregates available information about past and current promotion policies in districts across the nation; identifies similarities and differences between the policies as well as common trends and challenges in their implementation; and highlights key lessons learned from evaluations of promotion policies.

The New Wave Over the past five years, a new wave of efforts to end social promotion’ has begun in states and school districts across the nation. President Clinton called for an end to social promotion in our nation’s public schools during his 1999 State of the Union Address, and the U.S. Department of Education released a guidebook for educators and policy makers later that year. [9] Many states and districts were already addressing this issue, and others have been quick to jump on the bandwagon. How extensive is this trend? Fourteen states have adopted policies designed to end social promotion as students move from grade to grade, and 28 states have adopted high school exit exams based on rigorous standards, according to a report released by the American Federation of Teachers (AFT) in 1999. [10] The Education Commission of the States (ECS) has identified 29 states with promotion/retention policies in place. However, the implementation and impact of these policies varies dramatically. [11] New survey data from researchers at Johns Hopkins University show that at least 51 of the nation’s 100 largest districts provide summer school as an alternative to social promotion, and 42 of these districts retain low-performing students who do not successfully complete the summer session. [12]

The Debate Efforts to end social promotion have been the subject of great controversy. Supporters of social promotion argue that it is preferable to retention, which leads to high dropout rates while providing few educational benefits to retained students. Opponents say that social promotion robs students of the motivation to achieve and the skills to participate in modern society. They also argue that students who are promoted without the necessary mastery of subject matter and skills "dumb down" the curriculum and instruction at higher grades, decreasing the quality of high school education.

The balance of scholarly research on grade retention and social promotion falls heavily in favor of social promotion, and comparatively little research supports retention. [13] Unfortunately, the research on both sides of the issue is somewhat biased, as Gregg Jackson pointed out in 1975, because few studies use a true experimental design with random assignment of students to promotion or retention. [14] Whole-school reform initiatives and efforts to identify and restructure low-performing schools have received more support among researchers because they have produced clear student achievement gains in some high-poverty schools.

Meanwhile, elected officials, superintendents and school boards across the nation continue to adopt strict new promotion policies that will lead to increased retention, at least in the short run. Why does support for retention continue in the face of extensive research documenting its negative effects? The concepts of effort and valid criteria for achievement resonate with the work ethic of American culture. Even an army of researchers would have a hard time revising the basic expectation that high school graduates should be well-prepared for college or employment.

As the movement to end social promotion continues, it is important to note that ending social promotion is not new. It has been tried in many districts across the nation over the last 25 years. This report is intended to help policy makers learn from the successes and failures of past promotion policies and use this experience to inform future policy making.

The Sample and Data Collection Methods This report compares policies ending social promotion in 27 school districts across the nation over the past 25 years, and it presents key lessons learned from the implementation of these policies, based on evaluations and outcome data from 11 of these districts. We define "policies ending social promotion’ as any policies that establish explicit student achievement criteria for promotion to the next grade or for high school graduation or any policies that provide extra help to low-achieving students with the goal of boosting student achievement to a desired promotion standard. (The matrix at the end of Part II summarizes the district promotion policies included in this study.)

The districts included in this study were identified through a review of recent education policy research, education periodicals, and newspaper articles. We also talked with staff in 18 states and more than 30 school districts to collect information and documents describing the implementation of past and present promotion policies. We originally planned to include state-level policies as well as district policies in this study, but we narrowed the focus to district policies because few states have implemented their new promotion policies for more than a year or two, and none of the states we contacted had released policy evaluations yet.

In our preliminary research and conversations with state and district staff, we discovered that state promotion polices vary dramatically in their implementation and impact at the local level. For example, some states have a promotion policy in place, but it is either too vague or too poorly enforced to have a consistent impact at the local level. Those that establish clear promotion criteria including standardized test scores seem to have a stronger impact on instructional practices and promotion decisions at the district level. Lastly, some states take a "just services" approach to ending social promotion. They require extra services for low-achieving students to boost their performance to a desired level, but the policy does not establish strict promotion criteria. The relationship between state-level promotion policies and local practices deserves more careful analysis, but is beyond the scope of this report.

In examining district-level promotion policies, our key research questions were the following:

• What has been tried? What is the range of policy experimentation?

• What has been learned? How have these policies worked in practice?

• Do recent promotion policies reflect lessons learned from past experimentation?

2. SIMILARITIES IN SCHOOL DISTRICT EXPERIENCES

In comparing the promotion policies implemented in 27 districts over the past 25 years (and the available outcome and evaluation data), we discovered the following nine similarities in the policies themselves and also in their implementation and impact over time. Information collected by the AFT, the ECS, and researchers at Johns Hopkins University, regarding district and state promotion policies supports some of these findings. [15]

Large urban districts are taking the lead. The districts experimenting with promotion policies are predominantly large, urban districts. Twenty-four of the 27 school districts included in our study are among the nation’s 100 largest school districts. [16] It is interesting to note that, in some cases, the adoption of a strict promotion policy is preceded by severe problems or failures in these large, urban education systems. For example, students in Baltimore, Chicago and Washington DC were consistently scoring far below national norms on standardized tests in reading and math prior to the adoption of tougher promotion policies in these three districts (in 1978-79, 1985-86, and 1998-99, respectively). The districts that adopt strict promotion policies also tend to serve high-poverty and high-minority student populations, and many of these districts enroll a large number of students who speak English as a second language as well.

Retention rates spike upward initially, when rigorous promotion policies are first implemented, and then decrease. Rigorous promotion policies often create dramatic spikes in retention when they are first implemented, raising rates far above the national average of about 6 to 8 percent retention a year in grades 1-12. For example: [17]

• In Philadelphia, retention rates for students in grades 1-8 rose dramatically after implementation of a tough new promotion policy in 1985, from 7 percent in 1982 to 21.1 percent in 1986. [18]
• In Houston, 45 percent of students in grades 1-3 failed to meet one or more of the three promotion standards during the first year under the district’s new promotion policy in 1998-99. Only 20 percent of students were actually retained because some students were promoted after summer school or were promoted to the next grade even though they did not meet the standards. [19]
• In Chicago, 20 percent of eligible 3rd graders and 10 percent of eligible 6th and 8th graders were retained in 1997 and 1998, the first two years under the district’s new policy, and almost 1,600 students were retained for a second time in 1998. [20]
• In Boston, the retention rate for middle school students (grades 6 to 8) rose from 13.5 percent in 1983 to 16.8 percent in 1985, when the district’s new promotion policy was implemented, and 30 percent of first graders were retained in 1985. [21]
• In Pinellas County, Florida retention rates more than doubled when a rigorous new promotion policy was implemented for students in grades 1-8, rising from about 4 percent in 1977 to 10.4 percent in 1978. [22]

As implementation continues, retention rates tend to decrease over time.

• In Chicago, data from the first two years of latest promotion policy show that an increasing percentage of students in grades 6 and 8 achieved the minimum test-score cutoff for promotion in 1997 and 1998, as compared to student performance in 1995 before the policy was instituted. [23]
• In Philadelphia fewer students in grades 1-8 were retained each year that the district’s policy was in place, dropping from a peak of 21.2 percent in 1986 to 12.6 percent in 1991. [24]
• In Pinellas County, retention rates also dropped steadily in the after an initial spike, from 10.4 percent in 1978 to 5.9 percent in 1981. [25]

These steady declines in retention may indicate that more students are reaching high standards as a result of rigorous promotion policies or that students and staff became more familiar with the tests.

However, decreases in the retention rate after an initial spike may not always reflect gains in student achievement or test familiarity, but instead may represent "backpeddling." During the implementation of Washington DC’s 1981 promotion policy, the retention rate dropped consistently each year, from 19 percent in 1982 to 7 percent in 1986, but a rising proportion of students were promoted "with deficiencies" each year, representing 18 percent of students promoted in 1986. Further, the proportion of students promoted with deficiencies increased in the upper grades, suggesting that these deficiencies were compounding rather than being addressed after promotion. [26]

Minority students are disproportionately retained. The impact of tough new promotion policies falls most heavily upon minority students, who are consistently retained at higher rates than other students under these policies. This trend likely reflects the fact that minority students are often concentrated in low-performing, high-poverty schools that do not have the financial resources, qualified teachers, or effective administrators needed to prepare students for rigorous promotion standards.

• African-American students in Chicago were 4.5 times more likely to be retained than White students in 1997 (18 percent vs. 4 percent retention rates), and Hispanic students were nearly three times more likely to be retained than White students (11 percent vs. 4 percent retention rates). [27]
• Pinellas County, Florida also retained minority students at much higher rates than other students after its promotion policy was implemented. Non-White students in grades 1 to 5 were retained at three times the rate of White students (28 percent vs. 9 percent) in 1978, but the gap closed somewhat in following years (11 percent vs. 5 percent, respectively in 1982). [28]
• Houston’s new promotion policy produced similar outcomes in 1999, 24 percent of African-American students in grades 1 to 3 were retained, as compared to 19 percent of Hispanic students and 16 percent of White students. ^[29]
• Philadelphia’s 1985 promotion policy also retained minority students at higher rates than their White counterparts. In 1989, 31 percent of Hispanic students in grades 1 through 8 and 23 percent of African-American students were retained, as compared to 18 percent of White students. [30]

Higher retention is followed by dramatic increases in dropout rates. Most of the data available come from the "first wave" of policies, with the exception of Chicago, because more recent policies have only been in place for two or three years. Districts that conducted longitudinal policy evaluations reported that dropout rates increase over time after rigorous promotion policies are implemented. Perhaps the most often cited example comes from an evaluation of New York’s 1981 Promotional Gates Program, which showed that approximately 40 percent of students retained in the 8th grade later dropped out of school, while only 25 percent of students who were not retained dropped out during that time. [31]

• The evaluation of Chicago’s latest promotion policy showed that dropout rates for low-achieving 8th graders increased after the implementation of the district’s new policy, from 8 percent in 1995 to 10 percent in 1997, and the dropout rate among 8th graders retained in 1997 was even higher at 16 percent. [32] An older study conducted in Chicago, during the time that a rigorous promotion policy was adopted in the early 1980s, reported that the students who had already been retained once upon entry to high school were more likely than other students to drop out before completing high school. [33]
• Another longitudinal study of students in Miami-Dade County in the 1980s showed that students who had been retained once dropped out at twice the rate of students who progressed normally. [34]
• Boston’s drop out rate, which was already high, also increased during the time that the district was implementing a tough new promotion policy, from 36 percent in 1982 to 43 percent in 1985. [35]

We cannot assume that promotion policies cause increased dropout rates, because a number of different factors influence each student’s decision to drop out. But there is clearly a correlation between grade retention and likelihood of dropping out. The well-known Youth in Transition study found that if students repeat a grade once during their schooling, their chances of dropping out increase by 40 to 50 percent, and if students are held back twice, their chances of dropping out increase by 90 percent. [36]

Data problems hinder the implementation of many promotion policies. Problems with data collection and student tracking riddle the implementation of many promotion policies and limit the quality of policy evaluations.

• An evaluation of Houston’s new promotion policy identified data problems as the main challenge in the first year implementation of the policy in 1999. Of the 21,000 students eligible for the inclusion in the policy, 7.2 percent of students had missing data for every promotion standard, and only 71 percent of students were assessed on all three required standards. Among the students who were assessed on all three standards, 3.2 percent passed one or none of the three required standards but were "incorrectly promoted despite their lack of academic success." Further problems with missing data made it impossible for the evaluators to determine whether students promoted after summer school had actually earned their promotions or not. [37]
• The Baltimore public school district didn’t officially evaluate its 1978 promotion policy and changed the standardized tests used to measure student achievement a few years after implementing the policy, making it difficult to measure long-term progress in student achievement. [38]
• The Corpus Christi school district in Texas has reported difficulties in maintaining course completion records for students who are promoted during the regular school year, and they have also reported needing better analysis and more timely dissemination of student test data to support their promotion policy.

Many districts have difficulty getting student achievement data early enough in the school year to identify struggling students and provide the extra instructional services they need to avoid retention. Very few of the districts in our sample officially identify students at-risk of retention before the middle of the school year (January), and many districts wait until spring to notify parents that their children may not meet the promotional criteria. Often, delays and problems in identifying students at risk of retention are caused by the schedule of standardized tests. Because it is difficult to identify students at-risk of retention early on, many districts focus on remediation and other "catch up" services to address educational problems after students have been designated for retention, rather than beforehand. This may explain why summer school remediation is such a popular support strategy, because it is not until the late spring that testing data becomes available.

Attendance is a serious problem for most summer school programs that support promotion policies. Many districts implementing tough new promotion policies have problems with non-attendance in summer school programs intended to boost the performance of low-achievers. For example:

• Less than half (42 percent) of the students in Houston who were designated for summer school in 1998-99 actually enrolled. [39]
• In Cincinnati, only 52 percent of students identified for retention in 1994 attended summer school. [40]
• The school district of Hartford, Connecticut had less than 40 percent attendance in summer school in 1999 among students identified for the program in connection with their promotion policy. [41]
• During the same summer, nearly half of the 14,000 students identified for summer school in Oakland, California chose not to attend. [42]

Some evaluators and district officials point to late or ineffective communication with parents as a key source of attendance and participation problems. For example, parents in some districts were not notified about mandatory summer school until the end of the spring semester. Earlier and more effective communications with parents to encourage summer school participation may help resolve attendance problems.

Promotion policies cost big bucks. District spending on promotion policies and the services to support them varies dramatically across the districts included in this study.

• For example, New York City spent about $300 million on summer school for over 319,000 students in 2000.
• Chicago budgeted about $86.5 million on its promotion policy and related services including after-school and summer programs in 2000-2001. [43]
• Los Angeles planned to spend $71 million on its new promotion policy in the 1999-2000 school year but eventually scaled back the program.
• Washington DC spent $10 million on its summer school program for 20,000 students in 1998.

Some of the most obvious expenses related to strict promotion policies are the extra educational services provided to low-achieving students, such as after-school and extended day programs, Saturday school, summer school, and reading and math specialists. For example, San Diego’s new "Blueprint for Student Success," which establishes three promotional gates, will provide numerous services to students throughout the elementary and middle school system at a cost of approximately $61 million in the 2000-01 school year alone, more than five percent of the district’s $1 billion annual budget.

Promotion policies have significant hidden costs as well. The total cost is not limited to summer school and extra services, but also includes additional spending for extra years of schooling for retained students. Lorrie Shepard, one of the nation’s leading experts on social promotion and grade retention, estimated in 1989 that our nation spends almost $10 billion annually on retention in the public schools.

Efforts to end social promotion are often sporadic and short-lived, as some districts try over and over again to end social promotion. At least nine out of the 28 districts we studied have adopted policies to end social promotion two or more times in the last 25 years. Some of the largest districts, Chicago and Philadelphia, have been the most dogged in their efforts, implementing three or four different promotion policies during this time. Other districts have tried at least two different policies, including, New York, Baltimore, Boston, Milwaukee, San Diego, and Washington DC.Most of the "multiple experimenters" established their first promotion policy sometime in the 1980s, when the standards movement was beginning, and discontinued the policy later (often due to negative student outcomes and/or high costs of retention). Then, they tried again in the late 1990s as part of an effort to improve student achievement. These multiple experimenters provide valuable information about how promotion policies are evolving over time. As discussed in the next section, we believe recent promotion policies are beginning to reflect lessons learned from past experiments.

Many districts we studied implemented promotion policies for just a few years at a time. This short time frame for experimentation may be a result of the politics surrounding education policy. Elected officials and school board members largely determine state and local education policy, so policy can change quickly when new officials sweep into office and want to take action. For example, Milwaukee implemented a new district promotion policy during the 1999-00 school year. But the district is already replacing its policy, in response to a newly enacted state law, with a different one scheduled to take effect in 2002 and 2003.

The departure of key personnel from schools or districts can also make it difficult for districts to maintain continuity in their efforts to end social promotion. For example, Seattle’s recent effort to end social promotion was described as a pet project of Superintendent John Stanford. The district implemented a rigorous promotion policy under his leadership in 1996, establishing a system of "exit profiles" for students in grades 5, 8, and 11. Unfortunately, Superintendent Stanford died a few years later, and although many of his reforms have been continued, the promotion policy was eliminated in 1998.

Some districts’ promotion policies improve over time. Even when data is collected in a timely and accurate manner, outcome and evaluation information may not be used effectively to inform classroom practices or later policy making. In the course of this research, we talked with staff in 18 state education agencies and more than 30 school districts across the nation, and the lack of institutional memory about past policy initiatives was striking and consistent across the board. It was common for staff in areas such as policy planning, evaluation, assessment, and accountability to have little knowledge of past promotion policies and no easily accessible record of these initiatives, much less their evaluations. The Virginia state legislature actually eliminated the State Education Agency’s Office of Evaluation in the mid-1990s. This loss of institutional memory may make it difficult for lessons learned from past experiences to be incorporated into the implementation of new policies.

Despite the absence of "institutional memory" and the lack of continuity in many districts, we find that some district policies are evolving over time in more promising directions. The following trends seem to be emerging in the evolution of rigorous promotion policies:

• Recent promotion policies often make use of multiple measures of student achievement, whereas older policies commonly relied upon a single standardized test score;
• Recent promotion policies include a variety of support services for low-achieving students, whereas older policies typically provided few extra services for struggling students. However, there is still room for improvement in the effectiveness and consistent delivery of support services;
• Recent promotion policies provide more alternatives to retention, such as summer school, transition classes, and mid-year promotions, for students who do not meet the promotion standards, whereas older policies relied upon repetition of the same courses through grade retention as the main response to academic failure; and
• Recent promotion policies require more extensive public reporting of student outcomes, whereas there was less consistent public reporting on the impacts of past policies. However, independent evaluations of promotion policies and their long-term effects are still rare.

These trends, although not universal, suggest that policy makers can learn from past experiences and use this knowledge to inform future policy making. New York’s introduction of two additional standards for promotion, attendance and teachers’ assessment of the quality of classroom work, are often perceived by the tabloid press as "loopholes." The use of multiple measurements represent a better targeting of resources to students who need help and an effort to eliminate more students whose home schools are low-achieving.

3. DIFFERENCES IN GRADE RETENTION POLICIES

Below, we discuss four key areas of variation in the both the characteristics and impacts of different promotion policies. This broad range of experimentation provides a variety of models, although still unproven, for educators and policy makers across the nation. (Appendix Tables 1 and 2 summarize all district- and state-level policies included in this study.)

Various approaches focus on students in different grades.

Gates: Establishing promotional standards for students at key "gateway grades" is a popular approach. (We will refer to these programs as "gates" programs.) About half of the districts examined in this study (14 of 27) have implemented promotional gates at various grades. The idea is to measure student achievement at key developmental stages to ensure that children have the basic skills needed to benefit from the next level of education. Third or fourth grade is often chosen as a gateway year to test reading skills, and later gateway grades are often chosen to test basic math and higher level literacy skills.

All Grades: Some districts have adopted promotion policies that apply to all grades or many consecutive grades. This type of policy is sometimes implemented as part of a larger effort to raise standards for students in every grade. For example, Corpus Christi (1-12) and Houston (1-8) are both currently undertaking substantial standards-based reform efforts. Corpus Christi is also implementing an extensive middle school reform, which complements the promotion policy, supported by a special foundation grant. Boston also phased-in a consecutive grade promotion policy in the mid-80s, after implementing an extensive standards-based reform initiative throughout the district.

Ungraded: Classroom structures that mix students of different ages in primary or secondary school are another, less common approach to ending social promotion. This requires teachers to work with multi-age groups of students, usually for at least two years, to ensure that they master certain content and performance standards during those years. The non-graded approach give students more time to master different concepts, allowing them to progress at their own pace, while avoiding the stigma of retention. Kentucky implemented non-graded primary classes (for 5 to 7 year olds) throughout the state as part of the Kentucky Education Reform Act of 1990. Philadelphia also briefly experimented with non-graded elementary and middle schools from 1993 to 1995, and Washington DC tried multi-age classes as part of its 1981 promotion policy, which allowed students to be promoted at the end of every semester. Evaluations of these policies show that teachers have a hard time working with multi-age classes and request more professional development and new teaching strategies to support this type of class structure. [45]

Exit Exams: Rigorous high school exit exams are gaining popularity, especially at the state level. Many states and districts have had minimum-competency tests for high school graduation in place for a number of years, but these exams typically test knowledge at a very basic level and have pass rates above 90 percent. [46] For example, New York City has required all high school seniors to pass the Regents Competency Tests (RCTs) since 1985, which tested knowledge and skills at an 8th grade level. The new exit exams being adopted across the nation are much more rigorous. Seven of the 27 school districts we studied are implementing new policies requiring students to pass standardized tests based on new state or district standards in order to earn a high school diploma.

Measures of achievement are the source of major controversy. Standardized test scores are by far the most common student achievement criterion in the promotion policies we studied. A key distinction between policies is whether test scores are the only criterion or one of many. Researchers tend to criticize the use of standardized test scores as the sole measurement of student achievement because these tests do not accurately measure the extent of a student’s knowledge and abilities. Because they are not precise measures, test-score cutoffs can lead to arbitrary promotion decisions. On any given day, some capable students may fail to meet the cutoff, while other students with serious academic deficiencies may get lucky. In addition, it is difficult to determine what portion of a test-score gain is due to genuine learning as opposed to increased familiarity with the test. As a result, many districts (and some states) have established "multiple measures" of student achievement in their latest promotion policies. Standardized test scores are usually one of the measures.

Almost every district in our sample assesses student achievement in reading and math under their promotion policy. Some districts, such as Corpus Christi and Milwaukee, require student performance data in as many as four or five different subjects for promotion decisions. However, the more subjects required, the more assessments students must undergo. Teachers, parents and school administrators are already balking at the amount of time students spend in testing.

If the promotional criteria are not well aligned with the curriculum, two problems may develop. First, teachers may focus primarily on the material and skills required for promotional tests and sacrifice other elements of the curriculum or deeper exploration of topics. Alternatively, teachers may focus on the curriculum without emphasizing the content and skills required for promotion. In this case, the promotional test is not a valid measure of what students have learned. Alignment issues have been raised in a number of evaluations as a major challenge in implementing promotion policies, because the standardized tests many districts use to measure student achievement, such as the Iowa Test of Basic Skills, the Stanford-9, and the Metropolitan Achievement Test do not reflect the specific curriculum and instructional practices of individual districts. [47]

Some districts do not use standardized tests at all; instead, they rely upon teacher evaluations of student performance for promotion decisions. Cincinnati and Corpus Christi have both developed extensive guidance and scoring "rubrics" to standardize teacher evaluations, and Long Beach is also developing clear scoring guidelines for teachers under its new promotion policy. Washington DC was one of the pioneers of this approach with its 1981 promotion policy based on teacher judgments rather than standardized test scores. However, these programs often have problems ensuring the validity and reliability of teacher judgments. After four years of implementing and refining a promotion policy based on teacher evaluations, researchers reported that the Cincinnati school district’s rubric system was not producing accurate, consistent measurements of student achievement and was therefore "not suitable for making individual student decisions." [48] An assessment of Washington DC’s 1981 policy also seriously questioned the validity and accuracy of teacher evaluations of achievement. [49]

Districts and states are trying a variety of support services and alternatives to retention.

Summer School: Summer school is the most common supplemental service used to boost the performance of low-achieving students, among districts in our sample. Every one of the 27 districts we studied provided summer school programs to students who did not meet promotional standards at the end of the school year. Many of the summer school programs in districts we studied provide: smaller class sizes than during the regular school year; a focus on remedial instruction in math and reading; and more individualized instruction. Some districts also emphasize after-school and extended day programs during the regular school year to provide more "time on task" for students who are struggling with basic skills. For example, Chicago’s latest promotion policy is supported by the "Lighthouse After-School" program, and the district has recruited thousands of volunteer tutors, including retired teachers and community members, to help staff the program.

Summer and Mid-Year Promotion: Districts are also developing innovative alternatives to retention for students who do not meet the promotion standards. The most common alternative among the district and state policies we studied is promotion after summer school. Some districts promote all students who "successfully complete" summer school, which is usually defined as a combination of attendance and/or classroom performance. However more districts are shifting towards retesting at the end of the summer and retaining students who do not reach the promotional standards. Other districts allow retained students to be retested and promoted at different times during the regular the school year. For example, the districts of Tacoma, Chicago and Corpus Christi all allow retainees or transition students to be promoted during the regular school year.

Transition Classes: Transition classes or transition centers are another popular an alternative to retention for students who have been retained before or are significantly overage for their grade. Transition programs are an effort to find a middle ground between retention and social promotion. Sometimes these programs allow students to be promoted along with their peers, but they require students to continue working on the standards that they have not yet achieved. In most cases, transition programs have a heavy focus on remediation of basic skills in math and reading.

Phase-in Period: Additionally, a number of school districts take a more supportive approach by phasing-in a promotion policy slowly over time, so that teachers have time to prepare for the new standards. For example, Durham, NC began its current gates program at the 8th grade in 1995, added the 5th grade two years later, and then applied the policy to 3rd grade students in 1999. The Long Beach school district phased in its three promotional gates at grades 8, 5 and 3, over four years from 1997-98 to 2000-20001, and each year they added new supplemental services for struggling students in non-gateway grades as well.

Districts and states use different combinations of caps, waivers, exclusion and backpedaling to soften the impact of rigorous promotion policies.

Caps: Many promotion policies establish a cap on the number of times a student can be retained in each grade or in a certain set of grades. Among the 29 districts we studied, 24 districts had a cap on retention. These caps commonly set a maximum of two retentions in the grades affected by the promotion policy, or a maximum of one retention per grade. Caps on retention reflect concerns both that classmates who are substantially over-age could negatively affect the learning environment for other students, and that students who have been retained many times are much more likely to dropout of school.

Waivers: Another method of softening the effects of strict promotion policies is to "waive" the promotion requirements for certain students. Waivers are especially popular with districts that rely on standardized test scores as their only criterion for promotion decisions. For example, the Chicago and Durham school districts both use a single test to determine whether students have met the promotion criteria, and they have both instituted waiver or appeals processes that allow teachers, principals and parents to argue, based on other measures of achievement, for the promotion of students who do not meet the test score cutoffs.

In some districts, waivers have been used to avoid very high retention rates among minority students who attend low-performing schools. The rationale is that these students have not had the educational opportunities and preparation necessary to meet rigorous new promotion standards, and waivers are considered preferable to massive student retentions at these schools. However, if the use of waivers holds steady over time, this could signal that waivers are being used as a safety valve to reduce retentions without actually improving student achievement at low-performing schools. (On the other hand, students who received waivers under Chicago’s most recent promotion policy have performed similarly to students who were retained. For example, almost 2,000 6th grade students in Chicago were promoted with waivers in 1997. About 60 percent of those students were able to meet the 8th grade promotional standards two years later. [50] )

Exclusions: Excluding certain groups of students from the promotion policy is another popular way to "end social promotion" without retaining huge proportions of students. The vast majority of promotion policies we studied exempt some or all special education students from the promotional requirements, depending on whether their Individual Education Plans (IEPs) state that they will be tested with all the other students. Also, districts often exclude students who are English-language learners (ELL) from promotion policies for the first two years they are enrolled in the district, and some districts continue to exclude students for additional years if their English skills have not reached a certain level. These exclusions can account for a large percentage of students in some districts. For example, 31 percent of 3rd grade students and 20 percent of 6th and 8th grade students were not subject to Chicago’s latest promotion policy during the first two years of implementation in 1997 and 1998, based primarily on special education and bilingual exclusions. [51]

Backpedaling: Lastly, "backpedaling" is another common strategy for districts and states that have second thoughts after realizing the potential impact of their promotion policies. For example, Los Angeles dramatically scaled-back the implementation of its new promotion policy in 1999, from grades 2, 3, 4, 5 and 8 to only grades 2 and 8, when it became clear that about 40 percent of the districts students would not meet the promotion standards. [52] Gwinnett County, Georgia also decided in 1998 to ease off its initial plans of implementing gateway exit exams for student promotion until it had better estimates of the students at risk of retention and the extra services needed for these students. Similarly, many of the first states to adopt high school exit exams have backed off their original implementation plans or lowered the minimum passing scores after early testing indicated that large proportions of students might fail the exams. Maryland pushed back the implementation date of its high school exit exam from 2005 to 2007;

Virginia eased its new exit exam requirements, allowing students who fail the state test to graduate anyway if they pass other standardized tests; and Massachusetts lowered the minimum required score on its high school exit exam in 1999 to just above the "failing" category.

4. SUMMARY OF FINDINGS FROM EVALUATION REPORTS

As part of this research effort, we reviewed the evaluations of past and present promotion policies implemented in 11 districts over the past 25 years. (A list of these evaluations is included in the matrix at the end of Part II.) In this section we present five key findings from these reports, including common outcomes of promotion policies, difficulties in implementation, and recommendations of evaluators.

Rigorous promotion policies do not always improve the achievement of retained students. The reports on achievement of retained students are mixed and highly disputed. Some evaluations report that retained students make substantial educational gains during the year of retention. For example:

• An evaluation of the Pinellas County promotion policy showed that students retained in 1978, especially those retained in the early grades, made substantial gains during a year of retention. Following the year of retention, students showed a median decrease in grade-equivalent deficit of six months in reading and one year in math. In fact, students who were retained in 4th grade had a median score of eight months above the national norm for 5th grade math at the end of their 5th grade year. [53]
• Under Philadelphia’s 1985 promotion policy, retained students also made gains, on average, in terms of normal curve equivalents (NCEs) on standardized tests by the end of the year of retention. However, a sizable minority of students who were retained in 1987 actually scored lower on standardized tests at the end of a year of retention in 1988. Overall, 23 percent of students retained in grades 1 to 8 had lower NCE scores in reading and 17 percent had lower NCE scores in math at the end of a year of retention. [54]

Other districts report that socially promoted students perform as well or better than students retained under rigorous promotion policies. For example:

• In an evaluation of New York City’s 1981 Promotional Gates program, Ernest House reported that students who had been retained under the policy had similar test scores in later grades to a matched group of low-performing students who had been socially promoted before the policy. [55]
• Similar findings are reported in the evaluation of Chicago’s latest promotion policy, which established control groups for retained students. The authors report that retained did not do better than previously socially promoted students. "Only one-fourth of retained 8th graders and one-third of retained 3rd and 6th graders in 1997 made "normal" progress during the following school year," meaning they stayed in school and passed the promotional cutoff. Further, retained third graders performed worse than their socially promoted counterparts. They gained only 1.2 grade-level equivalents over 1997 and 1998, while students with similar test scores who had been promoted prior to the policy made gained 1.5 grade-level equivalents during the same time period. [56]

Students on the cusp of promotion continue to struggle in later years. Many school districts provide intensive summer school programs to boost a number of low-performing students up to the promotion standard. But students that are promoted in this way tend to fall behind again in the next grade. For example, in Chicago 13 percent of 6th graders failed to meet the promotional criteria at the end of the 1996-97 school year, but achieved the promotion standard after the Summer Bridge program. However, these students returned to lower level achievement rates after being promoted and were again at-risk of being retained in 8th grade. The evaluators concluded, "While Summer Bridge raised students’ performance briefly, there is no evidence that it altered the pattern of school achievement for these students." They provide two possible explanations for the poor school year performance of students who make achievement gains during summer session. First, summer gains could reflect increased familiarity with promotional tests, rather than true learning. Second, these students may receive weak school-year instruction or they may require more sustained, additional help throughout their schooling, rather than one-shot summer school interventions. [57]

Data from New York City’s summer school program in 1999 follows the same pattern. About 57 percent of students who took the promotional test again after summer school passed and were promoted to the next grade, but these students had trouble keeping up during the regular school year. In fact, 76 percent of students promoted after the 1999 summer session were at-risk of retention again in the spring of 2000. [58]

The lowest achieving students do not seem to be helped by retention or the standard support services they receive. Rigorous promotion policies, by their very nature, identify a group of very low-performing students among retained students, who are unable to boost their performance after a year of retention and additional services. For example:

• 64 percent of 7th grade students in Philadelphia who were retained in 1988, still failed to meet the promotion standard at the end of a year of retention. Overall, almost 40 percent of students in grades 1 to 8 who were retained in 1988 did not meet the promotion standards at the end of a year of retention. [59]
• The pattern is similar in Chicago. Only about one-third of 3rd and 6th graders retained in 1997 passed the promotional standard in 1998 at the end of a year of retention. After attending summer school in 1998, only 43 percent of previously retained 3rd graders and 47 percent of retained 6th graders were able to meet the promotion criteria. In addition, the majority of students who were retained at each grade level under the new promotion policy in 1997 failed to reach the promotion standards in both reading and math, not just one subject. These students were further behind on average than students who failed only one subject, and they also benefited less from summer school. [60]
• A longitudinal evaluation of New York City’s 1981 Gates Program reported that 1982-83 "Gates students," who had been retained once in reading or math and who were still unable to meet the promotion criteria after a year of retention, showed "at best minimal gains" in both reading and mathematics by the 1986-1987 school year. Among students who had been retained more than once, "extension-eligible" students, less than 25 percent of students who were retained in reading were still "on-grade" by the 1986-87 school year, and less than 10 percent of extension-eligible students retained in math were on grade level by that time. [61]

Many of the lowest achieving students are designated for special education after repeated failure under rigorous promotion policies, but it is not clear that these students benefit from or belong in special education classes. For example, approximately one in ten third graders and sixth graders who were retained during the first year of Chicago’s new policy in 1997 were excluded from the policy in 1998 primarily because of special education designation. [62]

The repeated failure of this group of students suggests that they are not well served by the typical one-size-fits-all, supplemental services provided to low-achieving students in many district grade retention programs. Evaluators suggest that these students need more individualized attention and instruction geared specifically to their learning difficulties. The continuing failure of this group eventually leads to the cancellation of rigorous promotion policies over time, because the negative effects on the lowest-achieving students are so severe.

Transition programs, a popular alternative to retention, vary greatly in their approach and effects. Evaluation and outcome data from a variety of "transition" programs show that the details of these policies make a huge difference. Transition programs seem to work well when students are not isolated from regular classes or regular schools, but instead receive supplemental services to help accelerate their learning.

• One example of this approach is the "Zero Period" program in Corpus Christi, Texas, where students who have not reached all of the required standards can be promoted on a probationary basis, provided that they come to school an hour early each day (during the zero period) to work on the concepts they have not yet mastered. In 1999-00, about 66 percent of students who participated in this transition class were eventually promoted during the following year, instead of being retained.
• The San Diego school district has also developed an innovative transition program that helps low-achieving students keep up with their peers by providing these students with a double-dose of classes in math and reading. The extra "parallel classes," which are aligned with the core math and reading classes that all students attend, focus on basic skills underlying the higher-level curriculum that are causing problems for low-achieving students.
• The Long Beach school district has also had much success with its special transition center, the Long Beach Preparatory Academy, a special school for 8th graders who failed two or more subjects. Instead of being promoted to high school, low-achieving 8th grade students spend a year at the Academy, where they receive extra attention from school staff, small class sizes and extra long class periods. The Academy opened in 1997-98 and reported a 93 percent promotion rate (266 students) for the first class that attended the school.

In other school districts, transition classes seem to create a "slow track" for failing students, reducing the expectations for these students. In Chicago, where students attend Regional Transition Centers for over-age 8th graders on separate school campuses, the district’s lowest passing and promotion rates in 1998 were those of 8th graders and Transition Center students. Only 27 percent of retained or Transition Center 8th graders met the promotional criteria in 1998, and approximately 38 percent of these students reached the criteria after summer school in 1998, but this is still substantially lower than the 47 percent pass rate of retained 6th graders in 1998. In addition, about 15 percent of retained 8th graders and Transition Center students moved or transferred to another school district between fall 1997 and fall 1998, and approximately 16 percent of the remaining retained 8th grade students and Transition Center students dropped out of school during the same time period. In comparison, only 6 percent of all 8th graders in the district dropped out of school between 1997 and 1998. [63]

Students who are struggling do not always receive the supplemental services they need to be successful in school. After students have been retained, they do not always receive support services or remediation to address their academic difficulties.

• For example, an evaluation of Washington DC’s 1981 promotion policy found that although remediation and other academic support services were available, very few of the retained students received these services. The vast majority of students who were promoted received assistance from math and reading specialists and teachers aides five days a week (74 percent of students who passed in reading and 75 percent of students who passed in math), while few students who were retained used the services at this high rate (only 28 percent of students retained in reading and 23 percent of students retained in math).