University of Chicago report finds district-wide reform making progress in Chicago high schools, despite teacher uncertainty

The system-wide effort to improve student performance in Chicago public high schools is yielding promising results, yet skepticism among teachers may impede long-term success, according to a new study published by researchers affiliated with the Department of Education and the Harris Graduate School of Public Policy Studies at the University of Chicago.

In 1995, the Illinois legislature enacted the Chicago School Reform Amendatory Act, which helped centralize authority and enabled the district to launch an educational accountability agenda that focuses on establishing and enforcing system-wide school and student performance standards. The Chicago Public Schools Educational Accountability Agenda requires schools with less than 15 percent of students scoring at national norms to be placed on probation. During the first year of the reform program 38 of the district's non-specialty high schools (61 percent) were placed on probation.

The University of Chicago study, "Implementation of an Educational Accountability Agenda: Integrated Governance in the Chicago Public Schools Enters its Fourth Year," evaluates how the district, schools, and teachers have implemented key components of the accountability agenda (such as academic promotions, probation/reconstitution, academies, and student advisories), and the effects of this implementation on high-school teaching.

"Our study found that while teachers in probationary schools are complying with district policies that focus on student performance, high school teachers are not willing to give up their control over the curriculum," said lead investigator and Associate Professor Kenneth Wong, who noted that this lack of compliance trend was consistent among teachers in sanctioned and non-sanctioned schools. "Overall, teacher responses to formal sanctions have not led to desired changes in the broader curriculum and instructional practices."

Despite varying degrees of implementation of reform strategies by teachers, the study identified encouraging signs of high school student improvement. "The gap in standardized test scores between reconstituted schools and non-probationary or average schools narrowed by nearly 8 percent in reading and 6 percent in math between 1996 and 1998," said Wong, who cautioned that standardized tests represent a crude and questionable measure of school performance.

"The Mayor of Chicago's bold move to centralize and integrate authority over the public schools and to focus on fixing bad schools has made promising inroads on a recalcitrant system," according to Laurence E. Lynn, Jr., a co-author of the study and the Sydney Stein, Jr., Professor of Public Management in the Harris Graduate School of Public Policy Studies. "The leaders of other urban school systems will do well to take a good look at what is happening in Chicago."

Among the study's other findings:

• English teachers observed in the study allocate an average of 7 percent of instructional time to "test practice" and 29 percent to "test skills development," which include activities that engage students in skimming reading passages and the daily 10-minute grammar exercises to determine academic needs. Teachers in sanctioned schools allocate a higher percentage of instructional time to test-related practices.

• Sanctions do not reduce teacher discretion over broader curricular and instructional practices. It is not clear if probation/reconstitution has produced a significant reevaluation of the effectiveness of current curricular and instructional practices.

• While principals find external partners helpful as technical consultants, high school teachers in the study report that the external partners have little effect on classroom practices and are a challenge to their instructional authority. This finding may be the result of a conflict between the central objective of the probation policy (improving test scores) and that of the external partners, who are charged with school-wide, long-term improvements.

• The district's use of formal sanctions, in combination with its efforts to attract and retain middle-class students, appears to create market-like competition among schools. Additionally, the stigma attached to schools under probation may impede faculty and student recruiting efforts, efforts as well as reduce the accompanying resources that are necessary for long-term improvement.

"Chicago's 'educational accountability agenda' and 'integrated governance' are not especially popular with school reform groups who believe that emphasizing test scores distorts the goals of education and prefer that change and improvement be controlled at the school site and in communities rather than at the school board level," said Lynn. "Despite some resistance, the central administration's strong emphasis on improving student performance and on raising achievement at the lowest performing schools is bringing a welcome focus on what and how well students are learning to Chicago Public Schools."

According to Wong, the district may need to consider more comprehensive alternatives to formal sanctions to encourage and support teachers in sustaining, evaluating, and developing effective instructional practices.

"Because the current efforts to direct improvements are greatly circumscribed by teacher discretion, the district must devise consistent policies and sanctions as well as improve the quality of professional development," said Wong. "The district's emphasis on test-taking may need to be better coordinated with other district-funded support strategies as part of a viable process for improving school and student performance over the long-term."

Wong and his colleagues also recommend that the Chicago Public Schools:

• Narrow the role of external partners to addressing the district's central mission of immediate test score improvement, refocusing on long-term improvements after this initial objective is met.

• Examine how policies and changing demographics contribute to the movement of faculty, students, and resources away from probationary schools and how the district will ensure students in different types of schools receive adequate resources and opportunities.

• Develop a more comprehensive program to improve elementary school performance, with a stronger focus on reading at the early grades to meet the expectations of the district's academic promotion policy.

• Using local universities, including their arts and sciences programs, to create teacher recruitment/induction programs, which may result in long-term, district-wide instructional improvements.

The study is based on results from a survey completed by 66 percent of the district's 62 non-specialty high school principals that gathered information on student enrollment, demographics, teacher recruitment/retention, implementation of restructuring efforts, and levels of district support. The survey was augmented by more than 12,000 minutes of classroom observations in four sample high schools as well as interviews with principals, administrators, and math and English teachers that were completed during the 1996-97 and 1997-98 school year.

"To assess objectively the success of the educational reforms, our researchers went into the schools to trace how the high school redesign was being implemented and its effect on teaching," said Robert Dreeben, a study co-author and Professor in the Department of Education. "We took these added steps to supplement commonly used measures of reform such as standardized test scores and general surveys based on people's perceptions."

This project was funded in part by the Joyce Foundation and the U.S. Department of Education. The study's other co-authors are Dorothea Anagnostopoulos and Stacey Rutledge, doctoral students in the Department of Education at the University of Chicago.

The Irving B. Harris Graduate School of Public Policy Studies at the University of Chicago is among the leading public policy research and training programs in the nation, with core concentrations in such policy areas as children and families, health policy, international policy, public management, and poverty and inequality. Established in 1988, the Harris School currently trains nearly 200 master's and Ph.D. students annually for careers in public service and private sector policy analysis.