School Ratings: Reflecting Student Background More Than Learning Gains
When it comes to evaluating the effectiveness of schools, it turns out that test scores may not tell the whole story. According to a study led by MIT scholars, school quality ratings often mirror the academic preparedness and family backgrounds of their students rather than the actual educational value the schools provide. This revelation suggests that many schools with lower ratings might be performing much better than previously thought.
The Influence of Student Demographics
The research, co-authored by MIT economist and Nobel Prize winner Josh Angrist, dives into the heart of how school ratings are determined. These ratings, commonly found on various education-focused websites, tend to be closely linked to the racial composition of the student body. Specifically, schools with a higher percentage of white students often receive better ratings than those serving predominantly Black and Hispanic communities.
“A school’s average outcomes reflect, to some extent, the demographic mix of the population it serves,” Angrist explains. This means that current school ratings might not accurately reflect a school’s contributions to student learning. Instead, they could be misleading parents and policymakers by suggesting that schools in more affluent or white areas are inherently better.
Misleading Measures of School Quality
The study, carried out in the Denver and New York City school districts, adopted a novel approach to measure school quality. By focusing on changes in test scores and incorporating statistical adjustments for racial composition, the researchers were able to develop a more accurate measure of a school’s impact on its students. This method aims to provide a clearer picture of how much learning gain a school genuinely contributes, regardless of the student demographics it serves.
This research highlights a significant issue with how school quality is currently measured. Simple, aggregate measures of school quality, such as average test scores, are not only highly correlated with the racial makeup of the student body but also “tend to be a misleading guide of what you can expect by sending your child to that school,” according to Angrist.
A Path Forward
The team behind the study is optimistic that their findings will encourage school districts to reevaluate their approach to rating school quality. By shifting towards measures that accurately account for student preparation and improvement, they believe ratings can more genuinely reflect a school’s contributions to its students’ education.
The New York City Department of Education is already collaborating with MIT’s Blueprint Labs — co-founded by Angrist and study co-author Parag Pathak — to pilot a new ratings system. This system aims to avoid the pitfalls of current rating methodologies by providing a more nuanced and accurate assessment of school performance.
This effort is supported by the Walton Foundation and the National Science Foundation, indicating a growing recognition of the need for more accurate measures of school quality. As more districts consider updating their rating systems, there is hope that families will have access to better information, allowing them to make more informed choices about their children’s education.
In the end, this research underscores a critical challenge in American education: overcoming selection bias. With more sophisticated rating systems, parents, educators, and policymakers can work together to support all schools in their mission to provide high-quality education — regardless of the demographic backgrounds of their students.
Divya 
Deepika 
Mehek