Test maker accuses Boston schools of misusing exam

Process hurts disadvantaged students seeking admission to coveted schools, company says

Pat Greenhouse/Globe Staff

[Sign up to receive a newsletter for The Great Divide, a new investigative series that explores educational inequality in Boston and statewide. And please reach out to us at [e-mail] with story ideas and tips.]

Boston Public Schools have for years misused the test results that help determine admissions to its coveted exam schools in a way that makes it harder for “underrepresented” students to gain entry, according to the organization that administers the controversial exam.

As a result, the Education Records Bureau decided last spring to sever its relationship with the city’s school district — its largest client — after 25 years, according to an e-mail the Globe obtained from the organization’s president Tom Rochon. The e-mail was sent Tuesday to some of the bureau’s other clients, including 30 independent schools in the Boston area that use the test in admissions decisions.

“District leaders have not yet chosen to make this information public, but when they do so we want to be sure you have the necessary background,” Rochon wrote. He said his organization has been trying to get the district’s attention on the issue over the last eight years.

Boston school officials, however, strongly maintain that they are the ones who walked away from the relationship in search of a fairer and more equitable test. Superintendent Brenda Cassellius, who joined the district in July, said that she has “often and publicly” noted that this is the final year of the district’s contract with the Education Records Bureau. Under the current year-long contract, the district is paying the organization about $600,000.

"Boston public schools is committed, and actively working, to expand equitable access to our exam schools,'' Cassellius said in a statement.

The fairness of the admissions process to the three exam schools — Boston Latin School, Boston Latin Academy, and the John D. O’Bryant School of Mathematics and Science — has been a contentious subject in recent years. Several civil rights groups and community organizations have argued that the admissions process, based half on student grades and half on their scores on the test, called the Independent School Entrance Exam (ISEE), has disadvantaged low-income students, particularly Blacks and Latinos.

And now, the test’s administrator appears to be confirming some of those fears. The district’s “misapplication of ISEE scores has been one factor in perpetuating admissions outcomes that disproportionately affect students belonging to underrepresented groups,” Rochon wrote. The dispute raises questions about who is more to blame for a potentially biased process — the test creator or the district that uses the test.

Civil rights leaders called the letter’s revelations “damning” to Boston public schools and City Hall, saying the district appears to have knowingly left in place an admissions process biased against students of color. Tanisha Sullivan, president of the Boston chapter of the NAACP , demanded an investigation into Boston’s exam schools process, saying the city needs “a moratorium on any test for exam schools admissions until that investigation is completed.”

“This is a shocking development in our fight for exam schools equity,'' she added.

Cassellius said her administration will release a request for proposals within the week for a test that has been shown to be free of bias and is more aligned with state standards than the ISEE. Boston’s contract with the records bureau expires June 30.

“Almost immediately upon my arrival in Boston, it was brought to my attention that there were concerns that the ISEE test was potentially creating barriers for some students seeking admission to BPS’ exam schools, particularly underrepresented students,” Cassellius said in the statement.

Black and Latino students combined make up just 20 percent of the student body at Boston Latin, the most competitive of the three schools, compared to 72 percent of the school district. They constitute 66 percent of the students at the O’Bryant and 47 percent at Boston Latin Academy.

Previous public debate about the test has focused on the fact that it is not aligned with most Boston public schools’ curriculum — or state standards—and as a result privileges private school applicants, who are disproportionately white and whose schools frequently sync their teaching and curriculum to the test. It also provides an advantage to wealthier Boston Public Schools students whose families can afford to hire private tutors or others to help prepare them for the ISEE.

"Given that it’s an exam that is completely foreign to students and requires parents and outside resources to help prepare them, it doesn’t make sense as a tool for identifying which students are going to be able to succeed in a rigorous academic environment,” said Joshua Goodman, an associate professor at Brandeis University. Goodman authored a 2018 study that found the school system’s reliance on the ISEE potentially blocked thousands of students of color from accessing the exam schools. The study recommended using Massachusetts’ standardized test, the MCAS, to boost diversity at the schools.

But Rochon spelled out other problems in his e-mail, claiming that for years the records bureau has asked district leaders to use the test scores in an “appropriate way” — for instance stopping their practice of summing student results in the different sections (verbal reasoning, quantitative reasoning, mathematical achievement, and reading comprehension) into a single score.

Instead, Rochon said in an interview, that the four sections are intended to be considered individually as part of a broader assessment of an applicant’s merits — which ideally would take into account their past educational experience. Too much emphasis on a summary test score can disadvantage students from more marginalized groups. But Rochon added that “it is obviously up to the citizens of Boston to decide how to weight academic achievement ... with the really important issues around equity and access.”

Charlie Drane, vice president of enrollment at the private Boston College High School, said his school uses the ISEE exam, along with factors including teacher recommendations and grades, when judging applicants for seventh grade.

School counselors consider students’ backgrounds when weighing their ISEE scores, Drane said. For example, if a student is a recent immigrant with limited fluency in English but has excellent teacher recommendations, the school may not give much weight to a low reading score on the ISEE. If "his recommendations say he’s a hard worker, he’s going to do great here and we’re going to go after him,” Drane said.

Rochon said the organization offered over the last eight years to fund research studies for BPS to determine the fairest way to weight the test scores. “But we were always rebuffed,’’ Rochon wrote. After what Rochon describes as the most “recent refusal,” the bureau notified Boston officials in April 2019 that they would cut ties within a year.

Boston school officials acknowledged that the Education Records Bureau approached them about funding a study. But Cassellius said the district, already eager to find a new test, declined the bureau’s offer to avoid giving undue advantage to “any one particular vendor.”

School officials provided copies of exchanges between Rochon and district leaders shortly after Cassellius took charge. In one e-mail dated in August, the superintendent said she was "hesitant to do any collaboration with a vendor that could eventually seek advantage on a future RFP.”

District officials said they had no record of the bureau reaching out to them about studying the test process several years ago. They added that they have recently studied whether weighting one subject over another could lead to a more equitable process and found that it would have no impact on exam school demographics.

The blame for the controversy falls on both sides, said Michael Contompasis, who served as headmaster at Boston Latin School from 1976 to 1998, and again on an interim basis from 2016 to 2018. Contompasis said the bureau pushed the district to study the test’s fairness after a string of media reports questioned whether it was biased against Black and Latino students.

“The district should have done a validity study on the exam when it was first asked," he said. "It should be something that is periodically done.” And the bureau should have raised their concerns more forcefully well before severing ties.

Partial funding for this initiative is provided by the Barr Foundation, a Boston-based foundation that has made student success in high school and beyond a top priority. The Globe has complete editorial control over story selection, reporting, and editing.