Report finds BPS data is inaccurate and incomplete, with wide-ranging consequences

The exterior of Boston Public School headquarters.Jessica Rinaldi/Globe Staff

Boston Public Schools has poor data collection and reporting processes, resulting in inaccurate, incomplete, and otherwise unreliable data in many areas, including bus performance and graduation rates, according to a review released Friday as part of the district’s state-mandated improvement plan.

The report found that the district often puts English learners — who make up nearly one-third of students — in classes that don’t match their skill level. BPS doesn’t review education plans for about a quarter of students with disabilities on time, leaving those children with services that may be out of date.

The district also fails to report to the state many incidents where staff physically restrain students, the assessment found. And it left nearly one-third of all bullying investigations open at the end of the school year — either because the complaints weren’t resolved or because the schools failed to update central office on the status.

The third-party data review by Ernst and Young consultants largely confirmed state concerns about district data, such as undercounting late buses and possibly overestimating high school graduation rates. The review was one of the requirements of the improvement plan Mayor Michelle Wu agreed to last summer, averting a state takeover of the city’s schools.

The report is based in large part on a review of information from the 2021-22 school year, and doesn’t reflect some of the progress the district has made since.

In a statement, Superintendent Mary Skipper and School Committee Chair Jeri Robinson acknowledged the need for reliable data, thanked the state for its review, and said those improvements are continuing in earnest.

“We have already begun to address several of the issues highlighted . . . and are committed to resolving the remaining issues with urgency as we work to make BPS the high functioning district our students and families need and deserve.”

The Department of Elementary and Secondary Education released the report Friday ahead of a state education board meeting scheduled for Tuesday.

The report cited a multitude of problems with the district’s approach to collecting and reporting information. BPS typically reports data with no internal review for accuracy, it found; the district often uses manual reporting processes vulnerable to errors, fails to train and communicate adequately around data procedures, and doesn’t give staff who work with data the authority and support they need to be effective.

In a statement, Education Commissioner Jeff Riley said the department appreciated Skipper’s willingness to endorse the state’s recommendations and to fix the problems.

But, he warned, the improvements “will require a systems-level approach that cannot be done by a single individual alone. A better functioning central office as well as a commitment from elected officials and the BPS School Committee is required to enable the change our students deserve.”

The two biggest problem areas in district data identified by the auditor were tracking students leaving schools — the driver of the graduation rate problem — and physical restraints of students.

According to a sample of 100 student withdrawals, about 80 percent of the time when a student left a school, Boston did not have the required supporting documentation to explain where the student went. That information is important because if the student transferred schools, they would be excluded from graduation rate calculations; if they dropped out, the school’s graduation rate would fall.

The auditor identified 89 incidents of physical restraints reported in the district’s internal Aspen data system but not reported to the state — almost one-third of the restraints reported in total by the district last year.

Other problem areas included:

• On-time bus performance: Because the district does not have a systematic way to report on routes with missing arrival data, roughly one-quarter of planned bus routes each morning are completely omitted from its reports to the state on on-time performance. That means BPS buses may be even less reliable than the district’s official data suggest.
• Individualized education programs for students with disabilities: The district takes too long to update IEPs about one-quarter of the time, meaning students might not be getting services they now need that were not noted when their plans were first developed.
• Parental complaints: Nearly a third of bullying reports to the district remained open at the end of last school year, even though district policy says they should be closed within five days of receipt.

The consultant also examined data reporting on student discipline, English learners, and student enrollment and said all three were low-risk areas, but they did find that nearly half of English learners this year have English language development levels that don’t match their English language assessment scores — potentially resulting in them being put in the wrong English classes.

In some areas, the report noted, interviews suggest the district is already making progress. BPS has created a new standard student withdrawal form and has taken other steps to ensure adequate documentation of student transfers. The district has hired a new analyst to track bus performance, and it put in place a new helpline in May 2022 to address parental complaints about bullying. The reviews in those areas do not account for improvements made for the current school year.

BPS officials said the district has started to conduct internal audits of its student withdrawal data in an effort to ensure the appropriate documentation is being included. In a recent review, officials found that 70 percent of 775 student withdrawal records had acceptable documentation.

Officials also said that starting this spring, the district will begin auditing its physical restraint data to make sure schools are reporting the same information to the state and the district.

In a statement, Wu said that in order to deliver needed improvements, the district needs to be able to track its results.

“I look forward to continued partnership with the state and our school communities to set the right foundation for our data systems as we build the rigorous, nurturing learning opportunities for all our young people,” she said.