Test isn’t better at gauging readiness, study says

A test promoted as a more effective tool than the MCAS at measuring students' college readiness is no better at predicting performance than the longstanding assessment, according to a new study commissioned by the state.

The results of the study are sure to fuel debate as the state Board of Elementary and Secondary Education votes next month on whether to dump the Massachusetts Comprehensive Assessment System and replace it with the PARCC — or the Partnership for Assessment of Readiness for College and Careers exam.

Such a change would mean a daunting adjustment for many of the state's public schools and nearly 1 million students, as they shift to a test designed to be taken on a computer, rather than paper and pencil. The new test would eventually replace the MCAS as a graduation requirement.

The study, released Tuesday, is the first concrete analysis of the PARCC's ability to predict college preparedness, compared with the MCAS, which the state adopted in late 1990s. The results elicited a range of reactions from advocates and opponents of the new test.

Paul S. Reville, who was state education secretary under former governor Deval Patrick, said the PARCC is a better and more comprehensive test than the MCAS and that its ability to predict college success is one of many factors for the board to consider — and not the most important.

"The rubber really hits the road when you adopt assessments, and the assessments drive instruction," Reville said. The message that education officials send through the standardized test they choose "is more important than anything else we do in signaling to teachers the kind of instruction that we want to see in classrooms," he said.

Monty Neill, executive director of FairTest, the National Center for Fair & Open Testing, said the study's finding that both tests fared about as well at predicting performance as the SAT, commonly used as a factor in college acceptance, showed that all three are flawed.

"If the MCAS and the PARCC aren't any better than SAT, then relying on the tests for college prediction puts Massachusetts really in a weak position," Neill said.

FairTest, a Boston-based advocacy group, supports placing a moratorium on the use of standardized tests as a graduation requirement and developing a different and more comprehensive assessment system.

The PARCC has been developed by a consortium of states led by Massachusetts. Based on a national set of academic standards known as Common Core, the test is under consideration by 10 other states and the District of Columbia.

The PARCC aims to delve into how students arrived at their answers and require more critical-thinking skills than the MCAS, state officials say.

About 54 percent of school districts statewide administered the PARCC exam this spring, in the second year of a two-year tryout. The rest remained with the MCAS.

Mitchell Chester, the state's commissioner of elementary and secondary education, said Wednesday that despite his deep involvement in developing the PARCC, he was prepared to fairly weigh the benefits and drawbacks of the PARCC and the MCAS as he prepares to make a recommendation to the board of education.

Chester said predicting college readiness was a factor the board should consider, but that the study did not closely examine the PARCC's more difficult performance standards.

"All of the evidence there is that there's substantially different expectations for student performance and what students have learned, with the PARCC being much more demanding," Chester said.

Although the study, by New Jersey-based Mathematica Policy Research, found the tests were about equally good at predicting student grades in their first year of college, it said the PARCC standards for college readiness were a better predictor of whether students could make A's or B's in math than the MCAS proficiency standard.

In the study, researchers studied 866 college freshmen who were enrolled at 11 public institutions in Massachusetts and had graduated from high schools across the state.

According to Brian Gill, a Mathematica senior fellow who worked on the study, "Massachusetts is in an enviable position. There are two good choices here."

Secretary of Education James A. Peyser, who is a voting member of the board that will decide PARCC's fate, said he was awaiting further information about both tests before making up his mind.

"I don't think this [study] makes the decision for us," Peyser said. "This is going to be a decision that ultimately requires some qualitative judgments and some trade-offs to be made. It's not necessarily an easy or a close call."