State higher education officials detailed a new project Tuesday in which Massachusetts is leading a group of nine states in developing a way to measure and compare what students learn in college by looking at their actual work, from term papers to lab reports, rather than using a standardized test.
The goal is to create a reliable standard to compare the success of Massachusetts public colleges to those of other states, from Connecticut to Kentucky to Oregon, and to help professors teach better in the process.
Also driving the plan is a fear that if states do not find good ways to hold themselves accountable, they will be vulnerable to a more simplistic effort to impose national standards.
“There is tremendous interest in this nationally, because everybody in higher education knows, if this doesn’t work, the next answer is a standardized test probably imposed by the federal government or by states,” Commissioner Richard M. Freeland said at a state Board of Higher Education meeting where the plan was discussed Tuesday.
The project, overseen by the Association of American Colleges and Universities, recently received $1 million in funding from the Bill & Melinda Gates Foundation.
President Obama deeply unsettled many educators with a proposal last summer that Congress eventually link how much colleges receive in financial aid to their ratings on factors including graduation rates and even graduates’ earnings.
At the moment, the only measures that compare the abilities of Massachusetts public college graduates to those of other states are passing rates on national licensing exams, state officials say. Students from the Commonwealth tend to do about average or worse.
In an interview Tuesday, Freeland said his concern about what students learn in college goes back to his years as president of Northeastern University, when he conferred degrees on thousands of graduates at commencement every spring.
“I’d be thinking to myself, so what do they really know? What do I know about what they know? I always had to be honest and say to myself, ‘I don’t really know, and nobody else does either,’ ” he said.
“I just think that has become increasingly unacceptable,” Freeland added, “knowing how high the stakes are for the state and the country in terms of producing a well-educated, competent workforce in a highly competitive economy.”
Getting support from professors, who are often concerned about the wrong standards being applied from on high, is a major focus of the effort. Paul F. Toner, president of the Massachusetts Teachers Association and a higher education board member, said state officials appear to be on the right track, but pushed them at Tuesday’s meeting to reach out to union leaders on each campus.
Some professors are worried that campuses or even individual instructors may be punished for poor results when they are doing their best to help students who arrived on campus underprepared, Toner said in an interview.
“I think there’s just a concern that they’re going to be held accountable for things beyond their control,” he said.
Before reaching out to other states, Massachusetts conducted a pilot project last spring. Seven campuses — including several community colleges, Framingham and Salem state universities, and the University of Massachusetts Lowell — gathered about 350 samples of assignments students who were nearing graduation had completed for classes.
Then a group of 22 professors spent three days over spring break at Framingham State evaluating the work for what it showed about each student’s abilities in written communication, quantitative literacy, or critical thinking, said Bonnie Orcutt, director of learning outcomes assessment for the Department of Higher Education. They used a method developed by the Association of American Colleges and Universities that scores work on a scale of zero to four.
Two professors reviewed each piece of work, and tiebreakers were brought in when their assessments differed significantly. The evaluators did not see students’ names or the institutions they attended.
Not every class assignment turned out to be a good opportunity for measuring a student’s skills. But Orcutt said the faculty members came away with new ideas for how to teach and how to assign their students meaningful work.
The nine states will now essentially repeat the experiment on a bigger scale. Massachusetts officials, the Association of American Colleges and Universities, and the State Higher Education Executive Officers Association brought together the group of states: Connecticut, Indiana, Kentucky, Missouri, Minnesota, Oregon, Rhode Island, and Utah. They will spend this fall gathering student work and plan to meet next spring to evaluate it.
If the multistate project goes well, Freeland said. he hopes to begin comparing Massachusetts to the other states within a couple of years.
He hopes that students will eventually be evaluated a number of times over their college careers, so professors can help them address their weaknesses. The faculty on each campus will have the final say about whether their institution participates, Freeland said.