Over the years I have participated in dozens of graduation day ceremonies and had the privilege of placing diplomas in the hands of many beaming graduates. These are happy occasions, to be sure, but I have often left auditoriums and arenas with a nagging thought: Do we really know what these graduates have learned and are able to do?
The remarkable truth is that we in higher education have never had a reliable means by which to answer that question. Our lack of accountability on the matter dates back to a bygone era when higher education was the province of an elite few. The general presumption was that these students from mostly privileged backgrounds would simply learn what they needed to learn in due course. As access to higher education expanded and democratized following World War II, and as the number and diversity of colleges and universities increased, the question of what a degree really means came into question. Two more recent, albeit distinct factors — the competitive challenges faced by US employers in a globalized economy, and the struggles of families to pay rising college costs — have led to even greater demands for accountability by campuses and demonstrable, transferable skill sets by graduates.
Curiously, the heightened interest in accountability has until recently not generated much progress with respect to measuring the quality of student learning. The Obama administration’s proposed ratings system would offer consumer information on graduation rates and college costs, but would not require colleges to measure or report student learning outcomes. A recent report in the Chronicle of Higher Education found that many campuses are moving away from voluntary assessment efforts which have been largely based on standardized test results. With good reason, faculty have often ignored or revolted against efforts to impose high-stakes testing on their campuses. Such testing provides data of limited value and does little to help educators improve the quality of instruction.
Clearly we need a new mechanism by which to demonstrate the intellectual competence of our students. Without a common set of criteria by which to gauge the quality of student work, we can’t improve our programs, enhance curricular design, or effectively prepare students for future employment and civic engagement.
To fill this void, Massachusetts public colleges and universities have taken the lead in a national effort to create a new statewide framework for what is called “student learning outcomes assessment.” This work involves hundreds of dedicated faculty across the state and is a signature component of the Vision Project, the state’s strategic agenda to achieve national leadership among state systems of higher education. With Vision Project goals in mind — principally, the need to achieve higher levels of student learning through better assessment and to close achievement gaps among students from different racial, ethnic, and income groups, a top priority of the Patrick administration — we developed and piloted an assessment model utilizing student coursework.
This pilot effort — launched at seven community college, state university, and UMass campuses last year — assessed broad dimensions of liberal arts learning. Hundreds of student papers, lab reports, and other samples of written work were collected from a wide range of courses across many disciplines. Several dozen faculty scorers then used rubrics, or standards, developed by the American Association of Colleges and Universities to assess student work in three areas: written communication, critical thinking, and quantitative literacy.
More from The new U: The case for a liberal education
Our first attempt to implement the model proved extremely beneficial. We learned that appropriately trained faculty can achieve a high degree of consensus on the quality of student work, thus demonstrating a valid basis for comparison. This comprehensive approach to assessement revealed patterns and levels of learning across each rubric, not just a set of numerical scores. The experiment also provided faculty participants with important insights, among them the need to rethink the way assignments are made. Many faculty discovered that their assignments would need to be redesigned if their students were to be able to demonstrate the competencies spelled out in the rubrics.
The so-called Massachusetts Model is the first statewide attempt to assess college student learning without the use of standardized tests and has drawn substantial national interest. Eight states — Connecticut, Rhode Island, Indiana, Kentucky, Minnesota, Missouri, Oregon, and Utah — have recently joined the Commonwealth in a bid to produce cross-state comparisons of student learning outcomes. We are on the verge of being able to deliver a comprehensive answer to the question, “Is college worth it?” And we will do so not simply with cost calculators and earnings projections but with comprehensive evidence of whether our graduates are truly prepared to compete in and contribute to the world.
Read more from The new U: