fb-pixel Skip to main content

Pollsters should look hard at their Question 1 misfire

Surveys that are so wide of the mark can influence the outcomes of campaigns.

With hours to go until the polls closed, Tom Smith with Neighbors United for a Better East Boston was door knocking to ask people to vote yes on Questions 1 and 4.Jessica Rinaldi/Globe Staff

This year in Massachusetts, numerous public-opinion polls showed that Question 1, the ballot question to add a 4 percent surtax on incomes over $1 million, was far ahead, thereby suggesting it would pass by a landslide. When all the votes were tallied, that measure, whose approval was the final step in amending the state constitution, passed by a 4 percent margin.

Thus despite the public polling, Question 1 turned out to be the most competitive statewide contest in 2022 — and one that was decided by the smallest margin of any ballot question since 2012.

What happened?

We have a prolific ecosystem of opinion pollsters in Massachusetts, with no fewer than six universities and think tanks publishing surveys this year. And yet pollsters failed to correctly portray the electorate’s opinion about Question 1. They owe it to the public to reevaluate their methodologies to ensure their polls are more accurate moving forward.

Four polls showed Question 1 ahead by a margin that made its victory seem inevitable. In mid-October, Suffolk University had yes winning by 21 percent; University of Massachusetts Amherst by 26 percent; UMass Lowell by 27 percent; and the MassINC Polling Group by a whopping 28 percent. Each poll carried a margin of error between 3 percent and 5 percent.


Now, these pollsters may contend that attitudes toward Question 1 seismically shifted in the final weeks and argue that their polls were a true reflection of public opinion at the time they were conducted.

But internal data from the no campaign does not support that position. My firm conducted polling for the coalition opposing Question 1, and six statewide surveys in the eight weeks before the election; all showed a highly competitive race with single-digit margins.

Poll results that are disseminated for public and media consumption in the lead-up to an election can have a large influence on the final outcome. Voters are not motivated to get out and vote in a contest that is deemed uncompetitive. Donors do not invest their money in causes that seem destined to lose by a landslide. Media organizations take polling into account when deciding which races deserve coverage and when making endorsement decisions (the Globe was one of the sponsors of the Suffolk University poll in question and endorsed yes).


So it is certainly fair to wonder if inaccurate public polling changed the result of Question 1.

There are technical and mundane reasons why these polls were incorrect. Voter turnout was at just 51.4 percent, the second lowest percentage turnout in the past 75 years.

A poll that interviews all registered voters, or that depends on voters to honestly self-report their intention to stay at home, as several of these did, will not screen out the half of the electorate that does not vote in midterms. Low voter turnout is a recurring feature of Massachusetts midterm elections and should have been accounted for in these polls’ sampling methods.

There are also broader methodological problems with these four polls. MassINC likely introduced a selection bias by asking 27 consecutive questions about transportation, including 12 questions about the MBTA. Respondents who agreed to participate and then completed MassINC’s long online questionnaire probably care more about public transit — and implementing a surtax to nominally fund it — than the average voter. Among other problems, UMass Amherst grossly misrepresented the partisan makeup of the electorate by failing to weight its poll properly: 59 percent of poll respondents identified as Democrats, whereas Democrats currently make up less than 30 percent of registered voters.


The Question 1 surtax is now enshrined within the Massachusetts Constitution. Tens of thousands of homeowners, small-business owners, and retirees will be forced to pay a tax that received a yes vote from only one-quarter of the registered voters in the Commonwealth, the proceeds from which can be spent on priorities that are unrelated to education and transportation.

Public opinion pollsters have a critical role in telling our new governor and the Legislature how their constituents want this revenue allocated and what forms of relief taxpayers may demand to offset the lamentable consequences of Question 1. But unless public pollsters and their sponsors reexamine and recalibrate their polling in the wake of this past election, they may fail to accurately report the opinions of the residents of Massachusetts in 2023 and beyond.

Brian Wynne is the founder of Mass Political Marketing, an opinion research firm, and served as pollster for the Coalition to Stop the Tax Hike Amendment. He was 2018 campaign manager for Governor Charlie Baker and is the former executive director of the Massachusetts Republican Party.