Opinion

opinion | Heather Hopp-Bruce

Rats! You can’t always trust statistics

IT’S WELL-KNOWN that statistics can be wildly manipulated. But what about a simple list of numbers? Can a basic chart be misleading? Sadly, yes. Case in point: Open rodent complaints in Boston. We started with a simple data set off the city’s website. On their face, the numbers appeared to demonstrate an alarming spike in cases. From April to September of this year, the number of complaints rose sharply. That’s not a seasonal issue, either — last month there were many more reports than the previous September. So should Bostonians be concerned? Is the city being overrun by rats? To find out, we crunched the numbers — and then misused a variety of graphing techniques. We rotated things, speculated using random events, and correlated the rise with seemingly unrelated data sets. Keeping our eye throughout, of course, on the real question: Where are all these critters coming from? (Spoiler alert: The real answer is at the bottom of the page).
Image problem

It looks like the plague is upon us! Rodent complaints nearly doubled from August to September, as shown in this simple line graph. But, wait there’s something wrong with this picture.

Problem: The data reflect number of complaints, not number of rodents. Furthermore, not all rodents are rats.

Gaming the reason

Significant changes in the number of complaints seemed to be linked to major developments in the Massachussetts casino saga. Has the pro-casino lobby’s promise of prosperity been widely accepted in the rodent community?

Problem: Cherry-picking events that coincidentally correspond to a line chart — and leaving out those that don’t — is entirely subjective.

Cause and (bad) effect

Boston’s open rodent complaints seem to have started to follow a similar uptick in the city’s open graffiti complaints. Conclusion: There’s a new graffiti artist in town with a huge fan base of rodents, like Banksy for the hip vermin set. Who say, by the way, they knew about him before you did.

RODENT COMPLAINTS IN RED, GRAFFITI COMPLAINTS IN BLUE

Problem: Or not. A causal relationship shouldn’t be asserted without more evidence, be that data or anecdotes. Also, these data don’t even really correlate that well.

Axis of evil

Oh, wait: What happened to the rodents? Now we’re starting to feel sorry for them.

Problem: It’s dangerously easy to flip a chart on its side. Since we’re so well-trained (maze, anyone?) to read a chart that spans time horizontally, adjusting to a different set of rules is difficult. This effect is more commonly achieved by expanding the axis values so the curve is more dramatic — or less.

So, what is really going on?

The first thing to remember? The data here reflect the number of complaints, not actual rats. (Yes, even using a rodent image to represent a complaint is just plain wrong.) But not even the dramatic uptick in complaints is so straightforward. According to John Meaney, Boston’s director of environmental services, the change is due to two things. First, the department has embarked on a public awareness campaign in which it solicits complaints. More importantly, though, city inspectors have changed the way they record sightings. If rodents are spotted in an alley, now a complaint is registered on behalf of every building on the block, rather than just the resident making the report. Both of these new intiatives started in, you guessed it . . . May. Meaney would like to remind you to secure your garbage and, if you do see a rodent, please register a complaint at www.cityofboston.gov or call Boston Inspectional Services at 617-961-3422.

DATA: City of Boston

Heather Hopp-Bruce, Luke Knox/Globe Staff; Illustrations by Francis Blake for The Boston Globe

Heather Hopp-Bruce is the Globe’s op-ed design supervisor. Follow her on Twitter @H_HoppBruce.
Loading comments...
Real journalists. Real journalism. Subscribe to The Boston Globe today.