Once every minute and 44 seconds.
That is, on average, how often NBA officials are getting calls wrong or missing calls at the end of close games, a Globe analysis found. But they are still correct 86 percent of the time in the final two minutes of close games.
As part of a transparency initiative, the NBA began releasing its “Last Two Minutes” officiating reports to the public on March 2. The reports contain reviews of plays that occur in the final two minutes of regulation or overtime of close games. Only games where the score is within 5 points heading into the final two minutes are officially reviewed.
The reports, which are publicly available, review every material play, including the call type, committing and disadvantaged players, comments, the review decision, and a link to a video clip of the play.
There are four categories of calls in the reports.
Correct call: An official whistles a player for the appropriate violation.
Correct non-call: A play is perceived as including a violation, but an official rightly determines no such violation occurred and doesn’t blow the whistle.
Incorrect call: An official whistles a player for a violation when one did not occur.
Incorrect non-call: An official misses a violation altogether. (We’ll refer to these as “missed call,” for clarity.)
Through the conference semifinals, there had been 135 games reviewed. Though each individual report offers a small sample, the data collectively reveals overarching trends that paint a more complete picture.
Here are some findings by looking at them in aggregate:
How often are calls incorrect or missed?
Of the 1,864 plays reviewed since the league launched the initiative through the conference semifinals, 250 were ruled to be incorrect or missed calls, about 13.4 percent. Eight out of 10 games included at least one incorrect or missed call. Officials are about eight times more likely to miss a call than get it wrong.
Though these mistakes on average happen less than once (0.6) per minute, the data show that officials are still likely to either miss a call or make an incorrect call at least once in the final two minutes of a close game. Officials are also likely to make about three officiating mistakes for each period of overtime.
What has been the worst officiated game thus far?
The league reviewed 34 calls from the final two minutes of Denver’s 118-111 double-overtime victory over New Orleans on March 15. There were 10 mistakes, the most in a single game of those reviewed. All 10 were missed calls: seven fouls, two traveling violations, and one defensive three-second violation.
Two other games of those reviewed went into double overtime and one went into triple overtime.
In the Pacers’ 99-95 double-OT win over the Wizards on April 14, the league concluded that officials missed six calls of the 42 reviewed. The Mavericks’ 144-143 double-OT nail-biter against the Nuggets on April 10 included two missed calls and one incorrect call out of the 22 reviewed.
The percentage of officiating mistakes in those two games was about average, and better than the 29.4 percent from the worst officiated game.
The triple-overtime battle, which the Nets won, 129-127, over the Bucks on March 20, featured the lowest percentage of officiating mistakes among the four double- and triple-overtime games. Forty-seven calls were examined and just five were ruled as missed, good for 10.6 percent of reviewed plays.
There was just one incorrect call in these four games. The rest were missed calls.
What plays are officials getting wrong most often?
Fouls of all types represent the majority of incorrect or missed calls, at 70.8 percent. More interesting is that traveling, a common complaint among some fans, is admittedly missed or called incorrect as much as any specific type of foul, accounting for 16 percent of officiating mistakes.
|Call type||Total incorrect||Missed calls||Incorrect calls|
|Foul: Loose Ball||40||37||3|
|Turnover: 5 Second Inbound||6||5||1|
|Foul: Defense 3 Second||4||4||0|
|Turnover: Out of Bounds||3||2||1|
|Turnover: 3 Second Violation||3||3||0|
Do officials make more decisions in favor of the home team?
The numbers do not support this notion. In the available data, 51.4 percent of all incorrect or missed calls favor the home team, rendering the perceived advantage virtually nonexistent.
The data represent only about 4 percent of total minutes in a game, from a sampling of games. If the NBA continues this initiative through next season, some clearer patterns may emerge, such as player or team biases.
Who does the reports?
It is a process of speed and accuracy, said Steven Angel, the NBA’s senior vice president of referee operations and analytics.
Each report is turned around by 5 p.m. the day after a game, but not before going through an extensive review by senior staffers.
A spotter watches the end of close games, noting all calls and looking for missed calls. That spotter then enters the data into a system that sends it to Bob Delaney, vice president of referee operations, and Don Vaden, vice president and director of officials.
“They provide their expertise on the accuracy on that first review and to make sure that we’re capturing not just the rating but also some explanation that makes some sense in the context of the play,” Angel said.
Delaney and Vaden make any edits in the system before the reports come across Angel’s desk. Once Angel signs off, the reports go to Mike Bantom, executive vice president of referee operations, for final approval.
“It’s a very top-heavy process, with a lot of senior people, because at the end of the day, it’s really a statement about how well we did,” Angel said.
What does a report look like?
The league uploads each report individually as a PDF. As a sample, here is what is included in the report from a Celtics game. Click the video links to watch snippets of each play:
|Period||Time||Call type||Committing player||Disadvantaged player||Review decision||Video|
Comment: Dedmon (ORL) sets the screen on Bradley (BOS) and gives him the opportunity to stop and/or change direction.
Comment: Pressey (BOS) makes contact with Payton's (ORL) arm and causes him to lose the ball.
Comment: Pressey (BOS) maintains legal guarding position and makes contact with the ball while defending Payton's (ORL) drive.
Comment: Vucevic (ORL) sets the screen on Bradley (BOS) without giving him the opportunity to stop and/or change direction.
Comment: Bass (BOS) cleanly blocks the layup attempt by Oladipo (ORL).
Comment: Harkless (ORL) makes contact with Bass' (BOS) arm as he dunks the ball.
Comment: Oladipo (ORL) and Harkless (ORL) trap Turner (BOS) and force the turnover.
Comment: Turner (BOS) cleanly strips the ball from Payton (ORL) on the play to the basket.
Comment: Crowder (BOS) makes contact with Payton's (ORL) arm on the play to the basket.
Comment: Gordon (ORL) wraps up Turner (BOS) following the inbound pass.
|Q4||00:25.1||Foul: Offensive||Nikola Vucevic||Avery Bradley||INC||Video|
Comment: Vucevic (ORL) holds Bradley (BOS) while setting the screen and prevents him from contesting the jump shot.
Comment: Harris (ORL) commits a take foul on Smart (BOS).
In the game, Orlando’s Nikola Vucevic actually got away with the same type of foul twice. The second time the referees missed the violation, Vucevic knocked Avery Bradley to the hardwood:
The two missed fouls on Vucevic were the only mistakes officials made in the final two minutes out of the 12 plays reviewed.
How have the referees responded to the reports?
At the end of the first round of the playoffs, the National Basketball Referees Association did its own analysis of the available data and provided an explanation in its own report, released May 6. The NBRA, which represents the league’s 63 active referees, found that its officials were correct 97 percent of the time when blowing the whistle, same as the Globe. That figure did not include plays that were later categorized as incorrect non-calls — violations missed by the referees — which brings the correct decision total to the Globe’s finding of 86 percent.
The report argues that because fouls are “subjective,” non-calls later ruled as missed calls don’t hold a lot of validity.
“Flow of game, how the possible contact influenced the play, speed of play and having just six eyes watching the greatest athletes in the world — all of these factors influence the referees’ decision to blow the whistle,” the report reads.
Lee Seham, the NBRA’s general counsel, said it is easy to second-guess.
“Officiating evaluators and even fans have many advantages over a referee,” Seham said in the report. “Refereeing fast-moving live action with just their eyes, knowledge and instinct — it’s easy to judge their profession, but a very select few of qualified individuals can actually do it, much less do it correctly 97 percent of the time.”
Delaney said moving toward a more data-driven review caused some nervousness from referees, but he has used his experience to help them understand the importance of the analytics as a tool.
“From my view, it’s invaluable,” he said.
As they have grown more comfortable with the process, Delaney said, they have been able to focus on the key questions.
“Why did we get it right or why did we get it wrong and how can we repeat positive actions and how can we tweak to make sure that we get it right the next time?” he said.
Is the league incorporating the data into referee performance reviews and development?
Yes. The league internally also logs all calls in every game, not just those at the end of close games.
Delaney, who was a referee for 25 years before becoming a supervisor in the executive ranks, said he remembers sliding ketchup bottles around tables with fellow referees after a game, trying to diagram different angles and brainstorming on how they could make better calls.
Not anymore. The league not only keeps logs and videos of each call, but it also tracks referee positioning, helping patterns emerge for individual referees. Delaney equated positioning on the court to how a passenger in a car views the speedometer differently from the driver.
“We’ve all done that, where we’ve sat in the passenger seat, we look over and you’re going 90 m.p.h. when really the person’s really only going 75 or 70,” he said.
Angel said the combination of the data, call logs, and video creates a “very powerful tool” when helping referees dissect their performances.
“Now [Delaney] also has data to know whether or not that individual referee just simply missed that play — and that will happen from time to time — or whether or not that’s part of a bigger picture or bigger pattern of performance that that referee is exhibiting and they’ll ask Bob to then diagnose with the referees,” Angel said.
Delaney said the process not only acts as a teaching tool, it also provides positive reinforcement.
“I’m able to sit down with an official and underline to them that what they do as an NBA referee, the larger percentage of times they’re getting things right,” he said.
“We’re going in such a way that we’re improving officiating by leaps and bounds.”
See our methodology and data here .