Amazon.com Inc.’s analysis of COVID-19 infection rates among its workers has several flaws and falls short of assessing whether the world’s biggest online retailer did a good job protecting its workforce through the pandemic, according to infectious disease experts who track pandemics.
Last week, Amazon said that almost 20,000 of its US workers had tested positive for the coronavirus during a six-and-a-half-month period. Amazon, one of only a few companies to provide such data, said the infection rate in its ranks was lower than that of most states, a finding it cited as evidence that investments in sanitation, temperature checks and protective equipment were keeping workers safe.
But three experts interviewed by Bloomberg said the data was unhelpful because it failed to reveal whether the infection rate was improving or growing worse. One said Amazon’s comparison of its workforce to the general population is fundamentally flawed. So while the announcement may have helped assuage some critics who say Amazon hasn’t done enough to protect workers toiling through a pandemic, it was essentially useless for employees trying to assess whether it’s safe to show up for work, they said.
“It looks like someone just put a bunch of numbers together, and what we can determine from those numbers, I just don’t know,” said Preeti Malani, an infectious disease expert, physician and chief health officer at the University of Michigan. “I have no idea what to make of them. At best, it’s incomplete.”
Amazon, through a spokeswoman, declined to reveal further details about its analysis. Amazon acknowledged the difficulty in assessing the effectiveness of workplace safety measures in a blog post on Oct. 1.
”This information would be more powerful if there were similar data from other major employers to compare it to,” Amazon said in the post. “Wide availability of data would allow us to benchmark our progress and share best practices across businesses and industries. Unfortunately, there are no standards for reporting or sharing this data, and there’s very little comparable information about infection rates and quarantine rates available from other companies.”
Amazon announced that 19,816 of its US employees had tested positive for COVID-19 or were presumed positive between March 1 and Sept. 19. Beyond the total number of cases nationally, Amazon released an analysis comparing the infection rate of its workforce in 40 states and Washington with the infection rate of the overall population in those areas.
The employee infection rate the company provided was below that of the overall population in all places studied except Minnesota, New Hampshire, and West Virginia. In Kansas, Amazon’s employee infection rate was the same as the overall population. Amazon excluded 10 states where it employs fewer than 1,000 people.
That Amazon compared its worker population to the general population at all reveals a lack of understanding of epidemiology, said Irva Hertz-Picciotto, an environmental epidemiologist and professor at the University of California, Davis.
The comparison ignores the “healthy worker effect,” a bias that occurs when you compare a cohort of working people with another group with a mix of employed and unemployed people. Statistical groups that exclude the unemployed tend to be more healthy than ones that include jobless people, she said, which is why scientists typically avoid such comparisons.
Amazon also provided data on the “expected rate of infection per 1,000 workers” and then said its actual rate was lower. Hertz-Picciotto called the expected rates calculation ”completely suspicious” because it exceeded the actual state rate in every state analyzed.
“I don’t think this analysis could make it into a scientific journal without a lot of further explanation,” she said. “It would just fall apart.”
Amazon disclosed data aggregated over a period of months without disclosing any trends about when the number of cases reported each day or week went up or down, which makes the information useless from a public health standpoint, said Ron Fricker, a statistics professor at Virginia Tech and co-author of “Monitoring the Health of Populations by Tracking Disease Outbreaks: Saving Humanity from the Next Plague.”
Amazon also tallied worker data at the state level, which could obscure outbreaks isolated to some facilities, just the way the coronavirus can rip through different parts of a state at different times, he said. Amazon’s aggregated data contrasts with more meaningful metrics regularly released by public health agencies that focus on new cases reported daily and averages over one- and two-week periods that indicate whether the number of new cases is increasing or decreasing.
“There’s not enough detail to say if Amazon is doing well or not. It’s just too aggregated,” Fricker said. “Tracking COVID is a week-by-week thing. You can have a big outbreak and not see that in six months of data or you could be getting better and not see that either. If I’m working in one of those facilities, I want to know if things are getting better. Based on this information, I don’t know.”
Amazon counted in its workforce anyone who had worked for the company at any time during the period studied. That approach could inflate the overall number of workers used to calculate the infection rate, making the infection rate lower.