At hundreds of construction sites, a safety inspector named Vinnie is watching the workers, keeping tabs on those who show up without work gloves or use the wrong kind of ladder. Vinnie doesn’t miss much.
That’s because Vinnie is software — an all-seeing artificial intelligence system developed by a Cambridge startup, Smartvid.io. Chief executive Josh Kanner named the program after a man he knows, a New Jersey construction superintendent with a photographic memory. He sounds a lot like the digital Vinnie, which analyzes photographs at construction sites to spot hazardous conditions. And according to research conducted with the help of Boston’s Suffolk Construction — an investor in Smartvid.io — Vinnie can predict workplace accidents before they happen.
Like other image-recognition systems, Vinnie has been trained using real-world images. Contractors routinely shoot digital photos and videos at every stage of construction, as a way to verify the work has been properly done. Builders often use time-lapse cameras mounted in fixed positions that take photos at regular intervals. Safety inspectors and project managers carry cameras of their own, and some building projects are shot from the sky using remotely-controlled drones.
Suffolk and other big construction companies compile vast libraries of these photos and other valuable records. Until recently, said Suffolk’s chief data officer, Jit Kee Chin, builders rarely made use of the information.
“When a project closes, often people don’t go back and look at the data,” said Chin, who holds a doctorate in physics from the Massachusetts Institute of Technology. Indeed, a 2018 study from McKinsey & Co. found that construction companies have lagged other industries in analyzing their data files to improve efficiency and boost profits.
Chin wanted to change this, and chose workplace safety as her first priority. For one thing, it’s a critical problem: Nearly 1,000 construction workers died on the job in 2017, according to the US Department of Labor, and another 198,000 were injured. Reducing that toll would benefit both workers and contractors. And, because these injuries are closely tracked by the federal government, there’s plenty of data to study.
Smartvid.io obtained millions of worksite photographs from Suffolk and other sources. The images were tagged by human specialists to identify possible safety hazards, and then used to teach Vinnie what to look for, such as missing gloves or hard hats, or a pile of debris that could cause a slip-and-fall accident.
“This is not a ‘gotcha’ type alerting system,” said Kanner “It is more subtle and powerful than that. It is a system that is gathering data to better predict risk in coming days and weeks.”
Vinnie isn’t designed to single out individual workers and punish them for safety violations; indeed, the company explicitly warns customers not to use it that way. Instead, the software system issues regular reports on the number of safety lapses it detects. Contractors can improve their scores through better training of workers and closer monitoring by supervisors.
Vinnie proved its worth earlier this year, by reducing injuries at a Suffolk construction site.
“After two hand lacerations on a project site, we used Vinnie to provide a tagged series of pictures observing workers on the project site not wearing their safety gloves,” Chin said.
Suffolk declared a “safety stand-down” at the project to train workers about the importance of wearing gloves. That happened five months ago, and there have been no more hand injuries, Chin said.
Kanner and Chin also believed Vinnie could foresee workplace accidents through “predictive analytics,” a way of studying huge databases of past events to predict the future.
If you use the e-mail service Gmail, you’ve seen a simple example of predictive analytics. Notice how the software sometimes tries to finish the sentence you’ve begun typing. Google’s study of billions of e-mail messages lets it guess what you probably mean to say. It’s not perfect, but it’s often right.
Kanner and Chin fed Vinnie a decade’s worth of data from Suffolk’s files, including every on-the-job accident, or “incidents as Suffolk calls them, and 750,000 photographs. Vinnie was also taught to factor in data that might seem irrelevant, such as weather conditions, or the type of building under construction. Those factors can make a big difference. Building a skyscraper poses different risks from erecting a warehouse. As for weather, rain and snow raise the risk of slip-and-fall accidents, while hot temperatures may tempt workers to strip off some of their safety gear.
Once the model was constructed, Kanner and Chin tested it against three years of construction project data that Vinnie had never seen before, including projects where workers had been injured For 20 percent of the actual accidents, Vinnie correctly predicted that somebody was about to get hurt.
That might sound like a poor success rate, but according to Suffolk and Smartvid.io, each prevented accident would save a contractor an average of $36,000. That’s a lot, considering that Suffolk may have 100 to 150 projects running at any given time.
Vinnie has captured the attention of major contractors nationwide. About 40 companies now use the service, including Boston’s Shawmut Design and Construction, a nationwide contractor with 2018 revenue of $1.4 billion. Shawmut began with a test of Vinnie in June 2018.
“That pilot quickly escalated from three projects to 50,” said Shaun Carvalho, Shawmut’s vice president of safety. “Today we have just about 200 active projects under construction” that are using the software.
Smartvid.io has begun developing customized accident-prediction services for specific clients and is developing a standard version of the product that can be used at any construction site.
Suffolk, Shawmut, and nine other construction companies, along with the property insurer Aon, have formed a consortium to promote the use of AI-based safety systems. Each will share 10 years of data to build better predictive models, which the companies can then use.
With more data to work with, Kanner said Vinnie might be able to make more precise predictions — such as what kind of accident is about to happen.
This story has been updated to reflect the correct spelling of the Smartvid.io chief executive.