Today’s cameras can shoot excellent video when there’s plenty of light and the camera isn’t moving. But try it at midnight from a moving car, and things can get ugly. That’s a massive problem for engineers trying to build things like self-driving vehicles capable of seeing pedestrians and road signs even when it’s dark.
Sebastian Bauer and Tristan Swedish think they’ve got the answer. Their Boston-based company Ubicept uses a radically different kind of digital camera that can shoot razor-sharp images under the most challenging conditions, and can even see around corners.
“We think that this technology is going to replace a large part of conventional imaging,” said Bauer, who earned an electrical engineering doctorate at Germany’s Karlsruhe Institute of Technology. “It’s a paradigm change.”
Cofounder Swedish, who earned a doctorate from the Massachusetts Institute of Technology’s Media Lab, said that the Ubicept system is especially good for shooting sharp images of fast-moving objects, even in low light.
Ubicept is based on technology developed at MIT and the University of Wisconsin, where Bauer did postdoctoral research. The company is based at The Engine, an MIT-affiliated incubator for companies tackling unusually tough technical challenges; part of the $8 million it raised in venture funding last summer came from MIT’s E14 Fund and the Wisconsin Alumni Research Foundation.
The key advance: Ubicept abandons the chip technology called CMOS, or complementary metal-oxide semiconductor, that’s used in nearly all digital cameras, in favor of a newer kind of sensor.
A CMOS camera sensor is covered with thousands of pixels which are assigned an electrical voltage depending on light intensity. Another chip converts these analog voltage levels to digital signals which are processed to create the image. But the conversion generates digital noise that degrades image quality.
In addition, CMOS chips use electronic shutters that shoot images one frame at a time. If the shutter speed is relatively slow, moving objects look blurred. Speeding up the shutter to prevent blurring means that less light hits the sensor. This can make the image too dark, especially at night.
Bauer and Swedish chose single photon avalanche detection or SPAD sensors, the kind used in lidar systems. Lidar, which stands for light detection and ranging, uses rapid pulses of light to accurately measure distances; the technology is found in many late-model iPhones.
SPAD sensors don’t need shutters and don’t measure the brightness of light. Instead, each pixel notes whether it’s been struck by an individual photon of light. The system can then calculate the time between the arrival of various photons to estimate the distance to a target.
Bauer and Swedish use the same principle to generate ultra-sharp pictures. As photons enter the camera and hit the sensor, software calculates their arrival time and uses the data to map the shapes of objects and display them on a screen. By measuring one photon at a time, the Ubicept camera can generate clear images even in very low light conditions, and even when the camera is moving. And it’s an all-digital process, assuring higher image quality.
In a demo video, the company showed images shot at night from a moving car by a standard camera and an Ubicept camera. The Ubicept camera’s image was far brighter, revealing trees and other objects that were invisible to the standard camera. Passing road signs were merely a blur to the regular camera; in the Ubicept camera, the signs were clear enough to read.
Another startling demo showed a person standing behind an obstacle that hid him from the camera’s field of view. The camera detected him anyway, because light reflecting from his body hit objects that the camera could see. These photons bounced off the objects and entered the camera’s lens. By measuring the photons’ time of arrival, the camera generated an image of the person’s body.
Bauer said this feature has so far only been tested in the lab. But it someday could enable a self-driving car, say, to hit the brakes because it’s detected a child lurking between two parked vehicles — a danger that would be invisible to a human driver.
Ubicept won’t have the SPAD camera market to itself. Earlier this year, Japanese photo company Canon released a $25,000 SPAD-based camera designed for high-security applications, such as guarding military facilities. Canon says the camera’s SPAD sensor can shoot color video in near-total darkness.
Steve Hoffenberg, a technology industry analyst at VDC Research in Southborough, said it’ll be a long time before SPAD-based cameras will be cost-competitive with today’s CMOS devices, although it’s not impossible.
“CMOS sensors used to be really expensive too,” Hoffenberg said. But he added that SPAD systems already make sense for industrial applications, like inspecting products for defects on an assembly line or monitoring machines while they’re running. “This super-high-speed capture capability could have lots of use cases,” he said.
Ubicept’s founders agree. They’re targeting industrial automation systems, as well as the automobile market. Even if self-driving cars are a long way off, there’ll still be demand for a night-vision system that could warn drivers of danger just around the corner.