Ideas

Brainiac

Making movies, video games sound as good as they look

Wikimedia Commons

Earlier this month I wrote a story for Ideas on an equation that describes how people move in crowds. Afterward I received an e-mail that said, in effect, if you’re interested in things that are hard to model, look at how sound waves bounce around a room.

That e-mail came from Dinesh Manocha, a computer scientist at the University of North Carolina who works on “three-dimensional audio” — creating technology that produces audio for video games, movies, and virtual reality environments that mimics the way we hear real noises in the real world. It’s an improvement that Manocha says has been a long time coming.

Advertisement

“Visual effects in movies and games are stunning, but audio is about 20-30 years behind,” he says, noting that unlike visual effects, which can be computer-generated, audio effects are still physically recorded.

There are a few reasons for the lag. One is simply that we care about visual effects more — we’re want to see a dazzling high-definition movie, but don’t mind listening to it on tinny earbuds. A second is that real-to-life audio is harder to generate. Standing in the middle of a concert hall, sound waves strike our left and right ears differently and what we end up hearing depends on subtleties like the shape of our head and the exact direction the sound waves are traveling when they reach us. Plus, there are all sorts of environmental effects that change the way a note from a trumpet moves — or propagates — through space, long before we register it as music.

Get Arguable with Jeff Jacoby in your inbox:
Our conservative columnist offers a weekly take on everything from politics to pet peeves.
Thank you for signing up! Sign up for more newsletters here

“Audio sounds in movies and games are pre-recorded with simple manipulations like increasing the bass, but the echoes and diffractions are not right,” Manocha says. “We need a simple, automatic technology that overcomes that barrier for propagation.”

To do this, Manocha’s lab, which he runs with his colleague Ming Lin, is creating software that uses the wave equation from physics to model how sounds move in different environments. The software takes into account complexities like the materials the sound waves interact with (a shiny window reflects more sound than a rough carpet), the rate at which a sound-emitting object is moving relative to the person who hears it (the Doppler effect), and differences between how sound strikes your (virtual) left and right ears. These are relatively simple calculations to perform in isolation from each other, and given a lot of time, but they’re extremely difficult to perform at the real-time pace of a video game.

“Physics tells us how we do it fast in huge environments like a city,” Manocha says.

Advertisement

Two of Manocha’s former students have started a company, Impulsonic, that provides three-dimensional audio technology to video game designers and architects (who might want to test the sound properties of building features). There’s still a lot of work to do, though, before simulated environments sound as good as they look. Researchers like Manocha are working on expanding the database of synthetic noises — the raw material behind sound propagation in virtual environments — and they have their eyes, or ears, set on one group of sounds in particular.

“Many sound effects in nature we can’t simulate,” Manocha says. “We want to figure out how these ambient sounds are generated.”

Kevin Hartnett is a writer in South Carolina. He can be reached at kshartnett18@gmail.com.
Loading comments...
Real journalists. Real journalism. Subscribe to The Boston Globe today.
You're reading  1 of 5 free articles.
Get UNLIMITED access for only 99¢ per week Subscribe Now >
You're reading1 of 5 free articles.Keep scrolling to see more articles recomended for you Subscribe now
We hope you've enjoyed your 5 free articles.
Continue reading by subscribing to Globe.com for just 99¢.
 Already a member? Log in Home
Subscriber Log In

We hope you've enjoyed your 5 free articles'

Stay informed with unlimited access to Boston’s trusted news source.

  • High-quality journalism from the region’s largest newsroom
  • Convenient access across all of your devices
  • Today’s Headlines daily newsletter
  • Subscriber-only access to exclusive offers, events, contests, eBooks, and more
  • Less than 25¢ a week
Marketing image of BostonGlobe.com
Marketing image of BostonGlobe.com
Already a subscriber?
Your city. Your stories. Your Globe.
Yours FREE for two weeks.
Enjoy free unlimited access to Globe.com for the next two weeks.
Limited time only - No credit card required!
BostonGlobe.com complimentary digital access has been provided to you, without a subscription, for free starting today and ending in 14 days. After the free trial period, your free BostonGlobe.com digital access will stop immediately unless you sign up for BostonGlobe.com digital subscription. Current print and digital subscribers are not eligible for the free trial.
Thanks & Welcome to Globe.com
You now have unlimited access for the next two weeks.
BostonGlobe.com complimentary digital access has been provided to you, without a subscription, for free starting today and ending in 14 days. After the free trial period, your free BostonGlobe.com digital access will stop immediately unless you sign up for BostonGlobe.com digital subscription. Current print and digital subscribers are not eligible for the free trial.