To continue getting breaking news and the full stories from The Boston Globe, subscribe today.

The Boston Globe

Ideas

When computers listen to music, what do they hear?

A new generation of scholars is turning music into data—and uncovering truths beyond human ears.

Soon after the release of the first iPhone five years ago, an astonishing new ritual began to be performed in cafes and restaurants across the country. It centered on an app called Shazam. When the phone was held up to a radio, Shazam would almost instantly identify whatever song happened to be on, causing any iPhone skeptics in the vicinity to gulp in bewilderment and awe.

There was something unspeakably impressive about a machine that could listen to a snippet of a random hit from 1981, pick out its melody and beat, and somehow cross-reference them against a database that seemed to contain the totality of all recorded music. Seeing it happen for the first time was revelatory. By translating a song into a string of numbers, and identifying what made it different from every other song ever written, Shazam forced us to confront the fact that a computer could hear and process music in a way that we humans simply can’t.

Comments

Shazam can recognize a snippet of music and identify the name and artist of the song if there is a match in it's database. What's astonishing is that it can do this and it's database is so large, not that it's something humans can't do, with the exception of the database. Recognizing a song is a child's play game for a human. "I can tell you the name of that song in one note". Shazam can't recognize a melody if you sang it or played it on an instrument., while if someone who knows the song hears the melody they can tell you it's name whatever else they know about it, even if sung in a different key. People can transcribe songs and hear melodies and harmonies beyond the ability of any computer if they're trained with the musical skills to do that kind of analysis. So you could say that a human can recognize musical information in an INFINITE variety of ways that a computer can't do. What data driven analysis gives is a list of data that has little aesthetic meaning to the development of music, and what a song means to any particular individual. Can the data account for why one person likes a piece of music and another person detests the same music? No. Can the data justify a subjective response? Probably never.