“The easiest way to think about it is – data visualization with sound instead of visual images, basically trying to convey information via sound,” said Michael Corey, senior data editor at Reveal. I spoke with Corey about sonification storytelling and he shared insights into how we can be producing musical journalism in our newsrooms.

Michael Corey leads a team of data journalists at Reveal who turn datasets into compelling stories that can easily understood by readers. Learn more about him and his team here. (Photo courtesy of Michael Corey)

How did you get interested in trying sonification?

One of our challenges has been how to do data on the radio in a compelling way. The story I was working on, earthquakes in Oklahoma, the radio producer asked if there was some way we could illustrate this phenomenon with sound. He had seen it done through radiolab and there’s actually a reference to data sonification in Samuel Pepys diary going back to the 1600s, so we certainly did not come up with the idea of sonification.

Have you seen other interesting examples of sonification in journalism?

Not that many newsrooms are doing it but obviously newsrooms have done it before us. I’ve been really happy that people are using the software we wrote to do our earthquake sonification. Other examples of sonification projects –  Inside Energy did a data sonfication on coal production that was really cool, Radiolab did one on the color vision of animals and WNYC created the Harlem Heat Project.

Tell me a little bit about your Oklahoma shakes project

We wanted to illustrate a pretty simple phenomenon which was that there were more earthquakes in Oklahoma than there were in California, by quite a lot. Not just that but how that didn’t used to be the case. Oklahoma used to have almost no earthquakes and was now leading the nation. We could have just said that in text but it felt like there was a more emotional way to say that with sound.

What was the process for the project?

The first stage was “What would that sound like? What are we actually trying to express?” What we decided upon was basically: there used to be no earthquakes, now there’s a ton of earthquakes and it happened very quickly. So we did a time series sonification showing the changes in earthquakes over time by playing a note every time an earthquake happened in a compressed space of time, we moved through 5 years of data in a minute.

So you guys built your own software to do this project, is that because of the lack of software for this or because you wanted the challenge of building it yourself?

I really think if there is a product out there that does what you want, you should use that. I started out thinking I’d use some available software, but I found some had fallen into disrepair, the code didn’t work anymore or they weren’t written in a language I understood.

What it came down to was that because we are radio show and our engineers are really high quality professional composers, I wanted to think through a way to get this to them so they can take advantage of all their tools.

What is your advice for newsrooms that want to try it?

Start from what you want your end product to be. Think through what you want the music to sound like before you even pick up any code. I think if you can’t come up with a compelling idea for a song, then don’t dive into it.

So you thought of the song first, not the data first? The process isn’t “I want to see what this data sounds like?”

Exactly. It’s analogous to how we do data journalism generally. It’s not like we just say – we’ll throw this data around and see what comes out. It’s I have a question and I want to know what the answer to this question is. It’s musically, what am I trying to accomplish here? Is there a pattern I expect to see here? And sometimes you don’t know exactly what it’s going to be, but you should have a good idea of what success will sound like.

So for example with the earthquakes, we could visualize in our heads that there used to be none and now there are a lot – so as you go through time it would get louder and louder and louder. We could visualize it in our heads, so it made sense to do it.

You shouldn’t start from “I want to do some sonification” it should be that there is music in your head that you want to get out.

If you were to take it a step further, is there something else you’d like to do with sonification?

My big idea that I’d like someone to do, hopefully me but could be someone else, is data data revolution where you can dance the data in an in-person performance kind of experience. A lot of my ideas are that I really want to bring data into meetspace, into the real world. I’d also like to try a surround sound environment where the data is geographical. I think it’d be interesting to see if it’s easy for an ear to discern information based on geographical location. I’d also love to try a live performance of data sonification. I think you could create, similar to a church bell choir, where a light would flash to tell the person to ring the bell. You could train people on the fly to perform music that way. I have tons of ideas I’d love to try.

Those sound really interesting, especially the dancing, what would be the challenges?

I think the challenge is, what is the dance? Dance is a pattern that repeats. So what’s the pattern? There is data that is much more cyclical, such as I think you could probably use climate change data. It would mostly be cyclical but something may go higher or faster with time. I think that’s one of the most interesting challenges, when you think about how is music different from the data we interact with. Most data we interact with is linear, but music really lends itself to cyclical data so there’s a lot of interesting ways to play with that.

Do you think sonification will become more common and more accessible as a way for newsrooms to tell stories?

I think it is something you should only use when it is adding something. It is good for when you have something emotional you want to convey about the data, with a really simple concept you want to illustrate. I don’t think sonification is going anywhere, it won’t be the next twitter, but it’s a good tool to have in your arsenal.

Is there anything else you can think of that you’d like to share with newsrooms who might want to try this?

Nothing else, but hit me up! I’d love to help and I’d love to see more people doing this kind of work.

——

Interested in trying sonification? Take a look at Reveal’s MIDI tool to turn your stories into sound.

Editor’s note: This interview has been edited for length and clarity.