Exploring the relationship between data sonification and information design

Journalism and information design: A fast friendship

In this age of flourishing digital media, the work of journalists has become deeply intertwined with information design. Long gone are the days of stories characterized only by text on a page. Consumers of media now absorb news content through social media, interactive digital articles, podcasts, and fast-cut videos. Because of this evolution, journalists have had to consider the broader design of their stories. Publications have begun to diversify their teams, bringing on UI experts and programmers to craft multimodal journalistic content. Just look at digital publications like The Pudding, who create interactive audio-visual stories driven by data (and some particularly unique audio stories).

In previous articles, I’ve discussed why journalists should take up data sonification as a creative storytelling method. But embedded in the practice of sonification is its design-oriented nature, which journalists will inevitably navigate when creating a sonic work. 

A journalist must make a series of design choices to solve the problem of creatively conveying some message or piece of information. They must consider (1) their own perspective in crafting a story, (2) the experience of the user or audience, and (3) the [potentially fallible] truths embedded in the data. 

The aesthetic and contextual framing of their creative output can play an especially influential role in whether a listener finds a story to be salient.

Why is design-conscious sonification important?

Researchers Sara Lenzi and Paolo Ciuccarelli (co-founders of the Data Sonification Archive) have delved into the synergy between design and data sonification. They point out that design-conscious sonification can be incredibly impactful, especially for communicating social issues. They speak to journalism in particular as a field in which the sonic communication of sensitive societal topics can have broad reach. In their 2020 paper on this topic, they state: “In data-intensive societies, with the complexity of social issues increasingly evident and inevitable and the visual channel often overstimulated, sonification can expand the domain of action…” 

They argue that well-designed sonification can improve the communication of social issues to a broad audience in two significant ways. 

• First, it can accompany data visualizations and unveil certain invisible dimensions of the data. 
• Second, it can awaken engagement of the public through a “captivating and compelling experience” (2).

This research illuminates how ominous or even unpleasant sounds can be used strategically to grab the attention of a listener and solidify a solemn message — for example, the danger of poor public policy, the threat of gun violence, or the acceleration of climate change. 

Pitching sonification to information designers

I recently had the opportunity to give presentations about data sonification at two design conferences. The first was the AIGA Design Conference, an annual gathering of designers connected to AIGA, the American Institute of Graphic Arts. 

This year, the conference took place virtually, and the theme was “Margins: exploring the uncharted territories of design.” Embracing this theme in my presentation, I encouraged the idea that well-crafted sonification can extend the perceived boundaries of information design. I emphasized that audio can be a powerful tool for communicators through the avenues of emotional impact and intentional chronology in a sonic story. 

I uploaded the recorded presentation to YouTube to make it accessible for anyone interested in watching:

I also created a follow-up resource document containing links to data sonification cited, resources and guides, sonification tools, and sonification communities. 

The second conference where I gave a similar presentation about data sonification in the context of information design was the Cumulus Design Conference, hosted in-person by the Cumulus Association in Monterrey, Mexico. I had the wonderful opportunity to interact with product designers, user experience designers, architects, writers, artists, and educators. I enjoyed sharing the concept of data sonification with them and hearing their varied but overall enthusiastic reactions. 

These presentations were aimed at communicators and designers, but that group also includes journalists, who hold an important place in the landscape of public communication. Furthermore, journalists have a responsibility to consider design as they experiment with sonification and craft data-driven stories through audio. 

Hastily-generated sonifications may land flat — instead, listeners are more likely to get on board with sonification when it contains captivating elements of context, musicality, and dynamics. With intention, a sonification creator can ensure that a message is memorably and artfully conveyed.

Tool of the month

This month’s featured sonification tool is csv-to-midi, a simple no-code web application created by Evan King that allows a user to upload a CSV file, adjust audio parameters, and export a MIDI file. MIDI stands for “musical instrument digital interface” — essentially, this format instructs a digital music program how to play a set of musical notes by timing and pitch. The great thing about a MIDI file is that you can drop it right into a digital audio workstation (DAW) and adjust the selected software instrument. You can make your notes sound like a piano, a synth, a saxophone, or anything you can imagine!

This method has a catch – it is not a standalone tool. If you use this approach, you will need to take a second step of importing your MIDI file into an audio editing program. By doing this, you will be able to transform the MIDI values into audible notes which can be exported as audio for your project. 

Digital audio workstations and editors for you to consider:

• Garageband (free with Mac)
• Logic Pro X (Mac, $)
• Audacity (free, open-source)
• Ableton (Windows or Mac, $)
• Soundation (free, online editor)

Try it out! Here is a data set that you can use — female representation in the Olympics during the last century (from the 2024 IOC Factsheet). Upload this CSV file to the csv-to-midi web application. When the parameters appear, try adjusting them to your liking.

I used Logic Pro X to convert the MIDI file to audio, but you can use a program of your choice. See some options listed above for reference. After some experimenting with the sound, you can export your sonification in your desired audio format.

For inspiration, it’s worth checking out Evan King’s sonification of sea level data in a project called “Bait/Switch.” It is a creative ambient composition transforming sea level data during the year 1967 near Okinawa, Japan into a digital underwater soundscape. 

Example of the month

The featured example this month is from BBC Sounds, called “The sound of a falling currency” (World at One, July 8th, 2016). It conveys the decrease in value of the British Pound (as compared with the U.S. Dollar) leading up to Brexit, when the UK withdrew from the EU. One can hear a resounding bell with each passing day, with a higher pitch indicating a stronger currency value. In the run-up to the vote, the pitch increases, and immediately following the vote, there is a distinct drop in pitch.

This sonification is a good example of how to guide the listener with narration — the sonification plays while the reporter explains the timeline. It even includes excerpts of news segments and the voices of government officials with contrasting reactions to the change. This sonification briefly and effectively reveals the economic impact of this historic shift.

Contribute your thoughts

As always, I appreciate reading your thoughts about data sonification. Additionally, as I develop a sonification toolkit for journalists, I have been brainstorming about its content. I’ve created a brainstorming site that lays out the high-level content of the toolkit. It is simply a way to explore what chapters the toolkit could contain, and to get feedback about the structure. 

If you have any questions or thoughts about my fellowship project, feel free to send me an email at aurawalmer@gmail.com. Thank you! 

---

Cite this article

Walmer, Aura (2024, Nov. 4). Why journalists should care about design-conscious sonification. Reynolds Journalism Institute. Retrieved from: https://rjionline.org/news/why-journalists-should-care-about-design-conscious-sonification/