How Many Words Is A Sound Recording Worth?

“Seeing is believing, but hearing is hearsay” — Julian Henriques, Sonic Bodies (2011)

Like a lot of people, I’m a fan of Instamatic, a smartphone photo app, because it makes me feel like a skilled photographer. The app is essentially photo editing software that allows you to quickly–really quickly, with the tap of a virtual button, actually–apply a range of filters to your photo to make it look sharper and just plain cooler. Cooler because the filters make your photo look old, moody, vintage, over-saturated, analog old-school, or as Leonard Koren might describe the worn aesthetic–wabi-sabi (you can read more about this Japanese concept here). Instamatic is also a social app, allowing you to share pics with other Instamatic users. So I imagine there are billions of Instamatic photos floating out in the ether, capturing and constructing cool wherever we users happen to point our camera-phones. It feels that easy to use: point, shoot, add filter. Here is one of my pics:

You get the idea.

Thinking about it, it strikes me that Instamatic encourages people to think photographically–to be on their toes, ready for cool sights to capture and render. This is a good thing in that the app pushes us to be visual ethnographers, documenting the passing, strange, and wonderful of our everyday lives. When we take pictures, we see the world differently for a few moments. We think about how we can “frame” what we observe around us, and how we can compress the seen into a visual document for posterity or for our friends. A lot is captured in a photo too. Consider the adage “a picture is worth a thousand words” which refers to the idea that a complex idea can be conveyed through a single image. If this isn’t literally true, then pictures at least provide the illusion of capturing ideas because in pictures we notice so many details, contrasts, and relationships.

But here are my questions: How many words is a sound worth? And if there were an Instagram-like app for capturing, filtering, and sharing sound (calling all app developers out there!), would have it have the same emotional resonance as Instagram?

***

I decided to experiment with taking a field recording I made on my iPhone and running it through some effects processing to try to create some different aural moods. The recording is of a man playing the accordian on a subway platform at Grand Central Station in New York City. After listening to him for a while, I donated some money and he turned to face me so I could take a photo. I liked the music he was playing, and liked it even more as I fiddled around with adding effects to it.

Here is my original field recording:

Here is the clip with delay added so that it sounds like the instrument is bouncing off walls, all dubby and mysterious:

Here is the clip with static added so that it sounds somewhat like an old recording:

Here is the clip with chorusing and flanging added:

Here is the clip with distortion added:

Here is the clip with reverb added so that it sounds like it’s in a cave or large cathedral:

Here is the clip with a glitch-y effect added:

You get the idea. (My favorite, by the way, is the clip with delay added.)

Now one question is whether or not the application of these audio effects affects the mood of the original recording. The answer, I think, is yes, but exactly how the effects alter the mood is somewhat hard to describe. A second and even more fundamental question: Do the audio clips convey as much as the photo of the accordian player conveys? I think so, though the things they tell us are of a different order. Given that I have included a photo of the musician above, we can directly compare the different kinds of information the visual and aural media bring to our awareness. Looking at his photo, you might notice how he’s dressed (casually in jeans, and with a baseball cap), the expression on his face as he plays (pretty neutral, though he was smiling for my picture), or even the heft of his instrument–it almost looks like it’s a struggle just to get those accordian bellows to wheeze into life. My photo captures these bits of information, but it leaves other things out. By contrast, the audio recording isn’t static like the photo is. Instead, it captures the accordian player’s performance over time (about a minute and half of time, in fact). This allows us to experience the non-musical ambient sounds of the subway too–for example, the sounds of other people, and the sounds of spare change landing in the musician’s open instrument case. Without anything to see, we can only focus on the audible information, but we’re granted the luxury of letting that information reveal itself over a duration. Finally, what’s most striking to me when comparing the photograph of the accordian player with a recording of his music is that the music seems more melancholy and expressive than its maker actually looks. The music seems to tell a story that we might miss if we were only looking.