Electronic Music and Gaming Theory

In this week’s New Yorker there is an article by  Nick Paumgarten on the Japanese video game designer Shigeru Miyamoto that unpacks the magic behind such Miyamoto game creations such as Super Mario Bros. and Legend Of Zelda.  Game designing is a creative endeavor that few people besides Miyamoto have mastered.  (Though the American Will Wright, designer of Sims and Spore, also comes to mind.). One key to designing a good game, notes Paumgarten, is to make sure it has a complexity and dynamic depth to it that is cognitively challenging yet also charming and fun to engage with.  Achieving this complexity and dynamism means designing the game around a few elements that can endlessly remix themselves in different combinations to keep things fresh.  Describing the source of Super Mario Bros.’s appeal Paumgarten writes:

“The game had just fifteen or twenty dynamics in it…yet they combined in such a way to produce a seemingly limitless array of experiences and moves, and to provide opportunities for an alternative, idiosyncratic style of play, which brings to mind nothing so much as chess” (92).

To me, there is a similarity between the experiences of playing and designing videogames and making electronic music with a computer.  Specifically, I am thinking of the way software such as Ableton Live (which I happen to really enjoy using–or should I say playing?) is configured.  For those readers who have never used it, this is what it looks like:

One of the software’s two viewing pages, Session View (what is shown in the pic above), is arranged like a mixer, with each sound given its own vertical track.  Within each track, one can stack discrete chunks of audio or midi called clips.  So for a single track of say, percussion, one can have a few dozen clips of different lengths.  Each clip can be looped, played back, and triggered in any order the musician wishes.  And that’s just one track; imagine the “limitless array of experiences and moves” available to a musician with a dozen tracks, each with two dozen clips.  That’s a lot of ways to combine sounds, and we haven’t even begun to consider effects processing (e.g. ways to alter, distort and enhance a sound such as distortion or reverb effects, etc.).

So, for electronic musicians who perform using a laptop running Ableton’s software, part of the pre-performing process is a little like Miyamtoto’s designing complexity and dynamism into his games. And the pleasure comes later when musicians get to (literally) play their music, improvising different combinations of sounds, and figuring out on the fly in what direction to head in.  Like the experience of interacting with a videogame, electronic music allows a musician to explore virtual worlds that strike a balance between adventure and play…

And speaking of play, Paumgarten also cites Johan Huizinga’s Homo Ludens (1938), a famous study of this essential human activity, in order to further unpack the joys of videogaming.  For Huizinga, play has five attributes:

1. it’s free,
2. it takes place outside the realm of everyday life,
3. it is, materially speaking, unproductive,
4. it follows an agreed upon protocol of constraints and rules,
5. its outcome is uncertain and therefore it encourages improvisation from its players

This sounds like an awfully apt characterization of music too, doesn’t it?

You Are The Controller

Last week Microsoft released the Kinect controller for their XBox video game console.  The Kinect is being hailed/hyped as the next step in gaming technology as it does away with the most annoying part of the gaming experience: those little handheld controllers that serve as an interface between the player and the game.  Nintendo’s Wii got us part of the way there with their handheld controllers that respond to body movement.  So what makes Kinect on another level?  It scans the player’s body movements in real-time, making the human body the controller.  No more wires, no more joysticks, no more buttons to press, nothing to hold.  In the words of the XBox commercial: “You don’t need to know anything you don’t already know.  Or do anything you don’t already do.  All you have to do is be you.  You are the controller.”

I imagine that the Kinect technology will have resonance for many electronic musicians because musical controllers have long been something that we need to address when composing and performing music.  Pick up a music store catalog and you’ll see lots of controllers for sale, each of them offering the musician the prospect of ever better “control” over their music.  Controllers are always aiming for the kind of almost perfect transparency demonstrated by an acoustic musician at his or her instrument–with maybe only a pair of drumsticks or a violin bow or a mouthpiece or set of piano keys as the “interface” between player and expression.  In my conversations with electronic musicians over the years, one recurring theme is the tantalizing prospect of having nothing come between them and their music.  It’s the dream of having one’s physical (and possibly mental) gestures directly translated into sound, a situation where, as Kinect puts it, you are the controller.