“Coldness is about more than just a sound and a look, and it’s more than the coldness of a technological being, too. Coldness is what we fear lies beyond human capability. Coldness is the gap between human intentions and outcomes. It’s the uncanny valley of the human reflected in the non-human.”–Adam Harper
In his marvelous essay “On The Digital Animation Of The Face” (from Uncanny Valley: Adventures In The Narrative, 2011), Lawrence Weschler writes about his experiences hanging out with the programmers and animators at Industrial Light & Magic and DreamWorks to learn about their work in digitally rendering the human form. One of their steepest challenges is how to realistically render faces. Faces and facial expressions are complicated to digitally model, in part because they’re complicated. For instance, we have many small muscles that are (unconsciously) deployed in thousands of ways in the service of even the simplest of expressions. These micro movements need to be noticed, “captured and programmed” (8) by animators. Also, there is the question of skin tone, texture, and its myriad hues, of “the subtlety and complexity of the way light radiates out from the inside” of a person’s face (5). In their complexities and subtleties, our faces seem to radiate our consciousness. And if the past failures of AI (artificial intelligence) are any indication, consciousness is tough–impossible?–to simulate.
Weschler also observes that we’re finicky when it comes to assessing the realism or fakeness of our digital models. In fact, we seem to have a tolerance threshold for the simulated real: when something seems realistic but is still slightly off, we get creeped out. Here, Weschler draws on The Buddha In The Robot (1981), a pioneering book by Japanese robot engineer Masahiro Mori. Mori coined the term “uncanny valley” to describe that small yet large gap between the simulated/machine entity and the real/living entity. Weschler sums up Mori’s concept:
“When a replicant’s almost completely human, the slightest variance, the 1 percent that’s not quite right, suddenly looms up enormously, rendering the entire effect somehow creepy and monstrously alien (no longer, that is, an incredible lifelike machine but rather a human being with something inexplicably wrong–part of Mori’s point being how incredibly finely attuned we humans are at perceiving those infinitesimally disquieting failings)” (15).
In sum, the purported goal of digital animators is to transcend the monstrously alien and move their work ever closer to looking and feeling real. And for that to happen, as Pixar founder Alvy Ray Smith notes, “we’ll only be able to get there using human actors, with all their idiosyncratic mannerisms and specificities, as our models” (17).
***
As I read Weschler’s essay I thought about what has been happening in electronic music since at least the early 1980s. For thirty years, electronic musicians have grappled with the question of how to make their music sound more realistic–how to make MIDI (musical instrument digital interface) sequencers, quantization, sampling, and other technologies work in a way that doesn’t sound artificial but as if a real person is behind them. Pick up a music magazine–then or now–and you’ll find story after story about techniques aimed towards these ends. Moreover, each generation of technology promises ever better quality, ever more realism, ever higher sampling bit rates, and sounds that feel less cold and more warm–like the real thing. The race for perfect simulation and modelling is ongoing, yet how close do we want to get? Is the goal to fool ourselves, to make the electronic sound acoustic?
I have written before on this blog about electronic music and the real. One post cites a passage by Jaron Lanier about the limitations of music post-MIDI. Another post describes some of my frustrations with the sounds of electronic music. In my own work, I had the experience over the past year of spending time listening to different sounds in my software and finding that the ones I liked best were those that fooled me into thinking that they were real instruments. But these sounds were few and far between. Tired of searching for glimpses of realism within my computer, I finally sampled one of my percussion instruments. And what do you know? I was surprised at how engaging its sound was, made of fundamental, overtones, and little imperfections. It sounded real because it is real. And so I began to work with it.
One of the animators at DreamWorks that Weschler spoke with, Lucia Modesto, opined that the quest for creating a believable digital human has essentially become a quest for quest’s sake. But for her, animation “ought to be about what you can’t get in reality” (13). Like say, creating Shrek. Maybe that’s the best case scenario for electronic music too.
brettworks 