Document Sample
peck Powered By Docstoc
					Beyond the Library: Applying film post-production techniques
to game sound design
Author: Nick Peck

It has been my experience that game companies often rely on commerical CD sound effects
libraries for the majority of their raw sound material. While these libraries are very useful, there
are other methods of collecting sound material that can achieve excellent creative results. The
film industry often uses custom field and foley recording to give each project a personal and
unique flavor, augmenting their needs with CD libraries where appropriate. In this session,
techniques of field and foley recording will be discussed, using examples and parallels
between the film Being John Malkovich and the game Escape from Monkey Island. Audio
portions of the game will be broken down track by track, showing how the dialog, music, hard
sfx, foley, and ambient layers combine to create a unified sound experience.

Let’s whet our appetite by hearing some great film sound. Here are snippets from three very
different films with excellent sound: Apocalypse Now, The Exorcist, and Castaway.

Our challenge is to meet this level. Games don’t sound like that. Even taking into account the
differences in the medium, games often don’t sound as good as they could. Why not? Four
reasons immediately come to mind: time, money, communication and delay across the team,
and accepted work techniques.

How can we improve the situation?

Not having enough time or money are always a huge problem. The solution is to budget more
of both for sound! Sound Designers are often limited by having poor, outdated equipment, not
enough off-the-shelf sound libraries, but most importantly, not enough time to go out and get
new, original sounds for the game project. Remember: SOUND IS ART. To make a game
sound artful, let the sound designers have the time and money to practice their art!

The next problem is poor communication with the rest of the team, and delays in production
propogating to further diminish the amount of time to develop sound for the game. We can say
that effluent follows the laws of gravity in changing state from higher potential energy to lower
potential energy. The translation will be left to the reader, but the point is that sound is a post-
production process. We are at the end of the line, where everyone is out of money, out of time,
and out of patience. This is true in film as well. By the time sound is done, the programmers
are burned out, the deadlines are absurdly close, and it is very hard to get the sound wired up
with the level of detail you’d like. You can address this somewhat by clearly communicating
your needs early on. If all else fails, become a programmer.

The final problem limiting sound production in games that I’d like to touch on is accepted work
techniques. It seems that pulling most or all raw sound materials from commercial SFX
libraries is often the primary approach. It is a model that is well-understood, and easy to
implement. While libraries are hugely useful, though, they do limit your creativity, and give you
the same raw sound as everyone else. It is true that the crafty sound designer will take these
materials as a starting point and manipulate them, often to the point of unrecognizability, but
there are still only so many wind recordings in the Sound Ideas 6000 series library, and most
every game company owns that library and uses it.

To some degree, these problems will always be there. Bringing awareness of sound needs to
the people that pay the bills is not always easy. But there are ways that we can improve game
audio incrementally: by bringing more film post-production techniques into game audio.

Why apply film techniques to games? Simply put, the movie industry has been around for a
long time. Film sound designers have honed their craft and figured out what works. They know
how to make films sound unique and interesting. As game sound designers, we can steal their
ideas to make games sound unique and interesting too.

Film sound is broken into a series of layers: dialog, music, hard SFX, foley, and ambience.
Let’s examine these each rather briefly, looking at their relation to the greater whole.

Dialog comes first. Always remember that dialog is king. It must be intelligible above all else, or
your story is lost. In game audio, this usually means that the dialog is compressed and limited
severely, to make sure it reads above the music and SFX within the limited dynamic range we
have to work with. Film dialog is not compressed as much, because the sound is carefully
massaged at the mixing stage to make sure of intelligibility.

The music sets the emotional context of the project. It tells the player what to feel, whether a
moment is placid or tense, majestic or scary. Music and sound effects share the same space,
and work together in it (or not). Both film and games have the same problem of these elements
competing with each other. The best compromise is to try to make both audible. This can be a
tight-rope act, particularly in interactive settings. I have found that having greater dynamic
range, particularly in the music, allows the elements to rise and fall in audibility, poking through
each other when appropriate.

Hard SFX are the meat and potatoes of game sound. These are spells, weapon hits, engine
loops, door slams, and all other foreground sound material. The sonic character of the game is
most strongly defined by the choices the sound designer makes in creating the hard SFX.

Foley is sound made by humans: footsteps, clothing rustles, the manipulation of props and
tools. In film, foley is recorded to picture to cover the movements the characters made on the
screen that could not be picked up by production microphones, due to the noise on the set.
Games are primarily animated, so of course there is no production recording to be used, and
all movements are recorded after the fact. Or more often, not recorded at all. Foley is the
sound layer that brings subtle realism to film. We can bring it to games as well.

Practically by definition, foley work involves recording custom material every time. There are
limited footsteps and clothing rustles available in some SFX libraries, that are often used by
game sound designers to fill in some movement. In my experience, foley recording sounds
better than using canned material, and is cheaper than editing it as well. It actually takes less
time to walk footsteps against picture than to edit library footsteps against it. Of course, foley
recording does require a foley pit with multiple surfaces for different types of footsteps, as well
as a good-sized prop collection and an extremely quiet recording environment. But in my
opinion, any game with a reasonably sized budget should do at least some custom foley

The last layer of sound is ambience. Ambience is the background recording of a particular
place that identifies it aurally. Swamp ambiences are filled with birds and frogs, beach
ambiences have the endless rumble of waves crashing on the shore, cave ambiences might
have a slow, reverberant dripping of water, restaurant ambiences might have muffled
conversation and the clatter of silverware on plates, and factory ambiences would have low
rumbling and the clatter of huge machines in the background. If music sets the mood,
ambience brings the location to life.

Ambiences have two components: The ambient loop, which is a long, streaming, stereo
recording that can be mixed with the music track, and specifics, which are separate, short
elements (bird chirps, foghorns, etc) that trigger randomly to break up repetition.

The way to bring convincing ambiences into your game is through field recording. Portable
DAT machines and stereo microphones expand your horizons to the end of the Earth. Field
recording is great fun, and rewards you with original material that has never been used in a
game before. As an added bonus, it is a great way to get out of the office for a while. I went on
vacation to Canada right before beginning production on Escape from Monkey Island. I took
the opportunity to record every type of water setting I could, from every distance and at
different times of day: waterfalls, beaches, gentle harbors, and streams. All of this material
made it into Monkey, and the result is a rich aural environment.

I’d like to show an example of how these elements fit together in film by using a scene I sound
designed for the film Being John Malkovich. In this scene, Malkovich enters his own mind and
ends up in a restaurant, where everyone he sees is a version of himself. I’ll show how foley,
hard sfx, ambience, music, and dialog fit together to create a complete picture.

Games have two different types of segments: Linear segments, often called animations or
cutscenes, and interactive segments. Each of these can benefit from a filmic approach.

Linear segments are short, animated movies with no interactive elements. The approach is
clear: Make sure to create all the layers of sound described above to create a rich experience.

I’d like to show the GMRR (Giant Monkey Robot) cutscene from Escape from Monkey Island
as a case study. Just as in the restaurant scene from Being John Malkovich, I will play back
the scene several times, soloing the ambient, foley, hard sfx, and music tracks separately. I will
then play back a mix, to show how all the elements fit together.

Interactive segments are, of course, the part of the game where the player is actually making
things happen. Events are not completely predictable, and take place as a result of the player’s
decisions. Clearly, the interactive portions of games are very different animals than linear film.
But there are still concepts to steal that can improve these segments.

Filmic improvements to interactive game sound would include: Filling the environment with
ambient loops and specifics, minimizing repetition by having as many alternate SFX as they
will let you (especially for footsteps and weapon hits), and letting the soundtrack have some
dynamics. Don’t compress/limit the life out of everything! Finally, to repeat my main theme, do
as much custom recording as possible! Make new footsteps, grunts, hits, weapons fires, UI
clicks, and anything else you can. It will sound different than other games, and will be a lot
more fun as well.

As a case study in interactive segments, I’d like to look at the sushi boat puzzle from Escape
from Monkey Island. This complex puzzle involves a lot of careful timing and thinking outside
the box by the player. There are various mechanisms at work, each with their own sounds. By
carefully working with these different sounds, turning them on and off and changing their
volume relative to decisions the player makes, the audio soundtrack enhances the puzzle
logic, and gives the player clues to help solve the puzzle.

To put my ideas in a nutshell: We can improve the sound of games by borrowing techniques
that have been widely used in film production for a long time, and adapting them to our needs.
I suggest making liberal use of foley, field recording of custom ambiences, and recording as
many hard SFX from scratch in the studio as possible, rather than relying on sound effect
libraries. Be careful not to over-compress the audio, and strive for the best mix of dialog,
music, and effects possible. The results will be game soundtracks that are more unique,
interesting, and beautiful to listen to.

Nick Peck
Perceptive Sound Design
37 Matilda Ave, Mill Valley, CA 94941
Tel/Fax: 415-388-2628
Nick Peck owns and operates Perceptive Sound Design, a firm specializing in audio post-
production for the game and film industries. His sound design projects have included such
games as Escape from Monkey Island, Vampire the Masquerade: Redemption, Grim
Fandango, Star Wars Super Bombad Racing, and New Legends, as well as the films Being
John Malkovich and the remake of Vampire Hunter D. Peck is also a composer and
keyboardist, holding an MFA in Electronic Music from Mills College. He has released six
albums, ranging from avant garde electronic music to progressive rock, and performs
frequently with his quintetTen Ton Chicken. In March, 2000, Peck completed construction of a
new post production recording facility in Mill Valley, California. Featuring a foley pit, voiceover
booth, grand piano, 5.1 surround sound, 2 Pro Tools systems, high quality microphones,
synthesizers, and recording gear, excellent acoustics, and a 200 gigabyte online sound effects

Shared By: