"Revisiting Sound in Video Games: The Road to Total Cinema?" By Adam Slight

On March 28th of this year I presented an essay at the Carleton University Film Studies Undergraduate Colloquium. Allow me to briefly summarize this essay before I continue:

As technology develops, video games have become more and more cinematic (film-like) to the point where games are almost films that one controls. Arguably, there is little visual difference between a film and modern video game. However as far as sound design goes, I argued that video games have surpassed film.

Using mostly examples from the Nintendo Wii, I demonstrated how the interactivity of the video game medium has allowed greater freedom in sound design experimentation than film. Legend of Zelda: Twilight Princess has put the player in control of how the game score plays out depending on how the player performs (slow walking=subtle music, horse riding=epic music, monster fighting=dramatic music; all within the same scoring sequence) and Super Mario Galaxy has demonstrated how music, sound effect and player action can interact to create unique sonic experiences.

I also demonstrated how video games such as Medal of Honor can present realistic, interactive sonic environments. In games such as these, sound is put into the perspective of the character/player immersing the player in a life-like environment. This is reminiscent of André Bazin’s theory of Total Cinema. In this theory, Bazin argues that conventional cinema is only one step in a historical progression in which mankind attempts to build a life-like, all-encompassing recreation of reality (think The Matrix). Video games such as Medal of Honor represent the next step in this progression towards artificial reality.

Lastly, I mentioned how games such as Guitar Hero and Rock Band put the player in control of the video game’s soundtrack. The delivery of the soundtrack depends of the success of the player in playing the video game. I also briefly showcased the Nintendo DS game Elektroplankton. This game was designed with few goals beyond those of free-play of sound experimentation and manipulation.

During the question period proceeding the presentation I was made aware of several interesting arguments against some of my claims. Most arguments were directed towards my use of the word “interactivity” and addressed the actual extent to which interactivity can actually apply to video games. I was told to read Jesper Juul’s Half-Real: Video Games Between Real Rules and Fictional Worlds which discussing many founding theoretical ideas surrounding the emerging field of video game theory as well as game theory in general. I have since read this book and would like to defend and redefine my claim.

As the title of Juul’s book implies, video games can be seen as a system of real rules (in that they actually apply) as well as fictional worlds (the fictional elements that disguise the rules). The real rules come to define the fiction of the game, and the fiction defines the rules. This was brought to my attention during the question period. How can one truly interact with a progressive game such Super Mario Galaxy when the choices and options that the game presents are already pre-designed. When one plays Super Mario Galaxy they are playing within the confines of the rules. As far as sound design goes, the sounds within the game are pre-recorded and pre-set and are emitted when triggered within the game realm. Thus games are not truly interactive, nor can they offer realistic decisions (as action options are severely limited).

I would like to think otherwise. We live within a system of rules. That is the rules of physics define what we can and cannot do. To expand on this, the rules of physics also define the sonic environments that we live in. With this in mind it can be considered that the limited interactivity of video games merely represents simplified real-life physical rules.

The complexities of causal sound in reality could be applied to video games. Digital technology already allows us to convert real sound into digital representation. If real sound can be represented by digits then it is possible to create realistic sound from scratch in a digital realm. If every possible nuanced sound can be represented by digital code, then hypothetically it is possible to create a mathematical system in which different sound can be produced depending on digital variables. That is to say sound in video games can be unique depending on the in-game conditions they are triggered by. No samples would be used, only in-game conditions such as surface textures, environment acoustics, impact velocity and air density. These would all be conditions that would affect the artificial reproduction of the game sounds. This would be pure interactivity.

This could no doubt apply to video game visuals. Digital technology allows accurate visual representation in visual form. Who is to say with stronger graphics capacity a realistic interactive visual environment is not possible. This would no doubt bring us closer to the Total Cinema that Bazin forecasts. This idea of immersion in a video game is discussed in Janet Murray’s Hamlet on the Holodeck and is commonly referred to as virtual reality.

Juul addresses some pitfalls to this model of video game. He uses the example of entering a car in Grand Theft Auto III. In real life there are an infinite number of ways to enter a vehicle, however in GTAIII there is only one way: Press the triangle button on the game controller. If the game required a button combination for each task required to open a car door and enter the vehicle the game would be tedious and boring. Instead the fiction of the game is simplified and stylized for player enjoyment.

So while a Total Cinema approach to video games would be revolutionary, it is possible that it would ruin the game play of the game. So perhaps Total Cinema does not lie in the video game genre, but through the video game platform. Video games themselves should always remain as games, however the technology of video games may hold the key to a lifelike and interactive Total Cinema.