Augmented Reality and Avatar (Part One)

As anyone plugged into the matrix knows, James Cameron's "Avatar" is an event film because of the boundary-pushing technology that powers its spectacle.

— By | January 6, 2010

James Cameron‘s Avatar has by now very likely recouped and surpassed the exorbitant amount of money with which it was made, in everything from shooting and production costs, to mass-scale distribution and marketing. By some reports, the film was made for at least $300 million; others place it (including the distribution and marketing costs) above $450 million. Either way, Avatar is the most expensive film ever made, and it is on track to becoming the most lucrative film in history as well. These facts, however, are not enough to justify any considerable excitement over the film’s value; other less sophisticated movies (such as Pirates of the Caribbean or Spider-Man) have accomplished as much in their time. But Avatar has become a must-see event for other stranger reasons.

Avatar PosterAs anyone plugged into the matrix will know, Avatar is an event film for reason of the boundary-pushing technology that powers its spectacle. The technology I refer to is not so much the 3D aspect (the use of which has existed since the 1950s, though perhaps never as spectacularly, or as integrated into the bone of the narrative, as it is used in Cameron’s film); rather, it is the innovative virtual camera system that Cameron developed for the manipulation of live-action within a hybrid CGI world which has dramatically upped the standards for film and camera technologies.  Virtual camera technology is one unknown to most filmmakers because it has been mainly used to design video game environments in popular games like the Resident Evil series or the immersion-heavy Bioshock. Resident Evil is one such game that, to my mind, was greatly responsible for demolishing the old and ordinary standards of horror cinema. I can think of no horror film this past decade which consistently delivered such purely cinematic feats of terror as those that arise in the Resident Evil games. To the cineaste, such a statement will undoubtedly arouse indignation: video game art should not seriously be compared to film art, should it?

Fighting zombies in Resident Evil 5

Scene from Resident Evil 5

Yet ironically, Resident Evil plays out a consummately pastiche experience that summarizes more than 50 years worth of zombie and monster films in its absurd unfolding. The original game is remarkable for its selection of finely placed shots, lighting and sound effects, which create mood and terror (much like in a film) as you navigate the character along shadowy hallways and past ominous creaking doors. The ‘director’ of the game is not any one person but an army of designers who utilize an intelligently scripted camera system that switches and selects its shots as you proceed in the game. While the game is on the surface played by you, the player, the computer or AI-engine that creates the environment you interact with (much like how a film environment is taken, if momentarily, to be a real one in which you are a mere spectator of some one else’s vision) is at bottom the controlling agent responsible for the shaping of a sensory experience. The video game player will often find herself prodded through a world that is the fabrication of a different kind of demiurge, the thinking, dreaming AI computer.

Scene from Resident Evil 4

Scene from Resident Evil 4

If Resident Evil is a prime example of a video game property that was influenced by the movie culture of the past century, then it was also an exemplar of how video games have surpassed films in graphic technology, aesthetic design, and (in some countries) overall sales. The inferiority of the Resident Evil movies to their video game counterparts is evidence enough that popular films have for some time been unable to compete with the high-quality graphics and immersive experience that video games guarantee their players (so long as they willingly participate of course). But Avatar is significant for this reason: it is an instance in which film art has finally caught up with video game technology, indeed surpassing it, in producing an environment of what Jean Baudrillard and Umberto Eco call “hyperreality.” The effects on display in Avatar are such that no computer game can boast to replicate (at least not for any time soon).

Na'vi tracking the IkranJames Cameron’s synthesis of video game design with live-action and performance capture is one that invites metaphoric speculation. For one, there is the use of special stereoscopic cameras designed by Cameron and his digital crew to simulate human sight. The action on display in Avatar is never blurred or made indefinite by the 3D visualization: the action, whether involving live or computer-generated characters, is augmented by the 3D camera scope, so as to impress on the viewer the sensation that the CGI world is at one with the physical movements of the characters as well as with your own inert but conscious body. The use of the 3D stereoscope, much like in the mechanics of the human eye that distinguish distances between the foreground arising from the background in a continuous spatial plane, creates a metaphysical sense of the hyperreal, or the more-real-than-real. In Umberto Eco’s words (speaking of holography), “It isn’t cinema, but rather a kind of virtual object in three dimensions that exists even where you don’t see it, and if you move you can see it there, too.” Avatar‘s world continues to exist even when we don’t see it, after we’ve left the theater: it is a world that, for the average media-glutted consumer, diminishes the sense of wonder we have for our own. The 3D which we were forced to undergo for nearly 3 hours in the movie theater suddenly becomes noticeable again when we walk outside, astonished to remember that human sight was already set to three dimensions.


Is it a game or a movie?

Avatar, thus, is a video-game-movie that can not strictly be considered ‘cinema,’ but is still cinema in a hybrid sense. The hybridity is one that compounds the absolutely fake (the cinematic world in Avatar, symbolically named “Pandora”) with the totally real (in the film’s scenario, the use of real human actors alongside their CGI “avatars”; or in our case, the presentiment – or even the strong desire – that a magical moon-planet like Pandora can virtually exist in a way similar to our own Earth). For something to be more ‘real’ than real, it must be inverse, that is, perfectly fake. Eco explains that “to speak of things that one wants to connote as real, these things must seem real. The ‘completely real’ becomes identified with the ‘completely fake.’ Absolute unreality is offered as real presence.”



12 Responses to Augmented Reality and Avatar (Part One)

  1. Johannes Climacus on January 6, 2010 at 4:37 pm

    This is an excellent meditation on a pop culture phenomenon. Looking forward to part 2.

  2. Avatar and the Hyperreal : clusterflock on January 6, 2010 at 5:09 pm

    [...] story may be terrible, but the technology behind it is not: As anyone plugged into the matrix will know, Avatar is an event film for reason of the [...]

  3. [...] to Part One Written by Jose-Luis Moctezuma on January 6th, 2010 | Category: Movies | Leave a [...]

  4. Hamza van Boom on January 6, 2010 at 9:45 pm

    I never thought I’d see Umberto Eco and Resident Evil married in the same column. This is a very well written, insightful piece on video and film epistemology. I can’t wait for part 2.

  5. Franz Forrester on January 7, 2010 at 5:59 am

    Avatar is an extraordinary use, an astronomical mass, of artistic talent.
    It is also good business.

    Beyond the insightful articles about its technological groundbreaking, I was disappointed that Holywood-cheese philosophy remains completely in tact throughout.

    The film is grotesquely pantheistic. Odd spiritual sorties are in keeping with a pagan, magical world of make-
    believe. There is no wisdom in Pandora’s box, no rational context to the existence of the blue monkeys beyond keeping a big tree alive. Where modern man has erred in excesses and abuses of his natural resources, we are presented with a thesis which in many ways eradicates Man altogether and delivers him to a ‘better’ more perfect existence (which is utter nonsense as no intelligent man should swap ‘himself’ in favour of becoming a blue monkey.

    I’ll keep this one short!
    But I enjoyed the flick regardless as the artistry that went into making it is palpable.

  6. Vallie Frohling on January 7, 2010 at 9:00 pm

    Hes really driving through Thiago. Reminds of something a hs football coach would say to drive through the opponent

  7. Parker Kuckens on January 8, 2010 at 10:40 pm

    Man i’m so stoked about Gomi, he is one of my favs of all time

  8. praxhe to porite on January 27, 2010 at 2:54 pm

    Wharf this evening I watched the sun
    serpent blonde on the sea

    and the boat-sloughing shadows
    together or against each other and the way

    two guys went with a girl, which recurred
    every time we tried too

    to feed love’s lymphatics
    the spray of the sea; ocean violence

    in our silent hearts and contesting and young
    we met competing for a girl

    until we saw in each other
    the same envious narcissisms and saw

    eventually we’d do better as friends,
    winding in and through failure

    because nobody really cares
    when with a friend– those days sprawl

    to a summer whenever and moreso as suns
    burden early on the wharf.

  9. Jose-Luis Moctezuma on January 30, 2010 at 12:01 am

    Ah, that familiar summer letter from Praxhe. Still have a copy too. Eventually we did better as friends because that is what friends do… Want to thank Søren for his climactic statement… Mr. Van Boom, terrific thinker and writer, happier that you read it and enjoyed it… and Mr. Franz Forrester, penetrating comments. Last but not least, I appreciate the random football comments. How’d that happen?

  10. Horace Miller on January 30, 2010 at 12:27 pm

    methinks they not football comments but comments on muai thai fighters… how got here, rose only knows

  11. jon hung on March 8, 2010 at 2:21 am

    Its interesting that you consider watching the movie an Augmented experience. In a film context there’s no reality, just an experience mediated by the filmmaker. There were, however, examples of AR within the film – HUDs in the army’s machines, various other technologies, even Jake Sully’s control over his Navi body might be considered AR.

    The best talk on augmented reality I’ve seen: Kevin Chang of Twitter, comic behind OK/Cancel.

    What is AR, what is not AR, some cool examples of where it is going.

    The definition of AR is certainly expanding, but I’m not sure where film fits in all of it. Nice article, though ;)

  12. Jose-Luis Moctezuma on March 12, 2010 at 10:49 pm

    Hi Jon. Thanks for the link to Kevin Chang’s discussion of augmented reality. I updated a link on part two of this post with it.

    Indeed, according to the definition established by Kevin Chang, Avatar, or film as such, does not categorically fit the “augmented reality” brand. But I hadn’t heard of AR before reading about the strong marketing use of AR for Avatar-related products. It struck me while watching the movie that AR, at least theoretically, plays a major role in the conception of the Avatar storyline: you pointed out that onscreen there’s a lot of AR going on.

    AR hasn’t reached the level it is prognosticated to be at in scenes taken from Spielberg’s “Minority Report,” where Tom Cruise is accosted by holograms and 3D advertisements without the use of a worn device; so far, our only experience of AR is possible through a computer, television, camera, or theater screen or webcam. In this respect, Avatar fits the AR brand since we’re being fed a product (the Avatar movie concept itself) through the conceptual manifestation — in place of the actual use — of mixed reality (human life interacting with CGI, onscreen and off).

    I am inclined to believe that AR would not have achieved its current relevance if not for the metaphorical presence of film, which historically has paved the way, beyond the passage of photographic art, for our perception of reality to be augmented by a directorial manipulation of its primordial elements. The soul of cinema lies in the edit, the cut, and even when a scene does not cut, it is this intentionality that allows what is being seen to continue and endure — to remain a human decision. Film is a mixed composite of what we perceive to be scenes from the “real” (with human actors and sometimes, especially in contemporary world films, real geographic settings) overlaid with directorial decisions. These decisions are more often than not hidden perceptions of an inquisitive or allusive nature: watching film, as in reading its text, in search of what the author intends to do or is in fact performing.

    Thanks to computer technology, the hidden intentionality that traditionally has made an art of photography and film is now made naked and visible: AR is aesthetic or utilitarian intention made visible. (AR is something like footnotes suddenly rising up and draping the text of a book with highlighted background and foreground minutiae, somehow simplifying the text by making it less allusive, yet more dense with annotation.) I would call this gross visibility or demystification of cinema, if you allow my theoretical posturing, to be the conceptual birth of AR and what it will bring to us.

    All of this is just smoke and theory; at bottom, AR introduced to me a novel platform for analyzing Avatar without having to be snide or negative about its bluster and fatuity. Avatar may not be a great film, but it certainly is symptomatic of what mainstream filmgoers are coming to expect from cinema at the behest of our technologies. AR then would have much to do with film because it is indirectly modifying our expectations of what cinema can do. As you noted, AR’s gone a long way from just being a Boeing project that makes it “easier to assemble large bundles of electric wire for aircraft on the factory floor.” Utility has upgraded to metaphor.

Leave a Reply

Your email address will not be published. Required fields are marked *