Munchausen by Proxy

This man exhausts me. I see him nearly every week. His thoughts are like the ping pong balls that bounce around in a Lotto jar,  fast, erratic.  It goes on an on. The number never coming up. The balls forever jumping.

I find him first sitting in a chair hunched over in the hallway, a wadded up brown paper towel clutched in his hands. He has been crying, angry, bitter tears. The world is against him. People hate him. I know this from prior conversations. But, he looks different. Defeated? This is someone who faked a seizure in front of me. With him, the truth and a lie are a hairsbreadth apart.

I ask him what is wrong. I ask him would he like to see me. He jumps up saying, “yes,” and follows me back to the clinic. His speech is even faster, more pressured than usual. His mind is jumping from a height, a ball rebounding –boing  boing, splattering me with minute details of perceived injustices. My own mind follows one then another then collapses. “Stop, I say, I need to bring you back to here and now.” He quiets for a moment, then is off again, like a thoroughbred at the gate.

The officer comes and stands patiently just outside the opening to the concrete cubicle. “Do you want him to see mental health?” she asks. I realize that it is Friday and that mental health is still here since it is not yet the weekend. “Yes, of course, ” I say. First, I must get him to take his medication.He is on treatment for Hepatitis C. I lead his thoughts to the here and now,  a mother taking hold of her child’s hand.  He swallows the pill dutifully even as he is telling me he wants to die–he spits it out  like a child screaming that he will show everyone at his party–he will eat the cake but he will not enjoy it.

The officer leads him off. A few hours later he is back. I suspected he would be. They have done nothing for him. They hate him. They are fucking idiots. And, so he has decided to take sixteen naprosyn and  ten capsules of gabapentin, an antiepileptic used for neuropathy. He does not seem worried and I realize that it is possible he is making this up. I check the pharmacy records. I calculate that if he saved all his meds from a week ago, what he says might be true. I ask him if this was a plan, he admits, no, it was impulsive. He asks me what will happen to him.  I call poison control and urgent care. The former advises I send him to the hospital, the doctor at the urgent care guffaws, “get a fingerstick and an EKG– he doesn’t need to go out.” I think this is true. I suspect he hasn’t taken much. The scratches on his neck and abdomen are superficial, too. Thin red lines that stand out dramatically against his sallow white skin.  He broke up a radio to do it. No, it was not sharp enough to do him in, but yes, he wanted to do himself in. I ask him at one point if he has ever had a pet– the point I am trying to make is about concrete reality, about the effort required to really care for another creature or oneself. It is not an abstraction, it is not some distant philosophy. It is the concreteness of food and water and cleaning out the kitty litter box. This slows him. But, in the end, he lives epic narratives and can not escape. Still, his thoughts rarely go to the egotistically grandiose–the music career, the movie career he was just this close to becoming a star but his mother that fucking bitch who stole his money and thinks she is perfect she thinks she is perfect and she thinks I am trash but that is what it was like she always wanted to hurt me as a child they would hurt me it was Munchausen’s syndrome!

I am trying to read the mental health clinician’s note and listen to him at the same time. Perfect attention is a waste. It only adds energy to his ranting. My gaze is  fuel for the fire. I stare at the computer screen. “It’s Munchausen’s by proxy,” I say, not expecting him to stop.  “What?” he says.  I look up. “It’s Muchausen’s if you do it to yourself. It’s Munchausen’s by proxy if someone else does it to you.” This revelation stuns him. He does not talk. He is processing something. Something has changed. He says, “I have to write that down, I don’t like seeming stupid in front of people.” He searches for a non existent pen, dropping the pile of papers of different colors and sizes and states of folding and crumpledness that he holds in his lap, then forgets. And, we are off to the races again.

He signs the refusal to drink activated charcoal. On the paper I note that by not doing what is recommended, he risks “injury, disability, and death” That seems to cover it. We are making deals. He does not want to go on suicide watch. That will only provoke him. He wants to go to C71 to see the psychiatrist. I sigh. They will not take him. I know this. He is clearly suffering, but also insufferable. He is a great sucking mouth that belches insults. He will be right back here. I tell him I will try to reach the psychiatrist. I tell him in my experience it is tough to do at 11 at night. I do try. Now, he asks for Klonopin. I tell him, no, I cant do it since he says he took ten gabapentin which can cause central nervous system depression. This logic makes sense to him, he drops the subject. As he waits for the fingerstick and EKG, he begins insulting a C.O. I walk out and tells him. “Sh. Just be quiet. Just for a little. Just for tonight. You can do this. You don’t not have to be mad, just put it off until morning.”

On the way out, he is grateful. We look at each other for a moment, maybe five seconds, which in his world is a long time. I want to say I feel for you, you are so incredibly fucked up. But, I don’t. This is not a movie though, and certainly to look at it,  nothing has changed, but maybe in some small way, it has.

Before, he leaves, he stops by my cubicle. He has no thought that maybe he has spent too much time, or that I have other things to do. He is forever a neglected child. There will never be enough time, love, reassurance, and he will try to eat the world, shoving it like that birthday cake he never got. He will vomit it and feel guilty and then angry for being made to feel guilty and the whole thing will start again.

He has a final request, of course, he does. There is always another and another. Another thought, another word, another solution.  He asks me  to write down the Munchausen’s “thing”.  I scribble it on a piece of computer paper. He looks at it quietly as if trying to memorize it. Then satisfied, he adds it to his pile.

Atomic Vacation update

Really paring down and directing opening scenes. It becomes a form of multimedia poetry. The user opens a drawer and finds these “found” images: a picture of fake Chanel pink gloves and a picture I found on Pinterist of someone taking a picture of a Juliana Spahr love poem with their cell phone. I have never met her, but love her poetry and emailed her to see if she’d agree to let me use it. She loved the idea.

As is our style we use a lot of circulating and historical media and what I call an “ecstatic excess” of data to tell stories. Here, the data the user gets is randomized so noone will get the same story. The other options when you open the drawer are a hybrid computer poem that I composed from appropriated text and my own monologues put into a computer story algorithm and reedited. Shizuku the robot girl is perhaps in love with her teacher Rae, but what does that mean? That she is organizing her thoughts around her? That she “feels” something? That is up for the user to interpret. Tomi’s Depthkit scenes are so present and human and weird. Maybe what a robot girl would remember/fantasize about?

“when computers begin to think, we begin to choose, how could you not have understood that?”

Thank you TFI and MacArthur Foundation for awarding us a TAA Interactive Prototype Fund grant! Queerskins is coming to VR!

 

Queerskins: a love story is coming to VR.  Really pleased and still amazed to announce that Cyril Tsiboulski and I have been awarded a grant by Tribeca Film Institute and John D. and Catherine T. MacArthur Foundation as part of their nonfiction Tribeca All Access Interactive Prototype Fund to create a VR experience inspired by our online interactive work Queerskins (www.queerskins.com). The awesome Oscar Raby and Katy Morrison (VRTOV Studio) will act as executive producers. You can like and follow our progress at https://www.facebook.com/queerskins/

 

Depthkit Shoot

A few Saturdays ago, this happened. So many things could have gone wrong. This part of Atomic Vacation really pushed us to our full creative capacities. Japanese-American actress Tomi Heady did an incredible job with a really challenging script. Tomi plays Rae–the scientist/linguist charged with teaching Shizuku, the robot girl how to “be” human. After Shizuku learns of the nuclear apocalypse on Earth, she cuts off all communication, this causes an Easter Egg program to run in her–a selfie video farewell from Rae. Shizuku whose memories of Rae have been erased by her handlers tries to reconstruct her memories of her. The result is the odd fantasy/memory of a little robot girl from the future. The text for the monologues is mostly a hybrid of my own writing and appropriated text of philosophy, poetry, and science, put into a story algorithm and re-edited. In the birthday scene, Tomi speaks a piece from Gertrude Stein’s “Making of Americans,” making Stein perhaps the first poet whose work appears in VR?

Supreet Mahanti and Alex Wagner,  Depthkit expert and DP+ respectively, were incredibly professional and enthusiastic. We are now working on post-production of the scenes, but we have them. And, I can definitely say you haven’t seen anything like this–poignant and really disturbing posthuman future.

Beautiful Stills from Atomic Vacation

Amazing demo from Cyril Tsiboulski.  To see teaser go to our Atomic Vacation dev page on FB.

There is a point when you have spent years trying to research and make an artwork that you see what it will be. Before then, in some ways, it didn’t really exist. It is like some space alien dropped this thing, like the monolith in 2001 A Space Odyssey, in the middle of your bedroom. and your job is to figure out how to understand it and how to use it. Cyril Tsiboulski and my Atomic Vacation reached this point today. I’m near tears and it’s not just 3 hours sleep. It is a beautiful thing. Here are two teasers from in headset.

So much of this data is archival. For us it is important to ground the virtual and the future in the flow of history. We think this is the basis for human ethical action in the world. The voice you hear on the radio is archival: Oppenheimer interviewed by Edward R. Murrow. The music in the second is the exquisite piano composition “Po” based on Cantor sets by Brazilian composer Liduino Pitombeira, played by Luciana Noda who have graciously allowed us to use this music. The ending –stepping out the door into deep space will not be allowed until your 15 minutes of attempting to “prove” you are human is up. Until then you are locked in this perverse domestic space with ever accumulating amounts of data. (every object you pick up, toss, shoot, etc.) causes more data to be revealed and these, too, can be moved and become “matter” and structural elements in the virtual space. At the end, the door will be a release. You can see what we have lost. Oh, and eventually the gun will shoot rainbow teardrops. I want the whole damn room filled with rainbow teardrops. #thefutureisqueer #tech #gamedev @globalzero

Queerskins For Oculus Touch

queerskins-screen-shot-copy

You know how much I want to make this. Revisited the script to rewrite for
 Touch
and it is so beautiful. Love in all its forms.

Queerskins: a love story exploits the unique properties of VR, providing users with unprecedented opportunities to play with subject position, identity, and negotiate the interfaces between real/embodied and the virtual/imaginary spaces. On an abstract level, this piece encourages users to experience and contemplate the loss of the body and the dangers and pleasures of unmediated sensuality in our increasingly virtual and technologically interfaced world. We employ several  devices to create a dynamic between“ real” existence and virtual, imagined or remembered life and between embodied existence and the human desire for transcendence whether this be through faith, love, sex, or storytelling itself.  Most importantly, the combination of Oculus Touch and Depthkit create a formal device by which the desire for touch and human connection is both stimulated and frustrated. This is in keeping with the uneasy position of touch/contact in the age of AIDS. If you are interested in supporting this project, please connect via our FB page https://www.facebook.com/queerskins/


Storytelling at the End of the World: Cinema and Narrativity in Virtual Reality

zabriskie

Really pleased that my paper will be published in Transatlantica Journal, a biannual publication of the French Association for American Studies. This is based on a talk I gave in Paris last March.  This is useable under Creative Commons,  attribution, please. #VR #cinema #apocalypse #epistemology #narrative

The word “apocalypse” comes from the Greek apokaluptein meaning to uncover or reveal. In keeping with the word’s origins, I approach the concept of apocalypse not as the destruction of the material world, per se, but, epistemologically, as the end of the world as we have known it. If not the primary agent of this epistemic cataclysm, computer technology, which increasingly mediates human perception of reality, certainly reflects these changes. As might be expected, these technologies have also dramatically changed how we communicate about the world. In his essay, “Database as a Symbolic Form, ” new media theorist Lev Manovich describes this as a move from “narrative” to “database” storytelling.

As a cultural form, database represents the world as a list of items and it refuses to order this list. In contrast, a narrative creates a cause-and-effect trajectory of seemingly unordered items (events). Therefore, database and narrative are natural enemies. Competing for the same territory of human culture, each claims an exclusive right to make meaning out of the world (Manovich, 1999 “Database as a Symbolic Form”).

Manovich concludes his essay by praising the Soviet film director, Dziga Vertov whose Man With a Movie Camera, he argues, successfully creates a new kind of narrative by ordering a database of images around the “kino-eye” of the filmmaker.

This process of discovery is film’s main narrative and it is told through a catalog of discoveries being made. Thus, in the hands of Vertov, a database, this normally static and “objective” form, becomes dynamic and subjective. More importantly, Vertov is able to achieve something which new media designers still have to learn — how to merge database and narrative merge into a new form (Manovich, 1999, “Database as a Symbolic Form”).

Whereas, even today, both the novel and cinema remain mostly on the side of conventional narrative, many new media forms resist the linear arrangement of information. Of these, virtual reality (VR) most directly and completely mediates visual sense perception. In the VR headset, the world disappears, replaced by an alternative reality generated by a computer. Here, the user is no longer a passive consumer of someone else’s database. By exploring and interacting with the environment, she orients herself according to her own interests. One might even go so far as to say that she becomes a kind of film director framing close-ups and long-shots.

Philosopher Vilém Flusser has characterized the time we live in as an epistemic crisis brought about by the recognition that, “scientific research is not the gesture of a transcendent intellect.”  What Flusser calls “the gesture of searching,” for meaning or knowledge, has shifted from “a digging down for reasons” to an examination of the aesthetic and relational qualities of things and an exploration of how humans respond and attend to their environments as subjects and as objects (Flusser, 2014, “The Gesture of Searching”). Perhaps unsurprisingly, Flusser’s description of this gesture serves as an apt description of being in virtual reality, a reality which has become, in essence, a database.

…the researcher is embedded in an environment that interests (matters to) him, both at close range and at a distance. There are aspects of the environment that interest him intensely and others that hardly touch him. The more an aspect of the environment interests the researcher, the more ‘real’ it is for him (Flusser, 2014, “The Gesture of Searching).

In this essay, I will build upon Manovich’s observation that, “cinema language, which was originally an interface to narrative taking place in 3-D space, is now becoming an interface to all types of computer data and media” (Manovich, 2001, 326). I will argue that whatever the language of VR turns out to be, it will rely far less on symbols and signs (content)  and more on a procedural code (form) that elicits both the memory of the body’s movements and affects as well as a shared cultural memory, not of the lived world, but of cinema, itself.

What determines the authenticity or veracity of this kind of storytelling? Here, it is useful to consider one of the earliest, most widely distributed experiences in VR. The Tuscan Villa  was provided free with the developer kit for the Oculus Rift DK2 head mounted display. It offers users the chance to meander through a country villa and surrounding landscape. A generic looking computer-generated farm house, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. Significantly, the experience does not provide a traditional narrative. Rather, it  offers the vague sense of remembering a dream or retrieving something that has been lost which never actually existed.

Kitsch is nothing if not a suspended memory whose elusiveness is made ever more keen by its extreme iconicity […] kitsch is not an active commodity naively infused with a wish image, but rather a failed commodity […] a virtual image, existing in the impossibility of fully being. Here, for a second or even a few minutes, there reigns an illusion of completeness, a universe devoid of past and future […]. (Olalquiaga, 1998, 28).

In The Language of New Media, Manovich points out that as cinema becomes more and more reliant on software manipulation in postproduction, it functions less as “a record of perception,” and more like a computer screen— “as a record of memory” (Manovich, 2001, 325). In the case of the Tuscan Villa, cinema no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.

Flusser suggests that gesture, by which he means the “reality” of affective states made manifest through aesthetics, should be judged in terms of art or kitsch rather than whether the gesture is true or false. “The scale of values we use to evaluate (gesture) may not oscillate between truth and error or between truth and lies but must move between truth (authenticity) and kitsch (Flusser, 2014, The Gesture of Searching).

It is not artifice in itself that renders the Tuscan villa demo artless. To understand this,  it is instructive to compare the Oculus demo to Maria Menken’s  decidedly un-kitsch 1957 film Glimpse of the Garden . Even though both The Tuscan Villa demo and Menken’s film privilege surface effects over depth of meaning, the result is decidedly different. In her film, movement of the camera, a unifying looped soundtrack of birdsong, and color filters take precedence over content, which consists mostly of shots of cultivated nature. Despite the amateurish quality of the images, the result is strangely magical. Menken’s editing, which juxtaposes different scales, points of view, and speeds of movement, resists the Tuscan villa illusion of completeness. Thus, it is due to the authenticity of the filmmaking gesture itself, not its content, that Menken’s film can be called art whereas the Oculus demo remains kitsch.

If there is any redeeming gesture in the Villa, it is in its ability to invoke an uncanny sense of bodily presence, a being there in a digital world that is generated by the user’s wandering or, in Flusser’s terminology, “searching” through the environment. According to neuroscientists,  presence is a primordial mechanism by which organisms establish self and other, creates a motor map of the universe and uses memory to plan actions and interactions. In so far as the Tuscan Villa succeeds in creating this sense of presence, it sets up a queer dynamic in the user—an oscillation between interior and exterior, self and other, the “real” and the virtual. In this way it, perhaps inadvertently, succeeds in making the familiar strange. In other words, the odd and unresolvable experience of being in two places at once creates a kind of digital estrangement conceptually related to Bertolt Brecht’s Verfremdungseffekt, distancing effect, and Russian Formalist critic Viktor Shklovsky’s “making strange” (Russian: приём остранения priyom ostraneniya) whereby certain devices are used to disrupt the audience’s easy identification with an artwork.

To see how this might be productively used in VR, I turn to filmmaker Karen Cytter’s work for a contemporary example of estrangement. Cytter employs familiar, even kitsch codes of cinema, especially melodrama. But, rather than offer an illusion of completeness, she uses what Thomas LaMarre refers to as “internal montage” (LaMarre, 2009,125). LaMarre suggests that computer graphic imagery and digital effects have created a new emphasis on the animation technique of compositing —the creation of an image composed of multiple layers. If an artist suppresses the space between layers to create an illusion of wholeness, the result is either a unification in depth, a hyper-Cartesianism with a well defined vanishing point (e.g. Pixar type animations), or a flattening of all layers into a single “superflat” plane in which multiple frames of reference and sight lines coexist. Cytter’s films demonstrate a remarkable compositing of space, dramatic events, and sound. It is effectively a jumbled cinematic database in which voice, visual form, emotion, and plot nearly float free of each other on the level of meaning.

Surface effects are at least as important as content here. Through a process of making the familiar strange, she astutely reveals the underlying game of seduction that takes place in cinema. In Cytter’s work we are left with empty codes, empty signs which nevertheless incite emotional responses. Devoid of linear narrative, in Rose Garden , the mournful sound of the flute is not only a wry reference to Sergio Leone’s Once Upon A Time in the West, it codes the psychological tenor of the piece. Although there are no clear “characters,” as with avatars in video games, the human body remains the basis for emotional engagement. For instance, at the end of the film, when a boy is shot in the back, though we know nothing about him as a character, the violent collapse of a child’s body is still shocking and emotionally wrenching.

Rose Garden’s flattening out of time and space, and a move from depth to surface, which Flusser relates to the gesture of searching, resonates with Hiroki Azuma’s description of database narrativity. In his book, Otaku: Japan’s Database Animals, he notes a move from “grand narratives” to “grand non-narratives” or database (Azuma, 2009, 55). This is associated with the proliferation of small narratives the purpose of which is not to refer back to some greater or universal theme, but to invoke strong feeling or fulfill consumer preferences. Whereas, in the modernist era, personal fictions could be related in depth to a grand story about the nature or meaning of human life, now, there is a lateral move in which multiple small stories are spawned from the same database as in Cytter’s film. Likewise, in virtual reality, narrative unfolds as movements through space. However, distance is no longer measured in terms of meters or any other universal objective standard, but, as both Flusser and Azuma suggest, in terms of the user’s desires and interests interacting with the environment.

In Mark Amerika’s cell phone film Immobilité, one of the characters remarks that the world is disappearing before her eyes. Indeed, the opening scenes, a shaky, looped walk down a tree-canopied path, reveal just how the world will end— not in some fiery apocalypse, but rather by our recognition that the world was only ever virtual. In Immobilité, what remains of embodied experience, what Amerika calls “muscle memory,” finds its expression not in meaning, but in form. In effect, Immobilité a film that remixes some of Flussers thought into the subtitle track consists not so much as a series of images, but as a series of film gestures.

In Immobilité, the quality of the image is sacrificed to movement of the camera just as content is sacrificed to surface effects. Depth, in the form of philosophic musings, is separated from the images and brought to the surface as subtitle. What remains on the screen are vibrations, rhythms, and frequencies that resonate with or cancel each other out. In fact, in many parts of the film, the camera is decidedly unsteady or even appears to vibrate thereby creating a surface that destroys a Cartesian perspective and the static perceiving subject. “And then I would begin to lose myself, to play uncontrollably, becoming something like music,“ the unreliable narrator texts us from the beyond.

Heidegger’s concept of “profound boredom” operates here. This is state in which all beings “recede” into indifference and the one who is bored comes face to face with the experience of time itself (Heidegger, 2001, 80). Heidegger describes profound boredom as the ground for all the multifarious variations of being. Profound boredom leads us to suspect that reality is neither static (immobile) nor completely knowable. Concomitant with this loss of a fixed self, language fails to signify. Rather, it functions procedurally to carve up pure duration into pieces: life and death, existence and nonexistence, chaos and order. This resonates with Henri Bergson’s suggestion in Matter and Memory that homogeneous and universal perceptions of space and time are a trick of the mind, a refraction of pure duration into space. For Bergson, space and time do not exist anterior to perception and action. We do not act through space and time, rather we create these symbolic diagrams because we need to “divide the continuous, fix the becoming, and provide our activity with points to which it can be applied (Bergson, 1990. 212). Bergson suggests that our knowledge of the world is delimited by the concerns of the body and its possible actions and, perhaps, even more importantly, by our memories. If that is true, we are like Plato’s cave dwellers: always living in a virtual reality of our own making.

The key dynamic in Amerika’s film, one that will likely dominate in this time of apocalyptic narrative, is a tension between overt aesthetics and conventional notions of truth. Although Immobilité’s low production values and distinct lack of artful montage lend the film a “real-life” documentary feel, Amerika’s overt use of glitch, repetition, and jerky camera movement insists upon a kind of self-conscious performativity or aesthetics. When the narrator asks, “Was I authentic?” the question is equally what does “authentic” mean if everything is virtual and, ultimately, data, and, furthermore, who is “I”? If the world is disappearing, so is the singular author, that figure of the auteur which haunts French New Wave cinema, to which Amerika  pays homage in his use of subtitles. In Immobilité’,  Amerika, like Flusser, suggests that the question of “authenticity” lies not in a “true” identity but in the aesthetic value of the gesture itself.

Antonioni’s masterwork, Zabriskie Point, also operates by disavowing conventional notions of authenticity. In Zabriskie Point all is surface: from the setting, an almost abstract stretch of ancient desert, to the way images are composed in multiple layers without depth. There is also a strange flatness to the main characters. They seem to have no ambitions and no particular future. With exquisite conciseness, Antonioni destroys the epic march of history with the first line spoken by the male protagonist: “I’m willing to die (for the revolution), but not out of boredom.” For the characters in Zabriskie Point,  time operates as it does in profound boredom as a pure duration.

Even on a personal level, events have no apparent consequence. Though one of the young lovers dies violently, there seems to be little emotional repercussion for the lover that remains. This flatness of character is matched by the physical beauty of the actors. It does not matter that beneath they are voids, the viewer can still be seduced into watching. In the end, it is up to the viewer to decide if Zabriskie Point’s aesthetics, which privilege form over content, offer any meaning.

The last scene of the film, which is overtly apocalyptic, masterfully captures this dynamic. Here, the female lover imagines blowing up the ultra-modern house of her boss, a slick real estate developer who wants to produce a housing development in the desert. Antonioni’s slow motion explosion which shows mostly intact objects floating across a horizonless, flat blue surface of sky disrupts conventional Cartesian perspective as well as scale. Moreover, using slow motion and repeating the moment of the explosion from multiple points of view, Antonioni disrupts the conventional flow of time and a singular perceiving subject. However, as with the rest of the movie, nothing actually transpires. The film concludes with the woman, a virtual terrorist, driving away from the intact house without a clear destination.

The strange physics of Antonioni’s explosion brings to mind the procedural physics operative in the software used to create VR experiences. These are the rules by which objects behave in a virtual 3D landscape, rules of interaction put into play by an algorithm in a game engine. As Baudrillard notes in his odd little book Seduction, games are subversive precisely because they allow for a reversibility and fluidity of meaning that operates outside of law (Baudrillard, 1991,131-157). Because procedural physics operates outside of natural law,  it offers the potential for radically new forms of narrative.

As the quote below from Donald MacKay, one of the key figures at the Macy Conferences on Cybernetics, points out: meaning, itself, acts as an “organizing function” for a perceiving subject. Significantly, MacKay defines information according to its functional impact, without respect to whether it is “true” or “false,”  an idea which resonates with Flusser’s insistence that aesthetics determines the truth of a gesture.

INFORMATION can now be defined as that which does logical work on the organism’s orientation (whether correctly or not, and whether by adding to, replacing, or confirming the functional linkages of the orienting system). Thus, we leave open the question whether the information is true, false, fresh, corrective or confirmatory, and so on…The MEANING of an indicative item of information to the organism may now be defined as its selective function on the range of the organism’s possible states of orientation (Wilden, 1982, 236).

Discussing filmmaker Peter Greenaway’s move from film to installation, Lev Manovich suggests that spatialization allows Greenaway to construct pure database narrative, something which film does not allow.

No longer having to conform to the linear medium of film, the elements of a database are spatialized within a museum or even the whole city. This move can be read as the desire to create a database at its most pure form: the set of elements not ordered in any way. If the elements exist in one dimension (time of a film, list on a page), they will be inevitably ordered. So the only way to create a pure database is to spatialize it, distributing the elements in space. (Manovich, “Database as a Symbolic Form”).

Following Bergson’s emphasis on the body as the starting point for perception, I would counter that this does not constitute pure database. At least in the “real world,” the narrative structure of installation arises out of the interaction of bodies with the architectural environment. As Lakoff and Johnson, and others have shown, human language, also revolves around the body. However, in the virtual world, this is not necessarily so. Here, even gravity need not operate according to natural law and, furthermore, reading text can often induce nausea. Yet, even in virtual reality, meaning-making requires some form of structure.

One way that virtual reality narratives might recoup meaning without resorting to conventional cause and effect is by using procedural physics to structure narrative and to create asymmetries in the environment (database) which allow for meaning-making according to MacKay’s definition.

Artist Rachel Rossin’s virtual reality works illustrate this possibility of a new kind of narrative, one that speaks to and through virtualized, cyborg bodies. In an early work n=7/The Wake in Heat of Collapse, Rossin uses the procedural physics of gravity itself to shape narrative. Here, users progress through three contemporary Dantesque landscapes. Gravity in the landscape decreases algorithmically as the user descends. Invoking the virtual body of the user, Rossin notes:

In video games you have a certain type of hermeneutics…what you understand as the sort of “body language” or instinct that come with the world. Gravity – especially – falling off of things usually means death. We get this sad little feeling when we watch ourselves die in a video game and there is a pause that actually feels like critical distance and so I used that as a loose Virgil… the thing that pulls you through the layers of the separate worlds (Rossin, personal email correspondence December, 2015).

In a more recent work, I Came and Went as a Ghost Hand, Rossin scanned intimate photos of her domestic surroundings then manipulated these and used them to create a virtual reality landscape. As a result, the viewer sees familiar things differently, not just because of the data loss, which reduces some objects to their raw qualities: color, shape, volume, and which causes erasures, but because of the unsettling experience itself: moving through objects, speeding up, slowing down, torque. In lieu of a narrator, Rossin offers a reassuringly banal, reoccurring icon of a white gloved hand, a kitsch gesture that acts as a point of continuity for viewers as they traverse the disorienting, horizonless landscape.

In his essay, “Envisioning the Virtual,”  Brian Massumi suggests that the virtual should be considered as the formative or potential dimension of reality, not as something which is in opposition to actuality. Analyzing an optical illusion where “Pac-man” circles cause a virtual triangle to appear, Massumi proposes that it is the tension between different forms of perception which produces “a field of intensity” within which the “pressure for resolution” acts as “a formative force.”

The tensions are between modes of existence proposing themselves to the experience[…] The modes do not add up to a form. They are tensely, incommensurably different. Their incommensurability exerts a differential pressure. Something has to give (Massumi, 2014, 56).

Massumi is describing a situation in which different ways of looking at the world seek to coexist in time. Cinema attempted to resolve this with montage, VR offers a different solution: the spatialization of time itself. How humans will organize that time-space in a meaningful way with respect to virtual bodies is evolving. Moreover, the political and social implications of this are as yet unknown. However, as I have shown, the memory of the real, preserved as a visual code borrowed from shared cinematic experience and the memory of the body, preserved as codified gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen. 

Bibliography

AZUMA, Hiroki, Otaku: Japan’s Database Animals, translated from the Japanese by Jonathan E. Abel and Shion Kono, Minneapolis (MN), University of Minnesota Press, 2009.

BAUDRILLARD, Jean, Seduction, translated from the French by Brian Singer, New York, Palgrave Macmilan,1991.

BERGSON, Henri, Matter and Memory, translated from the French by N.M. Paul and W.S. Palmer, Zone Books, Cambridge (MA), MIT Press, 1990.

FLUSSER, Vilém, Gestures, Kindle edition, translated from the German by Nancy Ann Roth, Minneapolis (MN), University of Minnesota Press, 2014.

HEIDEGGER, Martin, The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, translated from the German by William McNeill and Nicholas Walker, Bloomington (IN), Indiana University Press, 2001.

LAKOFF, George and Mark JOHNSON, Metaphors We Live By, Chicago (IL), University of Chicago Press, 2008.

LAMARRE, Thomas, The Anime Machine: A Media Theory of Animation, Minneapolis (MN), University of Minnesota Press, 2009.

MANOVICH, Lev, “Database as a Symbolic Form,” http://www.mfj-online.org/journalPages/MFJ34/Manovich_Database_FrameSet.html, accessed online 22/01/2017.

———, The Language of New Media, Cambridge (MA), MIT Press, 2001.

MASSUMI, Brian, “Envisioning the Virtual,” Oxford Handbook of Virtuality, Oxford, Oxford University Press, 2014.

OLALQUIAGA, Celeste, The Artificial Kingdom: On the Kitsch Experience, Minneapolis (MN), University of Minnesota Press, 2002.

WILDEN, Anthony,  System and Structure: Essays in Communication and Exchange, New York, Tavistock Publications, 1980.