Prototyping in VR
Using virtual reality as a medium came from the goal of developing a reading experience focused on performance, affect and non-traditional meaning making.
When I first started designing the amorphous poetry anthology/archive/exhibition space I was working with the web VR framework A-Frame. While these early experiments weren't very successful as outcomes, they were invaluable as an introduction to the infrastructure of working in virtual reality, especially since I had (and still have) a limited understanding of code.
From an early stage I began focusing on bodies and bodily sensation, trying to push against the idea of neutral, clean technology. Moving to Unity - a 3D videogame engine - was intimidating but rapidly opened up the options, especially once I became familiar enough with the program to start working with free assets.
The structure of Unity greatly shaped the final piece in many ways. The infrastructure of Unity and the huge community developing assets for it provided many of the assets that have been collaged, rejigged or repurposed for other means. As a game engine, Unity uses a system of objects, colliders and physics that are highly regimented and delineated, and working against and amongst this infrastructure while trying to create an unstructured, gooey free-for-all was a constant negotiation.
Early - very early! - experiments in web VR and Unity
Softbody physics system
WIth a focus on "congealed bodies" and larger masses made from smaller elements, I began experimenting with systems in Unity for emulating softness and liquidity. I began with Unity's cloth mechanics, and while it created a lot of interesting effects, it was difficult to control and achieve the balance of legibility and squishiness that I wanted.
Throughout the constant fiddling to figure out what "looked" good I was also figuring out what "meant" good. As I started to build out these elements I was realising the compatibility of simulated slime with my queer theory framework. When trying to emulate sticky, gooey substances like slime in a digital environment, I couldn't decide - a softbody physics system for solids or a particle system for liquids?
This inability to decide reflected the way that slime is a sort of in between - refusing to be categorised, causing discomfort and fascination, a material that doesn't belong. "Slime is queer" is a neat metaphor, but through the prototyping process I was able to demonstrate that materially; while simulating stickiness, oozing and gooey textures was eventually possible, it involved pushing against the allowance of Unity. I chose a softbody physics system, although I'm sure a liquids system would have been just as if not more effective.
As I began to explore the idea of slime, goo and unstable solids, I was starting to understand that I was designing an experience, not a movie. This sounds simple, but took a lot to get my head around. The entirety of the world design was important and part of the process of making meaning, and the interactions that users had were a vital place to be making my arguments.
I had been thinking very textually, relying on heavy narration and complicated story to try and get my point across. Over time I realised my goal was more about building a world that someone wants to stay in, rather than making a point by saying it. I focused on encouraging certain emotions, sensations, and crafting a space for contemplation rather than handing over a neat message.
What became clear throughout the process was how effective little things could be for shifting the experience. Adding wobbles to text suddenly made it "feel" like slime, and a rainbow skybox suddenly made the environment feel incredibly overwhelming. Sound design became incredibly important for shifting how users thought of digital materials they were interacting with. Narration was important for introducing the user to the space and encouraging spatial literacy in VR, especially since many people have not used interactive VR or have only used it for games that have different modes of interaction.
"It is not a question of leaving the body behind but rather of extending embodied awareness in highly specific, local, and material ways that would be impossible without electronic prosthesis."
(Hayles 1999, pp. 291)
It was key to my goals to cling to the tensions inherent to virtual reality. I was particularly drawn to digital humanities as a field that is concerned with critically analysing technologies and their relationships with society. I didn't want to make a "floating head" experience of disembodiment. By focusing on an intermingling of technologies, bodies and knowledges, I hoped to present the cyborg splice that virtual reality embodies as a site of radical potential, acknowledging the complicated discourses surrounding technologies such as virtual reality while also arguing for its success as a way of presenting often ignored stories, histories and existences.
One of the key elements of the interactions is the user being able to pick up text that, after a certain point, moves from their hands to the centre of their body and is no longer able to be touched. This literal "body of text" is an essential part of modelling a cyborg knowledge system, understanding the body as a network that includes the physical flesh, the complicated technologies that make VR possible, and the digital body that is layered on and amongst the "real" body.
Since the phrases that become their body are the phrases they've touched, the user's decisions are what shape the form of their body; this is an analogy of Sara Ahmed's model of sexuality as as spatiotemporal orientation, in which bodies and identities are culminations of desires and decisions (and the desires and decisions that are put out of reach).
Understanding interactive VR as a liminal dimension that sidesteps the real world as it simultaneously remains inextricably tangled with it opens up many possibilities for the medium. VR is good at immersion, but asking how, and for what purpose, means we can use it as a modelling system to investigate social phenomena and prototype theoretical positions in practice.
Lastly, I'm interested to know how the other parts of the experience other than the bit we design digitally can shape the meaning of a piece. How do users put on the headset? What greets them when they take it off? What do they take away from the experience? What crosses that threshold with them?
This project involved a lot of very quick learning and wishing I knew the systems I was working with in closer detail or with more familiarity during the process. Moving away from an understanding of textual or image based communication into a space where experience can be a way of making an argument has been been fascinating, and I'm interested in how we can talk about virtual reality in a way that does the medium justice as more than a 360 degree image or video.