TMU: Do you think that VR is set to merge/compete with cinema in the near future?
RS: I think VR is set to merge more with gaming - we haven't yet solved the problem of creating interactivity in video. I think the real future of VR is in gaming, and using gaming engines to render the cinematic experience. The power of cinema is that it's a director-led experience; games are a user-led experience. The expectation when you play a game is that you participate in the storytelling process, whereas in cinema you just let it unfold. But they're different types of entertainment that fulfil different needs - sometimes actively participating gets exhausting.
TMU: How do you think new technologies such as VR are impacting the way we tell and receive narratives?
RS: Humans have a unique storytelling instinct _ our ability to fictionalise and gossip is a way of protecting ourselves and building effective communities. I think one area of change is the way in which storytelling embodies the 'I'. We're no longer content being told stories about people that sit outside our lived experience. People look at an image _ VR or games or cinema - and long to see a version of themselves reflected back.
AI and machine learning may help us find that reflection by allowing us to create complex worlds that we can't on our own. Working with machines will enable us to build branching, interactive, and complex stories with more immersive experiences.
To extrapolate from there, with story genres growing more niche, machine learning may get to a point where experiences are tailored specifically to the individual. We'll see these technologies allowing for every individual to be a creator; for everyone to be the showrunner of their own world.
TMU: In your article for Quartz you point out that many of the techniques that are used in VR are not new, and are often borrowed cross-medium, such as POV and stream of consciousness. Could you elaborate on this?
RS: The novel is a powerful medium, and can do things that film can't. As audiences and filmmakers we've become accustomed to the classic Hollywood 'hero's journe' structure _ the hero goes out into the world, gets something, comes back and they've changed. But techniques from the novel can inform different ways of structuring story. For example in television recently (and as far back as The Wire) writers have been using techniques such as POV to tell a story from multiple perspectives, rather than just one singular protagonist. That sort of thinking can be translated to VR to teach us how we can use different points of view to navigate a story.
I'm not sure what actual stream of consciousness would look like in Virtual Reality, but one of the ways we could think about it is through sound - We have a graduate student at VUW School of Design who did this. Blind people play games by listening - they have headsets that provide haptic feedback and soundtracks that are spatialized to their surroundings. The experience is meditative, and one that can take storytelling from the page or image and into a physical storytelling space.
TMU: One of the key ingredients of storytelling is the suspension of disbelief - the concealing of artificial production elements to make the experience immersive. Do you think that this is possible in VR work?
RS: There's a few ways to do it, such as building the rig so it sits directly below the camera, and using practical lighting that's concealed behind or inside objects in the space. We're seeing a lot of VR films shot outside at the moment, because it's much easier to shoot under natural light in that sort of situation. There's also light-field technology being made which essentially allows you to 'crop' out physical elements that you don't want by depth-mapping everything in shot. Of course it's still very expensive and won't be on the consumer market any time soon.
We're still learning how to 'look' in VR - viewers tend to look forward (based on their orientation), so that's where we put most of the action. We're trying to find ways to make people look differently, to look up, look down, to pivot - and to not have to be cued to do it. I'd like to see some experimental VR where the apparatus is part of the storytelling, or a statement about moving beyond the apparatus. I'd like to see the Dogville of VR.
TMU: Could you tell us a bit more about your 'Circle Versus Square' research project? How did the idea come about?
RS: Cinema is currently experiencing a sort of spiral - we're only making one kind of movie, the kind of big-budget franchise film, and the result is audience fatigue. So naturally we're looking for new ways to tell stories. That's where VR comes in. I wanted to understand what is essential to both film and VR, and something that came up is the language of the square versus the language of the circle.
There's thousands of years of storytelling built up around understanding what happens when we look at a square that has an image in it. So what happens when we break it apart and turn it into a sphere?
The language of the square goes back to painting, and before that even cave drawings. The way we learned to create and consider art was to think about the frame, and what's inside the frame. There's thousands of years of storytelling that's built up around understanding what happens when we look at a square that has an image in it. So what happens when we break it apart and turn it into a sphere? That's what the project came out of - thinking about how we can reteach ourselves, not only to tell stories in a new format, but how to interpret them. The circle lies outside the edges of the square. It's the apparatus of the image and the bigger story world.
TMU: You've got a huge VFX catalogue behind you _ what was your favourite project to work on? Why?
RS: Definitely Avatar. I knew it was a special movie, but I didn't realise it was going to change the way that movies looked. It wasn't until the first crew screening - and I haven't had this experience since then - where I walked out and saw the same look on the faces of other people that I felt inside. We had collectively experienced an alternate reality. The vision for creating the world was of course led by James Cameron, he really thinks like a VFX artist and respects the process, so it was fun to work on the film for that reason as well.
It felt like a real world, like someone was there with a real camera just recording it all.
The other thing that was quite stunning about Avatar was that it was a true hybrid feature animated and live-action film. When I had previously worked on animated films we made environments that were entirely in focus, because we'd spent so much time making them that we wanted people to see everything. But what happened on Avatar is we spent all this time making complex environments and then someone (a compositor) came along and blurred much of it. We wondered why they took all of our hard work and did that, until we saw the final result. It felt like a real world, like someone was there with a real camera just recording it all. It pushed the visual language further than it had gone before.
TMU: By the numbers, women are significantly underrepresented in film both on-screen and behind the scenes. Do you see the same problem in the VFX community? If so, what needs to happen to create change?
RS: I do see that problem, and I don't think there aren't enough talented women to take up those roles. I think it's a failure of leadership. People who lead companies need to see the value of a diverse work force. In my entire career as a VFX artist I never worked on a team that was lead by a woman VFX supervisor. We need more women in upper-management and leadership roles so that younger women can aspire to those roles.
Diversity is one of those words that's important but gets thrown around so casually now that it has ceased to be meaningful. We have to think about what makes people ideologically diverse. Diversity rises up out of a variety of experiences based on things like race, class, gender, sexuality. People with difference life experiences bring different ways of looking at a problem to a team. And a team of people who approach problem solving in unique ways will solve problems more effectively.
TMU: Given the chance, what is one piece of advice you'd give to your younger self?
RS: No matter what your job is, keep working on your own projects. Practising your own art is like training a muscle - keep it alive alongside the craft that you do for a living, because they come together and inform each other in unpredictable ways.
TMU: Finally, what is your #1 must-do for anyone visiting Wellington?
RS: Drink coffee - it's much better here than in other parts of the world. Also, walk around the bays. They're beautiful and we have unique public access to the sea that you don't get in other parts of the world. Also, don't bother with the umbrella.
Find more of Raqi's work and a full list of her publications here