The 360º revolution

By Rhodri Marsden

Imagine you’re cruising around downtown Los Angeles, looking for a pleasant watering hole, when suddenly, out of the night sky, a barrage of meteors hurtles towards the city, flattening buildings and gouging out clods of tarmac. Wondering where, precisely, to fix your gaze would be low down your to-do list. But witness such a spectacle in a virtual-reality film, and the 360º perspective demands you investigate the ensuing chaos in detail. You can follow those fleeing or look on dumbstruck in horror. You can hew close to the arriving police, scrutinise the skies for incoming fireballs, or inch towards the alien emerging from a crater in the road. As this particular five-minute short (entitled “Help”) unfolds, Justin Lin, its director, uses subtle cues of sound, light and colour to guide our attention and keep us facing the action: a squeal, a flash, the sound of footsteps. But the techniques he uses have been hard won. “Help”, released last April, was one of the first VR productions to successfully grapple with the problem of narrative storytelling.

Viewers (and even the creators) of VR productions might have expected the medium to be a natural extension of traditional, rectilinear film-making, but that couldn’t be further from the truth. Directors describe exhausting processes of trial and error, after which entire projects still end up being junked. “There are no experts in VR,” says Henry Cowling, the creative director of Unit9, a London-based production company. “And there won’t be for years. Of course some people are more advanced, but everything is in flux because we’re all trying to figure this stuff out. And that makes it super interesting.”

Cowling acknowledges the “narrative anxiety” – the viewer’s worry that they might be missing something going on behind them – that arises when watching a film in 360º, one of many problems faced by creators of VR entertainment. The rhythmic, artful techniques we associate with traditional film – fast edits, close-ups, reaction shots – are rendered useless by the viewer’s capacity to choose his own experience.

“You’re not dictating a point of view,” says Patrick Milling-Smith, co-founder of 360º-video company Here Be Dragons. “You have to give people’s senses a chance to calibrate and become part of the experience. We’ve recently been working with directors Kathryn Bigelow and Henry-Alex Rubin, and what you get from these great film makers is a voice, a point of view, an understanding of design. But everything else has to be forgotten. Everything we do is a form of research.”

Milling-Smith cites “C’était un rendez-vous” (1976), a short by Claude Lelouch, as a source of inspiration from traditional film-making. A camera, mounted on a car, documents an eight-minute hurtle at breakneck speed through the streets of Paris. It’s filmed in one take, and you feel as if you are in the front seat. The challenge for VR film-makers, according to Milling-Smith, is to find that kind of truth. “False notes are really glaring,” he says. “They smack you around the head.”

Catherine Allen, a director who works with the BBC on VR, is more blunt: “Essentially, it’s harder to lie to your audience.” Directors have to be careful not to stretch boundaries of credibility and destroy the feeling of immersion that VR creates. Even actors have to modify their approach. “Imagine that you went to dinner with two people and they were both acting,” says Allen. “Of course you’d notice – and it’s the same in VR, because it simulates dropping you into a situation. It’s not just bad acting that can look bad, all kinds of acting can.”

Something as simple as the distance an actor stands from the camera can drastically affect a scene. According to theories of interpersonal distance, we’re likely to speak with strangers in the street at a distance of between four and 12 feet, but we interact with friends much closer. These rules stay true in VR, according to Edward Miller, an interactive-imagery specialist. “We’ve done tests using the same dialogue spoken by actors at 50cm increments from the camera,” he says. “You almost have to act as a psychologist as well as a director to ensure that these social norms aren’t violated.”

The difficult negotiation required to meet the expectations of reality in a virtual world has been particularly fraught in the sphere of VR gaming. Existing controls or game mechanics don’t work as you would expect them to in VR, if, indeed, they work at all, according to Peter Pashley, a game designer at digital product studio Ustwo. Photorealistic graphics, he says, are never realistic enough: “you notice all the things that are wrong.” Studios that are known for their photorealism, such as Ubisoft (the creators of “Assassin’s Creed” and “Far Cry”), have been developing non-realistic art styles that are still consistent and immersive. For the VR game “Eagle Flight” (pictured top), Ubisoft chose a cartoon-like aesthetic. “It’s an interesting challenge,” says Pashley, “to suggest the shape of things rather than the detail – for example not representing every blade of grass. That kind of high-frequency detail can also accentuate motion sickness.”

The problem of disorientation and sickness runs throughout VR. Badly made entertainment can provoke nausea and headaches. This is particularly problematic with gaming, which traditionally requires first-person movement: if your field of vision doesn’t tally with the signals the inner ear is sending to the brain, you can quickly start to feel queasy. Pashley and his colleagues wrestled with the problem of moving a player around a VR environment during the production of “Land’s End”, a problem-solving game for the Samsung GearVR.

“For us, it felt like a once-in-a-lifetime opportunity to help to define a whole medium,” he says. The mechanic they came up with involves prompting the player to focus on a point in the distance, and then slowly moving them to that point. “But if you wanted to use that method in, say, a first-person shooter,” says Pashley, “it would suck.” A preview of a VR version of the classic game “Doom”, showcased by Bethesda, its publisher, over the summer, features a teleportation technique where the player is moved while surrounding action slows down to “bullet time”. It’s another solution to this problem, but as Pashley notes, developers’ fondness for creating hyper-fast versions of real experiences will continue to come up against problems in VR. “You end up being constrained by your physical body, by reality,” he says.

In a VR game the player will have a set of tasks to perform, but in 360º video the absence of a role for the viewer can feel disorientating. If there isn’t any form of interactivity, explains Catherine Allen, users can feel constrained and powerless. “It’s almost like they’re a ghost,” she says. “This really affects creative direction, and I’m seeing a trend where that feeling of powerlessness is played upon.” She cites “Clouds over Sidra” (above), a short film by Milling-Smith’s Here Be Dragons, about life in a Jordanian refugee camp. “The closing sequence, where refugees are dancing around you and looking up at you, prompts a very raw, emotional reaction.”

The ethics of exploiting that immersive vulnerability is also something that Allen has wrestled with. “Viewers might not have the critical distance they would normally have,” she says. “When you move from representation into simulation, content creators suddenly have a lot of power. Making emotionally powerful content is inevitably defined as a success, but I think that with that power comes responsibility.”

That issue feels low on the list of priorities. At last summer’s Cannes Lions festival, the founder of Wired, Kevin Kelly, spoke of the “palpable uncertainty” regarding VR’s future, as producers experiment to discover what does and does not work. “For example, I haven’t seen anything funny in VR yet,” says Milling-Smith, “because you can’t cut to a reaction to let an audience know that something is supposed to be funny.” But according to Pashley, testing established genres in VR is approaching the problem back-to-front. “VR requires whole new genres,” he says.

These will emerge only by building on oustanding work, and some of those successes – such as “Help”, “Clouds over Sidra” and “Land’s End” – are now emerging as creators discover techniques that resonate with audiences. The future of VR is likely to resemble gaming, in that we’ll be able to control our surroundings. It will resemble film, in that there will be directed storytelling. And we’re likely to be doing it socially, with other people represented as avatars. But it’s precisely this nebulous future that drives the likes of Henry Cowling forward. “There’s this wide open territory of new things to be discovered,” he says, “and that’s where so much energy is coming from. Everyone is competing, and that’s why it’s so exciting, so deeply addictive.”