How Virtual Reality Technology is Changing Documentary Filmmaking
At this year’s New Frontier exhibition at the Sundance Film Festival, doubts about the staying power of virtual reality are fading away. Visitors wait in long lines and the exhibition floors are fully packed for a technology that has been in the making for decades but hasn’t yet been able to quite get off the ground. Of the 15 works presented, 11 incorporate a VR headset, either via 360 degree video or computer-generated 3D environments. Though it has long been the purview of gaming, the film world is now also paying close attention to VR. There is a palpable buzz about how this new medium will change the mediascape and revolutionize cinema.
It is hard to ignore that this excitement is partially caused by the novelty of the technology. This makes the critical distance required for thoughtful criticism and reflection difficult to achieve; however, even if the VR does not live up to all expectations, it is here to stay as a new platform for storytelling that demands new ways of constructing stories and shaping experiences. What are the potential futures of this emerging medium? VR requires interactive and real-time storytelling (without cuts), suggesting a closer dialogue between video games and film. This convergence has already manifested in several documentary pieces in the New Frontier exhibition.
“Project Syria,” “1979 Revolution,” and “Assent” are documentaries that do not use photographic images. Instead, they turn to game mechanics and 3D modeling to represent sociopolitical, historical, and autobiographical realities. With the increasing accessibility of VR headsets, 3D scanning and photogrammetry, interactive 3D is becoming a viable platform for documentary filmmakers while simultaneously challenging the conventions of documentary.
Nonny de la Peña, a pioneer in VR storytelling, presented “Project Syria” at this year’s New Frontiers section. The experience opens in a 3D modeled street in Aleppo. After a brief exploration of the scene, the street is struck by a rocket and the user finds herself in a cloud of smoke and debris, ears ringing and surrounded by screams. “Project Syria” uses the VR headset and infrared sensors for body tracking, allowing the user to walk within the virtual 3D environment.
This free-form exploration creates a sense of presence and immerses the user in the environment. The interactivity is limited to walking and looking, so that the story does not shift. Yet this limited interactivity is enough to create what de la Peña calls a “spatial narrative,” which she suggests can elicit a strong visceral response from the user. Another haunting aspect of the project is that its virtual 3D environment signifies to a lack of journalistic information coming from Syria, hinting at the unknown and grave human rights violations taking place.
“1979 Revolution” seeks to combine the aesthetics and storytelling of triple-A video games with documentary filmmaking. A project of Navid Khonsari, formerly the cinematic director of “Grand Theft Auto,” and Vassiliki Khonsari, a visual anthropologist and documentary filmmaker, “1979” suggests alternatives to realism in documentary by borrowing from game design. The game allows the player to experience the events that took place during the Iranian revolution which led to the Islamic Republic.
Navid was raised in Iran and wanted to tell his story of the revolution in a medium that was able to accommodate the subtleties and personal stories that get overlooked. The video game form, with forking and detailed narratives, and an explorable environment, yielded an ideal platform to represent multiple perspectives of different groups on a historical narrative.
In one of the four scenes of “1979 Revolution” presented at New Frontier, the player goes through an interrogation, given several choices about how to answer the interrogator. No matter how the player answers, the interrogation becomes increasingly violent. Ultimately, there is only one way to continue the game: by cooperating.
This scene exemplifies what game scholar Ian Bogost terms “procedural reality,” which is based in the idea that by reconstructing the dynamics and behaviors of a certain circumstance, a player who goes through this can gain a type of knowledge that is inaccessible simply through depiction or linear narrative. This type of representation is well-suited to a deep understanding of systems and situations that arise within them. In non-fiction storytelling, this approach to interactivity has been relatively unexplored, though independent games like “dys4ia,” “September 12th,” “Papers Please” and “Depression Quest” have experimented with it.
The realism achieved through interaction design gives room to move away from a realistic representation. Rendered in a visual style that is both 3D and reminiscent of comic-book illustration, 1979 “[liberates] documentary from photorealism” in the hope that this might facilitate its reception. Navid Khonsari notes that “if the world of the game is a few degrees separated [from reality], the content might be more digestible and the audience might be more accepting.” By moving away from photorealism, Navid and Vassiliki also make it clear that they are after a greater truth, the patterns and the intricacies of a revolution. The game attempts to reflect on the severe contingencies of revolutions, echoing movements from the Arab Spring to the Ukranian Revolution.
“Assent” is a documentary piece by multimedia artist Oscar Raby, about the strange status of the witness as well as transgenerational trauma. As a young man, Raby’fos father witnessed a mass execution while a soldier in the aftermath of the 1973 Chilean military coup. He carried this trauma and the consequences of this throughout his life. The work is deeply personal and innovative in its use of 3D environment and interactivity in order to tell its story. The visible polygons and artifacts of the 3D environment create a structure that is close to the imperfections of memory.
In “Assent,” the user is brought back to the day that Raby’s father witnessed the execution. The user sees through the eyes of Raby’s father and proceeds within the experience only by maintaining eye contact with certain objects and locations. Only by looking, she gets sucked in and becomes part of the event. This interaction mechanic closely resembles Raby’s father’s experience as a witness without agency, implicating the user in the philosophical challenges of the experience.
This year’s New Frontier exhibition hints at a new form of documentary storytelling that will further be complemented by VR technology. Drawing on the meaningful interaction design found in some experimental and independent games will be an important component, as will be finding ways to navigate the visual intricacies of the 3D medium.
To some extent, acceptance of 3D and interactivity as methods to depict serious documentary content is a matter of slowly changing audience attitudes. However, the examples at New Frontiers this year also suggest that meaningful manipulation of the medium, through intentional abstraction or poetic control of the interaction, may distinguish this form as a new genre, distinct from aesthetics and practices of mainstream video games and traditional documentary.
Deniz Tortum is a filmmaker and a research assistant at MIT’s Open Documentary Lab.
