Multi-Device Storyboards for Cinematic Narratives in VR

The Problem

Virtual Reality (VR) narratives have the unprecedented potential to connect with an audience through presence, placing viewers within the narrative. The onset of consumer VR has resulted in an explosion of interest in immersive storytelling. Planning narratives for VR, however, is a grand challenge due to its unique affordances, its evolving cinematic vocabulary, and most importantly the lack of supporting tools to explore the creative process in VR.

Traditional tools such as storyboards - a comic book like representation of the story - work well for traditional film, but are less effective for VR as there are concepts and constraints that are unique to this medium. To complicate matters, storyboard artists work incredibly quickly. The fundamental requirement at the earliest stages, while creating storyboards, is to capture ideas, concepts and thoughts as quickly as they occur. This means that setting up complex infrastructures and underlying surfaces to build 3D or VR scene is too slow.

The Analysis

To better understand planning for VR, we talked with industry professionals to determine the differences between planning traditional film and VR stories. From this, we identified unique features that affect planning for this medium.

Based on these planning concepts, the director's needs, and the artist's workflow, we propose a multi-device solution to enable storyboard artists to sketch in a fast and familiar method, yet allow film teams to see the results surrounding them. By treating the viewing environment as a stack of concentric cylinders, the artist can draw on these surfaces, unrolled, shown as flat panoramic panels - a surface that is easier for artists to mentally unroll. These panels can be rolled back into cylinders to be displayed immersively.

The Prototype

We built a multi-device prototype system that networks a computer tablet and a VR Head Mounted Display (HMD). This setup simultaneously exploits the sketching abilities of a tablet, and immersive viewing experience of an HMD. Through this "best of both worlds" approach, the artist is able to draw on the tablet, while the director instantly views the results in VR (or the artist can work directory in VR them self).

On the tablet, the artist has several tools at their disposal. Similar to the work of Storeoboard, there is a panoramic stereoscopic sketch surface where a user can easily sketch at the depth they wish to draw, allowing the artists to quickly build a scene surrounding the user. There is a panoramic storyboard view providing an overview of the scene, with gradient shading and boxes to show a viewer's intended focus. Finally, there is an overhead view where teams can quickly discuss and adjust content and layout of a scene from above.

Similarly, in the VR environment, a user can view the results of the sketched scene virtually all around them, or display a virtual reality storyboard prototype. This storyboard wraps around the user, showing miniature dioramas of each panel. Finally, by providing visual feedback of the pen and sketch surface's position, artists can draw while fully immersed in VR, allowing for many different possible workflows.