While arguments rage about the relevance of Virtual Reality (VR) for news audiences, we spend our days huddled up and surrounded by a tangle of wires in a New York co-working space, where we focus on quite a different way to deal with reality. 

We are the team behind a series of Mixed Reality (MR) explainers for NBC Left Field — a fever dream of brightly coloured infographics hand-drawn live in VR, but viewable without a headset because we’re using VR to create the graphics for linear video.

This initiative started as a desire to merge news and cutting-edge technology, then culminated in a deep dive into practical possibilities for newsrooms to utilise VR.

We already had some experience developing news stories for VR, but we were frustrated that only audiences with access to a headset could watch them. We wanted to reach more people and began wondering: How could we use VR to express ideas to people outside the headset?  

Our answer to that question is Google Tiltbrush, a VR programme that enables drawing in 3D space. While experimenting with it, we also wondered: Could we film someone drawing in real space? Could we use this to tell stories our audience would want to watch? And could we make them look really good? 

There were already examples of people experimenting with MR online, mostly in the gaming world. So we looked to them and set out to build our own MR camera rig using a standard-issue FS5 and an off-the-shelf HTC Vive kit. The concept behind MR is to put a Vive sensor on the camera, which will render a virtual camera view that mimics the exact movements of the real camera. By compositing both views together, we can let a linear video audience observe the experience the VR user is having.

It wasn’t an easy process because documentation online is limited, and it involved some coding on our part to get all the elements aligned. Once the setup worked, we thought long and hard about how best to use this tool. Top of mind was data visualisation, a vital component of news, but also an area where it’s hard to stand out from the crowd and aesthetics have been heavily influenced by tools like After Effects.

Part of the reasoning for using MR for explainers was to offer the audience something they’d never seen before. We began to push the boundaries of an ordinary data visualisation. Being able to present information and then dissect and annotate it formed the basis of our “back of the envelope” explainers. 

The freedom to draw and illustrate allowed us to include cultural and historical context. Along the way, we discovered the power and authenticity of the “presenter as illustrator,” which means the ability for a personality to come out through doodles and handwriting.

From a production standpoint, it’s had quite a reach throughout the newsroom. Beyond the videos, we have also set up a system that allows producers to draw their own graphics and film them from within the headset. The producer becomes the graphics person. This self-service graphics component is one of many effects from the main experiment.

Looking ahead, the ability to export 3D models of each drawing has fascinating implications for the world of AR. VR continues to improve and increase its reach, and we are excited to see what will happen as the technology develops. In the meantime, we are thrilled to be capitalising on the opportunities VR provides not just to consume content, but also to create it. 

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .