Tim Kashani presenting at AWE 2017.

The Augmented World Expo is underway, transforming the Santa Clara Convention Center into a hub of AR and VR innovation, exploration and play (this year we’re geeking out at the AR+VR Playground, which has drone racing and an AR golf course). There’s a stellar lineup of movers and shakers who are using new immersive platforms to drive economic growth, re-imagine existing production pipelines, break boundaries of access, and democratize health and education. You name an industry, and someone with a lightbulb about how AR and VR can make improvements or reimagine conventional formulas is here representing it. We’re here amid the whirlwind of speakers, pitches, and demos, adding a voice representing the arts – theatre, performance, film, and the coalescence of all three – into the AWE mix. How can AR and VR tear down gatekeepers, expand modes of performance, help market effectively, and foster a sense of open source connectivity and sharing in the entertainment world? Those are questions we’re asking and tackling at this year’s conference.

Before we get into the meat of what we’re presenting this year, it’s worth mentioning that even though the AWE hosts big time tech names and legendary Silicon Valley players, it isn’t as inaccessibly techy as it seems. When you’re on the ground here actually talking to people about their ideas, the AWE feels pretty down-to-earth: what we’ve got is a huge, buzzing conversation about how to bring more experiences to more people.

So what’s on deck for us? This year’s AWE is a pretty big one for us. We’re presenting an exciting new project that we’ve been working on with Microsoft HoloLens. Lorraine Bardeen, one of the faces behind Windows HoloLens Experiences, is presenting a talk on mixed reality and 3-D (and how Microsoft plans to bring it to personal computing and daily life). That’s where we come in, as one of the partners working with HoloLens augmented reality technology in the context of theatre and new media performance. We’re here demo-ing a new tool we’ve been developing: our HoloLens Set Builder.

Our HoloLens Set Builder is a piece of technology we’ve been working on that brings AR and VR into the set-design process of theatre production. Working in a Unity development platform, we’ve created a tool that lets directors and designers plan and design sets using augmented reality. What this looks like is a voice responsive AR diorama tool that lets you control set design, scale, lighting and angles in a virtual theatre space beamed in front of you and your collaborators (who may be thousands of miles away). It uses algorithms to accurately scale sets so that you can virtually foresee technical issues (that door hits that chair when it opens!) and resolve them before brick-and-mortar set building is even a glimmer in your eye. The tool incorporates gestural manipulation (so no crazy controllers necessary, just a pinch of your finger and you can move the offending chair or scale it down). It’s super responsive, malleable, and maybe most importantly, it has the potential to make the design and production process more interactive, less opaque, and more conducive to crowd-sourced collaboration. With virtual set building, the door opens for more input, more often, from a wider set of eyes.

Early iterations of the Set Builder showed small scale, low-poly set pieces

In this sense, the HoloLens Set Builder interacts with and really nicely brings to life some of the fundamental marketing ideas that make up the backbone of our model. We’ve talked before about applying lean startup methods to the theatre production pipeline. This involves incorporating an iterative process of rapid prototyping, user-testing, and generating feedback. With a virtual set builder tool, you can bring the backend of theatre to the frontend. In other words, we’re interested in involving more eyes in the early set building process and in connecting with our audiences in an interactive way as we build a production. Maybe that looks like running fully fleshed out virtual sets by our intended audience before we move forward with a production. Maybe it looks like involving users and viewers in interactive teasers that give them a sense of what’s to come. With the HoloLens set builder, users and audiences can become involved in the world-building part of the production process. This early world-building is usually constrained to the eyes of designers and directors: what we’re imagining is a backend world-building process that more people can witness and participate in.

The Set Builder has progressed to scalable, room-sized props.

And this only scratches the surface for us: we’ve got hair-brained ideas, wandering minds and crazy dreams when it comes to virtual theatre. Let’s let our imaginations take over for a minute. A virtual reality set building tool lays the groundwork for a new, non physically constrained theatre and performance landscape. Lets think about beaming actors into virtual spaces for a minute. What about immersive virtual theatre that you can participate in? Does that then become a video game? Does there need to be a clear cut difference? What we’re into is blurring boundaries between different forms of media, performance and storytelling.

So there’s a taste of some of the craziness we’re presenting at the AWE this year. We’re here to get inspired, to share our vision, and to stay on the cutting edge of AR and VR across diverse industries. Stay up to date with us as we move into the launching phase and sign up for our newsletter to get the scoop on all things Apples and Oranges Arts Innovation!