Taking a cosmic trip with We Are Stars

We Are Stars – now available on Viveport – is an immersive, 360-degree science documentary that takes you to the very beginning of time and back again. We spoke with NSC Creative, the award-winning UK developers of We Are Stars, to find out more about this time-traveling cosmic trip!

First of all, please introduce yourself.

I’m Paul Mowbray, the Director of NSC Creative and the Producer on the We Are Stars project. I conceived the idea of doing it as a cross-platform immersive film, oversaw the production and have been driving the release strategy for the VR version. We are excited to be on Viveport, it has a laser focus on VR only, so we are excited to see what the userbase think of our experience.

We Are Stars lets you “explore the secrets of our cosmic chemistry”. For anyone who missed science class, can you sum up what that means in scientific terms?

We ask some pretty big questions, like what are we made of and where did it all come from? We join the Time Master narrated by Hollywood superstar Andy Serkis, a Victorian gent with his very own time tent who whisks us off on a 13.8 billion year adventure. We follow our Victorian time travelers as they learn about the origins of atoms.

We witness the formation of the first hydrogen atoms after the Big Bang and then watch how those hydrogen atoms gather into galaxies and stars. We learn how the stars forged all the atoms needed to make life. We see the creation of planet Earth and follow the atoms and molecules through evolution back to where the time travelers started their journey, all in time for some jellied eels and roasted chestnuts!

Was We Are Stars originally conceived as a ‘fulldome’ experience? If so, were any changes made for the VR version?

The project was originally conceived as a cross-platform project. We have been creating dome films for over 16 years so have lots of experience of storytelling in 360° and frameless environments. Due to the multiple delivery formats required the easiest ways for us to achieve this was to do it as a pre-rendered experience, plus there are some shots in there that just couldn’t be done in engine at the moment.

The original plan was to generate a 16k x 8k equirectangular image per eye for 3D at 60fps so we could re-purpose for dome and giant screen from one golden, pixel busting master! Back at the beginning of 2015 we quickly realised that this pipeline wasn’t going to get us where we wanted quick enough and so decided on a three stage render process. We did the the 4k x 4k 2D 30fps dome version first which we released to critical acclaim into the international planetarium market and then followed up with rendering out the 360 video VR version which was created at 4x x 2k per eye for 3D at 60fps.

This was an optimal resolution for render time, distribution, playback and HMD resolution limitations. The final stage was the 8k x 8k 3D 60fps dome version for the ultra high-end planetarium market. We have the show playing in the highest resolution 3D domes in the world. We kept the multiple versions pretty similar with minimal changes between dome and VR. With future projects we hope to explore the enhanced field of view that VR affords giving us a full sphere to play with rather than just a hemisphere, but ironically the FOV of the current gen of HMDs is what really jumps out as you as a difference when constantly testing in the dome also – which fills our natural horizontal FOV.

We also had to consider how we wanted the viewer to be physically positioned when watching the experience. We Are Stars is a seated, uni-directional experience which encourages people to maintain a reasonably static viewing position with just a bit of head-tracking for those explorative shots. We don’t want people breaking their necks to look behind them constantly.

‘Spinny chairs’ is another seating type that can work really well for 360 video experiences, but if you want to control the viewer’s attention for the majority of the experience then assuming a predominantly forward facing position helps. The stereoscopic 3D effect is one of the main elements to create the sense of immersion, which not many other 360 videos have due to the challenges with 3D live capture. As we are a computer generated experience we can really explore the possibilities of 3D film making in a spherical environment and how this can be used as another tool in support of the narrative.

The NSC Creative team hard at work!

How did your team’s experience in developing ‘fulldome’ experiences help with developing for VR?

We’ve been using VR as a production tool for many years for the complex immersive projection environments that we work in across theme parks and museums. So actually developing for VR as the end medium was a natural and logical step for us.

We Are Stars on display in a dome setting.

We coined the phrase ‘Domography’ in the dome world to help describe the new type of film language that is necessary to compose and edit in that environment. A lot of this translates into VR much more readily than traditional flat-screen cinematography techniques do. There are subtle and not so subtle differences between dome and VR sometimes a shot will prioritise VR over dome or the other way around as we don’t always have time to create a bespoke version for every format. If we did we’d still be working on it now!

Give us an idea of what to expect from We Are Stars – what’s the time scale explored? What will we see in VR?

We cover pretty much all the time! We go back 13.8 billion years to the beginning of time and the creation of everything. You will witness the big bang, early galaxy formation, exploding stars, nuclear fusion, the formation of our solar system, black smokers, the first life deep in the Earth’s ocean and the extinction of the dinosaurs.

It’s perfect for VR! Who wouldn’t want to have our amazing cosmos whizzing past their eyes in stereoscopic 3D at high-frame-rate in 360 degrees!

We Are Stars is a 360 video experience – have you any plans for more interactive VR experiences?

We think there are all kind of new genres that are going to be born as VR evolves. We are really interested in the line between passive experiences and gaming; once the user can interact with the environment are they always going to want too? Guided, semi-on-rail experiences which are responsive to the users gaze or position are interesting, do you allow the user to know they are influencing the experience or keep it from them? We are obsessed with room-scale VR and have two projects in development which you will be hearing more about soon.

The ObserVRatarium is the virtual museum of everything where we offer multiple play modes. It’s aimed at home users, but also at museums and attractions to make it part of their exhibitions. The first play mode is ‘manipulate and observe’, where we put artefacts in people’s hands that are normally behind glass or in a far away place. The ‘Investigate and Interact’ mode brings the objects to life by giving rich dynamic info when you look at them. In the ‘Scale’ mode we give things a human scale by using 1:1 mapping. For instance, how big is a Saturn V rocket when you stand at the foot of it, or when you see the ISS within arms reach? Finally ‘Transport’ mode enables the experiential power of VR to create presence and transport people to thematic environments so they can see things in a real-world context, often as a slice of time.

The other project is a complete departure from an educational setting and is called Vestige, directed by Aaron Bradbury. It is a fictional short film which explores multi-narrative in a room-scale environment diving into the subject of memory, love and loss, using volumetric live capture to bring real people into the world but in a highly stylised, emotionally charged, art directed way. We want to offer people a profound experience that gives them a new perspective on life, love and death.

For lucky visitors to the National Space Centre in the UK, as well as seeing We Are Stars in the dome – what else can they see?

We have two vertical rockets and moon rock! If that’s not enough we have the largest digital dome in the UK, the Sir Patrick Moore Planetarium. A 17-seat 3D simulator that takes you on a ice cracking romp to Jupiter’s moon Europa. A giant Soyuz and Gemini spacecraft hanging from the ceiling. An immersive wrap around panoramic tunnel that plummets you through Venus’s killer atmosphere. A recreation of an International Space Station module, loads of other cosmic stuff and of course a shop where you can buy out of this world space gifts like Astronaut Ice Cream!

How do you see virtual and augmented reality working in museums and dome experiences?

We definitely see AR and VR working with artefacts throughout the museum. We are planning to do a bunch of cross-section views to literally add layers of context. We’ll be able to see inside our Soyuz craft and appreciate how cramped it is, look at the layers inside an Astronaut suit and add rich information that bring the static objects to life.

Fulldome technology has finally got to a point where we can run game engines in them so that opens up all kinds of possibilities for interactivity and gaming in the dome on a massive scale. It really is a version of Social VR, 200 people all in the same immersive space. We are excited to see what can be done in this environment and the continued crossover and cross pollination between dome and VR.

Thanks very much for your time and answers, Paul!

We Are Stars is now available on Viveport. For more information on NSC Creative, you can visit their website, or find them on Twitter and Facebook.