Magic Leap: bringing augmented reality to film in 2015

The Age of Starlight Promotion picture
Magic Leap technology is to be “premiered” at a UK festival in 2015, in a special film / show entitled The Age of Starlight (image: Manchester International Festival)

Professor Brian Cox may not be a familiar name to everyone, but in the UK and for those with an eye for science on television, he has become something of England’s answer to Neil deGrasse Tyson.

Professor Brian Cox
Professor Brian Cox

Cox, who played keyboards in the pop group D:Ream whilst studying physics at the University of Manchester in the 1990s, started his television career in 2005, appearing on the BBC’s science and philosophy series, Horizon.

Since then, he has fronted a range of science programmes and series, as well as appearing on chats shows on both sides of the Atlantic. He’s even  had a guest starring role in the adventures of the very master of time and space itself, Doctor Who.

Now, the BBC reports, he will be presenting in a cutting edge show / film (which he is also scripting) entitled The Age of Starlight, telling the story of the universe, intended to be one of the focal events of the 2015 Manchester International Festival. The production will also feature visual effects by Framestore, the team that won an Oscar for their work on the 2013 George Clooney / Sandra Bullock sci-fi vehicle Gravity, and will be directed by Kevin MacDonald whose films include the Oscar-winning Last King of Scotland and One Day in September and the BAFTA-winning Touching the Void.

But what makes The Age of Starlight particularly interesting is that it will utilise augment reality technology being developed by Magic Leap, the company that hit the headlines in October 2014, when it received $542 million in funding from a broad range of investors.

For those of you who missed it, Magic Leap is the company behind a headset that uses augmented reality to combine realistic computer graphics with everything the wearer sees in real time, in what the company calls “cinematic reality”. The results can be startling, going on the available promotional material: tiny elephants in the palms of your hands, dragons flying among flocks of birds,  yellow submarines sailing through streets, humpback whales floating over crowded beaches, and more.

One of the Magic Leap promotional images: a yellow submarine apparently floats down a street the Magic Leap wearer is walking along
Magic Leap merges realistic computer graphics with everything the user sees in the real world, in what the company calls “cinematic reality”.

However, beyond the stunning promotional images and video, the company has publicly revealed very little about what it is up to. But what they have shown behind closed doors has been enough to get John Markoff from the New York times very excited, and has been sufficient to get Google to lead that US$542 million (£346 million) round of investment in October, which itself came on top of an initial $50 million of funding earlier in 2014.

Given all the apparent mystery surrounding Magic Leap, Sean Hollister over at Gizmodo, decided to spend a little time digging around trying to find out more on what Magic Leap is all about.

In his article, Hollister starts out by framing something of the company’s history, revealing that Magic Leap has been chipping away at things for quite a while. In a fascinating track through the company’s history, he references their 2011 collaboration with Weta Workshop on something called The Hour Blue, as reported by Dice (see the video, below). This still appears to be around today, although exactly what it is, isn’t clear. This collaboration may have been the reason why Weta’s co-founder, Richard Taylor, opted to make a personal investment in Magic Leap during the $50 million round of funding and now sits on the board of directors.

Making augmented reality of the kind Magic Leap is trying to achieve is a significant challenge, as Hollister explains:

If you’re looking at the real world, your eyes are focusing at a variety of different distances, not necessarily on a tiny piece of glass right in front of your face. The real world also reflects a lot of light into your eyes, which is why the images from heads-up displays like Google Glass appear transparent and ghostly. Because you need to see the real world, you obviously can’t have a projector covering the front of the glasses: that light has to be bounced in from the side, which generally results in a narrow field of view.

And of course, you need some way to track your head and your surroundings so that CG objects appear to occupy a real place in the world, instead of looking like a flat image— which, sadly, is how many existing augmented reality specs do it.

Given this, Hollister reasoned, the best way to understand what the company might actually be developing is to take a look at the patents they have filed and which address such challenges. In taking this line, he’s actually following the lead set by Tom Simonite, a bureau chief at MIT Technology Review.

Investigations by both Hollister and Simonite point to a series of patents filed by the company which offer the means to overcome all of these issues, and so could enable them to produce a headset “absolutely jam-packed with cameras and sensors to exactly know where it is, and which direction it’s pointing, inside a depth-mapped recreation of the real world,” has Hollister puts it. Such a device could accurately track the wearer’s movements and eye movements, accurately projecting CG objects into their field of view that seamlessly integrate with their view of the physical world. The Magic Leap patents further suggest the headgear might be linked via fibre-optic cable to a belt-worn rendering system / power unit.

Additional poking at the patents filed by Magic Leap so they draw heavily on work undertaken by Chunyu Gao, Hong Hua, and Yuxiang Lin (Augmented Vision Inc. / the College of Optical Sciences University of Arizona), with Chunyu Gao in particular credited as the author of the Magic Leap patents. These further suggest that the company is possibly aiming to package the technology in either a “regular” spectacles / sunglasses style of form-factor, or possibly in a “wraparound” style of form factor.

Patent filed for Magic Leap by , draws extensively on one of earlier patents for augmented reality headsets styled in both
Patent WO 2014043196 A1, filed for Magic Leap by Chunyu Gao, draws extensively on one of his earlier patents, filed as a part of his work with Hong Hua and Yuxiang Lin, for augmented reality headsets styled in both “regular glasses” and “wrap-around” form factors (US 20120162549 A1) – click for full size

In his article, Hollister highlights a few additional points of note:

  • The company is already working on a tactile glove which could potentially allow users to interact directly with the computer-generated objects seen in the headset
  • The headset apparently includes an ability to switch between a transparent augmented reality mode and an opaque virtual reality reality mode.
  • The company has applied for a trademark for “a web hosting platform for users to participate in and contribute to the creation of virtual worlds.”

Technology “Premiere”

So what about The Age of Starlight? According to Professor Cox, it will form the premiere of the Magic Leap technology, which he believes will be “transformative”.

During the 18 days of the Manchester Festival up to 50 people at a time will be able to experience The Age of Starlight, which, as Professor Cox explained to the BBC:

Will invite an audience to face the biggest questions about our existence, our place within the universe, and the origin of our universe itself, using new technology to create an experience beyond anything that has been possible before. Whereas in television documentary you’re attacking these ideas from a scientific perspective, in this case it’s much more of an emotional experience.

He also hopes the film will leave audiences slightly disoriented, saying, “It’s supposed to be beyond what you would get in a documentary. It’s not a science lecture, it’s not a science documentary. It’s a piece of art. I want people to stagger out and have to have a sit-down for a long time before they go home.”

I’m quite looking forward to popping up to Manchester and seeing it in July 2015!