A few weeks ago, I came across a great Rolling Stone article about a company named Magic Leap. I was excited because up until that article came out, it was unclear what the company even did.

Which was strange, because the company itself had raised almost $2 billion in financing, and is valued at around $6 billion. To put that number in perspective, the Dallas Cowboys, the most valuable NFL Franchise, are “only” worth $4.8 billion.

So, what does a company worth $6 billion do?

You wouldn’t have much of an idea just by looking at its YouTube channel, as there are only six videos on it, showing ethereal, and sometimes impossible-looking digital objects overlaying the real world. Prior to about a month ago, if you looked at Magicleap.com, you wouldn’t have found much more information beyond the claim that they were building a mixed-reality headset.

That all changed on December 20th of last year, when the company invited Rolling Stone to try out their tech and revealed that developer kits would be made available in early 2018.

Boom! Suddenly, we’re getting somewhere.

So, what exactly are they making?

A mixed-reality headset.

What is a mixed-reality headset?

Let’s put it this way: virtual reality is replacing your entire field of view with digital objects. Augmented reality is viewing digital objects overlaid on reality through a screen of some sort, like a smartphone. Mixed reality is viewing both digital objects and the real world through your entire field of view.

So, I wear a headset, and I can see digital objects around me in real life?

Yes!

What’s the purpose of that?

I’ll get to that in a minute.

Well, how does it work, then?

I’m glad you asked!

Magic Leap’s headset utilizes object-tracking, analysis of the light-field, and an augmented reality display made out of silicon photonic-lenses that add digital objects into the world you’re already seeing.

For better or for worse, Magic Leap’s CEO, Rony Abovitz, does most of the PR for the company. And I can’t tell if he’s just being cagey, trying to protect proprietary information, or is just not that great at explaining things in a simple manner. He did do a lot of the initial research and development himself, so it’s entirely possible he’s being overly technical without realizing it.

After doing some research, I think I can explain how it works.

First, object-tracking. The headset records a video of what you’re looking at through a camera mounted on the headset. It then runs the footage through software that detects objects in the scene, like walls, tables, the floor, and even your hands. This enables digital objects to interact with the real world, like setting a digital vase on a real table, bouncing a digital ball off the floor, or using your hands to manipulate a digital object. There’s also a sensor on the headset that tracks the motion of your head so that objects stay in the correct place even if you move or look away.

Second, the light field. A light field is a description of where light is in space, its intensity, and what direction it is headed. In fact, it’s the technology that lets iPhones refocus pictures after they’ve been taken, as shown prominently in this ad. Neglecting the light-field is part of what makes augmented reality look fake. Since digital objects don’t react to real environmental sources, they seemed unnaturally bright, or dark, and don’t cast shadows in the correct direction.

Rony Abovitz is really proud of what he calls the “digital light field,” which he makes sound really complicated, but I think essentially boils down to this: software in the headset is able to interpret the organic light field, and use it to light digital objects so that they look like they’re correctly lit by the environment around them.

And, finally, the lenses that utilize the principles of silicon photonics. The lenses are made out of lots of microscopic bits of silicon that under normal circumstances are clear so that you can see the real world through them. They’re then stimulated in places which causes them to produce the digital images, blocking out the background light that would otherwise have passed through the spot.

You don’t really understand silicon photonics, do you?

I really don’t. When the Wikipedia page starts to talk about nonlinear Schrödinger equations and Brillouin scattering, I know I’m out of my depth.

So, it’s basically black magic?

Possibly.

But, how it works isn’t the most important thing about it. Ultimately, the magic in Magic Leap is dependent on how well it works. And, according to the journalist who got to try the tech firsthand, it works pretty well. In my opinion, the most promising detail was that when he got closer to things he was able to see more detail, much as we would if we moved something up to our eyes in real life. This is probably an effect of the digital light signal that the CEO loves so much.

But, even if the tech still has a way to go, it has the potential to one day be just as transformative as the smartphone was.

For example, when the tech gets good enough, we’ll wear it all day, every day, just as we’re now nearly inseparable from our smartphones. Furthermore, screens will become obsolete. With tech like this, you could use any old wall, or surface for that matter, and transform it into a digital display. In fact, you don’t even need a surface. You can just plop a screen in midair, and adjust it with your hands until it’s floating just where you want it—something I’ve always wished I could do with recipes while I’m cooking.

Smartphones will be the next to go, replaced by software that emulates it. A microphone in the goggles will suffice for talking, while the number pad will float in front of us, seemingly hanging on thin air, while elsewhere we can type on a keyboard that doesn’t actually exist, but sure looks like it does, searching for that particular contact whose number we can never remember.

And the rest of the smartphones’ functions will be superseded as well. Games, checking email, watching videos, scrolling through social media will all be made better by the digital screens that we plop around ourselves.

But, beyond that, advertising will be utterly revolutionized, bringing the targeted ads of the internet into the real world. Advertisements will just be little blank spaces with bits of code asking Google to serve up a targeted ad to you via your Magic Leap headset, meaning that fifty-thousand people might look at a given spot in a day, and each sees a different ad.

Furthermore, decoration in public spaces might go the way of the dinosaur: why paint the walls, hang paintings and art, and keep expensive fountains running, if you can just digitally create them instead, and no one would ever be the wiser?

That being said, it’s still a tech in the early stages of its development. It’s not yet revolutionary—but it one day could be. Ultimately, mixed reality has the potential to bring about the kind of future we see in science fiction, where humans interact with their computers without ever using a mouse and keyboard, merely using voice commands and gestures to get their tech to do things.

With the rise of Siri, Alexa, and Google Assistant, the voice command part is already well underway. What remains to be developed is the gesture control, and Magic Leap might be the company to get us there.

If you liked this article, you should check out:
Gunmantics
The Anatomy of a Political Joke
Schools Shouldn’t Sponsor Protests