More Intelligent Tomorrow: a DataRobot Podcast

Seeking Deeper Connection through Augmented Reality - Nicolas Robbe

June 24, 2022 DataRobot
More Intelligent Tomorrow: a DataRobot Podcast
Seeking Deeper Connection through Augmented Reality - Nicolas Robbe
Show Notes Transcript

Nicolas Robbe is the CEO of Hoverlay in Acton, Massachusetts. In 2016, he left his job as Chief Marketing Officer at Dynatrace and re-engaged with one of his passions—augmented reality. The space was going through a profound transformation, and his goal was to create new technology and cool new tools for the public. 

In this episode of More Intelligent Tomorrow, host Dave Anderson talks to Nicolas about his work bringing augmented reality to screens. Nicolas describes how AR can bring deeper meaning to our experiences, associated opportunities and challenges, and the need for technologies that have a positive influence—promoting empathy and understanding instead of distancing people from one another.

AR, he says, has the power to activate cognitive circuitry to convey emotion, promoting community connection and generating empathy and understanding. 

“As we live our lives, we're seeking meaning, looking for a deeper connection with the places we go and with the people around us. This medium is designed extremely well for that task, the same way the web is well-suited for sharing information and transacting.”

Nicolas has always had a passion for understanding the nature of interactions between humans and systems and reducing the cognitive load on people when trying to transmit information to them. 

He describes multiple layers. One involves figuring out how to use visual metaphors to communicate and bring information into the physical world through a screen. Another involves getting off the screen to where the camera becomes a sort of browser—something a user can place content into just as they place images on a webpage. 

He explains how it works from a user perspective: They hold their phone, open the app, and see the video feed go through—as if they were taking photos. However, with AR, the camera is able to insert pixels and content into the field. 

“The magic is that the user creates the illusion of presence to help them feel the content. There are many techniques for creating the illusion so the brain accepts that it’s real. The user holds their phone, sees the content, and plays along. The content is usually meaningful to their location.”

Asked how virtual reality compares with augmented reality, Nicolas says, “VR is about taking you away from your reality and giving you, perhaps, a better reality or an experience you couldn’t have in the physical world, with everything that entails. AR, on the other hand, tries to reconnect you with your physical environment. It could be a park, a building, (or) a kitchen. But it starts and ends with the location—the context—then augments it so your experience of that moment and location is more meaningful, more fun, more engaged. It's trying to connect you with the moment and the space versus taking you away from it.”

Listen to this episode of More Intelligent Tomorrow to learn more about:

  • How augmented reality deepens our understanding of history
  • How AR can be used to create connection and understanding
  • How context affects the experience and retention of information
  • How augmented reality is different from virtual reality
  • How ethical concerns relate to AR, including fake news and virtual trespassing



NR: So, it's growing exponentially fast and that's what's so fascinating is that the barriers to publishing are falling, which were, I think, the major barrier to people being able to take advantage of that technology. The tech in itself has progressed is ready. The core tech. What was really missing is how do you participate? How do you go and augment murals? How do you do a tour of your city? That's the barrier that we're attacking now and that's falling extremely fast.”

[INTRODUCTION]

[00:00:35] ANNOUNCER: On today's episode, host Dave Anderson sits down with Nicolas Robbe, CEO of Hoverlay. 

[INTERVIEW]

[00:00:44] DA: Nicolas Robbe, welcome to the More Intelligent Tomorrow Podcast. How are you, good sir?

[00:00:50] NR: Thank you. I'm doing great. How are you doing?

[00:00:52] DA: I'm really good. A disclaimer alert for everyone. We used to work together and you are responsible for sending me from one side of the planet to the other on a world tour. Thank you.

[00:01:04] NR: You're so lucky. Now, you do appreciate you live in Australia, thanks to me.

[00:01:06] DA: I do. Well, we did. We worked together at Dynatrace. I was only reminiscing the other day about the fact that you nearly convinced me to move to Paris instead of London. We were laughing about what life would have been like if that sliding door moment had have happened, because you're French, right?

[00:01:30] NR: That's right. Nobody's perfect.

[00:01:33] DA: So, where are you in the world?

[00:01:34] NR: I'm in Boston right now.

[00:01:36] DA: And what are you doing there?

[00:01:37] NR: After Dynatrace, I decided to jump into one of my passions, which was augmented reality, right at that moment where that space really went through a rather profound transformation. This space has been around for a long time, for about 20 years, computer vision and had different names. And then Apple and Google decided to jump right in 2017, with their own essentially, middleware for augmented reality. So, it was right at the moment where seeing that coming, I thought, this is the perfect time to now re-engage in that space and bring some new technology and cool tools for the public.

[00:02:25] DA: So, 2017, we're talking sort of four or five years ago, but you said you've been interested in it for a long time. What was your interest? And then what did you go off to do? Because you went from – just to give people context, you were one of those tech execs, that was very successful CMO, that decided to go and follow a passion in a startup. There's probably a lot of people listening, that go, how you gave up a really solid income with a young family to go and follow your dream. Are you crazy?

[00:03:03] NR: Well, you there are different ways to look at that. I came from, like you said, the tech background. I mean, my field of study was AI back in mid ‘90s, where this was actually very hot before it went through some kind of near-death experience, late ‘90s, early 2000, nobody wanted to even mention AI. It had a lot of bad press and failures, and so on. Then it went through this revival later on.

But pretty much my entire career, I've been dealing with different fields of computer science that had some tie to AI, even though it was not called AI. So, I've always been in that field and I've always had a passion, again, from the beginning around how people interact with systems. What is the nature of interactions between us as humans and software, computers, machines? That has different kind of layers. One of them being how we use visual metaphors to communicate and basically bring information into our physical world, through a screen, for the most part. That went from alphanumerical screens to Windows, graphics, and so on, and limited 3D, and now getting off the screen. So, where essentially the camera becomes a browser in a way so something you can program into, and you can place content into just like you place images on a webpage.

It's always been a very profound center of interest for me to understand how the brain relates to this information in how we take that information and what are the ways we can reduce the cognitive load on people when we’re trying to transmit information to them, through systems. So, long story short, we only live once. Being an executive seems very glamorous and when you're in it, maybe it's not that glamorous. It can be sometimes a lot of stress. There are always two sides to every job, every story. And so, I felt this was the right time. Timing is very important when you're trying to launch something, so I felt this was the moment where I needed to jump in.

[00:05:15] DA: Yeah. So, what you're doing, so people have the context now is you bring an augmented reality to screen so you can start to place a virtual reality. You're merging these worlds of a physical world with a digital world with an application called Hoverlay. Dod you want to explain sort of how that works and what those use cases are?

[00:05:37] NR: So really, from a user perspective, you're looking at it from the experience of somebody consuming the content. They're basically holding the phone, they opening up an app, which is essentially like your phone camera, just very simple. You see the video feed go through just like if you were taking photos. However, that camera is able to insert pixels, and content into the field of view. The magic is that you create the illusion of presence. You feel this content is there. There are lots of techniques that are used to mimic the real lights, for instance, on the inserted objects, or if you have a reflective object, like a metallic object, which is a 3D model to create a reflection map that mimics the actual field, the actual color pattern that's around you, so that your brain is tricked in believing that this content is there.

Your start, when you're experiencing this, your brain very quickly accepts this illusion as real, because you're using other form of circuitry, using your lizard brain to consume that content. So, that's the user experience. You're holding your phone, you see content and then you’re playing along. That content usually is meaningful to your location.

I’ll give you a very simple example. There is a very famous monument in Boston called the Shaw Memorial. It's a Civil War monument and it was created to honor the first African American regiment that was raised during the Civil War. If you pass by this monument there this just like any big walls, monument, and you don't really notice I understand what it means. But the National Park Service and the conservancy that is in charge of the Boston Common decided that they were going to play some holograms around this monument to explain the story of the monuments.

So, as you're watching, as you're there, right by the monument, you're seeing life size people telling you the story of the regiment. Telling you about how groundbreaking this was, from an artist perspective to put African American soldiers in monuments. It's no big deal now. But if you go back when that was done, it was a major departure from everything that was done in terms of public art. And then somebody tells you about the political role of this monument. How it relates to today's issues around racism and certain social elements that are still battles to this day in the United States. So, it's a very much an enriched experience that you have at this particular spot. As you’re actually looking at the monument itself, and holding your phone, you're getting this additional information around you to relate, understand, bring meaning to your experience.

[00:08:27] DA: It's a really great example for me personally, as well, because I used to live in Boston and I took my mom and my two kids who were six and eight for a walk around Boston Common and did all the Civil War – sorry, not the Civil War, it was the war of independence against it. When you throw all the tea into that, well they throw all the tea and said, “We don't want to drink tea anymore. We're going to be the world's best coffee country”, which didn't quite eventuate. But you get what I mean of sort of the birth of modern America.

I'm walking around Boston Common and there's all these monuments and I have a physical map, and I'm reading out of a book, and I'm staring at this monument, which may or may not look anything like what it used to look like, I guess it sort of does. And you're reading out to the kids and the kids are just staring at you like whatever, dad. Immediately, when you were telling that story, I was thinking the kids now and people already have their phones in their hand. They've just got them facing the wrong way. They're facing themselves and going look at me standing in front of this, as opposed to the sensory overload you can get from pointing a camera and going back in time and going, “Oh, my god, it used to look like this”, and casting the people or the image into that scene would be really powerful.

What you said also, was about bringing that empathy, so people can get a level of understanding whether that's like accepting traditional land holders and things like that, as I think about like Australian heritage and I think about well, this is what the land used to look like, and these are the traditional people. It would bring a closeness of people who don't fully understand closer to a source to really picture it and get that visual. It's really powerful.

[00:10:11] NR: Yeah, absolutely. Again, it's because I believe that this medium has, and like the web, it has the power to unleash emotions, activating different circuitry, cognitive circuitry and conveying different forms of emotions. It's difficult until you experience it to realize what that means. But it's a bit like VR. Unless you put a headset on your head, you can explain all day to someone what it means. But once you have it, you're like, “Oh, my gosh. This is different, I can see this is – and it's hard to put into words.” Same thing with augmented reality, when it's done well, and when it's actually meaningful.

I'm really insisting on that point, because I believe that the opportunity here is not necessarily entertainment, per se. It's bringing meaning to experiences. I think we all – as we travel, as we leave our lives, I mean, we're seeking meaning. We're looking for a deeper connection with the places where we go, with the people around us. I feel that medium is, that medium is actually designed or is actually extremely well suited for that particular task. The same way the web was very well suited for sharing information and transacting. The web, as we know it today. It was designed to share textual, mostly information, and then it evolved into what it is. But it's not extremely good at conveying emotion. I mean, the best way you can convey emotion, maybe is watching video on YouTube and that's pretty much. But it hasn't reinvented the way we are emotionally touched and exposed to content.

Those new media, this new era of immersive media that's upon us, is actually very good at that. It might be very bad at actually doing transactions, but it's extremely good at – it's the web of emotion. The web of empathy, of meaning. It's not a web of transactions and data. So, I feel there's a new space, and it's a little bit like when cinema was invented. At first, what did people do well? They put people on the stage. They were trying to redo theater scenes, and people would come in and out. They would enter the stage and so on. But everything was fixed. It was a fixed frame, and you were just trying to move what you knew how to do, into this new medium. I think we've arrived at this moment, we're starting to understand that this medium is actually very good at other things. It's not just about doing what you would do in entertainment on the video and do it in AR. It's about something else. I think we're – that's what's so exciting right now is because we're starting to realize the power that this medium can have in terms of community connection and understanding and generating empathy and emotions. So, it's an interesting moment in time.

[00:13:09] DA: It's a really interesting moment in time, because we're also at the moment of generation of just junk scrolling, clickbait, and irrational decision making. We've seen politically, and socially, just a generation of people making decisions just on a whim. They just thought, let's just go that way because I don't like that way. It's not an informed decision. I think what you're suggesting, and what you're saying is, if you're going to shape the future of a city, of children, of a society, you've got to really put things into context, and you've got to help them learn. This particular application can go deep enough for someone to be able to say, “This is what a city could look like. This is how you can better understand our history, so you can better shape the future.” It's these powerful applications. It's not, I wonder what a table would look like in this room, which is definitely a use case.

[00:14:08] NR: It is a use case. Absolutely. I think you're spot on. I think what we are starting to realize is that it is a local, community centric medium, as much as it is – the web has been – I mean, for the most part, is a global media. You put something so that you can get as many eyeballs no matter where almost, and you're trying to go for the masses. Everything's open 24/7, accessible anywhere, and you want it to be as quick from accessing it from Australia as France, as Boston.

Here, I think it's actually a little bit reverse. It should be like FM radio when that started. It was this weird thing, it's okay why would I have a radio that's local to my city? And then also realize, “Oh, okay, well, it's different kind of shows. People can call in. Participation is a sense of community.” There are things are specific, your local sports. So, it starts to feel a certain void in terms of how people interact with one another in terms of local news. Things like that.

So, here, I'm sensing a very similar, very similar aspect. It's really focused around how do I help neighborhoods tell their stories? How do I help people maybe see things they're not seeing in their neighborhoods? Give you an example, right after the George Floyd events, there is a high school in Cambridge, that has a STEM program. So, these kids, they’re part of a program called Innovators for Purpose. Again, high schoolers, they decided to create an exhibit in augmented reality on the Cambridge lawn, to tell their view about the George Floyd defense, what it meant to them as teenagers, and their perspective about social racism in Cambridge, and systemic racism. Things like how redistricting affected COVID cases, and things that are actually very interesting. But they’re not for anybody but people from Cambridge. That's really the audience. And these are kids you may run into when you're just walking down the street. But it gives them a forum to actually communicate with local people about their perspective, when you think that Cambridge is this over the top, very modern, very affluent city, that, in fact, there are issues and they are kids or teenagers that are feeling it.

I feel that this is a very powerful vehicle, as you're watching these exhibits, these augmented reality exhibits, and they're recorded themselves on green screen and you can see them talk, you can hear the voice, and they’re telling like it is. It's not a Hollywood show. It's our movie. It's a very kind of – it's very much homegrown, kind of grassroots type of exhibits, but they're taking over an area that's very central to Cambridge and telling the story. Again, this is new, and I feel there's a lot of potential around that way to empower local groups to tell stories to people around them.

[00:17:09] DA: So, what you're doing is you're bringing virtual reality into the real world. You're bringing pixels into the real world. If you think about what's going on now with Meta, they decided, well, let's just create a new world, like a fantasy world is, sort of like a digital Disneyland. That is a far stretch from inserting pixels into a real world. What's your take on this digital Disneyland?

[00:17:37] NR: It's a great point. The technologies are the same, very similar. It all takes its roots in gaming technology, a lot of computer vision, understanding spatial placement of your body and space to be able to kind of augment your vision and have the sense of immersion, or most an extension of your physical movements with content. So, it's a very similar field. But you're right that they feel at the opposite end of the spectrum. What VR is about is, it's about taking you away from your reality, and giving you, maybe, a better reality or experiences that you could not have in the physical world, with everything that that entails. Okay. I don't need to go into more details about what's there. But the good, the bad, the ugly, you're going to see it in the metaverse. It's just bound to happen.

On the other hand, augmented reality is trying to reconnect you with your physical environments. The starting point is the physical environment. It could be a park, it could be a building, it could be a kitchen. But it's starting and ending with the location, the context, and then augmenting that so that your experience of that moment in time, a dark particular location is more meaningful, is more fun, is more engaged. So, it's trying to reconnect you with the moment and the space versus trying to take you away from it.

I feel that this is fundamental purpose, I would say. I think they could they could both coexist, but they serve very different intents and what I've seen, it's almost, when you start something that new, you don't know the markets does not exist. There is no market for AR. I mean, outside of maybe putting furnitures in your kitchen or your living room, and so on. But for the most part, you push it out there and you see where you feel attraction and where people are actually saying, “Hold on, we want to take this and move it forward.” And what we're seeing is a lot of storytelling. A lot of how do I take that story and bring it to life so that others can actually understand the depth of it. Understand and have this connection with this space, with this with this street, with his neighborhood and its history, with something that's going to be happening here in the future.

It's very interesting. The group, we’re seeing a lot of cities. We’re seeing park owners. People in charge of botanic and park. People for which the web is there, but it's not that interesting. All of this, this is their web. Artists, public art artists, people doing murals, people want to tell stories around their murals. Campuses. Today, I'm at Northeastern University preparing their commencement week, next week. All the deans of all the colleges, just in the past week, recorded themselves in front of a green screen to create virtual themselves on the campus so that they can be with the students and they can have photos and be with them. It's this way to think about – it's not for them to be halfway around the world. It's for this event, it's to create more connection, deeper sense of connection between the deans that have very limited time. All these students are going to be in the families, they're going to be around the campus. The body language, the anxiety, everything that come through without, that you cannot put in words, because we're again, like as you very well know, we process a lot of information, nonverbal information and it's amazing that, again, this is something you get to experience. But seeing someone appear in front of you holding your phone, in life size, your brain is like drawn into that experience in a way where your brain wants to believe in it. Your brain is not trying to take you away from it, it wants to participate. It wants to play the game and I feel this is what's so interesting about this and so experimental about it, that that is wow, this actually works. Even for me, who has been in this now for years, every once in a while, we do something like, “Oh, my gosh, this is amazing.”

[00:21:53] DA: Yeah, I think of schools. I just think a school. Sorry to cut you off. But I think of the schools and you go, “We're not that dissimilar in age”, and probably people listening are probably pretty similar that they didn't have a lot of technology, when they were learning. Even the kids today, they've got the internet, and they've got iPads so they can connect. But it's not immersive. It's not – I remember sitting and reading a book about war and I didn't really – you had to visualize it in your head. You go, “I don't really know. I've seen some things on a TV.” And you’re sort of reading, trying to piece it together.

I remember actually going to Reims in France and it's where the war was unofficially signed at the end of the war. I sat in Eisenhower's headquarters, which was an old school and they've retained that as a museum and we sat in the room where they basically walked in to sign the end of the war. The thing that made that spine chilling, was that the room has been kept exactly the same, and they were filming in the room while we were in the room, sort of. So, you actually could get a visual of well, they're the tanks coming down the street. And then they did the camera angle of like you looking to the side. I was like, “Okay, that was the troops walking past the door.” Okay, I'm now sitting in the room where this happened. I just remember the moment, every hair stood up on my arm and just went, “Oh, my God. This war isn't something that was on the TV. This war was in this room.” And then it made me go, we as a society, obviously, it's so ridiculously powerful, right?

We've got the context now and I think everyone now can appreciate, “Holy crap. This is – you want the kids to learn properly about things, make them part of the world in which this thing happened and you really educate them.” What I want to know is, what do we need to do to get to that? Because we're still in a world today where the kids still got amazing technology with iPads and they can call other kids around the world, and they can watch TV and they can have interactive whiteboards, but they're still not bringing this context, this history into their world. There's huge potential. What do we need to do to close that gap?

[00:24:16] NR: Well, I think, so what we've been doing on our end, kind of our vision was to say, alright, there's this new medium coming up, extremely powerful, but inaccessible to most because the typical workflow to produce an output required development, required coding and gaming skills, and so on. My belief was that we could actually make it as easy to publish content in the physical world as it is to upload a video on YouTube. You remember, it's not because of YouTube, that we were able to stream videos on the web. I mean, you could have videos on the web way before YouTube, that wasn't the problem. The problem was, how do you and me and everybody else who cannot go and ask someone to set up a streaming server and don't know about codecs and this and that, how do you do that?

So, YouTube was really an empowerment not so much from the streaming aspect, but more from the ability for the Average Joe, like me, to go somewhere, load up whatever format, not knowing anything about compression, and you get a URL back that you can send to your friends. I mentioned earlier, Hoverlay and kind of the user experience at the very end of the chain. But really, the big breakthrough that we've been able to make is actually the workflow for publishing content that you can, right now, go to Hoverlay Spaces, which is the name of the publishing platform, pick a location in the world, in Australia, somewhere around your house, the next street, whatever, take a photo, take a green screen video of yourself, and place it there. From your computer, you have a way to go through this workflow without any coding, any understanding of the technicalities behind it, and how you turn things into 3D and how you turn them into AR and you just publish them just like think of it as you were publishing your own Pokémon, location you want, and you choose what the Pokémon is. If it's you, if it's a piece of content, and so on.

And then you get a link back that you invite people to go experience on your channel, on your augmented reality channel and they walk and they get that content, they get a map to know where to go, and they get that piece of content in the location that you've picked. Just to go back to your initial point, I think this was – it is a major breakthrough and that's why we're seeing a lot of interest from universities right now and even high schools. I mean, tomorrow morning, I'm going to high school in Concord. They’re celebrating their centennial, and they've done a tour of the CD with the kids. The kids have created the content, the students are creating the content, high school, by themselves. They've created two tours, 20th century, 19th century, and you can go around the city and they work with a local museum. They scan, I believe, the 3D scan. I'm not even sure, exactly, everything they've done. But I think it's 3D scan some historical pieces that they've placed. They've recorded green screen of stories about buildings. And that's a way to learn history, because now you're in this – you're telling the story. You're out there.

So, I feel these are the barriers are falling very, very fast. They used to be significant, I would say even 12 months ago, this was not something you would even consider. Now, 12 months later, you have a high school that is doing a tour and on their own. It's growing exponentially fast and that's what's so fascinating is that the barriers to publishing are falling, which were I think, the major barrier to people being able to take advantage of that technology. The tech in itself has progressed, it’s ready. The core tech. What was really missing is how do you participate? How do you go and augment murals? How do you do a tour of your city? That's the barrier that we're attacking now and that's falling extremely fast.

[00:28:20] DA: Is there a saying the best way to learn is to teach? is that an actual saying? Or did I just maybe, party made it up. But it makes sense.

[00:28:29] NR: Not that I would know.

[00:28:30] DA: It makes sense to me, because if you need to learn something, you got to have to learn it well enough to be able to teach it. So, it's a forcing function. I was just thinking through the kids. Yeah, they're being forced to learn the history so they can teach the history. And then I had to go to this school play the other night and everyone said, “Oh, it was amazing. It was so good.” I was like, “Yeah, it's pretty good.” But I'm like, I'm sitting there and they're going through like some history thing. I'll admit, I was browsing my phone a little bit because the football was on at the same time.

But I was also thinking through, it was really hard to get everyone together because of COVID. But people were wandering the streets. I'm like, this would have been the best thing for kids to be able to film themselves on a green screen, and then people can go and walk around the city, and they can learn different things from all different parts. And the kids in the end, get an output that is far better and far more exciting than what they would – if they just I mean, yeah, you can stand on stage. You'll get some rounds of applause at the end. Sure. But this is still like, it's really geeky cool. Every kid, they're learning both digital development and augmented reality at the same time as learning history. It's new tech mixing in with history, and it's exciting and it's exactly what the curriculum should be designed for, to inspire kids to think about something radically different in the future. How can we apply this technology to something else?

[00:29:59] NR: Right. There’s another side to this which we also are just starting to really understand better, but it's retention of information. It's always memorization techniques where you walk yourself through a space, and you're memorizing words in a building, and so on. These techniques that mimic the way our brain functions in a physical environment. There’s a super interesting study that was done by MIT a few years back where they took two groups of students, and they gave him 30 years of Super Bowl winners. Okay, you're 1967, ‘68, and so on. I don’t remember the years, but you get my point.

So, same data. One group took the list and had to memorize the list. Another group had an AR experience where from the subway to the Media Lab, they had placed in order an icon that represented the winning team. So, the Packers, it would be somebody holding packages, and the Jets would be a big jet. So, you get out of the elevator, and you see this big jet, and so on, so forth. They said, okay, well, we take these two test groups, and we'll see what happens. So right after, if you see retention, right after you've done the exercise, the results about the same. Group A is about the same result of group B in terms of their ability to memorize the results.

But if you wait 90 days, the results are completely different. The group that learned with the list, their retention completely collapsed. The group that had done the AR experience, their retention remains almost the same at around 95 percent retention. I don't recall the exact number, and I'm sure we could maybe put the link to the actual paper, in copy somewhere of the podcast here or email us if you're interested to know more about this. But it was a fascinating result, that the fact that you had experienced the contents, the information this way, in context of a certain location, that your brain was able to retain that information effortlessly over a long period of time.

I feel there's something there, going back to your point about history, that I feel is, you're tapping into what your brain does well without effort. And so that has also benefited from that perspective, and trying to help us learn better by utilizing the brain the way it functions best.

[00:32:34] DA: Every company should have this, because I don't read the instructions on any instruction manual, despite the fact that there are warnings all over the box going do not operate this thing without reading the instruction manual. My automated lawnmower, as a perfect example, which did run over one of the cables the other day and cut the power off to the house. But that's a completely separate story. I'm sure I probably read that also posted bury the cable a little deeper than what I did. But I can immediately picture things that are quite technical, you can hover your phone over it, and someone would just pop up and basically go, “So, if you want to increase the volume on here, you just do this and you can push this to do that and do that to do this.” And it gets rid of 45 pages of black and white text that no one is ever reading unless they're crazy. And bring this improved instruction manual with an actual person to be.

I mean, I know that's a really simple use case. But then they also I'm thinking like advertising. You can bring advertising if you're reading something. I nearly said magazines. I can't remember the last time I bought a magazine. When? Why are you talking about magazines? I can't remember the last time you read a magazine. But yeah, you came to my point on just like you can bring – photos, you can bring photos to life. Is this like part of – have you seen recently people saying that with artificial intelligence, they can sort of bring people to life? But imagine, with Google Photos, there are often like, there are videos of scenes, and then you might have a photo of that particular scene. Imagine being able to like go into your house and then just hold the phone up and go, “Oh, yeah, let's have a look at that day.” All of a sudden, it pops up and, on your phone, and you got a memory of that day. Old people who could like – it's not just for the kids. It's for the elderly as well. Oh my god, I'm rambling, but I'm getting really excited by it.

[00:34:29] NR: I think you're onto something because again, AR is the delivery mechanism for a lot of that content. It's the delivery system that will make you believe in what you're saying. Now, upstream from that, you have things like deep nostalgia, which is kind of a deep fake that he's able to recreate out of a photo, a very realistic posture, movement, eye movement of somebody and you combine that with a full size rendering of that person and you put them in your room and I tell you, it's going to feel – it's going to feel the person is there, in a way that is difficult to explain. In a way that you cannot get again on a YouTube video, because your brain is not participating. Your brain is separating from the content.

[00:35:19] DA: So, my next-door neighbor, we've got – this has got me blown away now, because I've got this idea where I know that this one political party, if they apply this, they would win the election easy. If you could apply this technology very quickly, because like all over us, we've got a federal election coming up. If you follow Australian politics, it's basically like, which moron are we going to elect into the arc, into power? They're both basically, the same people and whichever spin they can get. I mean, politics is the same all over the world, right? Well, we go into the US, history of politics, because it's very polarizing.

But yeah, you get my point. So, my next-door neighbor has a picture of this lady, and she's running for whatever seat we're in. Because I'm new to the suburb, and she has three words underneath it. I stand for –you've worked in marketing, they're like generic tag lines of like, “I stand for this”, and you're like, “Really?” Your banner is going to do that, just a picture of this head is going to make that difference. She apparently stands for the environment. But she also has a car driving around constantly. It's not an electric car. Advertising her head constantly, which was I'm sitting there eating breakfast going, the irony of this isn't lost on me.

But imagine, you want to go – actually, I do want to learn a little more about what this person stands for and I hold my phone up over her picture and she stands there and she tells me what she stands for. If she was doing that, and the guy –

[00:36:43] NR: She’s in your living room, telling you their story.

[00:36:45] DA: Yeah, telling me a little bit about it and I'm in control of it, and I’m like, “Nah, I’m done with you.” I turned her off. But at least then, I've got an experience where the politicians on Election Day will stand at the school, and they'll walk up to as many people as they can to try and influence them to vote for them at the very last minute. You're talking about school deans going around and duplicating themselves a million times over, so every kid can get an experience with it. This for politics would be exceptional.

[00:37:15] NR: I believe so. It's a medium that is, again, it's in its infancy. But again, go back to emotions. I mean, you vote for somebody you trust, you feel comfortable with. What about seeing someone in front of you? That changes your perception of that individual. Even if it's through an augmented reality version of that person, the fact that they're there, they’re in your kitchen, around your furnitures. They're standing up and opening their arms to you. Your brain is going to process this in ways you don't even know yourself and you got to make good use of that. But my point is that it's a very powerful way to touch people and to break those barriers that we have.

[00:38:03] DA: But this also has the potential fake news, now that I've said that.

[00:38:05] NR: Of course. So, I think like everything else, there's a lot of things to pay attention to, maybe even before fake news is utilization of space. Who owns the virtual space? I can put something on your lawn. I can put something on the White House. Okay, who owns that virtual space? Yes, it's like radio, because it's opt-in. So, same thing. I mean, radio waves, they go through buildings, they go through the White House, you can say whatever you want on radio, and go through buildings and so on. Unless you turn on the radio, you don't hear it. And the same thing here. Unless you tune in to those channels of virtual content, you're not going to be able to see this content or experience that content. But it raises all kinds of questions around utilization of space.

If you're a big brand and you want to launch let's say you're a big shoe brand and you want to launch a new shoe in Central Park. So, you're going to place this giant version of LeBron dribbling and holding a shoe in the air, 100 feet in the air. You have to be there to experience it. Well, is that an okay use of land? It's a park. There’s no fee. You can use the space you want.

[00:39:21] DA: You’ve ruined my brain this morning. I can imagine, for some reason I was pitching, you said the White House. So, everyone's standing in front of the White House with their phones up already. What's stopping someone from uploading an image of like a political image that's maybe even negative of like war criminal or like environmental criminal, or whatever for the opposition political party? Is there a way – do you have to copyright the image of the White House to stop people doing it? Or you haven't reached a tipping point yet where there are morons like me, who’ll find opportunities to do really silly things, which I'm not going to. But there's part of my brain that goes, “That would be pretty crazy.”

[00:40:08] NR: It will happen. There is no way you can stop that right now. Because this image – in fact, you don't even need an image of the White House. You can place it in relation to wherever it's pointing to White House. Instruction might be okay, hold your phone toward the White House and press go. And then when you press go, the fence is that I don't know, 100 feet from the White House, you're going to play something on top of it. That's it. You have spatial arrangement. The phone is able to understand space, understands measurements, it understands distances. So, your phone is capable, you can tell your phone play something 100 feet from me to my right, floating to my left. The creation of those experiences are blended between the physical world, and essentially kind of a gaming scene. Think of it like a gaming, technology element, which is very advanced. The blending of the two is coming. That's what's happening.

So, the limit in terms of the technical limits are not particularly significant. The legal limits are inexistent right now. I feel it's going to take a long time. But it will. It's bound to happen that something is going to be – it's going to feel either from a moral standpoint, or an economical standpoint, that somebody is taking advantage of a space that's not there to do some kind of activation. Somebody else carrying the cost of bringing people to Central Park, launching this big – doing a big brand activation, where you're utilizing the space without paying for it. Things like that, I think is going to be raising issues of virtual trespassing. I mean, the term is actually even starting to make it. Virtually trespassing of space. But again, who knows? Who owns – do you own the virtual space above your backyard? Who owns this?

[00:41:55] DA: Yeah, this whole NFT and artists, and that's starting to push the boundaries of like, who owns digital rights to certain things. But digital property is an interesting one. You've like exploded my brain and immediately I'm like, I had this vision of I was thinking about football, again. I had this vision of like being at a football and you're watching your star forward, have a shot at goal. Now, when you're watching it on TV, they have augmented reality pop up things that happen when the guy's doing it. But they're easily controlled and they go, well, he's kicked six through here this season, and two over there and one over there. And everyone close up has already got their phones, hovering over the dude. And they can immediately go, he’s not going to kick this, because they get the same Hoverlay experience, same augmented reality experience. Same for someone shooting free throws. You hold the thing up against the guy shooting the free throw, and he goes, he's 89 percent from free throw for blah, blah, blah today.

The opportunity is endless. And then you go, if you kind of have your phone up that often, for this amount of application, you might as well just put them into your glasses, which is where Google was probably going and where Microsoft was going with HoloLens. This is sort of like maybe the tipping point of where the glasses are going to come in to the mix. You told me ages ago, also, at an event, you're giving me – I'm just thinking through event experiences and you were saying to me a long time ago, imagine having the LinkedIn profile could pop up over the person. Or you could have that related content, like you talked about the MIT study that came out, like if you were talking, and the MIT study could come through, and I could sort of virtually flick through the MIT thing in a speech bubble over your head while you're talking. Or you could bring in someone from the history that's passed away to help enrich your presentation.

I could be talking on stage about some famous Mahatma Gandhi quote, and I could just be Mahatma Gandhi, if you've got your glasses on, and you'd stand next to me and go, “Well, this is what Mahatma Gandhi had to say.” He happens to be standing next to me delivering the actual presentation. He's not a slide on the screen behind me. This immersive technology. Is this the glasses like I would do foresee? Foresee, get it?

[00:44:03] NR: I do.

[00:44:03] DA: I didn't even mean to do that. That was really accidental. Well, I don't know, accidental, probably. Do you think this is where the tipping point comes is when this technology – when our world really collides, we will start to get into more wearables instead of just holding this phone up in front of our face?

[00:44:22] NR: I think that's looking at how much wearables of progress over the past five years, is also something that's quite unbelievable. That again, you see, not the if, but when, kind of moments where, wow, it's there. It's just how it would be socially accepted? How would it be rolled out? And what is the content strategy around this? I think the concentrations we've been discussing, you know, until now is you take the web and you start to put things out of pages and you put them back into the physical world, and where that's meaningful and curated based on location in time. Okay, so that is what movement.

The glasses movement, a feel has an interesting play, which is not to start with the scary part. But simply start by replacing current glasses that are just physical. Your prescription glasses, whatever with a pair of E glasses, that can just do a better job of than your regular prescription glasses. So, if you can already just imagine that now, you can adjust the correction of your glasses because the electronic is able to magnify, give you correction without you having to go back and replace your glasses. That's a benefit. If you can improve vision at night because you now have sensors and you can – you’re driving your car, you put your glasses on, all of a sudden you have night vision, because you can see contrast better.

If you can see when there is smoke, because now you can show objects because you have other ways to sense than just vision, you have LIDAR cameras now on phones, that sense physics of objects, not through vision, but through other technology. Imagine that in a pair of glasses. Forget about the football games. Think about just what if you have a pair of glasses that are just better. And then that is your jump point into, wow, I can use this because it's – and the form factor now, the thing that's also has changed a lot is that the form factor resembles a pair of sunglasses with a bit of a heavy frame, but they don't look anything like Google glasses back 10 years ago. They look like regular glasses.

[00:46:45] DA: This is rose colored glasses. You're going to artificially make people see things better than they really are. It’s a rose-colored glass analogy. And when you said, forget about football, I think it literally was just thinking about football, just for a split second. I was like, “God, my team is playing badly.” Imagine putting these glasses on and they could just start, I get sort of like a replay of another game as though I'm already there and they win. And I'm like, “What do you mean? I don't know. My team won. I don't know what the problem is.”

So anyway, but I digress. But yeah, you're talking about –

[00:47:19] NR: I do, about adoption steps.

[00:47:21] DA: Yeah. Wow. We started off this going imagine the audio processing could automatically be processed, so you sound like you're coming from Abbey Road Studios and everyone sounds amazing all the time on every podcast. You're applying exactly the same principle and going Imagine if you could see everything perfectly or even better than perfect, better than anyone's ever seen before. Because we all see differently, and you can adjust everything to whatever you want. You could turn a gray day with rain into sunshine?

[00:47:52] NR: Or bring some snow and if in case you miss it, in case you miss Boston, I could send you some snow there, virtually. It's going to take some evolution to get there. My point is that I think they are entry, the art strategies that can be applied to enter gradually into this world in a way that is accepted, because just like you get hearing aids when you're getting old, well, why don't you get glasses that do the same thing? They help you see better all the time. They improve your vision in many ways and you could think of glasses as being a piece of technology already. I mean, a lens is super sophisticated. So, why not another form of tech you put on your head, but that gives you now the tool – it opens up. Everything we've been talking about becomes now a natural evolution of that device. Once you put your glasses on, you see the world differently, and I think these are the big tipping points that when and where and how it's going to happen.

I mean, obviously everybody's watching rumors around Apple's introduction of some form of headsets, it's going to be VR first, going to be AR. I'm not sure everybody knows. I'm not even sure they know exactly what they want to do and it's still fitting it out. But I think in our lifetime, I think, we will see major steps toward some kind of wearable type of device and we'll be opening up now everybody to this new era of contents where your car can hold the manual and the phone number of the next dealership. If you look at it, and the car is going to tell you, okay, tap here to call the next guy over. Natural things that you would say. “Oh, well, why do I need to go google this?” I can just look at my car. My car's going to tell me who to call.

[00:49:50] DA: When you think every technology is as bad as good as it gets, every day, I've sort of taught myself to be content, and the way I can find happiness is by going, “Geez, we're really lucky. We've come a long way.” Zoom is amazing. The fact that we can Zoom to each other, and you're in Massachusetts, or wherever you are over in the US, and I'm here in Australia, and we're talking in real time. But sitting here at my desk, I have a round desk with another microphone across from me and sometimes I physically have someone in the room. As you're talking about this, I'm sitting here thinking about, why would these glasses – you would actually be sitting right there. You would be in the room. You'd be a digital version of you, but you'd be a three dimensional you or maybe you're sort of semi two dimensional, whereas I see you. But I would be seeing with my eyes you sitting across from me, and that would bring people together to have – think about the workplace. You have virtual meetings where you're then seeing each other properly. You would just need the camera to shoot you back into the same frame. It’s – oh, gosh.

[00:50:54] NR: It's, the tech is getting there. The tech is getting there. The question is how it's going to materialize? Who the players are going to be? Is it going to be an open system? Is it going to be regulated? Is it going to be like the web? Or is it going to be vendor stacks and battle between the big player. They’re all kind of unknowns, and I think things that could actually slow down the growth of that space. But I feel looking, having been to space now, for a few years, the path the trajectory is very clear to me. The progress made by the technology, the investments that are being made by the big vendors. I mean, you look at phones now, what do they differentiate on what is always the main big feature of cameras? It's not the browser. I mean, it's like, okay, why is that? Where does that go next? You feel well, they're the visual world, the world of images and cameras, and it's central to those devices.

So, there is a trajectory that you can see, if you're looking at those different threads and bits and pieces. How's it going to come together is still, the jury, I think is still out and to a large extent is what people are going to make of it.

[00:52:06] DA: The other cameras are like the shavers now. It was like two blades is good. Now, we're going to have three blades and we're going to do four blades. And then we're going to do five blades and the cameras are the same. It's like we're going to have one camera lens. Now we're going to have two camera lenses. Now we've got three, and this one's got four. So yeah, just keep adding it and they're just get heavier and heavier and heavier. But that is the point of difference. Did you say Apple was coming out with an augmented reality headset? Did I hear that?

[00:52:30] NR: Yeah, there are rumors about – I mean, it hasn't been a complete secret that they're working on some form of display wearable. They are competing visions about what that should be internally to Apple and again, the open questions we're having now, I'm sure people at Apple have the same, plus probably bigger questions around privacy, around what does that mean if Alexa can record your voice? And if people are concerned about that? What about some kind of things you put on your head and you're going around with some electronic device on your eyes that can see and sense things?

Okay, so I think there are all kinds of aspects to that story that's a company that size has to be extremely thorough about in how they do it and when. But I can tell you that they are other vendors out there and you can see and try their devices. I mean, starting with Microsoft with the HoloLens, that was an amazing piece of hardware. It's more of a kind of visor kind of system. It's bigger, it's not small glasses, but you try the sun, and you go okay, now I understand what this means to have something virtually appearing in my space and I feel like it's there. It's holding steady and I can move around it and it's there. That I think is what we're starting to now see, the kind of the goalpost is set, and the technology is ready to a large extent.

[00:54:02] DA: In this podcast, in this conversation with you, the penny finally dropped for me. I sort of got it and I'm like, I have a, not a HoloLens. I have one of those Oculus Quests and I was sort of like, yeah, I get it. You can disappear into another world. But I think bringing that current world into a digital world is where the power is. And then the application of that in terms of communication and influence. It would be so remiss of me not to mention Peloton because I do this every single time on nearly every podcast because I'm that brainwashed, because I do it all the time. And I think they've gone into my psyche.

But again, I'm just picturing being with a headset on and I'm in the studio, and it's safe enough so that I can still see my surroundings. But I've also got this three-dimensional instructor telling me to get moving. The days of just this flat screen could be numbered with what's the potential. I think this is the moment for me, you've successfully got me really flowing where I'm like, “Oh, my gosh, the potential for this is unbelievable.” It's the old saying goes, it's like, Daniel Burrow said this, “If you don't do it, someone else will.” It's there for us to take. I think there's so much potential. People listening to this podcast could potentially just go political campaigns, schools, education, so many applications for this. You can make a living out of doing it.

As we wrap up, firstly, I wanted to thank you for significantly mind altering my perception of the potential for augmented reality. And thank you for your patience in communicating that to me, and eventually having me get it. I wanted to pose one question to you. How do we get to a more intelligent tomorrow?

[00:55:52] NR: Well, that's a great question. I feel it that’s, for me, depends what you mean by intelligence. I feel for me, the intelligence we probably need right now is probably more of a human intelligence of some form. So, I feel like technologies that can actually positively influence that, help people see each other the way they are, not that distance from each other, and that's adversarial to each other. Anything that will raise empathy, understanding, meaning, I feel, and help people feel more secure and confident in their own environment, I feel, is really where that stepping stone is going to be toward everything else working better.

The tech is already ahead of where we are as humans, so I feel like it's more us catching up and figuring out how do we prepare ourselves for that abundance of technology and capacity? How do we do this right, so that it doesn't end up as a few dominating the rest? But something that's more of an evolution of the technology toward a more harmonious and shared future with everybody. I feel like, I know, it's a bit a Barney type of vision, but I feel that's the moment where we got to rethink this – I think the power that this technology enables, needs to be translated into something for the greater good and feel like it will require – it's not something we can lean on governments to regulate, because there's just too far ahead. It's difficult to understand. They've got all the things to think about. So, I think it's more of a collective. I think anybody working in this tech industry has to apply their judgment and understand that this is great technology, but it needs to be put to good use. So that would be my parting words, in terms of how we move forward here.

[00:57:54] DA: It's powerful. It's a harmonious society, enrich society, with a very powerful technology that comes with a warning, that we need to do it correctly, in order for us to achieve that harmonious, enriched community and not deviate the other way. That's the same with any of these new technologies, whether it's AI, when they're this powerful, they can be that disruptive. So, very important message and very enlightening conversation. Thank you for taking the time to chat with us and hopefully, people can take away some ideas. I definitely have a couple.

[00:58:34] NR: Thanks for having me. It was a pleasure.

[00:58:35] DA: Awesome. Thanks, Nicolas.

[END OF INTERVIEW]

[00:58:40] ANNOUNCER: Thank you for joining us on this More Intelligent Tomorrow journey. Discover more and join the conversation at moreintelligent.ai. The future is closer than we think.

[END]