How does the will to survive bring clarity to the human experience? What would you sacrifice to achieve social justice? How do we find meaning and purpose in a world dominated by technology? These are the kinds of spiritual, social, and philosophical questions posed by today’s guest in his futuristic novel, Unfettered Journey, which follows the story of an AI scientist who seeks to create true robot consciousness.
We’re joined today by writer, philosopher, and technologist, Gary F. Bengier, who has followed a fascinating and uniquely non-linear career path. Before turning to speculative fiction writing, Gary spent nearly 30 years immersed in tech in Silicon Valley, including a stint as eBay’s Chief Financial Officer. He then pursued passion projects, studying astrophysics and philosophy and devoting much of the last two decades to thinking about how to live a balanced, meaningful life in a rapidly evolving technological world. This self-reflective journey infuses his novel with insights about our future and the challenges we will all face in finding purpose.
How does the will to survive bring clarity to the human experience? What would you sacrifice to achieve social justice? How do we find meaning and purpose in a world dominated by technology? These are the kinds of spiritual, social, and philosophical questions posed by today’s guest in his futuristic novel, Unfettered Journey, which follows the story of an AI scientist who seeks to create true robot consciousness.
We’re joined today by writer, philosopher, and technologist, Gary F. Bengier, who has followed a fascinating and uniquely non-linear career path. Before turning to speculative fiction writing, Gary spent nearly 30 years immersed in tech in Silicon Valley, including a stint as eBay’s Chief Financial Officer. He then pursued passion projects, studying astrophysics and philosophy and devoting much of the last two decades to thinking about how to live a balanced, meaningful life in a rapidly evolving technological world. This self-reflective journey infuses his novel with insights about our future and the challenges we will all face in finding purpose.
SEASON 2 EPISODE 26
[INTRODUCTION]
[00:00:00] GB: Major theme in my book, and that came about because of my thinking around my thesis on what is the mind, really? And what is human consciousness? That then can lead to the question of, can you actually create consciousness in a machine, right? I'm a little bit of a doubter, I'd have to say. I am less worried about the singularity, where suddenly you have an AGI, an advanced general intelligence machine that, nanoseconds later, multiplies itself, and the next thing you know, they've taken over.
[00:00:32] ANNOUNCER: Welcome to More Intelligent Tomorrow, a podcast about our emerging AI-driven world. Critical conversations about tomorrow's technology today.
[INTERVIEW]
[00:00:46] AK: Hey, Gary. Welcome to the podcast.
[00:00:48] GB: Hi, Ari. I'm delighted to be here. Thanks.
[00:00:50] AK: Yeah. I am really excited to have you on. I am a huge sci-fi enthusiast, as are a lot of our listeners. You are one of the bestselling sci-fi authors, but you have one of the most unique life journeys that I've seen; CFO at eBay. You've had many adventures around the world, but why don't we start by telling us a little bit about what have you done in your career and what brought you to today?
[00:01:18] GB: Well, the careers are a random walk, as we all know, right? It's a non-linear adventure for all of us. I had an undergraduate in computer science and decision theory, Harvard Business School. Then, I was a management consultant for a short time. Then, I was in tech in Silicon Valley for nearly 30 years. I had the great fortune to be in a lot of different technologies, that's including bioscience in the early 80s, computer peripherals, chip design, streaming video over the internet, and also, the internet with eBay.
After the company has got to be 100 billion dollars and more sold and thousands of employees, then I had some opportunity to do some other things, so I did some philanthropy and I went back to school. I got a bachelors degree in astrophysics, and then in philosophy, and then I got a masters in philosophy with a thesis in theory of mind. That took me on a very different journey, thinking about what the mind is and human consciousness.
That was the basis for writing this book, Unfettered Journey, which is an adventure and love story that also focuses on the near future. That is a 140 years in the future. In that book, I tried to think about how will we find meaning and purpose in this next century and a half?
[00:02:42] AK: Wow. I would love to do a 20 to 30-hour podcast on each – an hour on each of those subjects, since they're so thrilling. There's so much going on in the world in philosophy, astrophysics, as things get automated, how will we find meaning? What is meaning? Psychologically, there's so much to go down in each of them. Where to start? One thing that jumps out, eBay CFO during their initial IPO. What was that like? That must have been so unique in history. They came up with such a great concept and were able to execute.
[00:03:18] GB: It was a tremendous ride. It was one of those hairs on fire time when you could see the company growing by as much as 30% per month. That just focuses one's attention. eBay is a complex, adaptive system. That's actually how I became associated with the Santa Fe Institute, looking at complexity science, non-linear dynamics. That's an element in how I think about the future, the paradigm for how does the world really work? I think it's a complex system.
[00:03:48] AK: Yeah, I love that topic, too. That's where I see, I don’t know about you, but the world of AI and data science go in the next couple of years. That, typically, AI has been, you have data that happened in the past, and let's just model it and then project it forward. People are dynamic. People, when a price goes – it's not linear. It goes above a certain price, they act a little bit differently. When they get hungry, they purchase things differently. When their child gets a cut, they act, I wouldn't say irrationally, but a different rationality, to spend more money or to go out of their way to get the band aid or the wound care. Complex systems. I don't know. How do you see that being defined and used?
[00:04:34] GB: I like that. In fact, just this past week, there was a paper shared from the Santa Fe Institute president about whether we are in a recession or not. Essentially, one of the points of the paper was, no one knows. Not even the top economists can really know, because it's such a complex system. In fact, that's one of the reasons why – that's one of the paradigms of how I think about the world. That's one of the reasons why I placed my book, Unfettered Journey, in the year 2161. Because I do think that it's so non-linear, day-by-day, who predicted the Ukraine war, right?
I think we can look at trends in technology. We can use that to make some more highly likely predictions about what the world will look like, in general terms, and we can make some predictions about what are the things we have to pay attention to? That's one of the hard science backbones to my whole project with the book.
[00:05:36] AK: Wonderful. Then one more thing, I'm very familiar with the Santa Fe Institute. Love the things coming up. Our audience may not be, but tell us a little bit about what that is what you've done with them.
[00:05:46] GB: Yeah, sure. It's been around for 40-plus years. It's an institute set up by several Nobel Prize winners, including Murray Gell-Mann, who passed away a few years ago. He was a great friend there, and others of that caliber. The idea was collaborative research across disciplines, because finding knowledge by looking at the cross-disciplinary interface, where the mathematician may be sitting down with a biochemist and finding things where the sciences overlap. That's what they're really good at. A lot of brilliant minds there. It's been a lot of fun being part of that. I'm on the board there for the last 15 years now.
[00:06:24] AK: Amazing. Murray Gell-Mann. Yeah.
[00:06:27] GB: Yeah. Murray. What a dynamic mind. So much fun.
[00:06:31] AK: Yeah. Amazing. I don't know if you knew, I was an undergrad at Cal Tech in the late 80s to early 90s, and did some research, actually on adaptive systems with Major League Baseball analytics. He became a fan. I don’t know if fan is the right word. He became naturally inquisitive. He worked – like his office was a trailer in a side parking lot outside of where Richard Feynman was. He always had this car, seemed to be at the time, the 1960s. I think was a Volkswagen and you could guess the license plate had five letters. It could be one of the Wordles these days, but QUARK.
[00:07:16] GB: Yes. QUARK. Yes. That's right.
[00:07:19] AK: Probably the hardest Wordle to guess.
[00:07:23] GB: Maybe, I'd love to talk about this hard science view of the future, because I think – and that fits with both data robot and what you're doing with AI. Also, Ari, you with your JPL engineering hard science background. I think we're on the same wavelength, looking at the future that way. Here's my big thesis as a meta view of this next century and a half or so, is that I think that, in this century, the two most important technologies affecting humankind are, one, bioscience, and two, AI and robotics.
With regard to the first one, I think that – I was in bioscience for half a dozen years in the early 80s, doing electrophoresis, chromatography, all the separations, chemistries, behind sequencing the human genome. On the finance side, not on the biochemist side, I must admit. But close enough to those industries. I think in this next century, we're going to see tremendous impacts on human life. We will, I think, reasonably likely solve cancer. We’ll cure cancer. We’ll have increases in lifespan, on average. I don't think we’ll live forever, but we will make amazing gains.
That said, I think that most people won't notice in the sense that if you live another 20, or 30 years longer, you live to a 100 on average, or a 110, or something, and you don't get these diseases, that’s just the way life will normally be. We don't get polio hardly at all. I think there was one case recently, a week ago, but we don't really see that. Whereas, 50 years ago, that was common among children, right? They were still concerned about it. That's bioscience.
It’s very important, but it will have a peripheral impact on our psyche. I think that the second, AI and robotics will make a dramatic change to the way life feels, because – Maybe I can talk about that a little bit. What’s the end result look like in say, a 100 to a 150 years? People have probably seen the Boston Dynamics dancing robots, and the Honda robot at the Olympics that was throwing perfect free throws from the centerline of the court. We can see that that's getting better.
In fact, I think we can see an A to Z engineering path from where we are today, to making that technology really developed. Then, I’ll also say that there there's a strong economic reason to make that happen. We've already had, over the last 30 years, automation replacing people in factories. The Tesla factory is trying to be fully automated. The Amazon warehouses are getting very, very automated. My view is that it's an A to Z engineering problem that is driven by positive economics. I do think it will take a lot longer than most people think, because using the analogy of the automobile and the automobile was developed over a 100 years ago, but it really took a long time for the automobile to be the automobile that we think of today.
You need roads, you need legal systems. We will need, for robots, legal system to deal with what happens when automated cars kill people, when planes crash that are on autopilot, etc. I think then, it will be longer, but eventually it will happen. Then, we'll have robots really walking around among us. That's my thesis. I think, they'll be our size. In my book, I imagined a couple of classes of robots. One that you can talk to that's a little but shorter than us, so it's not very intimidating. There's another taller robot that more of a working robot called a mecha. I think they'll be walking around among us, and they'll be doing more and more human jobs.
[00:11:11] AK: Yeah. This is all amazing and fascinating. Love to drill down a bit on each. Yeah, what were they called? The HIPAA bots and the mecha bots, reminds me, everyone today has the mobile phone in their pocket. I used to work at US robotics, who made the Palm Pilots and then the PDAs and people. This is only 20 something years ago. I would say, I will never need to check my email when I'm out of my home. It was looked at as just a calculator that has contacts in it. Now it changes your life. You can only imagine, or you can't even imagine when you have these adaptive robots, like personal assistants. I could use one traveling, carrying my luggage, so to speak. How cool could it be? I'd like to hear your thoughts on uses. But just to assist you physically, or walk the dog for you so you don't have to do repetitive tasks, unless you want to. Weed your garden. Any labor that people don't want to do, or that can help augment your just life walking around, going outside.
[00:12:17] GB: Maybe let me drill down on the economics, because I think that's – I think this is highly likely. It's going to be a change for humanity that we haven't seen since agriculture 10,000 years ago. Here's the thesis on that, is that we'll have this trend of continued automation. It will be driven by the economics. Then jobs will be going away. When I was at a workshop at the Santa Fe Institute three years ago, there was a presenter who described that changing landscape as a topology, with hills and mountain hills and valleys. The water level rising as the analogy for jobs going away.
Then one would ask, well, what jobs are at the top of the hills? Maybe it's your job, Ari, a podcaster. Maybe it's politician. Maybe it's strategist. I'd like to argue that one of those jobs is roofer, the guy that climbs up on the roof and nails the shingles on, because it's very hard to automate. Roofers will be making $400,000 a year. Then, there'll be an economic reason to replace the roofer. The point of that is that robots will do the easy things first.
At this conference, actually, Rodney Brooks, who invented the Roomba was there. He said to me, “Who would imagine that it would take 10 years to develop that little vacuum robot?” But it's hard. It's really hard to engineer things. Eventually, it gets done. We'll do the easy ones. Then the harder ones who will be the ones that requires the dexterity of the human, but that we’ll do that, right? We'll build military robots to save lives, and then that can lead to robots that walk around among us. Then, that roofer will be automated.
Then what happens? Two things. Two very important things. One is, there'll be no jobs. When the roofer is replaced, those jobs will have gone away, right? The servers in the restaurants will be robots. All those jobs will have gone away. At the same time, then, think about it, robots could be mining the metals, smelting them, making the steel, building the factories that build the robots. When that happens, you have robots building robots, and you have a tremendous amount of robots. The number of robots that is unconstrained by human labor. We think about labor productivity, how much output per person and that's, of course, a limitation of how much stuff each of us has.
What happens in the future? Well, we can have a ton of stuff, but we won't have any jobs. That's my prediction is a highly likely outcome. In fact, I'm a financial person and I ran a spreadsheet forward looking at modeling US in GDP per person, and global GDP per person in the year 2161, okay. If you run it forward taking, reasonable rates of productivity increase that we've experienced for the last 20, 30 years, it turns out, we'll have somewhere between 10 and 20 times as much stuff per person as we have today, assuming the population follows the demographic curves. That it's not a whole lot more than we have today.
We'll have a lot of stuff. That's even before you have robots making robots. Here's a world where we have lots of stuff, but no jobs. Then the question, the economic question, one of the questions is, who owns the robot factories? What do you think? Who owns the robot factories?
[00:15:47] AK: That's one of the fundamental questions. Like, what is meaning of life? Who decides it? Are humans even needed at that point? Does it matter? Even more simpler and basic, if a robot can make a better robot that is less rusty, or more agile, or can work on some other chemical, like sulfur, or whatever it is, then why not make the robot have control over that? If the robot has some weird idea that the human brain didn't think of and it's a gray zone, what's the practical use? Then, yeah, is that a human to make the decision, a robot? Is there a board of directors with a mix of them, where the robots come up with ideas and some human, economics or not makes that decision? That's a great –
[00:16:40] GB: You realize that with my career in tech – I am a capitalist. I believe that the capital system tends to produce more stuff and adds some benefits to human life. I mean, we have less poverty, etc. If you think about a system where the robots make robots, and there's this exponential increase in, potentially, the value of owning that, then it makes the system capitalist we have now even harder to sustain that way. You can imagine a world, what happens at the end of that? You have very few jobs and you have lots of stuff.
If that's where we end up and those who own the robot factories are a very tiny number, then you have a formula for complete social instability. I don't think, actually, that's sustainable. I think, you actually have to imagine some way that we have the robot factories owned by everyone. You can do that through taxation and a guaranteed income, etc. You will have a different system, because it's not sustainable. If you think about science fiction, there's a lot of science fiction that imagines all these dystopian worlds, right?
It's Ready Player One. There are folks living in these hovels. Yes, that could happen if the system continued where we had such a concentration of power. I just don't think those are sustainable. How do we cross that chasm and allow for a society that is stable? That's a fundamental challenge for us in this century, a century and a half.
[00:18:10] AK: Yeah. Amazing, fundamental questions. Is that society a pure human society, or a hybrid human/artificial society? Then, they may not be economically driven, or that new society may be less economically driven. From the dystopian standpoint, at least people today, you get concerned where you have a handful of super rich people, and then everyone else, and if everyone else doesn't have a job, then yeah, could be social instability. That could be good, that could be bad. The good way, like it's a utopia, where nobody has worries. They get all the food they want, all the resources they want. There's no struggle internationally for resources, so they could just focus on, if religion’s their thing, or taking care of family is their thing. But that could be very unstable on the dystopian side.
[00:19:06] GB: In my novel, Unfettered Journey, I have this conceit to drive the story forward, that whereas as these robots, after the climate wars in the year – now in the year 2161, now the robots now make robots and there are a lot of robots around. That, in other parts of the world, they pick more egalitarian answers. In the United States, because of our strong property rights culture, what happened is, as the oligarchs who own the robot factories were encouraged to give them up, they became owned collectively.
The quid pro quo for that was that they enacted a series of these things called the Levels Acts, where as everybody was assigned a level from one at the top, to 99 at the bottom. These are merit-driven. You can move up and down in levels, but you had to level and the levels would cause certain restrictions. That's the world in this future, again, to drive the plot, where there are these levels that are explicit. There's some folks that don't like levels and that drives a very strong social justice theme. Do we have levels today, Ari?
[00:20:17] AK: I was just thinking in China, they have, or I heard about some social status. If you do something good, you get rewarded. If you don't do – When I say good, something aligned with what the government deems as what they want, you level up or down, but I don't know –
[00:20:33] GB: Yea. They have a social credit score. Yeah, if you jaywalk then they face recognize you and they put your face up on a big billboard and you're publicly shamed, as well as getting demerits on your social credit score. Yeah.
[00:20:47] AK: Yeah. I watched, was it Black Mirror last night, and the whole episode was social influencers trying to do stuff to get followers. That seems to be a theme in some of the sci-fi right now. Yeah, one of the thoughts we're talking about in the future, dystopia, or utopia. We had only one other sci-fi writer, David Brin on our podcast. He said –
[00:21:12] GB: The Postman. Yeah.
[00:21:14] AK: Yeah. He had this really cool quote that science fiction authors don't try to predict the future. They try to prevent it.
[00:21:21] GB: Yes. Well, that's a good point. I mean, part of the reason for my hard science view in this world is, I think, first that it is more highly likely that these are where the main technologies will drive us. Those raise certain kinds of questions. They are the ones we should focus on, I think. In this case, what happens as jobs go away? How do we still maintain or recognize the differences between people's abilities and having some merit driving what happens. At the same time, having some equality, some ability to have equal access to the ability to use your own abilities and your own talents. How does that happen?
In that regard, I think that the future will have some of the same problems that we have today. They won't go away. Here's another tidbit from my world, this year of 2161. Since everyone, there's so much stuff and these are collectively owned, everything is free, except for the top 10% of goods. If you walk into a restaurant, you can sit down and eat. Then you can just wave goodbye to the robots and the meal is free, unless you wanted that special bottle of wine that's in the top 10%. Because the point was, is that it recognizes that humans are fundamentally competitive. It's hard to imagine that we will not find some basis to compete with one another on.
[00:22:45] AK: Exactly. It reminds me of the video games where it's free, but you have in-app purchases if you want to accelerate your progress, or get something new. Yeah, the restaurant, that makes sense. It could. Right now, it doesn't, but it could be a totally different model.
[00:23:00] GB: Well, and we have crazy things now. These NFTs that are being sold for outrageous amounts of money. Are you really buying something? Isn’t that in many ways silly? I think it is. Just follows on outrageous prices for certain kinds of artwork. Everyone wants to have the one thing that no one else can have. That's something driving human nature. In human nature that's driving that equation.
[00:23:22] AK: Yeah. I was just in Boston. They had a wonderful Bansky exhibit. He's an artist, performing artist, philosopher. I don't know if you've heard of him or seen it. Yeah, he's almost a prankster. His goal is to make weird art and see how the world reacts to it. Like a Paris Hilton album that he totally faked. It was illegal. You have copyright issues. He handed it out at concerts. Then, before long, there's hundreds of millions of people on the internet spreading around these fake things. It's like social prank experiments, planting things in a museum that don't belong there.
[00:24:01] GB: Emulating Banksy with his painting that sold for over a million bucks. Then, as soon as the hammer dropped, it shredded itself.
[00:24:08] AK: Yes. Yeah, that was the same one. Then the shredded document sold for 10 times as much, since it was associated with an event.
[00:24:16] GB: Ari, you mentioned something, though, about along the lines of what would the robots do? Maybe that's a segue to consciousness, and will robots be conscious? First, on a basic level, you can imagine if you have lots of robots, they could do some pretty cool things. For example, you could just tell a 1,000 of them to go off and plant a bunch of trees and they would be merely planting trees and picking up trash and suddenly, a couple months later, you got lots of trees and landscape is beautiful. Humans just set them in motion. I can imagine that we’ll have amazing multiplications of our ability to get things done.
Think about that in a weird way. Think about, a few 100 years ago, building a building, how long that took, yet, we're not surprised that if you walk down any cityscape and you see these giant cranes and think, how fast these buildings go up? Yet, we don't think anything about it. Well, just imagine that that's magnified again and in terms of what we can create. That's what happens with robots. Will they be sentient, which means have feelings, real feelings? Or will they be conscious? Can they actually think like us? That is a major theme in my book.
That came about because of my thinking around my thesis on what is the mind, really? What is human consciousness? That then can lead to the question of, can you actually create consciousness in a machine? I'm a little bit of a doubter, I have to say. I am less worried about the singularity where, suddenly, you have an AGI, an advanced general intelligence machine that, nanoseconds later, multiplies itself and, the next thing you know, they've taken over. I'm not too worried about that. I think, in fact, for the next century, rather than a Terminator scenario where we have to worry about them killing us, I frankly, think they'll just be annoying.
You could imagine trying to talk to the darn thing, and then it just gets it wrong. Then, we're constantly trying to correct them. Of course, that data is going to go into your deep learning algorithms. It's going to help try to fix them. They'll get better and better. I think that in my book, in the year 2161, there's this conventional wisdom that they are conscious. The main character, Joe, is an AI scientist, and he's working in the ministry that is tasked with truly making them conscious.
After five years of hitting his head against the wall, he goes off on a sabbatical. He has reached the conclusion that he can't figure it out. How can you make them really conscious? And that, in fact, it's cheap tricks all the way down. They have gotten better and better and better, and they so appear to the average person to be sentient and conscious.
[00:27:01] AK: One of my favorite movies growing up in the 80s was called Electric Dreams. It brings up the point when you said, annoying computers. At the beginning of the movie, the computer is semi-sentient. That's the whole thing of the movie, is it sentient or not? Asks his name. He says Miles and misheard him. And says, “Your name is Moles?” He goes, “No, Miles.” The rest of the movie, it's an annoying computer. He tries correcting it, “My name is not Miles. It’s Moles. Not Moles. It’s Miles.” It just won't stop the whole movie.
Yeah, I could see it being annoying. Build my mother a house. The robot says, “All right, I'll build her a mouse.” The interpretations, what is ironic are not, doing things that are totally opposite meaning. Yeah, that can be a big challenge. The goal is to have it fully integrated, where you don't know even it's assisting you. Just like, your life magically seems more cheerful, and things just happen around the house, or round your life to make it happen.
[00:28:02] GB: Yeah. Maybe let me segue to another part of my book about how different it is, and then let me come right back to that. In the year 2161, I've got a couple of devices that I imagine that we have. This gets to my cynicism on we've all become cyborgs, or we’ll it become integrated with the machines. I think that's overblown, and that won't really happen. I have two devices. The first is something called a NEST, it's a neural to external system translator. You can imagine that your iPhone, rather than carrying it around and looking at it, it's a chip that's inserted behind your left ear or something. It's connected to the internet, the net at times and directly, and you can just talk out loud and your voice recognition, the chip interprets it.
You can say, “Search for the closest pizza shop,” that kind of thing, and it gives you an answer. Then it's connected to a corneal transplant. If you just said that question, what could happen is you could have this thing called an ARMO, an augmented reality map overlay, which is a little red line painted on your cornea and that you look out and there's the path and walk up this way, and then turn left and turn right and there's the pizza shop. By the way, Ari, I saw that you were talking to Mike and Brian at Mojo Vision.
[00:29:24] AK: Yes. Brian and I went to CalTech together. He was in my brother's house. They're amazing. For the audience, like augmented reality on a contact lens. It actually works. It's not FDA approved. It's version 0.1, but it works and it will work and it will happen.
[00:29:44] GB: Okay, so I love that, because my book came out – well, certainly before I heard anything about that, and I love the fact that that real technology, which I think will take a long time to get all the way done. Again, these things always take longer, but you can imagine that your phone will be embedded in some fashion, right? I love that one. Okay, so that's one device I have. Now that sounds a little cyborg-like to some people, but you can imagine, you carry this phone around with you all the time. It's just a little bit easier. That's one.
Then, two, I've got this thing called the MEDFLOW. You imagine you got this little thing that's inserted by your hip, for example, some device. In microdoses various chemicals and it monitors your body, adds anti-aging chemicals and that sort of thing. That just automatically works. Now we've got those two devices in an average person walking around. Besides that, life feels the same, right?
One of the reviewers said that this future world, 2161, feels eerily authentic, in that we're still humans. We're not going to be that different, really, than we are now. It's not going to be some crazy cyborg world at all.
[00:30:55] AK: Yeah, I love those two ideas. I think, most people would jump on them. Certainly, the MEDFLOW. The insulin pump isn't that old, but it's something that you don't have to put a needle in your finger all the time and measure and insert. When you mentioned cancer could go away, or get cured, or get held at bay, like AIDS. It's a cocktail that doesn’t cure it, but keep it at bay. You have a choice. Do it and be healthy and energetic and vibrant, or don't. I think everyone would do that.
Then the ARMO, I'm a big Virtual Reality player. Huge. I would personally do that. I don't know if the whole population would. If it's a contact lens, if it's easy, and you have a choice, I see everyone Googling, you have a question, “When's the next movie? What date did we plan that trip?” You're always bringing out your phone. If you could just telepathically, or verbally say it, and it just pops up, I would do it. I would think a lot of people, wouldn’t. That, to your point, change the world, but it would still be the same world, just an easier world.
[00:32:04] GB: Yeah. You mentioned telepathically, that brings up Neuralink, the Elon Musk company. Again, I'm a bit of a doubter on that. In my book, I imagine that when you're a child, you learn maybe a 100 keywords that you can think of, and that will – you could think ‘search’, and it starts doing some search. It's certainly not as efficient as our audio, our voices and vision. Our human brain is evolved to be optimized for those sensory inputs. This other is going to be very clunky. It'll be, I suspect, it'll be very helpful for people with certain kinds of impairments. It'll be life changing for them. The average person, it'll be so clunky, so you won't really do it.
In my book, the character, Joe, is on an airplane and the social protocol is that you don't talk to yourself on a plane to annoy everyone also. He then uses this keyword thing, and he thinks a few keywords to look up some stuff. Or, as an example, if I met you on the street, Ari, and I don't know who you are exactly. I could just take a quick vid snap, a little video, a little image of your face and face recognize it. Now I know you and, on my corneal implant, I see a little bio, and I said, “Hi, Ari. Tell me about that last podcast you did with XYZ.” You can imagine that will fundamentally change our human interactions in some fashion.
[00:33:26] AK: That would be amazing. Then I could see, I walk down the street and people would say, “Can I get McLaren tickets? Can I get Formula One tickets? Can I get Formula One tickets?”
[00:33:34] GB: Minority Report. Yeah.
[00:33:36] AK: Exactly.
[00:33:38] GB: Right. Right. Exactly. Then the question is, okay, back to the robot, though. If you've got this world, but will robots really be conscious? What is that ‘I’ that's at the center of you, of everybody? What is that I, that conscious thing? Can we really put it in the machine? You and I both know, it's just code, right? I was a computer scientist at the beginning and wrote code, and it's just ones and zeros. Is that really thinking? Sentient?
[00:34:08] AK: Again, our understanding of sentience, I know, we're talking about Google earlier, but there were just that Google engineer that said, “What we've developed is sentient.” From you and my perspective, we think it's just a code that was written. It could be that, like you mentioned robots that create robots. Maybe it was code that created code so iteratively that it is sentient. Yeah, what do you think of that language model and that Google engineer that everyone's talking about?
[00:34:38] GB: I thought it was appropriate to diss the conversation, because we're so far away from having that conversation. I mean, I think in even 50 years, that we will – Certainly, we won't be there. I mean, we're dealing with the “hard problem of consciousness,” as David Chalmers, the philosopher said. Then the easier problems of consciousness. We don't even know what it is. I mean, as an example, philosophers talk about qualia. Qualia is the word that denotes the ‘what is it like’ experience. What is it like to taste an apple?
Now, think about that ‘what is it like’ experience, which philosophers tend to think, requires an embodied experience of the world. Then, how does one code that into a computer, into an AI, into a robot? It's mind-numbingly difficult to imagine how one would do that. Then, how would you know? So that that model that you mentioned, the Google engineer talked about, if you're talking to the AI and the AI says, “I am conscious.” You go, “Well, how do you know?” And then you have a conversation.
Well, you and I know what happened is that they took the, what is it? The models that Google now has go up to 500 billion vectors. I mean, it's an amazingly layered machine learning hierarchy. With that level, well, you can look up on the internet, all kinds of human interactions, and then you distill that down and come up with an answer. All you're doing is distilling statistically our human experience. Is that something real sentient? Does it really feel something? Is it subconscious that it really does? I think we're so long, far away from that.
[00:36:15] AK: Yeah. I'd love to hear our audience write in. When you said, what is the taste of an apple? I tried in three minutes to describe it while you were talking. I'm like, it releases, I don't know, serotonin in the brain, but it's like, robots don't have it. They won't understand that. Yeah, I'd love to hear thoughts from the audience on that as well. Also, got me thinking, like you were mentioning philosophers and sci-fi writers. Who are your favorite, or both sci-fi writers and philosophers and why?
[00:36:44] GB: Oh, that's a tough question. Well, I'm an old sci-fi lover from my childhood. Azimoff, those classic writers. What I liked about them is that they tried to stick to the rules of physics. Their science fiction tried to look at the future. We had, was it Ray Bradbury predicted satellites, geosynchronous satellites. I'm afraid what I think has happened in the certain parts of science fiction is that it goes to the extreme, it takes some idea to the absurd end. That in some way, they have given up on sticking to the rules of science. If you're all these faster than light speed spaceships, and we have all these apocalyptic worlds where we've got uploaded brains and terrifying robots.
I mean, on that point, for example of faster than light speed spaceships, I think the general wisdom – the conventional wisdom is, yeah, we're going to have all kinds of spaceships out there. In my world, in the year 2161, there is a space station orbiting the moon. Humankind has this world interstellar exploration project, the WISE orbital base is the key to it, and they're hoping to send XO planet probes. Yes, we have a couple of bases on Mars, we've got a couple bases on some asteroids, but we haven't gotten out of the solar system at all. That's in a 140 years.
Einstein's equation means that we have enormous difficulty moving any amount of inertial mass, any speed. Close to the speed of light. It's not going to happen. in fact, I use a word in there. Think about those folks in a 140 years who are out there exploring space, I described them as planeteers. A planeteer is someone who has spent at least 10 years, a decade, off-world, either in low orbit, or on another place. That person is going to be really unique, because you'll have to have enormous sacrifices to do all the exercise required to be in low gravity or no gravity environments, and being exposed to cosmic rays, and you're going to significantly shorten your life.
No, it's not going to be hundreds of thousands. It's going to be hundreds or a few thousand. It's going to be tiny. We're going to be stuck here on Earth, and so we better protect our own planet. Because I think that the world really will be more constrained, and we'll be less different than it is today than we imagined.
[00:39:16] AK: That's this sad constraint truth, at least for another 100 years. Then with a complex system, who knows? But probably, at least for many thousands of years. It got me thinking for the robots, it will be an interesting nearby asteroids to have them automatically send, I don't know, water. I just saw on the news, there's some giant mass of water trillions of times the amount of Earth just floating in space. Probably not in our solar system, but you could go to asteroids, or comets and mine stuff and send it to Earth and then have a receiving robot, space elevator, bring it down, or something like that. That could help with some of our limited resources and could be helpful. Then, is there anyone actively working on anything like that, you know of?
[00:40:06] GB: Well, I think even Google is investing some money and looking at mining asteroids. Those are extremely expensive projects and lots of engineering risks to get there. You can imagine that, as we keep on making steps, we're making great steps this last decade in terms of getting into space with some of the private companies doing a wonderful job, bringing down the cost of launches, that sort of thing. That's wonderful. Again, it's engineering. A little bit of the time, it takes a long time to make these things successful.
[00:40:34] AK: Yeah. You might call it skepticism, but being conservative of predicting the near future. What do you see other people are getting wrong when they predict the future?
[00:40:44] GB: Again, it's going off on these absurd extension of current trends today, and then worrying about the wrong problems. I mean, I think the problems we can see are clear. It is climate change. It is extremely important right now that we focus on. Then, I think that over this next century, the fundamental issue that we have to worry about is that, as robots, AI and robots become more ubiquitous and as jobs will inevitably go away, then how do we cross the chasm from the economic system we have now until then, how do we distribute the wealth that will be produced by all of our systems in an equitable way, so we have a stable society?
I mean, this could be a utopia, in the sense that we have lots of stuff. If we can figure out how to work through it, it can be a pretty positive future. It's just going to be very challenging, because we'll have so much change in this century to deal with. I'm optimistic.
[00:41:42] AK: If you could just skip and teleport into the future state, it would be more utopian. Sometimes when people lose jobs, or their economy changes, you have families to feed, then the intermediate steps could be chaotic.
[00:41:57] GB: Yeah. It's actually hard to get everyone on the planet on the same page about what the real problems are. Think how much problem we're having with climate change to try to focus on the fundamentals. I mean, The Economist had some articles recently, dealing with the fact that we should focus on emissions and measure that. We should have government policies and programs to limit carbon use. I think, in Stanley Kim Robinson's book, The Ministry of the Future, mentions that there's something on the order of 2,000 gigatons of carbon that we know it's out there, that will be pulled out of the ground.
If we use more than 500 of those giga tons, we're in serious problem because of the heating of the atmosphere and the climate change. We have to essentially decide consciously not to use the rest of that. In fact, produce energy from sustainable sources. I think those are obviously, solar and wind and fission and fusion. Fission, now, I strongly believe that the only four that you can think of hard are solar, wind, hydro, which has, it's hard to build more hydro without having serious ecological damage. Then, if we could get over our visceral fear of the atom, we could take advantage of some of the newer technologies that are far, far safer than anything before, and we could save our environment.
Environmentalists have to get with the program of people will want their phone to be charged. People will, in developing countries, will buy air-conditioning, and they will use energy. If we don't figure out how to move to those sustainable sources soon, we're going to have a problem. How do we get everyone on the same page and focus on the real problems? That's the issue.
[00:43:46] AK: Yeah. That's possibly the biggest challenge for humanity, surviving on our only planet we can probably live in realistically, for the next thousands of years. Yeah, I was so excited, then so disappointed with Pons and Fleischmann when they said, “We got cold fusion working,” in the late 80s and then it didn't work. That's win the lottery. That's where a banana peel could power a city for weeks. Yeah, short of that, nuclear fusion, fission, solar power. Compared to the damage that carbon causes. At some point and you see in the news now, the record heats everywhere, which will have economic ramifications on everything that people aren't even really focusing on now.
At some point, the cost of that, and that could be this year, it could be a few years from now, is going to, I hope, make people say, “Mercy. Let's do a timeout and change.”
[00:44:44] GB: Well, in Dr. Stanley Robinson's book, he imagines that we muddle through and, eventually, get our act together. His book is relatively hopeful, but it takes a while. Maybe my book is a bit of the same. In the year 2161, what has happened is the character, Joe, looks up, “Where are we today with this problem?” After the climate wars of fighting over scarce resources, the world has figured out a way to solve the problem. The statistics at that time suggest that we'll get back to baseline in 17 centuries. Because this is a very long-term problem. By that time, it's solved with carbon sequestration and lots of bio-engineered trees that can suck up lots more carbon, and solar wind and fission and now fusion dotting the landscape at that point. Unfortunately, Venice is gone. New Orleans is gone. Mumbai and Jakarta are gone. Yes, we have to deal with the fact that the climate is a lot hotter.
[00:45:51] AK: A sad story. I went to New Orleans, and got one of those ghost tours. It was just fun. The tour guide was crying, saying, “I'm really sad. I only have a few more years to give this tour. Our city is going to be flooded.” Wasn't even joking. Was just like, I see this as the future. I love my city. It's going to be gone. That was really deep. I had a lot of empathy.
[00:46:16] GB: Think of Venice, by the year 2100, most of the median of the predictions suggest will be up about a meter and sea level rise could be twice that. We don't know. A meter or so means that all those bridges in Venice, one cannot fit a gondola under. There goes that Disneyland part of the experience. I think, it will be gone, actually. It's very hard to save.
[00:46:40] AK: The whole food chain, things will – wheat and everything. Prices will go up so much.
[00:46:46] GB: But we have bioscience, right? I mean CRISPR Cas9 technology is just phenomenal. When I first started bioscience in the early 80s, at that time, they discovered the PCR technique. That was the major technology change in bioscience that revolutionized these last 30 years. CRISPR Cas9 is as revolutionary in terms of what it'll do for drug development, and allowing us to engineer all kinds of things. I'm hopeful that that part of technology will allow us to feed the planet and produce all kinds of new, amazing products that will change our lives.
Again, I'm an optimist about this. We have to solve these other big problems. The big problem is climate change and this economic problem of how do we make the transition from where we are now to a future when we have lots of stuff and lots of robots walking around among us and so few jobs? Then, what do people do?
[00:47:40] AK: I have thoughts on that. We were talking about all the different policies, regulations, and technology, is that something you foresee the US government, world body governments do, or private tech sector, the Musks of the world? Who's going to help with the policies and the technology, just for climate change alone?
[00:48:00] GB: Oh, that's really hard. Climate change, we’re muddling through. We're trying to – back to Robinson's book, he imagines all these UN/global government entities trying to set policies to get governments to do the right thing. It takes a while to get everyone organized., and so the climate gets worse. His book opens with, suddenly, there's a heatwave that kills 20 million people in India. These last couple of weeks with the heat in the US and Europe gives us an indication that you can imagine that would happen.
[00:48:31] AK: Sadly. Yeah. Hopefully not, but yeah.
[00:48:34] GB: Humankind does tend to come together now to find the solutions. The good news is the world is more transparent, and we're connected instantly. That has bad implications, too. It does mean that we can start seeing all the problems and start taking action, I think, collectively. Technology itself is a wonderful thing. It has pluses and minuses. It certainly will continue in the century to change our lives as human beings. I think we'll still be recognizable as human beings. We’ll have the same needs and wants and the way we think. I don't think robots will be conscious anytime soon, if ever.
[00:49:13] AK: How would you define consciousness?
[00:49:15] GB: That's a tough one. I'll just say, go to my book. There's a glossary terms, that there's a multiple-part answer that is a collection of philosophers of what they think consciousness is. It does include qualia. These qualia are these ‘what is it like’ kind of experiences in the world. I think it includes embodiment in the world. I think that consciousness creates meaning and that meaning is a relationship between how we interact with the world. That, when you and I are having this conversation and you say something about AI, we all have an understanding what the meaning is of that, based upon all the things that we had collectively read about that before.
Our audience understands it, and so we’ll understand in different ways. In general, I think the audience will be vectoring in on the same meaning created by what we've heard about that concept in the world. That's key to an idea of that, it’s what our consciousness does is it looks at ideas. What does that look like inside of an AI, or a robot? It’s just a bunch of ones and zeros. How does one turn that into that meaning? That's a deep problem.
[00:50:38] AK: Exactly. Well, that's a beautiful answer. It's a great one to think about. It's seeing the movie, Everything, Everywhere, All at Once, when they transform into rocks that exists for thousands of years, but can still talk with subtitles. I've seen other, or think of other things where a blob of water has no meaning until something sentient interacts with it, and then it has meaning. Then I go back and forth with this with my kids. If a tree falls in the forest, does it make a sound? It’s not, does it physically vibrate? They phrased that wrong, but does it have more meaning than less meaning if something sentient hears it? That's a matter of question.
Then in terms of religion, they asked the same thing. You could say that an inanimate object becomes holy or has meaning if it's used for a good deed or doing something positive, then that instrument has value. It's kind of wild. That all ties back to consciousness.
[00:51:37] GB: Maybe taking on some of the other themes in my book. Unfettered Journey is a cross genre, adventure and love story. It deals with consciousness. It deals with some of those questions about meaning and purpose. In fact, it's won seven awards now, and it's out in eight languages. Among the awards are Best Spiritual Fiction.
[00:51:57] AK: Oh, really? Oh, that's right. That's amazing.
[00:51:59] GB: And Best Science Fiction. It won the Eric Hoffer award. He's a famous noted philosopher who won the Medal of Freedom in the United States. I'm very pleased with those awards. I'm pleased to the fact that it has been recognized across several different fields, not just hard science to the future, but a whole lot more.
[00:52:18] AK: Yeah, everyone listening, absolutely, if you like this episode, absolutely has to read that. It's very rare. I actually don't know of any other, but maybe there are a few where sci-fi gets best philosophy as part of that and spirituality. Those three things sometimes are separate. When you can combine them, you get into deep thought. I had a question. 2161, you may have noticed, but that's a prime number, right? Why did you pick that?
[00:52:46] GB: Ari, okay, that's beautiful. Okay. I love talking to a mathematician. There are lots of Easter eggs hidden in the book. One of them, one might notice is that virtually, every single number in the book is a prime number.
[00:52:59] AK: Oh, wow.
[00:53:00] GB: From the numbers on the robots, etc., all the way down. They're all prime numbers. There are a couple of little easter eggs in there for AI, data scientists, folks thinking about the Turing test, for example. It's fun to do those things. I love when people like yourself suddenly pick some of the things out. It's great.
[00:53:18] AK: Well, awesome. Again, strong recommendation to read it. Yeah. Let me just pause for a second. We have probably 15 minutes left. I didn't even look at my notes. I had all these notes prepared, so this is an amazing conversation. I've tons of other topics. Just wanted to see what else would you want to talk about? Otherwise, I could just keep going.
[00:53:37] GB: Why don’t you go through your list. This is fun. We're having a great time here. It's fun to talk about the future. It's fun to, I think, again, my main thesis on this part is to think about what it will really most likely be like. To recap, I don't know what will happen tomorrow. The world is non-linear. I do think that we can look at some of these longer trends. That gives us a chance to think about what is more likely than not. Some of the things that are not likely are that we're going to have the aliens show up and start killing us, or that the Vulcans show up and we're going to find another intelligent race, and all be happy together.
I don't think any of those things are likely. We're just going to have to muddle along here by ourselves and be stuck in our own solar system, and making tiny changes in terms of getting out into space. Then, we better protect our own planet. Climate change is real. We have to solve that problem. Then, we're going to have to deal with the fact that, with AI and robotics, DataRobot, companies like this, making steps forward are going to change the world. I think, in a way that we’ll recognize. I think, what's critical is that they're going to change the economic system in a way that we will not recognize, because we'll have a lot of stuff.
Then, what worked in the past will stop working. Maybe I'll just talk about that for a moment, Ari. The economics, I'll go back to that, because I think this is crucial. As I said, I believe in a market system. If you have supply and demand, go back to your basic economics 101. If there's not enough supply, the price goes up, new companies come in, they increase the supply, when the demand is too high for the supply. There's a feedback loop there and it takes time.
Well, now, if you think about the data that for example, Amazon collects, or eBay, collects those companies and others, like Google know not only what we bought, but what we want to buy. Now, if you can imagine we have a system where robots have made robots and we got a ton of robots and the plans are automated, then statistically, you could produce almost in real-time what the demand will be, because you could predict that, and then you could predict what they want to buy.
Suddenly, you can imagine that this system actually will be more efficient than our market capitalism. Because it knows all that stuff. Now you can imagine, we have an economic system that is more efficient than capitalism. Then you have is the producers are not people necessarily at all. It's our machines. Our machines now can do all this stuff for us. Now, we are living in this world where we don't really have a need to do all this stuff. It's hard to imagine that world will exist without guaranteed income. Stuff will be mostly free in some fashion. But we won't have any jobs. Will we be spending all our time doing creative things? Well, that's a problem in some sense. I mean, I think today, there are on the order, of 4,000 books a day being printed in the US. My book being one of many. Please read mine and not the other 3,999 right now.
That's a problem, right? How much poetry can we write and consume? Then, who has those cool jobs? In my book, the person who manages that interstellar base circling the moon. That's a pretty cool job. Will the world change so that having a job is a privilege?
[00:57:15] AK: Yeah, amazing thoughts. Yeah, hopefully, we could teleport into that state, since if it's a point where you don't need the traditional job, you're not worried of the hunter-gatherer. You don't need to work to get food, work to pay your electric bills, then people will be satisfied, if they get what they want, and their children or friends get what they want, but the transition could be a challenge. You can't tell by my name, but I am Jewish, and from Chabad, which is one thing. They actually predict philosophically in the future that in the future, everyone will get the resources that the human body needs, so you could focus on love. However, we want to define what that is, and studying philosophy and religion.
That would be the utopia. But we would still be actual humans. What you're saying would be, yeah, you would still go to movies, still walk around, but don't worry or be concerned of what you call earthly wants and needs. That's one thing.
[00:58:16] GB: You mentioned, yeah, and think about the Jewish tradition is studying these deep texts and philosophy and the Torah and everything. In my book, the academics, well, they all have at least two doctorates, because they got plenty of time to study. You need to be in a couple of different disciplines to discover something new. Yes, you can imagine that that's one of the avenues people will invest their time.
Still, I think there'll be a – imagine that all the cool jobs are taken in this future world, the law requires that you cannot work more than 12 hours a week. The typical three days, four hours a week, and that's all you get. Because if you do too much, then there are fewer jobs. These are valuable things to – and you get to do some cool things, like manage all these robots, and give them the directives, so that they go off and do fabulous things. Who gets to do that stuff?
[00:59:10] AK: Yeah. It's interesting when you talk about economics. Like living in America, or Canada, North, the capitalist system that's what we were born into, and what we assumed as right. Historically, there's been all different types of systems, all different types of political and I don't like making – I don't want to bring up religion and politics at once. Not seeing any one government, or other, but that could be one other thing that changes in society is how you rule on a local level, on a country level, on an international level. I see so many things in the world getting reinvented. That could be one of them as well. That'll be interesting.
[00:59:52] GB: Right. Well, so think of how the United States might evolve into the future when we think about those changes. My book imagines a continuation, you can imagine getting to there from here. I mentioned the data that Amazon and an eBay and Google have, and that that could be added to great efficiency to your economics, but then that data will be held someplace. What does that say about data privacy, privacy in general? How will that be treated in the future? I mean, do we have a right to be forgotten? I think, which is an evolving right that they're looking to guarantee in Europe.
What happens in the US on that? How do we deal with that? Then how do we feel about a government? Will there be some concerns about the government using that data and controlling us? You mentioned, Ari, China, as an example already that one hopes that we do not turn into. I think, all of those are actually real issues to deal with. Part of the reason I raise them in my book, in part is to point them out as the key things that we really do have to worry about.
[01:00:57] AK: Wonderful. Let me ask to wrap up, one final question. If you could change one thing worldwide to get us to a more intelligent tomorrow, what would that be?
[01:01:08] GB: I think that the most – besides the transitions I've talked about, the most important transition is to deal with climate change earlier rather than later. I think that I mentioned that the sustainable ways to produce electricity and energy are wind, solar, hydro, and nuclear at this point. I think that if you think about the fact that the sun and the wind do not necessarily work well, at night. I was in the wind industry for a short period too, making high-tech windmills, so I know that one. It doesn't blow at night. To balance the load, our energy companies struggle with that.
They need something that can be turned on and off. If it's not going to be gas, it's going to be – or coal, it's going to be nuclear at the moment. What I would hope that happens is that the environmental movement gets over its fixation and worry about the atom, so that we can get those plants built to replace fossil fuels faster. That's the trade-off. I think that that has to happen soon. If we can get through that, I think we can deal with the issues of waste. I think then, we need government programs that subsidize that, that push for forms of plants that are can be small, and perhaps that don't cause nuclear proliferation of the more dangerous types of fuels. That we can cookie-cutter out, like the French did. The French still have something on the order of 70% of their energy comes from nuclear.
They've never had a single accident that caused any deaths, and hardly anything at all, and that's the old technology. I think that the current technologies are far safer. I think we've got this fixation in our minds, that is just the irrational and non-scientific. I think, we have to fix that. That is a key step to get through this window to stop the worst elements of climate change.
[01:03:13] AK: Yeah, totally agree.
[01:03:13] GB: That's what I would change. We're scientific, right Ari? I mean, that's the facts is let's not be emotional about this really important thing. Let's solve the problem and that will create far, far fewer problems for humanity than to kick the can down the road and to continue use fossil fuels.
[01:03:31] AK: Gary, how did you transition your business world, tech world to become a sci-fi writer? What inspired you?
[01:03:37] GB: There are so many lives that we can live, so many interesting things we can do. I also think it's important to repot yourself every once in a while to learn some new skill. It was a lot of fun to learn the craft of writing, and to work through that as a new process. That was mostly inspired by this journey, as I mentioned earlier, to go back to school to ultimately get a degree in philosophy of mind. Then, I had this urge to get these very deep, profound ideas out into the world. I wrote a very rigorous philosophical book called Unfettered Journey Appendices. It's actually just three papers directed at the philosophical community. Then, the novel is a more accessible version that deals with some of these issues for a wider audience.
I think these are important ideas to think about, because we all think about when we sit back, and we let the buzz of the social media get out of our heads, and actually ask ourselves, why are we here? What is the meaning of life? We ask the deep questions, and then we're starting to find important reasons for why we are here. What is the meaning of life? My character, Joe, I mentioned the levels, he's a level 42. If anyone knows the reference. It's a joke, but it's also something that I think is serious about the book.
[01:05:01] AK: Wonderful. Yeah, great way to end it. Well, Gary, thank you so much for being on the podcast. Where can we find more information on your book and your website?
[01:05:10] GB: Yes. Well, I'm delighted to tell your audience that the book is Unfettered Journey. You can find it on Amazon and wherever you buy books. As I said, it's in eight languages. It’s won seven awards. You can find my author website at garyfbengier, which is spelled B-E-N-G-I-E-R. That’s garyfbengier.com. I'd be delighted to have folks read the book and I'd like to hear about your comments on this world and these important issues. Thank you.
[END OF INTERVIEW]
[01:05:42] ANNOUNCER: Thank you for joining us on this More Intelligent Tomorrow journey. Discover more and join the conversation at moreintelligent.ai.
The future is closer than we think.
[END]