It’s hard to talk with Robert Scoble and not feel like you were left behind in a pre-historical era and have a lot of catching-up to do. He moves around with glasses for augmented reality (AR) and virtual reality (VR). In his new book, The Fourth Transformation, he describes the glasses as the fourth user interface since the start of the personal computer age. This wave of technology that is coming is really going to change human life,’ he told LSE Business Review managing editor Helena Vieira in an interview during a tech conference in New Orleans.
How can I describe you?
I’m a lover of the future. I’ve been at the forefront of a number of things. My life goal is to have a front-row seat on the future. I don’t know that I can build the future, because I’m not a coder, I’m not a scientist or a visionary that way. But I want to be there. Uber was created literally in front of me, when Travis and Garrett were creating it. That makes me happy. My life goal is to see the next Uber being created and be able to explain it to people and go ‘hey, there’s a new thing’.
Siri was launched in my son’s bedroom and I had the first ride in the first Tesla. Elon Musk got me a ride before he got his best friend a ride. Yeah, his best friend hooked us up. Shame on you, Jason. You should have taken the first ride. But he was racing us and I was livestreaming on a Nokia phone. So I’m a tech journalist, futurist, whatever you want to call me. And now I’m starting a variety of media businesses that are coming in this mixed reality space. I’m doing work with startups and brands to help them deal with this disruption that’s coming. This wave of technology that is coming is really going to change human life.
What exactly is it?
This is basically the fourth user interface since the start of the personal computer age. When we had Apple 2 we had a character mode interface. We had to type commands to the computer to copy a file from a folder to another, DOS. And then Windows came out and it got easier to use and a lot of people got into computing, because it was easier to learn. And you’re holding an iPhone here, with touch-screen technology. A lot more people can use an iPhone than can use a Windows computer. The next level is mixed reality or augmented reality, and that’s going to be a lot easier for people to use. There’s still 4 billion people who don’t even have a phone, and have never been on the internet.
Those people are probably going to get on the internet with these glasses because they work the same way the world works. A four-year-old child came over to my house and started VR without being prompted. She’d seen it on YouTube, I learned later. So she knew about it and she knew what she wanted to do with it, because she’d been shown Job Simulator, this game that a lot of kids love. She was able to do it because she knew how to pick up a glass of water off the table. Every four-year-old knows how… how the world works, how you manipulate things, how you open doors, how you move around the world. And that’s how VR works, AR works. That kid can know how it works without knowing English, without knowing how to read, without reading it in an instruction manual, basically watching a little video on YouTube and boom, you’re in. That means a lot more people are going to be able to compute than today, and it means the tech industry gets even more important.
How do these glasses work?
It has sensors that see the world in 3D, it has a LIDAR (Light Detection and Ranging, a remote sensing method) that sees the world in 3D, it has a brain that is comparing the real world to the artificial world, so we’re building a new kind of 3D map called SLAM (Simultaneous Localisation and Mapping). Facebook and Snap have just announced SLAM-based features for their AR, which means that the virtual thing can be locked to a table, for instance, locked onto the floor, so that you can have virtual things walking around your real world. The self driving car needs to see that 3D map. Mercedes has mapped out the streets in 3D and it’s comparing the real world to the map, to the 3D grid of pixels that it’s looking at. And if there’s a kid in the street, that’s new. So the AI engine goes, “oh, new thing!’ and focuses the LIDARs on the front bumper onto the kid to start doing AI and trying to figure out what that kid is doing. And then doing predictive software. ‘What is that kid’s likely next move? Is he going to run into the street? Is he going to stay on the sidewalk?’ They’re doing prediction based on past behaviour. They parked a self-driving car in front of a school and studied how children behave, and taught the engine: ‘this is a child of x age and this is their common behaviour,’ so that when a car sees a kid in the street they have a pretty good idea of whether or not that kid is going to be a problem to the car and what to do. ‘Should I slow down? Should I turn left? What should I do?’
What other applications do you see for this?
Because we’re building a 3D grid on top of the real world, a SLAM-based map. The technology was designed for the Mars rover. There’s a rover up on Mars right now. It takes 12 minutes for a packet to get up to Mars. So if it’s heading towards the cliff and you tell it ‘don’t go there’, it’s already gone over the cliff, because it hears your command 12 minutes after it needed to. So it needs to understand the surface of Mars itself and to understand where it’s located, to drive and navigate around without getting into trouble and without human intervention from Earth (because of the 12-minute lag). So we’re building an artificial copy of the real world. Uber is doing this. Mercedes is doing this. Apple is doing this. Facebook is doing this.
So everyone is going to have their own map?
Well, you’re going to put on a pair of glasses with a 3D sensor that sees the world in voxels, volumetric pixels, a point cloud with millions of little points around it and builds a 3D (landscape)… You can start seeing this stuff on the internet, how it works, the world as seen in 3D. We’re looking at a column in a conference room. That column has millions of voxels on it, and my sensor can see the real world and then can look up the same pixel that was mapped yesterday by somebody else sitting here and can see if something has changed about the real world. If it hasn’t, and that column is not going to change for many years, then it knows where the glasses are, based on that column, right? It’s doing trigonometry back to the glasses and doing all sorts of fun stuff. And then, now that we know the world, we can change the world, we can change it visually. We can put Sponge Bob on that column, we can put video wrapped around the column, we can put volumetric things, like baseball games on the kitchen table, stuff like that. That’s all coming in in the next few years. The guy who builds an 8-eyed camera company told me I’ll have baseball on my kitchen table in four years – if he’s wrong, eight years.
Amazing. Tell me about more applications.
Your drone is going to fly through the air based on this system, because once we’ve built an artificial copy of the world the drone will know where all the walls are and will be able to fly through the air without hitting any walls or hitting anything, If we had a robot serving us drinks right now, it would navigate to us based on this SLAM based map that the glasses owner are building for it. So we’re going to wear the glasses to see Sponge Bob on the floor and we’re going to be mapping out that world pretty quickly when enough people get the glasses. Then the robot can roll around and a drone can fly through here and a self driving car can see the world. The bleeding edge of AI and what these cars are really doing is recognising everything in the street. If it sees a detour sign, it needs to know ‘that’s a detour sign.’ So somebody had to train it to recognise detour signs, stoplights, stop signs and all the other things you would come across on the street. It gets more complex with humans, right? If there’s a cop in the street holding his hands in front of your face, you know that means ‘stop,’ and you stop, otherwise you get a ticket, and maybe even get arrested. The car has to recognise that gesture as well.
The glasses are going to know a lot about us as well, our behaviour, and then we can do all sorts of fun stuff. ‘Hey, Siri can you show me all the night clubs in town?’ and four night clubs will spray in the air, with video from last night, the average age, gender makeup, whether it’s mostly heterosexual or homosexual. We can start telling just by eye looks what kind of people are going to this night club. Everyone is wearing Apple watches, so we can tell how many people are dancing in the night club. So now finding a night club has changed, finding a restaurant, watching sports has changed.
So the likelihood that you’re choosing the right place increases…
Yes, because it’s going to know a lot about you, and a lot about what the experience of going to that nightclub is, and it can show it to you in video. So you can actually see how many people are dancing right now, and whether it’s a club you’re going to be entertained at.
I was struck by your description of applications in education, such as kids learning about World War II by watching VR videos…
Not watching. Walking around a battlefield. I have an app on my HoloLens that lets me walk around a piazza in Rome. Now, it’s not as sharp as really going there, and you can’t certainly have the same experience as really being there, but you get to experience what it’s like to be there… And you certainly see how pretty the architecture is and the kinds of people that are walking around It’s pretty cool. Just a week ago I got Facebook live 360. Now I can pull out a $200 camera, attach it to my iPhone and capture the scene in 360 and people can watch that live. I broadcast live in front of Preservation Hall last night ‘This is what it’s like to wait in line’, and now you see the street scenes, crazy street scenes, people dressed up and all kinds of stuff going on…And now you can explain the context of what it’s like to be on the street in front of Preservation Hall.
One thing you’ve written is that language may become obsolete. What does it mean?
I don’t believe that, but we are reading less. You see how posts are getting shorter and shorter, and becoming more visual, more video, more pictures, soon 360 video and soon volumetric video, where you actually have a 3D representation of the world you’re broadcasting.
It’s a scary thought to think that we’d be using language less and less.
No, I don’t think that’s going to happen. We’re still going to talk to each other (…) You’re still going to have to communicate, look things up and have to learn… particularly if you want a job. Jobs are going to get different and harder to get. The easy jobs will disappear because they will be done away with AI. And easy might even mean the white collar worker poring over spreadsheets right now. I can see a world where you can teach an AI assistant to look at spreadsheets for you and tell you whether you’re making a profit or a loss or if someone is stealing money from you, or what patterns are changing about your business… Whatever you’re looking for on your spreadsheets. Or if you’re getting a lot of sales with millenials, you may want to unpack that and understand it.
You’ll see that kind of pattern in 3D on a chart on your table. Now you and your executive team are going to walk around a 3D graph of what’s going on and it’ll be laid on top of the United States map and you can actually visualise your sales in real time and understand what the hell went wrong in California, for instance. And talk about that new pattern. Jobs are about to change quite a bit due to AI and there’s a whole raft of new jobs, plenty. It’s just that those will be the really hard jobs, the ones that you need to go to university for four years minimum, to learn computer science or optics design… You’re a chemist, a physicist, or you’re building software…
What will happen to smartphones?
I’m not using one in four years.
Really? What are you using instead?
The glasses. Self contained glasses that aren’t big and heavy like these HoloLenses. The glasses are going to weigh 3 oz, 4 oz, in the next 18 months.
So if I call you, you answer it here (pointing to the glasses)?
Yes. You might even have a virtual phone in your hand. Facebook Spaces is a new VR thing from Facebook, and I can call your phone, your real phone, inside VR and when you pick up, you’re a virtualised phone in my environment. And I can actually grab your phone with your video running on it and I can see you and talk to you. And I can put it anywhere, I can make it bigger, so I can see you a little bit better, and it’s crazy. And that’s just virtual reality, right? Wait until that is just here in your hand… I’ve held virtual Facebook posts and spun them around and it’s really crazy…
So every company will have somehow to learn how to use VR…
Yes, and that’s why I think that’s a positive sign for jobs that nobody is talking about. First of all you’ve got to get people to dream about a world where everyone is wearing smart glasses that are doing this kind of mixed reality, and then you’ve got to get them even further. We’re going to have education.. this technology is better at education than we have ever invented. It’s way better than a book. At Caterpillar, they’re putting these glasses on people and teaching them how to fix a 1 million dollar tractor. In real time on the tractor, so it shows you the four screws you have to undo, then the panel you have to undo, then shows you where the oil pump on the tractor is and how to undo the 15 screws, whatever, that are holding that oil pump and how to disconnect all the connectors. It teaches you in real time and checks your work while you’re doing it…
It sounds really crazy to me…
Sounds really crazy but it’s only three to four years away, and this is going to create millions of jobs. So if I’m at Coca Cola now, I have to have a team building virtual brand, a virtual staff for my brand, so that when you look at a Coca Cola can with these glasses on, squirts will jump out of the can onto your table, give you a performance for five, ten or twenty minutes, and then jump back in the can and disappear.
Think about how much work just to create 5 minutes on your kitchen table. Someone has to have a volumetric camera, somebody has to stitch all the images and all the digital things together, somebody has to programme it so that it works right and if you tell it to play a special song for you it can do that, and on and on. Millions of jobs are coming, but these are jobs that are creative at minimum, which means we have to take a truck driver who’s used to driving a truck around, because they’re going to lose their jobs in 7, 10, 15 years. Everything is going to be self driving eventually. And they’re going to start losing their jobs in 7 years. So we need to find a way to 1) educate that truck driver, 2) take care of their family for four years or some amount of time so that they can educate themselves and find something new to do.
It’s going to be hard. Truck driving is not a university-level job…
We need a new GI Bill. My family went from Pittsburgh coal mining to top in the world of technology. My dad was an engineer at Lockheed Martin doing military selling, so in one generation he did that, because he had the GI Bill. He could have six years to go to school and learn something, to make his life better, and make my life better (laughs), which I’m very appreciative of, that I’m not digging in a coal mine right now… So it can be done, one. Two, it takes some time. Three, we need a dream, a new American dream. We don’t have one right now. We’re fighting against each other and having a very divisive politics. But I think both sides will be amenable to that kind of argument. ‘Hey, we need a new education plan for America. We need to take a truck driver and give them a fair shot at a decent life. You can’t just put them on the street. What are you going to do? Millions of people on the street? That’s not going to work.
My job might even go away in seven years, so I have to learn new things, and it’s hard, but I have an education tool that is unparalleled now. I can learn new skills that might have some value somewhere soon. I’m a positive person about technology. I know that the Black Mirror stuff is there. I watch it too and I can tell you all about the negatives. Cars kill 30 thousand people a year, yet we all drive… Why? Because there’s utility to driving, a deep utility and it’s worth it taking the risk even if we might die today. And we have arguments about how weird society is going to be with mixed reality, or how much privacy is going to go away… But lack of privacy never killed anybody and yes, we still drive…
These glasses are going to educate us, immerse us in experience and new kinds of entertainment, all sorts of new things, new kinds of journalism, new kinds of sporting events. At a football game you’re going to see the stats from the players on top of them while they’re running down the field. So you know how fast he’s running, and is he beating everybody else in the league right now? I see millions of jobs coming, but we do need to figure out how to retrain people. I think these glasses are the trick. I can take you into a Stanford classroom with these, and you can have the same education that someone paying $70,000 a year at Stanford is having. That’s pretty cool, right? It democratises a bunch of things. It reduces the inequality in society that we’re seeing.
It depends on the walls they build around it and how much they charge.
They’re giving it away for free on YouTube, so I’m not too worried about it, you know? Educators like to educate people. Someone will figure out how to do this very inexpensively, because the scale is so nuts. Particularly if there’s a GI bill that’s helping you pay for this stuff, right? And helping the companies do better chemistry education, or better moviemaking.
- This Q&A is the eighth in a series of interviews with tech leaders during the Collision conference in New Orleans, 2-4 May 2017.
- The post gives the views of the interviewee, not the position of LSE Business Review or of the London School of Economics and Political Science.
- Before commenting, please read our Comment Policy.