What happens when AI moves off the screen and into your field of view?
Recorded live at Web Summit Qatar, this TechFirst episode dives deep into the real future of XR … beyond the hype cycles of VR headsets and toward smart glasses, physical AI, and contextual computing.
Host John Koetsier sits down with Amy Peck, founder of EndeavorXR, to unpack what’s actually working in AR, VR, and mixed reality, and why the most important XR breakthroughs may not be for humans at all.
- Get the deepest insights concisely on the TechFirst Substack newsletter
- Subscribe to the TechFirst YouTube channel to never miss an episode
And, watch our conversation here:
In this episode, we explore:
- Why VR was never “dead” but always a destination technology
- How physical AI and robotics rely heavily on AR and spatial understanding
- Why smart glasses are emerging as the next AI-native device
- What Meta, Apple, Google, and NVIDIA are really signaling with their hardware bets
- The hidden risks of eye tracking, biometric data, and always-on augmentation
- How digital twins, virtual training, and simulation are reshaping manufacturing and robotics
- The uncomfortable truth: we’re about to let Big Tech into our most private spaces
This is a candid, funny, and deeply informed conversation about where XR is going, who will control the data, and why augmentation, not immersion, is likely the future.
Transcript: XR post-hype
Note: this is a partially AI-generated transcript. It may not be 100% correct. Check the video for exact quotations.
John Koetsier: We are going to let some of the FAANG companies that we talked about into our bedrooms. We’re gonna let them into our bathrooms.
We’re going to wear— we’re gonna be augmented. People will choose not to do this, but most people will do it, I think, just because they want the benefits. It becomes their new normal.
Amy, I am so pumped to do this. We’re recording right here at Web Summit in Qatar, and we’re gonna talk about XR. We’re gonna talk about AR, VR— all this stuff. I want to know where in the world this stuff is right now. We know that Meta pulled back massively, and we talked about the billions and tens of billions they spent there.
AI is super high on the hype cycle right now, and that’s taking up all the oxygen in the room. But smart glasses are actually getting really, really good. And you know what? Meta’s selling a lot of those, and coming out with new ones and cool ones. We still have tons of other brands— including Chinese brands— coming into that.
Physical AI, shockingly, uses a lot of AR. They really care about that. We’re gonna talk about all this stuff: where we are, where we’re going. Let’s chat.
So I’m gonna introduce you real quick, but you gotta add a few things if I skim over stuff. This is Amy Peck. You’re with EndeavorXR. You have decades of experience in AR and VR and all this other “R” stuff.
Amy Peck: All the Rs.
John Koetsier: All the Rs, exactly. You’ve helped hundreds of companies build solutions around this, so you’re the right person to talk to.
Amy Peck: I hope so. I hope so. Um, yeah, it is interesting that even four years ago, five years ago, we were talking about VR— you know, “Is VR dead?” Right? I think I did a talk at MWC— not as good as Web Summit—
And that was the title of it: “VR Is Dead, Long Live VR.” The reality is that VR has always been a destination. It’s a very specific use case.
The early traction was in training, and then we moved into healthcare, particularly behavioral health. But it’s in transition.
Like every technology, because of AI, AI has kind of become that foundational element that’s powering all other technologies. And we haven’t really cracked the code on how to apply AI into every other technology, as well as into our own workflows. So it’s more that it’s a complex morass of technologies coming at us than any of the XR technologies not being relevant anymore.
John Koetsier: I think it’s a really good thing— in spite of it probably being super annoying for hundreds of millions of people— that we have so many Rs. I think it’s actually been a good thing because if we look at what’s going on right now, yeah, you have the destination: you can put on the headset, you can enter a reality, right?
You can do that. But there are so many other applications. I mentioned physical AI, and as we were talking about before we started rolling the camera: physical AI is huge right now. There are 150 humanoid robotics companies in China, not even to mention how many hundreds or thousands of others in general robotics, and you’ve got other robotic companies all over the world.
They all need to know how to navigate in the world. And guess what? AR has figured out a lot of that stuff. Where is this thing? Where is that thing? How far— if I move this thing— will it be in front of that thing or behind that thing? If I need to grab that thing, where do I move my hand?
Talk about that. That’s an interesting second life for some of the Rs.
Amy Peck: A hundred percent. And I do want to address, also, the “all the Rs,” because it’s confusing to the public. So it’s just for consultants— they need so many Rs to confuse everybody so they can make money. And you need, obviously, one PowerPoint slide per R, right?
But I look at it as a spectrum of reality. It’s not like we talk about whether we’re using our laptop or our mobile phone— you just pick one up or you use another. And as these technologies— and to your point, the AR glasses start to get really good— we’re not going to care about which reality we’re in.
We are going to be augmented to a certain degree, or not at all, with a digital asset in our field of view. So hopefully that moves away from some of the confusion.
But back to your question: I’m very excited about the applications of physical AI because it bleeds right into a lot of the work that I do with digital twins, which is another example of the “Rs” not being dead, right?
Such a critical piece of advanced manufacturing, the AEC industry. But where it really gets interesting with physical AI is not just training the robots in dexterity, but actually training them to navigate in the actual environment in which they will be deployed.
So if you imagine you have an Amazon warehouse: you ingest that entire environment, and then you can virtually deploy these robots, train them, run simulations, and that increases safety and efficiency.
John Koetsier: Absolutely huge. Digital twin stuff is growing like crazy as well.
Also virtual training, right? It’s expensive to train a robot. It takes a ton of time. It’s slow. It’s like real-world timescales— an hour is an hour. Virtual training on environment data that you capture can be… you can run thousands of simulations that robots can learn from.
And there are some really cool new technologies, including a tech-first that I haven’t published about yet— quick training based on virtual data. That’s really, really interesting. And you have to understand your reality in order to do that.
Amy Peck: Yeah, it’s interesting. It also coincides with some of the virtual production that’s happening in Hollywood, where you have the human in a motion-capture suit doing the activity that then is captured and used to train the robot.
So when we talk about “convergence,” we’ve been using that word for a long time, but it’s really here now.
John Koetsier: It’s so funny you brought up the Hollywood stuff because you see some of the behind-the-scenes footage of how they did a movie, and your respect for an actor just goes way up because you’re like— they’re in nothing like the scenario that it actually looks like in the real movie.
And somehow they’ve got to have natural reactions, motions, emotions— everything— on a green screen.
Amy Peck: And then they have all the little dots on their faces. There was a viral video around the making of Avatar, with all the actors being tortured with all of this stuff. They’re in their green suits with the little trackers— it’s great for evening wear. I would wear one of those to the Oscars, frankly.
John Koetsier: Let’s get those suits and wear them out here. Wear them at Web Summit. See how many stares and glances we get.
Amy Peck: Exactly.
John Koetsier: I want to hit Meta, because Meta has been sort of the biggest story maybe in VR— XR, I’m not so sure, but VR for sure— for half a decade or so. Investing tens of billions of dollars, putting together a platform, hardware, multiple iterations of the hardware, and then very publicly pulling back and saying, “Hey, we’re cutting— what was it— 10,000 jobs,” redistributing people.
How do you feel about that whole episode and where we are now? Is that horrible and awful, or is it maybe just fine for the future of XR?
Amy Peck: I think it’s just business. Woz was on— I think he did an “ask me anything” live on maybe Instagram— and there’s always sort of a spin component to these things. But in the scheme of things, Meta really jump-started the race in immersive technology, and I think even accelerated Apple coming to the party.
And what they did— which I thought was really smart and fascinating— was: you have a VR headset, but they didn’t say, “We’re gonna take that VR headset and build the smallest version of it we can for AR.” They said, “Let’s find something really fantastic that people want to wear, and we’ll put as much technology in it as we can.”
And between the Meta Wayfarers one and two— what a massive difference. So where they’ve come: they’ve accelerated the industry.
On pulling back, I think it’s just business. AI has kind of made everybody change their business decisions.
And NVIDIA has obviously been a huge marker in this world. Jensen Huang has gotten up on stage and said, “We are going to be iterating on these chipsets every year.” So throwing down the gauntlet and saying, “This is now the pace of innovation for every company in the world.”
And of course the FAANGs are gonna follow suit, right? Wouldn’t you love that term? “The things.” How’s the “don’t be evil” thing going?
John Koetsier: No, it’s so great.
Amy Peck: We’re just kidding. We’re just kidding.
John Koetsier: A couple things I want to get into after the Meta conversation, because a couple things come to mind.
One, I want to talk about Apple and the way they approached the market. And two, I want to get into smart glasses, which is seemingly where Meta is investing more energy and excitement right now.
Apple had this “we are gonna do it the best” approach, right? Super expensive hardware, and it’s a super amazing experience. I wanted to buy one. You probably did, because you’re in the space.
Amy Peck: You know, I didn’t, because I wear hard lenses and I would have to get soft lenses to do it— or get the insert.
John Koetsier: Yeah, yeah. I wanted to buy one— I wanted to want one. Let’s put it that way.
I’ve had several of the Meta headsets, and I just decided: they sit and collect dust. That’s just honest. If I have five minutes to chill, guess what? I can pick up a mobile game and chill. And I don’t have this half-hour— it’s not a half-hour, but psychologically it feels like half an hour.
Amy Peck: Yeah. It feels like a thing, because you gotta drag it out and put it on and get it set up.
John Koetsier: Where’s my boundary?
Amy Peck: Yeah, I know. I’m always hitting my hand on the wall. I have no more lamps in my house. So, exactly— all high hats now.
John Koetsier: Exactly. So let’s talk about Apple first and maybe what they did and where they’re going.
We know there’s a second version coming. It appears from the rumors that it’ll be— I don’t want to say a lower-spec machine— but they’re not trying to hit a grand slam. They’re trying to get something that’ll be more fun to wear and still deliver a good experience.
How do you feel about Apple vis-à-vis where Meta was and what Meta did?
Amy Peck: Yeah, I mean, the Vision Pro is a beautiful device. It’s incredible. And it’s also a workout device for Formula One drivers— strengthen the neck.
But it does make sense to do a lighter-weight version. Again: something people want to wear, and put as much functionality in it as it will allow at this moment in time. And that has already been accelerated quite a bit.
Even Google is working on their AI glasses. Samsung is purportedly working on another lighter-weight mixed reality device. These devices are coming.
In terms of Apple specifically: they do everything right for the most part. People argue that the Vision Pro was a complete miss. I don’t think so, because I would go and stalk all the Apple stores and the surprise and delight that people had— not $3,500 worth of surprise and delight. There was a cap. It was maybe $1,300 worth of surprise and delight.
But as it gets down to the price of a phone— and Apple, their ecosystem, and their privacy— what we all forget about and no one talks about is the eye tracking.
That is— there are some serious risks.
John Koetsier: This thing sees your whole world, and it sees you.
Amy Peck: It sees how you react to every input in ways that we are not even conscious of. So if I put a piece of advertising or an object in front of you, your pupils are gonna react before you cognitively make a decision about how you feel about it.
That information— for me— scary. I can sell you anything if I have that information about you.
John Koetsier: Anything? Well, challenge accepted.
Amy Peck: I’m—
John Koetsier: Starting some condo in Greenland.
Amy Peck: Yes, yes, exactly. Greenland, comma America.
John Koetsier: Let’s talk about smart glasses, and I want to highlight something you just said.
You said “Google’s AI glasses.” You didn’t say “Google’s smart glasses.” And that’s interesting, right? Because we see OpenAI supposedly coming out with some hardware that Jony Ive is working on with them, and we’ve seen other attempts to create AI hardware.
It may be the case— I’m gonna bet that it is the case— that smart glasses are the AI device, right? Of course this is also the AI device, right? The smartphone.
But the glasses will see what you see. Interesting. I was just chatting with the CEO of Oura— you know, the smart ring company?— and he said they will go to other form factors. And he said what’s most interesting is smart glasses because it sees what you see, and it’s connected to places that are sensitive. It can sense all kinds of motion and physical characteristics— like heartbeat and everything— on your ears and other places like that.
So talk about smart glasses, because that seems like… I almost bought some of the new Meta smart glasses. They’re still a bit chunky. They’re still not— you know— I like my Ray-Bans. Can you make it that small? Probably not.
Amy Peck: Right. Eventually, yes. Eventually, yes. And again, it’s down to functionality.
I don’t think we want these digital displays on all the time. In the same way that we— at least at some point during the day— put down our phones, we want this short, sharp shock of, “Oh, I have a message.” I can reply, or I can swipe it away.
I’m imagining people on the street doing this, and it’s gonna be like—
John Koetsier: Ready Player One, final scene. Everybody’s fighting.
Amy Peck: We’ll have all these weird little ticks. Like when you couldn’t see the person’s little Bluetooth headset and they were gesticulating wildly in the middle of an airport.
So we’re headed for that.
John Koetsier: It used to be the case that when you saw a person talking to themself on the street, they were crazy.
Amy Peck: Yes. Just so you know— yes. Yeah. I forgot that I’m dating myself talking about Bluetooth headsets.
But yeah, I think the glasses are very interesting. And I think we’re moving towards— you mentioned Oura— there are a lot of companies working on smart fabrics to detect more accurate biometric response.
Are we able to triangulate this data? In a perfect world— if I wake up and I’m queen of all things tomorrow—
John Koetsier: That’s happening.
Amy Peck: That could happen. I’m just saying. I will be a benevolent queen.
John Koetsier: Do we have to trust you on that?
Amy Peck: Pick me. Pick me.
But the important thing is: if we can manage that data— and make the effort to manage that data— we can take much more control of our own health and wellness.
So the benefits are great, but the dangers are equally as great. And I don’t hear enough people talking about the data, and we know this is a problem.
John Koetsier: It definitely is. We do have to have that conversation.
I was on stage twice yesterday at Web Summit Qatar, and the one thing that I really want from a great set of smart glasses— and I believe Meta announced this feature a couple months ago— is that I can have my notes right up here.
Because you know what I do? I take this up, and okay— there we go— and I put it down. I’d love to have nothing in my hands. And if you’ve noticed, I’m a hand-talker.
When you’re communicating with somebody— whether it’s an individual or an audience— you want eye contact. You want to be looking up, not looking down.
So that’s a killer app for me: being able to see my notes. I don’t need a lot. I don’t need to script. I don’t need to read everything. I just need my notes so I know where I’m going, where I was, all that stuff.
That would be amazing.
Amy Peck: Yeah. And the good news for parents in the future is that their kids will be able to ignore them by looking directly at them.
John Koetsier: Is this good news?
Amy Peck: They’ll be like, “Oh, here’s my Instagram feed,” and they’re not even paying attention.
And anyone out there who’s a builder— please build this for people who are meeting a thousand people at these events. Wouldn’t it be great if it recognized that person and pulled up the conversation you had with them a year ago?
It’s a little dystopian, but it would be so useful, because I feel like kind of a jerk when I don’t remember people I know.
John Koetsier: I know. And I feel like a jerk all the time.
I recognize you, and I’m really happy about that. But I have a thing— I don’t know what it is. Maybe there’s a condition— I sometimes don’t recognize faces very—
Amy Peck: No, I don’t.
John Koetsier: I suck at it.
Amy Peck: I really suck at it. It’s too many. We’re meeting too many people— that’s the problem.
I think I’m literally out of RAM. I have to lose a piece of information. And I’m not gonna lose my kids’ names, so it’s gonna be somebody’s face or somebody’s name. That’s what’s gonna happen.
John Koetsier: That is a real challenge for me. And remembering names is a real challenge for me.
And of course our relationship goes back like a decade. We’ve connected digitally when we haven’t met physically, so that worked out well.
But it would be amazing. I had it just the other day— and it happens at least once every conference— somebody walks up: “Hey, John, how’s it going?” And you’re just like…
And I just play. “Oh yeah, how you doing?” And I have no freaking clue who this is.
Amy Peck: I know. And it’s kind of embarrassing. And then you’re trying to surreptitiously look at the badge.
Yeah, it’s not good. It’s not good. I’m ready for the chip. I’m not a v1 chip person. I’m like v3.
Just like, “Okay, I’m on my way to Tokyo”— boop— “Ohio.” I’m ready for it.
John Koetsier: You’re downloading new skills, like in The Matrix.
Amy Peck: Yeah, I’m ready. Completely ready.
John Koetsier: Let’s talk a little bit about the future.
We’ve talked about all the Rs. We’ve talked about the hardware and the fact that there’s really cool hardware coming out. It isn’t necessarily immersive, but it is augmentative. Whether it places objects in your visual field or just places information over them, it augments your experience and your reality and your understanding of your meeting or other things like that.
As we move out from here, the next few years— I don’t see Meta pulling back on investment in smart glasses. You mentioned Google AI glasses. You’ve got to think Apple’s thinking about what they’re going to do in this area. And they always have like three or four products in the pipeline that you never hear about.
So what do you think the future holds for us?
Amy Peck: Yeah. Again, I think we should be looking at it from: how would we as consumers want to architect it?
Because the problem is these devices really haven’t had the uptake. Starting with Google Glass, there’s not a lot of utility yet that consumers are asking for because we don’t really know what we want.
But what Apple is really, really good at is putting a device out there on the market, then being very limber in their SDK and looking at what people are doing organically, and then baking that into the SDK.
So developers— and then you start to get this app ecosystem where the gestures are the same, the voice commands are the same, the swipes are the same— all those kinds of things.
John Koetsier: All my music is here, my mail is here, my messages are here.
Amy Peck: Exactly. It all just feels the same.
And where do you want it in your field of view? Everyone’s gonna be a little different. So I think where AI is also going to be a benefit is in really customizing the individual experience.
Like, I have contacts, so one eye is for distance and one eye is for close-up.
John Koetsier: Different colors there would be really cool. One red, one blue.
Amy Peck: Exactly. Your hand comes at me like this— I have super depth perception. It’s like “John sees 3D.”
But that sort of personalized experience is what we’re after. Again, the deal we’re gonna be making is: personalized experience means the deal with the data devil. Let’s trademark that: the data devil.
John Koetsier: Maybe it’s OpenAI, I’m not sure.
But that’s super interesting, and we should chat about that a little bit because the thing about AI— we all use it. Very few of us own the AI that we use.
I know some people are super hackers and they’ve hacked together their own LLMs. They’re running DeepSeek or something else locally on their devices, and they own their data in a way that none of us mere mortals ever will.
So if you’re going to partner with somebody— if you’re going to buy into somebody’s ecosystem— you better be certain you trust them.
Apple obviously comes out as one of the ones that is a sort of shining light there. I’m sure there will be others, and others will always say you’ll be safe, but will you really trust them?
That’s one of the challenges for Amazon. They sell a lot of stuff. That’s one of the challenges for Meta— they make money on ads. And one of the challenges for Google— same thing.
It’s a real challenge. We are going to let some of the FAANG companies into our bedrooms. We’re gonna let them into our bathrooms.
We are going to wear— we’re gonna be augmented. People will choose not to do this. But most people will do it, I think, just because they want the benefits. It becomes their new normal.
You saw how quickly smartphones became that three-foot device you can’t have more than three feet away from your body without entering a little panic attack.
Amy Peck: Where’s that? I have two.
John Koetsier: My precious, my precious.
Amy Peck: So I can use one to find the other one.
John Koetsier: Are you joking?
Amy Peck: No, I’m not. This is my $700 iPhone finder.
John Koetsier: I like your strap.
Amy Peck: Isn’t that good? Very fancy.
John Koetsier: I like my orange because I like orange as well, but holy cow.
Amy Peck: But that’s also very athleto, you know? You’re out there fighting the good fight.
John Koetsier: Totally, totally.
Amy Peck: Which way to the beach?
John Koetsier: What are you talking about? Beach? I don’t know— the beach gym.
This has been a ton of fun. It answers a few questions and raises a few more, as these things always do.
I do think the future is augmented. I think the future is intelligent. I think the future is contextual.
The key question will be: what device? Who owns the data? Where does it go?
Are we safe? Are we secure? Can the government get their hands on the data?
We’re in the Middle East— I’m going back to Europe— and they’re not looking at American tech companies as very trustworthy anymore because the American government can tell them to do something: turn something off, access something.
How do you make that safe? How do you make people feel like, “I will buy into that and give that company all the data about my life”?
Amy Peck: Yeah. A lot of the countries we’re working with in this region are talking about sovereign AI because they don’t want to be customers of foreign AI or foreign underlying infrastructure and technology.
But the level of investment to build your own foundation models is insane. So there’s going to be some sort of happy medium.
And a lot of it is going to boil down to trust. There’s a lack of trust in the geopolitical landscape.
And even as individuals, we can’t just trust these models, because also think about all the AI— I hate to use the term “slop” because it seems so dismissive—
John Koetsier: Some of it is very sloppy.
Amy Peck: But some of it is just not good. And that is now a wash throughout the internet.
John Koetsier: Go to LinkedIn.
Amy Peck: Go to LinkedIn. But the models are training on that as well.
So we have to start thinking about different ways of feeding these models new, fresh data, and being able to extrapolate from what we know into much more creative thought, as opposed to collapsing in on the average middle. That’s just sad.
John Koetsier: It’s sad.
New foundation model idea: we’ll train the LLM on only 17th- and 18th-century books. Everybody will sound like a pilgrim on LinkedIn.
Amy Peck: Exactly. I kind of like it. Good sir—
John Koetsier: Amy, this is why I do this podcast. I like talking to smart people who don’t take themselves too seriously, who have a lot of fun, know a lot of stuff, and can have a great conversation.
Thank you so much for this time.
Amy Peck: Thank you. It’s been great.
John Koetsier: And I can’t do this on most podcasts, but hey— fist bump.
Amy Peck: Boom. I like it. Thanks.
John Koetsier: John. Have a great day, everybody.