If I had a dollar for every 3D startup that has failed, I’d probably have … a few hundred bucks. Getting no-glasses no-headset 3D right is insanely difficult.
But Looking Glass Factory CEO Shawn Frayne says he’s done it. It’s shipping now, and I have friends who say it’s the best thing they’ve ever seen. He’s been working on it for literally half his life … and it has the potential to revolutionize photography, memories, and maybe more. Think collaborative work in 3D. Think your everyday monitor being 3D capable. Think living memories.
Or stop thinking. Just watch.
Support TechFirst: Become a $SMRT stakeholder, or check out $SMRT rewards
In this episode of TechFirst with John Koetsier, I chat with Frayne about why he built this holographic display, where he uses the 4K and 8K versions fitting in, and where he sees holography in 3-5 years.
Keep scrolling for the full video, to subscribe to the podcast, and a complete transcript …
Or, read the story in my post at Forbes …
Watch: are holographic displays the future of the metaverse?
(Subscribe to my YouTube channel so you’ll get notified when I go live with future guests, or see the videos later.)
Subscribe: this holographic display has been in development for decades
Apple Podcasts
Spotify
Google Podcasts
Overcast
Other podcasting platforms
Read: are holographic displays the future of computing … and the next major evolution in displays?
(This transcript has been lightly edited for length and clarity.)
John Koetsier: Didn’t someone once say that the road to hell is paved with 3D startups? 3D tech has always been ‘the next cool thing’ and has usually failed to deliver on its promises … but maybe not anymore.
Looking Glass Factory has built a very interesting holographic picture frame. At least one friend says it’s the best thing he’s ever seen, and it has the potential to revolutionize photography, memories, and maybe more. We’re talking to the founder today, Shawn Frayne. Welcome, Shawn!
Shawn Frayne: Hey John. Good to be here.
John Koetsier: Hey, great to have you. Give us the big picture: what is Looking Glass?
Shawn Frayne: So Looking Glass is a holographic platform, including our flagship product, The Looking Glass, which is the world’s first holographic interface that folks can use to look into a 3D world of content, whether that’s real-world content or something synthetic, and folks can do it without needing to put on a VR/AR headset.
John Koetsier: Interesting. So I can have a picture frame on my desk, on my wall, whatever, and it’s a 3D picture that changes as I move around?
Shawn Frayne, CEO & Co-founder, Looking Glass Factory
Shawn Frayne: That’s right. I mean, it’s in our name, the Looking Glass, and in the name of our company, Looking Glass Factory. It’s the idea of having this magical window that lets you peer into a three-dimensional space.
John Koetsier: You’ve been working on this a very long time — since you were a kid, in fact, with your dad. Tell us the story.
Shawn Frayne: Well, I got into the dream of the hologram like many folks did, I guess, when Marty McFly was getting gobbled up by the holographic shark.
John Koetsier: McFly! McFly!
Shawn Frayne: Exactly, in Back to the Future II. And, you know, that vision of the future hooked me. And I asked my parents if there was any way to get it. I was something like eight or nine at the time, and no one had holographic interfaces for sale at that time. So they got me a book called the Holography Handbook and found it under the Christmas tree one Christmas.
And then my dad and I used that to build a holographic studio at the foot of my bed in Tampa, Florida when I was in high school. And from there, figured out all of what was possible at the time and everything that was missing, and fast forward 15 years, and me and my co-founder Alex started a company called Looking Glass Factory to reach the full dream of the hologram.
John Koetsier: Insane. How often does that happen? I mean, you’ve got entrepreneurial people, you’ve got people who are in the startup world. How often does it happen that a passion that you had as a little kid comes to life in a new startup and actually successfully launches a product? Very impressive.
Shawn Frayne: Thank you.
John Koetsier: I want to go into the tech. I want to go into how it works, what it looks like, the use cases, all that stuff, and the future of it. But I want to take a step back for half a moment and just ask this question: why does a holography matter? What’s important about it? You actually wrote about it recently, you talked about memories and you talked about your brother. Can you answer that in that context?
Shawn Frayne: Sure. Yeah. So my brother Ryan, he passed away a few years ago. When he was still around, he was sick with pancreatic cancer — he was an inventor too — and one of the things that I had wanted to do was to capture a video message from him to his recently-born daughter.
We weren’t able to actually pull that off. We had the display prototypes that could kind of do it. We had some of the camera systems that could kind of do it, but at that exact moment in time, a few years ago, it all didn’t coalesce into a holographic recording and playback system. Now we actually have that, today, and so … I don’t know, there’s a bunch of things that are tied up into that memory.
Some of the capabilities of the new Looking Glass holographic display, according to the company.
And, you know, the importance of memories is one of those things, how we remember folks in our lives that are still here and also folks who are not here anymore, I think, is something that a new interface naturally plays a role in. There’s also, there are other elements of that as well, like how you connect with other folks that you’re not with physically in the same space but might be on the other side of the world. And I believe Looking Glass has a powerful role to play in all of that.
John Koetsier: Thank you. At least one of my friends, I mentioned off the top, has said this is his new favorite thing. And this is a friend who is in technology, he has every conceivable latest gadget, perhaps not an unlimited budget, but almost. You have a product that he’s got in his house right now that starts at $300 US, says it’s his favorite thing.
Why is that happening? What’s eliciting that response?
Shawn Frayne: You know, the closest thing I can equate it to is maybe the — and this’ll sound arrogant and maybe it is a little bit arrogant, but the shift from photograph to film.
So there was a time, a hundred plus years ago, where folks had memories and illustrations of imagined futures that they could see in photographs and in paintings, but it wasn’t, they were not alive in the way that things are alive in the real world. Then somebody came along and put 12 of those pictures in sequence in a second and they repeated that, and then a new medium, of film, was born. And that became closer to life, closer to what we see around us.
And the jump from flat media, flat computing devices, flat display to spatial systems like the Looking Glass, is at least as big a jump as the jump from photograph to film.
And so I think maybe what your friend is responding to is the fact that there’s never been something — never been a device, a platform that actually could represent both imagined realities and things from your own life, real captures, real photographs, real videos in a way that felt true-to-life dimensionally. And that’s a big step, I think.
John Koetsier: It’s really interesting that you put it that way, because it brings to mind some of those videos that we still have of people watching a movie for the first time …
Shawn Frayne: Right
John Koetsier: … of the locomotive, the train coming towards them out of the screen and they’re all backing away and shying away from the screen because it’s so real to them. It’s so out of their experience, something they’ve never imagined, that they feel like it’s actually coming to them out of that screen. It’s interesting because there’s also, it’s a different startup, but they’ve taken people’s old photos …
Shawn Frayne: Oh yeah, those guys.
John Koetsier: … of relatives. You’ve seen that, you’ve heard of that. And they’ve 3D’d them — that’s probably not a word [laughing] and that has been…
Shawn Frayne: It is now. It is now, yeah.
John Koetsier: Yes, exactly. And that has been really transformative for how people see their ancestors, remember a parent or a grandparent. I want to talk a little bit about your product range, because you’ve got a massive range. You’ve got this initial product that I think it’ll retail for $500 or something but it’s on a presale for $300, and you’ve got products up to $15,000. It’s a massive range. Talk us through what that is.
Shawn Frayne: Yeah. So we have the whole family of Looking Glass hardware which runs all of our software product, but the hardware part of the product lineup goes from small, medium, to large — Looking Glass Portrait, the 4K Gen2, and the 8K Gen2 — and those are at a pretty wide range of prices, which is a bit unusual.
But what we’re doing is fairly unusual with a big shift in the interface to something that people genuinely didn’t think was possible.
So, when we were planning out what we were going to do for this entire family of Looking Glass products, we decided we needed to have a bite-sized piece of what the Looking Glass could deliver. Something, it’s a 7.9 inch personal holographic display that you and your family and friends and colleagues can all see at the same time without needing to gear up. And it’s very high quality, it’s just smaller than the larger format systems, and we needed to get that onto folks’ desks because a lot of folks genuinely don’t think what our team has done is actually physically possible. And so we thought the best way to do that, especially during a time when it’s hard to have folks into our office and showrooms to see the technology, was to get them a piece of it on their desk.
So that’s where the Looking Glass Portrait, the smallest unit, comes from.
And then naturally some folks once they see the potential and have that moment, like you’re saying folks did with film where they see the train coming at them and they have the ‘aha’ that something has shifted, something is different now — then some of those folks decide they want to use it for work. They want to use it for larger demonstrations of something three-dimensional that they might’ve made in their lives, so … that’s where the lineup comes from.
Importantly, any piece of holographic media or any holographic app that runs in one of the Looking Glass units runs in all. So there is that hop between the size systems that’s possible.
John Koetsier: You also have, perhaps it’s one of your products that has interactivity so you can engage with it and you can move and it reacts to you. What’s going on with that?
Shawn Frayne: It’s great to interact directly with holograms. And so we have software to allow any peripheral, whether that’s a simple thing like a mouse, or VR or AR controllers that folks might use in other environments, or hand trackers to interact with holographic content.
So most of the time folks who are building holographic apps using platforms like Unity or Unreal, they can then attach these peripherals into their application and use that to directly interact with the hologram. And we have some examples that folks can download for free directly if they have a Looking Glass, for all of the size systems actually.
John Koetsier: So you’ve mentioned a couple of times that people look at your tech and they think that’s not possible, or it’s at least before they see it they think it’s not possible. Talk about the tech. What have you built? How are you making it happen?
Shawn Frayne: Yeah, so I’m kind of a nerdy guy, so this’ll come off from the nerdy angle of it.
John Koetsier: That’s okay, this is a fairly nerdy podcast.
Shawn Frayne: Okay, alright. So every display, including the one that we’re chatting with each other through right now, all of those for all time have always worked in fundamentally the same way: they shine with pixels, points of light, that have two properties of intensity and color.
But the real world, like these glasses or this cup of coffee, these are real to us in a way that’s different than what we see on a flat screen because of a third property, of directionality. So the light that’s bouncing through my glasses and bouncing off of that coffee cup and bouncing off of everything in this hotel room, that has this third property of directionality.
That gives things it’s dimensionality — I don’t know if that’s a word either, but just made that up — it gives the world specular detail when you see the glint off of someone’s eye or off of a river or what have you. That’s real because it’s three-dimensional, it’s specular, etc., etc., etc. And so what we’ve done is we’ve added that third property to millions — and in the case of our biggest system the Looking Glass 8K, a hundred million points of light — and we then can control that.
So we control intensity, color, and direction. Essentially synthesizing then, the field of light that represents reality, but now through a holographic computing interface. Is that nerdy enough for you?
John Koetsier: Yeah, that was good. I think that was the right mix of nerdiness and understandability. No, that was perfect. I want to talk a little bit about what people can put into it. You can take photos with depth. You can share photos with depth from an iPhone, a modern one with lidar, some other phones as well, share them to Facebook. Do those work? Do those have dimensionality on your platform?
Shawn Frayne: Totally. So most folks don’t know this, but every time, as you’re saying, every time someone snaps a portrait-mode photo, a depth map or a recording of the depth of that scene’s recorded, typically to blur out the background around the main subject of photo. But we can use that same depth map information to generate all of the different perspectives needed for a holographic version of that photograph.
So we have software, including in our flagship suite called HoloPlay Studio, that does that automatically based on portrait-mode photos. And of course, there’s a lot of folks in 3D land that work with tools like Unity, Unreal … creation platforms like Blender, Zbrush, etc. that can then take 3D creations from those platforms and have them essentially come into the real world through the Looking Glass.
John Koetsier: And if I have photos, can I just import them? Is there a website? Is there an app? How do I get them into my Looking Glass Portrait?
Shawn Frayne: Yeah. So, this is a prosumer device at this stage in its evolution. So you connect up your Looking Glass Portrait to a PC or Mac, and then get your photos onto that computer, and then through HoloPlay Studio you’ll see them appear instantly in the Looking Glass Portrait. If you want to then save them onto the device of the Portrait itself, then there’s a sync button that basically then processes the content to save onto the device.
John Koetsier: Can I save multiple? Can I save a thousand …
Shawn Frayne: Yeah, you can create playlists and things like that.
John Koetsier: Perfect. Let’s talk about the future a little bit. This sounds exciting. I want one. I want to get one. I want to play with it, have some fun and display some of my favorite photos. That’s great.
What can we expect in three to five years? Can we expect something significantly larger? Does it always have to have depth of the actual product in order to have that depth of light field? What’s it look like in three to five years?
Shawn Frayne: Yeah, so I’ll take the questions kind of in reverse. So, this newest generation of Looking Glass is actually hollow.
So it might look like there’s a block of material on the system, but in fact you can reach into the device and your hands, in this example, exist in the same physical space as the light representing the hologram. So if you have a hologram of a flower growing for instance, in the Looking Glass, your hand can be there in the same physical space as that holographic flower.
And that’s different than the prior generation of our dev kit, versions of the technology. Where this is going… I mean, I think … so we have the world’s only, at this point, holographic platform — hardware, software, the community.
That, when it’s combined with a global communication network, essentially allowing the Looking Glass systems to be networked to one another so that what I see in my Looking Glass you can see in your Looking Glass, whether that’s a person, a 3D model of something, a scan … I think that is when the next major shift occurs.
It is very similar to the shift that happened for personal computers as we all know very well, but in this case on a much tighter timeframe, because all of that infrastructure has already been developed and we’re essentially building this new interface on top of it.
John Koetsier: In other words, this sort of photo frame reality that we have right now, and of course you have a model that can be used in work at this moment or shipping soon, but this photo frame kind of reality transforms potentially to a reality in which, hey, I’m remote working, as so many people still are and may continue to do for the foreseeable future, but I have some level of 3D presence with my coworkers. We have some level of combined 3D presence with a model of something that we’re working on. This works in social engagements and interactions as well, correct?
Shawn Frayne: That’s right. So, one of the greatest benefits of the Looking Glass today is that you can collaborate more effectively around 3D content with someone standing right next to you in a work context. And that’s already being extended in the hybrid office situation that most of us are in now and expect to be, including our company Looking Glass, expect to be for the foreseeable future, where I’ve created something.
Let’s say, a molecular model of a new drug that I’m designing or a part that I’ve designed in CAD, and I want to have that represented in as true-to-life form as possible with my colleagues that are not in the headquarters; they’re working out of their home office and what have you. Right now, today, I can send them a hologram and they can view that in their Looking Glass, and we can get on the same page, have Zoom open separately and whatnot on the side, and have a conversation around this instantly generated essentially hologram of whatever it is we need to collaborate around — in this case, in-person and remotely.
That’s going to take another step in the not-too-distant future where I can have synchronous communication, so I can also be there with my colleagues represented in a holographic video call and what have you. So, that is much, much, much closer than I think most folks realize.
John Koetsier: Shawn, thank you for taking this time. It’s been a real pleasure and an eye-opener.
Shawn Frayne: Thanks a lot. Great to be here.
Interested in tech and AI and gadgets? So am I
Made it all the way down here? Who are you?!? 🙂
The TechFirst with John Koetsier podcast is about tech that is changing the world, including wearable tech, and innovators who are shaping the future. Guests include former Apple CEO John Scully. The head of Facebook gaming. Amazon’s head of robotics. GitHub’s CTO. Twitter’s chief information security officer, and much more. Scientists inventing smart contact lenses. Startup entrepreneurs. Google executives. Former Microsoft CTO Nathan Myhrvold. And much, much more.
Subscribe on your podcast platform of choice: