There’s a new app that can tell your heart rate, heart rate variability, oxygen saturation, respiration or breathing rate, and mental stress just by taking a short video. Soon, blood pressure is coming too.
Sounds crazy, right?
In the episode of TechFirst with John Koetsier, we’re joined by David Maman, CEO and co-founder of Binah.ai, to find out …
Get the full audio, video, and transcript of our conversation below …
Subscribe to the TechFirst podcast: digital health data by video
Apple Podcasts
Spotify
Google Podcasts
Overcast
Other podcasting platformsWatch: digital health data by video
Subscribe to my YouTube channel so you’ll get notified when I go live with future guests, or see the videos later.
Read: digital health data by video
John Koetsier: Can you monitor your health just by video … via an app? Welcome to TechFirst with John Koetsier.
There’s a new app out that can tell your heart rate, your heart rate variability, oxygen saturation, even your respiration or breathing rate, and perhaps even your mental stress … and soon, blood pressure, just by putting you on video. Sounds kind of crazy, right?
Well, to figure out if it actually is, we’re joined by David Maman — I should have asked how that’s pronounced — CEO and co-founder of Binah.ai. David, welcome!
David Maman: Hey John, great being here. Thank you for having me. I’m a fan.
John Koetsier: Excellent. And did I screw up your last name or is it pretty good?
David Maman: I usually use “Ma-man.” No, Maman is fine, I’m kidding.
John Koetsier: Okay. Okay, that would work too. Well, welcome to the show anyways. Let’s start here.
David Maman: Thank you.
John Koetsier: These are some pretty incredible claims. Tell me about your app, tell me how it works.
Binah.ai founder and CEO, David Maman
David Maman: So the app actually has been in production for way over a year now. That’s nothing new, we never released an app as a B2C solution, so it’s not for the public yet. We are a B2B company, so basically we are, I guess we can call it even a digital health enabler.
What do I mean by that? We sell our SDK and enable major organization[s]. For example, Sompo, among the largest insurance compan[ies] in Japan, already uses it in production and many others. How it works … you know, any smart watch owner knows those blinking lights at the back of your watch.
John Koetsier: Sure, absolutely.
David Maman: And those blinking light[s], doesn’t matter if you use an Apple Watch, Samsung Watch, it doesn’t matter, Fitbit …
… those blinking light[s] actually send a tiny light beam — an infrared, actually, light beam to your skin — and then it can monitor tiny color changes at your skin.
And those tiny color changes provid[e] an indication regarding the blood flow behind your skin. This technology …
John Koetsier: Absolutely, but that’s on your wrist, right?
David Maman: Exactly.
John Koetsier: And you’re actually talking about using the camera in a phone.
David Maman: Yes. So this technology [is] called actually PPG, photoplethysmography, and PPG have been out there … listen to that, since 1938. The first article about PPG is from 1938.
So instead of sending a light beam, what we are doing, we’re actually analyzing the light reflection intensity that comes from your cheeks to the camera lens.
John Koetsier: Interesting.
David Maman: And we can do it on a smartphone, on a laptop, on a tablet, even on a smart TV, basically. There’s just a specific distance that we can extract this kind of a light and doing hardcore signal processing capabilities we’re analyzing that, and then we’re able to get the same PPG quality and signal just out of the person’s face.
John Koetsier: Does it matter … what quality do you need in the camera?
David Maman: So the camera quality, we don’t need anything else we need just in a very high level we need actually 30 frames per second. You know, any webcam from the past 15 years is 30 frames per second, but we need a decent processing capability of the device.
So basically any iPhone 7 and above, any Android device from the past five years should work perfectly. We have over 7 million devices that already run our technology worldwide, so it’s not that new actually.
John Koetsier: Interesting. And does it matter what skin complexion you have? I mean …
David Maman: No, so as part of …
John Koetsier: I’m pretty lily-liver white, right, so it might work, but does it also work on darker skinned people?
David Maman: So it works at a completely extreme wide range of tones, and we’ve been able to validate this technology in India, Nepal, Botswana, Poland, Russia, because we wanted to go from the extreme white to the extreme dark tone complexity, in order to make sure that it’s fully supported. And not only that, we also tried it from age 4 until the age of 92.
John Koetsier: Wow.
David Maman: So any age works as well. We also, goes without saying, both genders, any gender basically, it’s fully supported.
The only thing we need is a reasonable light condition and then we can extract everything actually just from your cheeks.
John Koetsier: So let’s get into a couple of those things, like oxygen saturation. I’m guessing that influences the color of your blood, is that correct? Is that how you’re finding out, how are you finding that data?
David Maman: So there’s a lot of different articles that were published about how you can extract PPG from a PPG signal, your oxygen saturation. So we took actually what’s defined as the gold standard, and we had a lot of I guess we can call it challenges, how we can actually control the tiny — whoever understand[s] that — the DC changes in the color that is generated by an RGB camera. Keep in mind that a lot of sensors at the back of the watch use IR sensors.
John Koetsier: Yes.
David Maman: And we are using pure RGB capabilities. So when we’re thinking about oxygen saturation, you just probably you’ve heard as well, that Apple is providing an oxygen saturation with the Apple Watch 6. Samsung are providing it. Withings have announced that their ScanWatch will have oxygen saturation. So it’s completely doable.
John Koetsier: Yes.
David Maman: I truly believe that within the next two years, any wearable and any smart watch device will have oxygen saturation and additional features.
John Koetsier: Talk about blood pressure. How are you measuring that?
David Maman: So blood pressure is extremely interesting because we already submitted a few patents about this technology so I can share with you.
So, unlike blood pressure, the different vital signs that we’re extracting today — such as heart rate, oxygen saturation, respiration rate, mental stress level, which is based on the Baevsky Stress Index — those are pure signal processing capabilities.
So there is no statistical estimation. This is pure signal processing. Blood pressure, unlike those capabilities, is based on modeling.
So we worked very hard with seven different hospitals worldwide in order to create the dataset that we truly need, which is basically a person with invasive blood pressure measurement, or a cuff-based blood pressure measurement, and is face exposed to the camera with our own recording application.
And once we’ve been able to build this sufficient database of almost 13,000 people, then we’ve been able to actually model it correctly. So we are generating handcrafted 1,400 features out of the PPG signal. And yeah, I know that’s hardcore.
John Koetsier: Haha.
David Maman: And then we’re modeling, and the beautiful thing, it’s not a black box. It’s not just deep learning did the work for us. We are hard workers. You know, we have over a dozen PhDs in our team, so we have an extremely amazing team that we’ve built during the past few years. And we’ve been able to model it in a way that we’re getting to FDA accuracy requirements for blood pressure.
John Koetsier: So that’s really interesting. So what you’re basically telling me is that a lot of what you’re doing, the oxygen saturation, the blood, the heart rate, the heart rate variability, respiration — that’s a direct measurement, you’re seeing that and you’re representing that.
The actual blood pressure, that’s a derived measurement and you’d be able to basically experimentally see how all those other factors can trigger your learning on blood pressure. Is that correct?
David Maman: Yes. It’s not only that, it’s actually more than that, because once we are able to extract a perfect PPG signal over 25 DB, which means like a very high quality signal, we’re able to extract using [a] signal processing mechanism we’ve built, all the features that are needed from the core signal.
And to that, we’re also adding of course the respiration rates we’re reconstructing and also the heart rate and oxygen saturation, and we’ve built this beautiful model.
Very important fact about that, the entire technology that we have created is running on the device itself.
John Koetsier: Mm-hmm.
David Maman: You don’t need a cloud service and there’s no measurement running outside of the device. And the beautiful thing about that is that there is no privacy issue.
John Koetsier: Right.
David Maman: We as Binah, we never see the data that leaves the phone in any way. Everything is just for our customers. Blood pressure is not the end, blood pressure we’re finished with the research. We’re just in final checkups before releasing it, but we’re already working on hemoglobin levels to know whether you’re anemic or not.
We’re working already on alcohol blood level, and we truly hope before the end of the year to have the first version for alcohol blood level. That’s very exciting.
John Koetsier: Wow! Very interesting. You can check yourself really quickly before you leave the bar — if we’re ever going to a bar ever again.
David Maman: More than that, think about that, any Uber driver before he starts his shift should have a quick check. Any heavy labor operator, you know, you want to know that the person that works in your power station actually is not consuming any alcohol while he works.
John Koetsier: Yes.
David Maman: And having this available on a smartphone or any tablet or any, you know, any even smart mirror, is changing the industry today.
John Koetsier: Or corporate system that you have to access. Very, very …
David Maman: Exactly.
John Koetsier: … interesting. So you’re saying that this data is medical grade. That’s pretty interesting because a lot of data that we have out there with consumer devices is not, right? So Apple Watch ECG, that sort of thing, not medical grade. It’s like two points of contact versus seven or something like that in a traditional ECG.
David Maman: Yes.
John Koetsier: Talk about how it’s medical grade and how you’ve managed to achieve that.
David Maman: Yeah, so it was a really hard struggle about how we’re able to actually get the perfect level of signal with a perfect level of strength to isolate it and to dramatically increase the SNR, the signal to noise ratio. Y
ou know, basically when people don’t understand sometimes, but even their Apple Watch — and nothing to say about Apple, don’t get me wrong, it’s any watch — any kind of motion, any kind of movement, any kind of your watch just shaking a little bit on your wrist create a lot of noise.
And what people don’t understand many times [is] that the noise is about 90% of the signal that you’re getting.
John Koetsier: Wow.
David Maman: So normalizing motion, normalizing light condition[s], normalizing a lot of different exposures that are always happening, this was the toughest struggle that took us years to get to where we are now. And when we’re talking about medical accuracy, you know, each and every vital signs have its own specific reads of what is considered accurate.
So we took us a very long time and a lot of different clinical trials to validate that, for example, in oxygen saturation, we have only 2% error level, from oxygen saturation 70 until oxygen saturation 100. So it’s a very wide range of error level as well with respiration rate that it’s only up to three error level, heart rate up to two.
So, and once we’ve been able to conduct those clinical trials inside of major hospitals, we’ve been able to understand exactly the air level that we have, the standard deviations that we have, and we’ve been able to prove what’s the accuracy. I’m very happy to say that we’re in [the] final stages in order submitting it in a lot of different location[s]. One more thing that I would like to just … [crosstalk]
John Koetsier: Go ahead, go ahead.
David Maman: One more thing that is important to mention that we are an SDK provider. Our specific app, which is called Bvue, is not publicly available yet. But so we are basically an enabler.
Doesn’t matter if you’re [an] insurance company, a telemedicine company, even just a wellness platform. Instead of asking all your customers to buy a smart watch, you just can do everything just out of your phone. And one last thing, you know, there’s a lot of different conditions where light is not good enough. For example, you’re in bed, middle of the night, there is just not enough light to see your face.
And in this case, we know how to detect those conditions and we’re telling you right away if you cannot use your face, or move to a different location with a sufficient light condition. Just put your finger on the back camera and we can extract everything just out of the back camera.
John Koetsier: Very, very interesting. I was going to ask that next, actually, what’s the consumer experience? What’s the user experience? How long does it take? Is it 30 seconds? Is it two minutes? How long do you have to be there? And what’s that feel like?
David Maman: So it’s for each and every read, it’s a bit different. For the heart rate, in order to get it accurate, we need between 10 to 12 seconds. For oxygen saturation, [it’s] about 25 seconds. Respiration rate, about 32 seconds.
Mental stress level, and this is the official Baevsky Stress Index, we can do in about 45 to 50 seconds. The entire read, even including blood pressure, will take between 55 to 60 seconds and you will get all of them at the same time.
John Koetsier: So, this is interesting to me for a bunch of different reasons. First of all, I’m interested in health. I guess most of us are, in fitness …
David Maman: Yeah.
John Koetsier: … and other things like that. But I’ve been wondering for quite some time how we can get data about our health that’s being monitored. There’s a new class of data that’s available right now.
So I happen to have an Apple Watch, many people have a smart watch, or others have fitness apps on their phone and stuff like that, and they’re accumulating data on their health that could include heart rate, that could include …
David Maman: Everything.
John Koetsier: … variability in heart rate, exercise levels. Many people are capturing data on what they eat and so that you can see what nutrients they have. We seem to lack a way to get that data to doctors in a way that they can consume it. I mean, I talked to my doctor probably, it’s gotta be months ago now, and I said, ‘Hey, you know, here’s all my workouts and all that stuff’ and he says, ‘I don’t have time to look at that.’
And he’s pretty young and he’s pretty tech savvy, right. But we need some kind of system where we can feed all that data, maybe a medical AI or something like that, that can look at that and then maybe alert a doctor, ‘Hey, pay attention to this. Hey, pay attention to that.’
Do you see something like that coming out?
David Maman: So, definitely. I don’t think that we will ever have enough doctors or trained professionals that will be able to go through the entire data that we’re generating.
You know, we’re generating endless amount[s] of data and Apple Watch checks your status every 20 seconds, every 2 minutes, which means there’s thousands of reads.
And a lot of compan[ies] are not working about how to take this information and build a baseline, which will actually build even those kind of guidelines regarding when your health is getting better, or it’s getting worse, and you should maybe start practicing more, you should stop eating junk food, you know, that actually hurts you. And as you can see, I’m not a small guy, to say the least.
And the very interesting thing about what you just mentioned, this is the second layer, because according to the World Health Organization, within the next 10 years we’re going to have, we will be lacking 15 million doctors worldwide.
John Koetsier: Mm-hmm.
David Maman: And that’s not going to get any better, as we both know. And AI comes and, you know, AI is a buzzword, don’t get me wrong.
John Koetsier: Yes.
David Maman: But all the new technologies, and cloud resources, and statistical estimations, and everything comes in order to build us the personalized health graphs. Because when you come to the doctor and even if he gets your vital signs at the same second, that means nothing.
Maybe just at a middle of the night, or only when you train, something is really bad. And personalized health is going to change everything.
You know, if you’re going to have your reason, it doesn’t matter if it’s a smart watch or, you know, most people won’t be able to afford the thousand-dollar smart watch, but just by looking at your $200 smartphone, you’ll be able to get all your vitals ten times a day.
Then building this kind of a baseline will help those kinds of information savvy data centers, and all those kind of insurance companies, and medical service providers, specifically telemedicine providers, you know. There was a beautiful acquisition this week that Teladoc acquired Livongo.
John Koetsier: Mm-hmm.
David Maman: And this is an amazing integration between random people that need just to talk with a doctor, and Livongo, which is purely based on chronical condition patients. And this kind of a combination truly, I guess, evolved — that’s a beautiful evolvement to the world of telemedicine.
Because all of us, we will need remote patient monitoring and not just to talk with a doctor once a quarter whenever we’re feeling we’re getting a cold.
And I think that all those kind of technology that comes in order to have your body read at least a few times a day will build a beautiful timeline for us to see and to enable personalized health services.
John Koetsier: Mm-hmm, and that’ll give us very interesting data in health systems as well. So I happen to live in Vancouver, Canada. We have a provincial health system, a national health system, that sort of thing …
David Maman: Beautiful city. I actually used to live there.
John Koetsier: Oh, very good.
David Maman: Beautiful city.
John Koetsier: And so some countries like that you’ll have a very good sense of health across your nation, with privacy protections, obviously, but also maybe HMOs in the U.S. will be able to judge the health of a population in some way. Where do you see this in maybe about five years or so?
David Maman: Wow. That’s a hard question, but I think is that, you know, so I would like to break it down to three main insights. The first one is that we see today countries that [don’t] even have a sewer system, but everyone [has] a smartphone. You know, that’s unbelievable.
And this kind of an evolution, you know it was unexpected at first, but now we understand that that’s where the world is heading. So everyone will have a smartphone. So I think that the average person uses his phone between 3.5 to 5.5 hours every day.
And what if you just, while you’re going through Instagram, or watching a video on YouTube, or just checking your mail, you will be scanned because there’s a front facing camera that looks at you.
So if you can have your entire vital signs being extracted 50 times a day, so everyone that owns a smartphone will have this information, you know, we’ll live now in the new era of pandemic enabled society. And with a pandemic there’s a very, very long time that it takes for the World Health Organization to announce that a specific epidemic started in a specific location.
John Koetsier: Yes.
David Maman: What if all this information, just the metadata, yes, no identifiable features, will be then aggregated for the World Health Organization?
John Koetsier: Right, temperatures are rising an average of one to two degrees among about 50% of the population of this one town, something’s going on, check it out. Yup.
David Maman: Exactly, exactly. And then the entire world wouldn’t have this craziness that is happening now. I haven’t been on a[n] airplane for six months, you know how crazy it is?
John Koetsier: Yes.
David Maman: No, I’m kidding. But you understand what I’m saying. So moving forward in the next five years, medicine is going to be personalized, undoubtedly.
John Koetsier: Mm-hmm.
David Maman: Which means there is no just off-the-shelf treatment for each and every one of us. We are all different. The food we eat, the time that we spend on training or walking every day, the time that we spend sleeping, or the roots and the DNA structure that we have from our parents. Everyone is different.
John Koetsier: Yeah.
David Maman: And being able to extract those vitals — and I’m not talking about only the ones that we talked today, let’s talk about 15 different vital signs — in the next three years, just in Binah we’re planning to have over 20 different vital signs and we’re already working on many of them that will be able to be extracted within one minute, just whenever you use Instagram.
John Koetsier: Wow.
David Maman: And if we’re going to be able to extract 50 times a day those kind of vital signs or, you know, or an Apple watch that’s got nothing to do just with Binah, I think the evolution [has] started. Then all the treatment and everything will be actively created for us. What do I mean by that? If your vital signs are being read and, you know, telehealth or any other provider will get this information, a doctor should call you three days before you feel bad.
John Koetsier: Yes.
David Maman: And will automatically send you a recommendation, ‘Please take one pill, please stay one day in bed.’ And that’s it, we prevent even getting a cold.
John Koetsier: Haha, exactly. Your medication is in the mail. You don’t know you’re sick yet.
David Maman: Exactly.
John Koetsier: Just take one, there you go. Don’t need the visit. Why do we need the visit, we know what’s going on. Very very interesting. Okay. Let’s come back to the present …
David Maman: Yes please.
John Koetsier: … and your technology, where is it? Where do you expect it to be in the next few months?
David Maman: So, now we have five insurance companies worldwide already using it. And the insurance organization, you know, when we’re talking about insurance, doesn’t matter if it’s health insurance or life insurance, we’re talking about the option to manage risk correctly.
And whenever they are managing risk, they would like to know it during the underwriting, which means when you onboard a new client to insurance companies. So some companies already take your vital signs during the onboarding in order to add it to the actuary system, to define what’s the risk on insuring you.
John Koetsier: Wow.
David Maman: In others, for example, Sompo [has] released an app which is called Linkx — Sompo is the largest insurance company in Japan — that they’re measuring the stress level, the heart rate variability, and heart rate of all of their customers, almost on a daily basis to determine what is the risk, whether it’s increasing or decreasing.
John Koetsier: Mm-hmm.
David Maman: You know, in Japan there’s a really big problem with suicide, not only Japan, in Korea, Japan, and other countries — which I love by the way, I love Asian countries — and those countries have a massive problem with suicide, with depression, with lot of people, you know the number one …
John Koetsier: Yeah, isolation.
David Maman: Exactly. And whenever you’re able to measure the mental stress level, you can see degradating conditions of a person, and very quickly it will actually degradate to depression and then to much more severe situations. And what we’re able to do is to enable all of those organization[s] to, in one hour, spread it through their app. They don’t need to sell or send our app, we’re an enabler.
John Koetsier: Yeah.
David Maman: So we’re enabling to do that. Other companies, so we have telemedicine providers already use our technology. Think about that, instead of just having a video session with your doctor and just showing him ‘Yes, I have a rash here, what do you think about that?’ he can actually treat you like you’re in his clinic. You can get your vital signs …
John Koetsier: Yeah, check your pulse, all that stuff, right. Blood pressure.
David Maman: Everything. Exactly.
John Koetsier: Very, very interesting.
David Maman: Exactly, just through the video. And one last example, which is actually growing very, very fast, is the world of living in the new age of epidemic.
And we’ve actually created a system that’s called “Back to Work,” but it’s not just about work. It’s back to college, back to prayers, back to any environment that creates interaction between people. So every employee and every person can scan himself before he leaves his home in the morning and he can get an answer right away, you should stay at home today, or you can go to work.
And if it’s critical, you know, your oxygen saturation is below 92, go see a doctor, that’s really dangerous.
John Koetsier: Yes.
David Maman: Your respiration rate is above 30, listen, maybe you just ran five miles, but maybe you should see a doctor, something is not working well.
John Koetsier: Yeah, yeah.
David Maman: So those kinds of a “back to” system will be part of everything. Even before you’ll go to see a show you will be required to have your quick health check.
John Koetsier: Very, very interesting. Obviously it does bring up the privacy question. I’m assuming the apps that you’re integrated with have appropriate privacy provisions in place and other things like that.
But what about, could an app install the SDK and not let people know, and so people don’t know that they’re being scanned? Let’s say for instance, there’s a bad social network or whatever, that just wants more data on people’s health and other things like that so they can more accurately target ads. You know, is it possible that they could incorporate some of that technology and then use it to customize some of their user experience?
David Maman: I think we all have the beautiful rights of knowing when the camera is operated.
John Koetsier: Yes.
David Maman: You know, [we] have to allow it, even on Windows, on Mac, whenever an app or a website tries to…
John Koetsier: A little light goes up.
David Maman: Not only the light, but actually your computer tells you an app is trying to access the camera. So a person [has] these options to manage your rights, but, you know, elderly people, people that are not that tech savvy, have an issue, and for them everything’s okay. You know, they’re not paying attention for that.
So like any technology, and let’s be honest, if we are just a 50 people startup is able to do that, I don’t want to even imagine what armies and governments can actually do.
John Koetsier: Right.
David Maman: And I think because this kind of technology started in 1938, I’m sure that there are places that can actually hack your body. And when I’m saying hack, I mean, like, you know, they can extract …
John Koetsier: Data.
David Maman: … sensitive information. Everything’s about data, you know, and when you take a look at the darknet, medical information is the most expensive one. Not even financial information, not the army service. Medical information is the most expensive one.
John Koetsier: Wow.
David Maman: And yes, you know, what if I can just optimize the advertisement to your conditions just by knowing your medical conditions?
John Koetsier: Mm-hmm.
David Maman: If I will be able to know your glucose blood level, your cholesterol level, your hemoglobin level, you’ll start seeing commercial[s]. Well, the world is crazy, but what I’m trying to say, we worked very hard in order just to work with formally organized countries. We don’t think that we have a weapon, don’t get me wrong. We don’t.
John Koetsier: Yes.
David Maman: But we, I think we still have this level of responsibility to make sure that it will not be available just for everyone to abuse privacy.
John Koetsier: Yes, yes. Well, David, very interesting stuff. Very interesting app, a lot of power there, and I think that over time it will get combined and we will see our data stream, our health stream, and some AI analysis of that. Thank you so much for joining us on TechFirst.
David Maman: Thank you very much, John. Please keep safe, everyone, not just John. And thank you very much for your time. Again, I’m a real fan.
John Koetsier: Excellent. Well, everybody else, thank you for joining us on TechFirst. My name is John Koetsier … I appreciate you being along for the ride. There’ll be a full transcript of this podcast in about a week at JohnKoetsier.com. You’ll see the story on Forbes soon after that.
Plus, the full video will be available on my YouTube channel. Thanks again for joining, and hey, maybe share with a friend. Until next time … this is John Koetsier with TechFirst.