Robot hands are one of the biggest frontiers in AI, and according to RLWRLD Founder & CEO Junghee Ryu, the challenge isn’t just building better hardware. It’s building AI that can control it.
In this episode, I chat with Junghee Ryu to discuss RealDex1, RLWRLD’s foundation model built specifically for high-degree-of-freedom robot hands. Unlike companies that develop AI for a single robot platform, RLWRLD is creating a hardware-agnostic intelligence layer that works across a wide variety of robotic hands, combining vision, tactile sensing, force, torque, and motion information to achieve human-level dexterity.
Check it out:
In this episode:
- Why dexterous robot hands remain automation’s biggest challenge
- Inside RLWRLD’s RealDex1 foundation model
- Combining vision, touch, force, and torque into one AI system
- Training robots using teleoperation and human demonstration data
- Supporting dozens of different robot hand designs with one model
- Why hardware is becoming the next bottleneck in humanoid robotics
- Working with NVIDIA on dexterity benchmarks
- Building OpenHand and the future of open robotics standards
Transcript: Physical AI for high-DOF robot hands
John Koetsier:
Robot hands are super hot right now. We’re seeing such amazing technology. We’ve seen it from Kyber Labs. 1X blew me away with some private things I’ve seen from them on their NEO robot. Genesis AI is doing some super cool things that I saw recently in San Carlos at its headquarters in California. RLWRLD, which was just named a World Economic Forum Technology Pioneer for 2026, says dexterous manipulation remains the single greatest bottleneck and last-mile barrier in industrial automation.
So today, we’re chatting with the founder and CEO, Junghee Ryu. Welcome, Junghee. How are you doing?
Junghee Ryu:
Hi, good to see you. Thank you for having me. I’m doing well. Thank you.
John Koetsier:
Super pumped to have you. Awesome.
You just released some pretty cool news.
Junghee Ryu:
We invented our own foundation model to support multi-DOF robot hands. The model’s name is RLDX-1. It’s an upper-body foundation model, like other models out there from NVIDIA or Physical Intelligence.
John Koetsier:
Interesting. So you take any hands that come along and work with them. What requirements do you have for the hands you work with? Do they need a certain base level of specifications?
Junghee Ryu:
Once we started this company, we met with over 200 industry leaders around the world, mostly in East Asia, including Japan and Korea. Almost every company said that we need humanlike dexterity.
At that time, most companies were trying to solve problems in the workplace using just two fingers or three fingers. So we decided to provide human-level dexterity. We’re an AI company, but providing AI technology to control a multi-DOF, five-finger robot hand is extremely hard.
It took us under one year to accomplish everything, but now we’ve released our model. Regarding the hardware requirements you asked about, that’s why we’re now working together with NVIDIA to define dexterity and establish a benchmark for dexterity testing.
John Koetsier:
So how dexterous a hand can you work with? I’ve seen hands with 22 degrees of freedom—22 degrees of actuated freedom. They can literally exert force in all those degrees of freedom. How high-end does your model go? Does it meet that need?
Junghee Ryu:
That’s a good question. What we wanted to do from the starting point was understand the mechanics of a hand with more than 20 degrees of freedom.
As you know, degrees of freedom refer to the active joints inside the robot hand. If you have two fingers, the active joint count might only be one. But compared with that, a five-finger hand can have 22 to 24 degrees of freedom. That means the robot hand has more than 20 active actuators inside it.
To control that, we had to design our AI model in a totally different way. One of the approaches we provide is a multi-stream action transformer. That means that inside the structure of the transformer, we include not only visual information but also different physics streams.
We don’t just support controlling the robot hand. We also include tactile information coming from sensors on the fingertips and palm, along with torque and force sensing. That’s one approach we use.
We also provide motion-awareness functions. Usually, to provide dexterity using multiple fingers, we need to be aware of the motion of the object. That’s why we included that.
We also provide better memory. Because we support multiple degrees of freedom, we need to provide longer memory than in other cases. We included many modules inside our model to provide better control of high-degree-of-freedom hands.
John Koetsier:
So I’m trying to tease out whether the answer to my question is yes. You can support as many degrees of freedom as a hand might have. Is that correct?
Junghee Ryu:
Yes. Regarding the types of hands, we’re now working with almost every hand provider out there from the U.S., China, Korea, and Japan. We’ve tested many of them.
There are different characteristics among the hands. For example, one of the two main approaches to making a hand is tendon-based, using metal or synthetic wires to control the fingers and placing the actuators in the lower arm. That’s a typical approach used by Tesla Optimus and Figure.
We started this journey by collaborating with WIRobotics, one of the top-tier humanoid makers in Korea. We co-designed the lower arm and the whole body using tendons.
We’re also using some direct-control hands, such as Shadow or Wuji. But the mechanical characteristics of those two approaches are totally different. As a foundation model company, we should be able to control both approaches in the market. That’s why we’re now testing almost every hardware platform out there.
John Koetsier:
Now, you said you integrate multiple sensor streams, right? You’ll have many hands with various ways of understanding the world they’re operating in.
First, of course, there might be vision from a head or chest camera. There might also be a video stream from the hand itself, with a camera attached to or embedded in the hand.
There are probably also touch sensors that the hand may have. In some cases, the gearing itself can detect resistance or applied force. So you’re merging all those available data streams.
Junghee Ryu:
Exactly. That’s what we do. We don’t depend only on visual information, like a classic vision-language-action model. We use a lot of physical information and physics streams.
We don’t just use existing tactile sensors. Sometimes we use intrinsic signals, such as impedance signals coming from the actuators. Those signals can sometimes work like tactile sensors.
That’s why we designed our model to embrace not just one type of physical information, but any physics stream coming from the robot’s body.
John Koetsier:
Okay. It’s one thing, of course, to deal with one hand—what it’s doing, where it’s moving, and all that. It’s another thing to deal with two hands and how they might work together to touch, hold, or grab something, maybe holding one thing while the other performs an action on it. Talk about that level of complexity.
Junghee Ryu:
One hand might have 20 degrees of freedom, and the other hand might also have 20. Excluding the hands, the upper body might have only 14 degrees of freedom.
That means that when using simple two-finger grippers, the total number of degrees of freedom for the upper body might only be 16. But when using five fingers, the dimensionality increases by around 3.5 times.
That’s a huge increase in dimensionality, so my model needs to account for it in a totally different way. We also need to design not only the model but every pipeline, including the data pipeline.
That’s why we invented our own teleoperation system using MANUS gloves and VIVE trackers. We needed to establish precise positioning and understand what happens when a human interacts with an environment. That’s why we invented a teleoperation system focused on five-finger control.
John Koetsier:
That was super interesting. I’m not sure whether you understood my question. My question was about two hands working together to hold something. How does that change the equation? How does that change the complexity of what you’re doing?
Junghee Ryu:
No, we don’t use any explicit equation. That’s why I answered that way. We’re providing a model. This is a large model, like an LLM, so it’s end-to-end.
We don’t know exactly how it’s working. It’s a statistical engine. If you want to control two hands, you perform teleoperation at a much higher level.
In the case of ChatGPT, you don’t know—even the OpenAI engineers don’t know exactly how the model creates each sentence. That’s the meaning of a large-scale, end-to-end model. We don’t know exactly how it controls the two hands and five fingers.
John Koetsier:
Okay. That’s at a higher level. Understood.
Junghee Ryu:
Sometimes we think about adding basic layers to control the mechanics because we want to avoid collisions between the two hands and other objects. But basically, it’s the same model.
If you record teleoperation using two hands, the robot can mimic you. We don’t explicitly control the two hands.
John Koetsier:
Okay. Understood.
So I’m guessing from what you’re saying—and this is probably the source of our miscommunication—you provide a model. Somebody comes in with their hands, trains your model for those hands using teleoperation or other datasets, and then it goes from there.
I was just at Genesis AI in San Carlos, California. You’ve probably seen the demo online, right? There are two hands. They’ve got some wires they’re stringing together, and they’re putting tape around them. One is holding the wires while the other works.
That’s all training that happens after somebody makes a deal to get your model and train it, correct? Okay. That makes perfect sense.
Junghee Ryu:
Correct. That’s what I wanted to say. We don’t care about the mechanics or lower-level control.
John Koetsier:
Yes. It’s just a learning engine. It’s a learning engine. You take it and go with it. Okay, perfect.
Junghee Ryu:
That’s right. We’re making artificial intelligence. That’s why.
John Koetsier:
Very interesting. The question that comes to mind is that a lot of robotics companies want to own the full stack, right? They want to own the hardware, and they want to own the software.
How do you fit into that when you offer one piece or component—a training AI?
Junghee Ryu:
There are now a lot of companies, such as Figure, 1X, and others, that try to provide the robot body together with their AI. That’s a relatively easy approach because if you fix one hardware body, making the AI is easier.
But the reality is what happens inside a factory. We’re targeting industries such as factories, logistics, and hospitality services, and all their needs are different.
We can’t provide just one body for all those applications. That’s why we’ve taken this strategy. As a foundation model and AI company, we should support almost every hardware platform out there because our customers request many different specifications for their robot bodies.
John Koetsier:
Interesting. So I’m wondering—
Junghee Ryu:
For example, most companies don’t need bipedal walking. They use a wheeled lower body, such as an AMR. We want to give them the freedom to choose the lower body. That’s why.
John Koetsier:
So I’m wondering whether there are a bunch of robot companies I’ve talked to that say, “Hey, we develop our AI in-house. We develop our hardware and software in-house.” That’s true in a sense, but they’re using your foundation model to do it. Is that likely to be the case?
Junghee Ryu:
Yes. There are many layers inside the robot software stack. For example, ROS, the Robot Operating System, is in the lower layer and controls the robot.
My model is placed in the upper layers. My model generates robot action signals, and those robot action signals go to ROS. Then ROS can control the body of the humanoid.
There are standards for controlling humanoid robot bodies. That’s how we can control almost every robot body out there.
John Koetsier:
Gotcha. Nice. Talk about some of your clients. Who are some of your clients?
Junghee Ryu:
We’re now focusing on East Asia, especially Japan and Korea. I believe Japan and Korea are two of the top countries in terms of advanced manufacturing, logistics, and service environments.
Their need is also very urgent because they’re facing a severe population cliff. That’s why they really want to use humanoids in their workplaces. They’re struggling to hire younger generations.
We’re starting with Japanese and Korean companies, including manufacturing customers, large car manufacturers, hospitality companies such as hotel management companies, convenience stores, retailers, and logistics companies such as warehouse operators.
John Koetsier:
Can you name names, or are they confidential?
Junghee Ryu:
Most of them have signed NDAs with us, but I can mention names such as KDDI, one of the biggest mobile operators in Japan, and Lotte Hotel, the number-one hotel chain in Korea.
John Koetsier:
Gotcha. Very cool. What’s the next big challenge for you? What are you working on now that you’d like to solve?
Junghee Ryu:
It’s a very interesting situation because we’ve already outperformed almost every other model. Now the bottleneck isn’t the software. The bottleneck is the hardware.
That’s why we’re providing standards. We’re collaborating with NVIDIA to define better dexterity benchmarks because almost every benchmark out there doesn’t account for high-DOF, five-finger hands.
That’s why we need to define our own benchmark. But as a model provider, the better approach is to collaborate with other leading companies in this ecosystem. That’s why we’ve teamed up with NVIDIA to define it.
Another problem is that there aren’t good standards for data. To provide a large model at this level, we need tons of data.
Many startups are beginning to provide different kinds of data, including human-action data, simulation data, and other data for model companies like us. But there are no standards for that type of human-data pipeline. That’s why we want to define the standard.
The final issue is hardware. As you mentioned, there are so many hardware options out there, but most of them don’t account for actual actions.
That means that when we get a hand and test it extensively, we find problems when performing specific tasks in real workplaces. That’s why we defined DexBench, our own benchmark test.
Using those hand architectures inside a simulator, we test many of the skills required by the benchmark we’ve created. We see many failures.
Recently, we decided to make everything we’ve learned from our experience with robot hands open. The website is called All Hands Up! Within one or two weeks, we’ll put it online.
The website includes a lot of information about existing five-finger hands. A mini simulator is embedded in the site, and you can test them through the website. That’s our approach.
John Koetsier:
Interesting. Very cool.
It’s interesting that you say the bottleneck isn’t software but hardware, and I believe that to be the case in a lot of scenarios. I’ve seen a lot of grippers. I’ve seen a lot of bad hands—or maybe not bad hands, but early-stage hands.
But I’m also seeing some amazing hands emerge. I saw 1X’s hand for NEO, which they showed me, but I couldn’t share the video.
Junghee Ryu:
I saw that inside their office. I visited their office and saw it. It’s amazing.
John Koetsier:
How long ago were you there?
Junghee Ryu:
About two months ago.
John Koetsier:
Okay, they might have an even better one right now. I talked to their head of product and design about a week and a half ago. It has 22 degrees of actuated freedom, and it’s fast, quick, and very impressive.
That doesn’t mean it’s durable. That’s still an open question. They’re targeting the home, where durability will be less of a concern than in a factory, obviously, right? But it’s super interesting as well.
With hands like that—I was also just talking about Genesis AI. They’re targeting human-level hand capability. The fingers are different lengths, right? It’s not one finger unit repeated four times, plus a thumb. They have fingers of different lengths with different capabilities.
I think hardware might not be the bottleneck anymore, depending on reliability and how it works in the real world, because we probably don’t have a ton of training data for very good, highly actuated, five-finger hands. I’m guessing.
Junghee Ryu:
That’s true. That’s why we’re building a human-data pipeline and a synthetic-data pipeline.
The easiest way to gather data isn’t by using humanoid hardware but by using human hardware. It’s simply taking video of real humans in real workplaces. Human actions teach the humanoid.
John Koetsier:
Is it just video data, or is there also sensor data attached to people’s hands and fingers as they move?
Junghee Ryu:
It’s mostly video data. But as I mentioned, we also care about physical information, including tactile data, force, and torque. Sometimes we use gloves that provide tactile-capture capabilities.
John Koetsier:
Very cool. Well, thanks for taking this time. I really appreciate it. It’s cool to hear a little bit about what you’ve been working on and what you’re doing. I really appreciate your time.
Junghee Ryu:
Thank you. Thank you so much for having me.