Why It Matters
The open, ultra‑lightweight controller makes sophisticated humanoid robotics accessible on everyday devices, unlocking new possibilities for rescue, exploration, and consumer automation.
Key Takeaways
- •New teleoperated robot controller "Sonic" translates human motion to 3D joint commands.
- •System is multimodal: accepts video, voice, music, or text inputs.
- •Model runs on just 42 million parameters, runnable on phones or toasters.
- •Training used 100M motion frames, 128 GPUs for three days, then lightweight.
- •Open-source release enables broad adoption for rescue, exploration, and everyday tasks.
Summary
NVIDIA unveiled a new teleoperated robot controller, dubbed “Sonic,” that can watch a human perform a task and instantly translate those motions into precise joint commands for a robot. The system is not limited to visual cues; it accepts video, voice, music, or plain text, making it a truly multimodal interface for controlling humanoid machines.
The breakthrough rests on a lightweight 42‑million‑parameter neural network trained on 100 million frames of human motion. Training required 128 GPUs for three days, after which the model runs on a smartphone—or even a toaster—without noticeable latency. A novel “root‑trajectory spring model” smooths abrupt commands, preventing the robot from injuring itself while preserving fluid, human‑like movement.
Demonstrations showed the robot performing kung‑fu, crawling through tight spaces, and dancing to music, illustrating both its expressive range and practical utility for hazardous environments such as rubble rescue or planetary exploration. The research, led by NVIDIA’s Jim Fan and Professor Zhu, is being released openly, inviting developers worldwide to build on the technology.
By democratizing high‑fidelity robot control, NVIDIA lowers the barrier for advanced robotics applications, from disaster response to consumer assistants. Open‑source availability accelerates innovation, promising a rapid expansion of autonomous systems that can safely operate alongside humans.
Comments
Want to join the conversation?
Loading comments...