For robots to become useful and get deployed across the globe, their intelligence needs to generalize. A robot should be able to carry out tasks it hasn’t seen or work in unfamiliar environments. And ideally the intelligence can carry over to different embodiments of robots without starting from scratch.
Today, for robots to work in new tasks, environments or embodiments often requires more data collection and training.
We are excited to share our investment in Dream Labs which is aiming to change this, seeking to build general robot intelligence that can generalize across tasks, environments and embodiments.
We believe that for us to get to billions of robots doing useful across the world, we need to have robot intelligence that generalizes. As hardware has started to get cheaper and taking new forms, one of the missing pieces is intelligence for that hardware that works well out of the box. Reducing the data and training needed for every deployment expands the use cases of what robotics applications are practical, and reduces the barriers to deployment and increases the usefulness of the hardware.
Dream Labs’ approach builds on the team’s work on World Action Models, which are learned models that predict video and robot actions together. The model learns to visualize what carrying out instructions look like along with the movements needed to do so. We see this approach as a promising foundation for generalized robot intelligence, potentially when paired with larger LLMs for additional planning and reasoning.
The Dream Labs team brings a wealth of experience across both robotics and world models. The team brings experience across both robotics and world models. Joel Jang led world model work for NVIDIA’s GR00T, including DreamZero, while Yunhao Ge worked on the Cosmos world foundation models. You Liang Tan led GR00T’s open-source efforts and Scott Reed was a founding member of NVIDIA GEAR and worked on Gato and Genie at DeepMind. Their combination of model research and practical robotics experience is well suited to building generalized intelligence for robotics.
Today, as Dream Labs come out of stealth with $90M in seed funding, their first model, Dream-1 has demonstrated zero shot embodiment generalization, where it can control bimanual robots with parallel grippers it wasn’t trained at all on. The team has also shown adaptation to new robots using just five minutes of demonstration data. These are very exciting early steps towards the vision of making robot intelligence broadly available and accessible.
That idea of accessibility is also central to the company’s approach. Developers and robotics companies can bring their own hardware and use cases, while Dream Labs focuses on the intelligence and the infrastructure to serve that intelligence. To that end, Dream Labs is also one of the first companies to launch a generally available API for their foundation models, available over the cloud.
We’re proud to partner with Joel, Yunhao, You Liang, Scott, and the entire Dream Labs team as they work toward general robot intelligence. If you’re interested in joining them, Dream Labs is hiring.
