Training Robots in Virtual Environments

Robots are currently being trained in the Matrix, or something just like it.

Let me explain.

NVIDIA built a platform called IsaacSim, which is a hyper-realistic virtual environment

where robots can practice walking, picking things up,

and navigating spaces without any physical robots involved.

They also built something called NVIDIA Cosmos,

which is a world foundation model that can generate realistic environments on demand.

So anything from kitchens to warehouses and factory floors,

Cosmos can build any of them with realistic lighting and physics that match the real world.

Inside those generated worlds, there is a bunch of virtual robots training.

And it's not just one at a time.

There's thousands of copies of the same robot running in parallel,

and each one is experiencing slightly different conditions.

Now, because the simulation runs faster than real time,

a robot can experience the equivalent of 10 years of training in a single day.

Once the robots learn a skill in the simulation,

that skill gets loaded into a foundation model called Groot,

which is NVIDIA's brain for humanoid robots.

And then that brain can be downloaded into any of these physical robots

and deployed in the real world.

And so when a robot walks out of a warehouse for the first time,

it already has thousands of lifetimes of experience.