NVIDIA, a global leader in artificial intelligence (AI) and accelerated computing, has unveiled an exciting new AI platform called Cosmos.
Announced at the Consumer Electronics Show (CES) in Las Vegas, the platform is designed to revolutionize the development of robotics and autonomous vehicles (AVs) by harnessing generative AI.
Cosmos is built to address the challenges of training robots and AVs in complex real-world scenarios. At its core, the platform introduces world foundation models (WFMs)—advanced neural networks capable of generating precise video simulations.
These models are trained on an unprecedented dataset of 20 million hours of real-world robotics and driving footage, enabling developers to create lifelike, physics-accurate virtual environments.
Newsweek contacted NVIDIA via email for further comment.
What is NVIDIA Cosmos?
Cosmos supports synthetic data generation, allowing developers to simulate and test systems in virtual spaces without relying on massive real-world datasets.
This could be a game changer for industries like robotics and AVs, where collecting diverse, high-quality data—such as snowy road conditions or rare humanoid movements—has been a costly and time-consuming hurdle.
"Physical AI has a big data problem," Jim Fan, NVIDIA Senior Research Manager, said in a post on X (formerly Twitter). Synthetic data to the rescue! We apply Cosmos to large-scale synthetic data generation for robotics and autonomous driving, and now you can too! It's all yours to finetune."
NVIDIA has released Cosmos as open-source software via the Hugging Face platform. Early adopters include industry leaders like Uber, Agility Robotics and AV pioneers Waabi and Wayve.
How Does NVIDIA Cosmos Work
Cosmos combines generative AI with NVIDIA's powerful simulation technologies to help developers build robots and AVs. The platform is equipped with three categories of models: Nano, Super and Ultra, each designed for different performance levels.
The platform enables the creation of virtual environments that replicate real-world physics. For example, robots can be trained on scenarios such as detecting falling objects in warehouses or navigating unpredictable road conditions.
These simulations provide "next-token" predictions, helping robots and AVs anticipate and select optimal actions.
"A key innovation is its reliance on WFMs, which represent a major leap forward in bringing AI closer to physical interaction," Farshid Amirabdollahian, a professor of human-robot interactions at the University of Hertfordshire, U.K., told Newsweek.
"These models enable highly realistic simulations and provide training data that closely mimics real-world conditions."
Cosmos also introduces a suite of tools for data processing and model fine-tuning. Developers can process 20 million hours of data in as little as 14 days using NVIDIA's GPUs, compared to years with conventional methods.
Is This a Game-Changing Moment for AI and Robotics?
Rev Lebaredian, vice president of simulation technology at NVIDIA, told Fortune that Cosmos is "absolutely game-changing" for AV and robotics companies.
Meanwhile, CEO Jensen Huang said during his announcement on Monday that "the ChatGPT moment for robotics is just around the corner," a nod to the buzz surrounding OpenAI's text generation model a couple of years earlier.
Other experts were equally excited by the news. "Cosmos has the potential to be transformative," Amirabdollahian said.
"One of the biggest challenges in robotics is generalizing AI systems to real-world applications, and Cosmos addresses this directly. By improving real-time adaptability and streamlining development, it could accelerate progress across industries like healthcare, manufacturing and logistics."
Still, challenges remain. For robots to fully "understand" the real world, models will need even greater power and sophistication.
Cosmos can help make realistic models of the real world, but in much the same way as ChatGPT doesn't know why it's writing what it does, nor does Cosmos understand the laws of physics it replicates.
Other barriers, according to Amirabdollahian, include cost, safety, regulation and public perception.
Do you have a tip on a science story that Newsweek should be covering? Do you have a question about artificial intelligence? Let us know via science@newsweek.com.
English (US) ·