NVIDIA’s Cosmos 3 release adds four open-weight models for creating interactive simulations in robotics and autonomous driving. Transfer-Auto generates synchronized driving scenes across as many as 11 camera views, including fisheye cameras, and can produce aligned RGB and LiDAR data. Text prompts and structured scene controls let users vary appearance, weather, traffic, and road conditions. Sim-Auto turns this capability into a streaming simulator: it generates successive camera and LiDAR observations conditioned on driving controls and its own generated history. Sim-Transfer renders RGB observations from simulator-provided spatial controls such as depth, segmentation, and edges, leaving physical state and motion to the simulator. Sim-Bimanual predicts video from robot and camera actions, supporting two-arm YAM and AgiBot embodiments as well as viewpoint changes. The checkpoints are post-trained Cosmos 3 Nano models with 16 billion trainable parameters, released under the commercial and non-commercial OpenMDW1.1 terms.
Streaming models predict short video chunks using prompts, controls, and prior generated observations; causal attention prevents access to future frames, while caching supports continued generation. Their training progresses from learning domain-specific control-to-video relationships to teacher-forced causal prediction, four-step distillation, and self-forcing on model-generated histories to address compounding errors. Developers can adapt the models to their own sensor rigs, environments, or robot embodiments using provided post-training recipes, then distill driving models for faster interactive simulation. Running inference requires Linux, an NVIDIA GPU, Docker, and NVIDIA Container Toolkit; the documented setup was tested on a GB200, not specified as a minimum. The release offers a starting point for building and evaluating simulated worlds, while fidelity, limitations, and usefulness for policy training still require careful testing.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Richard Sutton argues that brute-force computation alone cannot produce intelligence and that AI needs clever new methods. He says the science of AI is far from finished. (twitter.com)
Anthropic is accused of charging Russian Claude users before identity checks, then cutting off access after verification and retaining biometric data. Critics say account bans can affect linked GitHub projects and Russian citizens living abroad, and accuse the company of failing to answer public questions. (twitter.com)
LLM coding agents boost workplace productivity but can prevent developers from learning through hands-on problem-solving and weaken their technical skills. Developers should preserve their expertise by building projects manually, studying fundamentals, and practicing without AI tools in their own time. (ducktape.blot.im)
A fabricated Newtype interview led The Diplomat to retract its article about a Korean startup’s AI war test in Ukraine. The retraction followed a diplomatic dispute. (thediplomat.com)
Microsoft Research’s AI Frontiers group brought together about 50 researchers to develop agent systems and foundational AI advances. After the team joined Microsoft AI in September, its research-first approach clashed with leadership’s emphasis on scaling. (hunch.net)
Microsoft researcher John Langford has left the company. Microsoft’s AI Frontiers group has been declared finished. (businessexaminer.ca)
Today's best Hacker News stories, summarized and screenshotted, one email a day.