On AI ‘World Models’ & Why They Matter

Alpha
December 06, 2024

TECH IN THE NEWS

Trump Picks — Trump nominates David Sacks as “AI and Crypto Czar”, pro-crypto advocate Paul Atkins for SEC chair, tech hawk, Gail Slater to lead DOJ antitrust and Musk ally and private space traveller, entrepreneur Jared Isaacman for NASA.

Bitcoin Milestone — Bitcoin surpasses $100,000 for the first time ever, with President-elect Trump congratulating crypto community amid market euphoria.

China Trade War — China responds to the US’ trade restrictions by banning US exports of tech-essential critical minerals including gallium, germanium & antimony.

12 days of "Shipmas" — OpenAI announces 12 days of daily product launches including a new o1 reasoning model and likely the Sora text-to-video tool. It is also weighing ad implementation and just added a $200 p/m ChatGPTPro subscription.

Meta's Nuclear Push — Meta is seeking nuclear power developers to add 1-4 GW capacity for AI data centers in the US to support its AI and sustainability efforts.

COMPANIES TO WATCH

SpaceX — Discussing an insider share sale that could value the company at $350B, putting it among the world's largest publicly traded companies.

Anduril — The defense tech startup partners with OpenAI to integrate AI into counterdrone systems, the AI giant’s first defense partnership after removing military-use restrictions from its terms of service.

WereBuilding.AI — Three key members of Google's NotebookLM team have departed to launch a stealth consumer-focused AI startup led by Raiza Martin.

Cake — Emerged from stealth with a $10M seed round led by Google's Gradient to help businesses integrate and secure open-source AI infrastructure components.

IN PARTNERSHIP WITH INSTANTLY.AI

Find, Contact & Close Your Ideal Clients.

Instantly turns leads into clients with Automated Outreach, Deliverability Network, Sales Engagement, B2B Lead Database & AI-Powered CRM

THE HOTTEST THING IN TECH
What Are AI ‘World Models’ & Why Do They Matter?

World models, AKA world simulators, are being touted as the next big step in AI. AI pioneer Fei-Fei Li’s World Labs, DeepMind’s Genie 2, and OpenAI’s Sora are all making waves this week with major announcements on their world models. These headlines focus on the immediate potential applications in generative entertainment and gaming. But far more exciting is the potential implications for fields like robotics and modeling/decision-making in the digital and physical world across industries. World models offer potential for diverse training environments, a crucial step towards developing more capable embodied AI agents and even AGI. Investors are taking notice too, with a lot of smart money in the AI sector backing startups like World Labs.

Why World Models Matter

Yann LeCun, Meta's chief AI scientist, believes that although LLMs are good at predicting the next word or pixel, they are far from achieving "human-level AI". Since current AI systems lack human-like 3D understanding of the world even after extensive training, LeCun and others believe world models are essential. However, LeCun admits there are significant challenges and estimates it could take a decade to make accurate models a reality. The primary challenge is computation. Training and running world models requires massive compute power even compared to the amount currently used by current models. While some of the latest LLMs can run on a modern smartphone, large world models like Sora and Genie would require thousands of GPUs to train and run, especially if their use becomes commonplace.

The World On A String

Fei Fei Li’s Silicon Valley startup, World Labs, just announced a new tool that turns 2D images into tiny, navigable 3D worlds. This AI feature creates scenes with persistent visual aspects, real-time control, and consistent geometry. This allows for precise camera movements within the generated environment with potential applications in film making, game design, and the creation of simulators. The tool is not live for the general public, but examples can be seen on the company's website.

In September the stealth startup raised a $230M Series B round led by Radical, Andreessen Horowitz and NEA at a pre-money (and pre-product) valuation at $900M and is currently valued at over $1B (over a 3x step-up from its Series A.) Other notable backers included Ashton Kutcher’s Sound Ventures, Nvidia's NVentures, Intel, AMD, Eric Schmidt, and Salesforce CEO Marc Benioff as well as AI pioneers like Geoffrey Hinton, Reid Hoffman and Andrej Karpathy.

Genie Out Of The Bottle

Google’s DeepMind research lab just unveiled Genie 2, an AI model that can generate interactive, real-time 3D worlds from a single image and text description. It was trained on videos, allowing it to simulate object interactions, animations, lighting, physics, reflections, and the behavior of non-player characters (NPCs). While Genie is positioned as a tool for research and creative prototyping, it could well have potential impact on the gaming industry, simulations and beyond. DeepMind also recently hired one of the creators of Sora, Tim Brooks, to work on “world simulators.”

Very Leaky Shipmas

Last week a group of artists claiming to be beta testers of OpenAI's Sora video generator leaked the model in protest. OpenAI reportedly considers Sora to be a world model, as it can simulate actions like a painter leaving brush strokes on a canvas or video game worlds. The company just announced it will be making 12 ‘shipmas’ releases in December, and Sora is expected to be one of them.

A Well Versed Startup

VERSES is another LA-based startup pioneering a novel approach to spatial AI through its Spatial Web Foundation and Intelligent Framework (VIF). It aims to create adaptive, biomimetic intelligence systems that can learn, self-organize, and operate across complex networked environments, inspired by biological intelligence. Its flagship product is a potential computational intelligence paradigm shift in AI development, focusing on creating intelligent ecosystems that can dynamically adapt and learn, rather than static computational models. VERSES is well positioned to deploy its framework across industries, with $50M raised from backers including Playground Global (founded by Peter Thiel) and other prominent tech VC funds.

Digital Twins Meets World Models

The digital twins sector and AI world models are closely interconnected. Both aim to create dynamic, predictive representations of systems. Digital twins rely on real-time data from sensors and IoT devices to simulate and optimize physical systems, while world models enable adaptive learning and scenario planning. Together, the two could enhance applications in smart cities, healthcare, and industrial automation by enabling real-time decision-making, predictive maintenance, and improved system resilience. This synergy forms a feedback loop where digital twins provide data to refine AI models, which in turn make digital twins more intelligent and adaptive. Startups like VERSES and larger enterprises like Siemens could be a play to get exposure to the intersection of these two trends as they develop industry together.

“We already have the ability to create virtual, interactive worlds, but it costs hundreds and hundreds of millions of dollars and a ton of development time. They will let you not just get an image or a clip out, but a fully simulated, vibrant, and interactive 3D world.”

— Justin Johnson, World Labs co-founder, on the a16z podcast..

TECH BUZZ DIRECTORY
Featured Products

SEO Bot — A fully autonomous "SEO Robot" with AI agents for busy founders.

Memelord — Meme Software for marketing memes, tech memes, & sales memes.

Otonomos — Incorporate your Delaware C-corp or tax advantaged foundation and get 5% OFF.

TRENDING TOOLS AND BUZZY TECH
Tools

ElevenLabs — New end-to-end conversational AI agents building service.

Google — Veo video generator beta. GenCast AI Agent 15-day weather forecasts.

The Browser Company — New AI Agent browser Dia with task automation in 2025.

Hume AI — New Voice Control feature with 10 sliders for custom AI voices.

AWS — Nova AI models revealed. Project Rainier supercomputer. Trainium2 chips now, Trainium3 in 2025. Apple to use AWS AI chips for search, training. Anthropic’s Claude now optimized for AWS Trainium2.