The success of Meta’s Muse personal assistant has convinced the AI industry that personal agents that proactively complete tasks are the next logical evolution of the technology, but it’s unclear if there is enough compute to power them—at least in the timeframe that these companies are betting on.
This week, Elon Musk said the cost of running Grok Bot, the personal AI assistant SpaceX introduced in August, is “unsustainably high relative to revenue.” He suggested the company was throttling back the compute allocation for the technology, causing users to hit their limits faster.
Last night, at an intimate dinner with reporters in New York City, the head of ChatGPT’s consumer business said that OpenAI’s Dots assistants are “on the compute heavy side.” Each Dot assistant, OpenAI’s Adam Fry said, has its own cloud computer, which it uses to execute tasks on the user’s behalf, accessing any accounts and services it has been given access to.
Dots are helping people read and respond to emails, find and book tickets to events, and a far range of other assistant-like tasks. Through new features like “background mode,” Fry said, Dots plan to become an always-available presence, quietly running our lives, and changing how we go about our daily lives.
The company is moving all the right chess pieces to make this possible—as long as it gets enough compute. OpenAI has invested heavily in voice AI models, whose humanlike responses make interacting with the technology more seamless. ChatGPT’s voice mode now has 250 million weekly active users, Fry said. OpenAI also claims that ChatGPT search volume, measured by queries with search intent, is now one-fifth of Google’s volume. Paired with its forthcoming consumer devices designed by legendary iPhone architect Jony Ive, set to debut this year, all of these pieces could come together in a powerful consumer experience.
The success of these initiatives hinges on whether OpenAI can find enough compute in the couch cushions. The availability of Dots is limited today to those with a Pro subscription, which costs a whopping $100 per month. Bringing Dots to the masses would require a breakthrough in compute efficiency, which Fry said he expects will happen, causing costs to fall rapidly, even as model performance improves. At the same time, the more powerful—and constantly running—personal AI agents become, the more compute they consume. Will OpenAI ever be able to catch up with itself?
Today, OpenAI announced it is introducing a system by which Dots will route different tasks to different underlying AI models, including its GPT-6.1 Terra model. Terra is smaller and uses less computing power than GPT-6 Astra, the larger model that originally powered all Dots tasks. This new automatic task router will allow OpenAI to reduce how compute intensive Dots is.
Meanwhile, the nationwide, bipartisan backlash to data centers has become so undeniable that OpenAI this week kicked off efforts to rebrand them “compute campuses.” Community pushback could cause permits to slow, or even kill facilities entirely, if the benefit to the community is not clear enough when it comes to job creation, environmental protection, and a host of other issues. In OpenAI’s inaugural post for the “compute campuses” lingo, it promised to reduce strain on local electrical grids and not take the limited supply of power from local families.
Anthropic’s S-1 filing gives us a sense of the astronomical sums the frontier labs are spending on compute. Anthropic has reportedly entered into various long-term and short-term purchase agreements, totalling $895.4 billion, “substantially all of which is related to compute and infrastructure,” an updated amendment to its S-1 from this week reads. Yet it may not be enough. Inability to secure even more compute is listed as a major risk factor that could cause Anthropic to be unsuccessful.
Although OpenAI has not yet filed its S-1, or confirmed IPO timing, sources tell us it almost certainly faces similar compute challenges, although it has some unique ones of its own. Unlike Anthropic, whose infrastructure includes four different types of chips, OpenAI’s fleet is largely dependent on Nvidia GPUs. The company is working on developing its own in-house chip, which it’s calling Jalapeno, set to scale up production in 2027.
OpenAI is also still searching for more funding, given that the company—like Anthropic—does not yet make a profit. Yet private funding may be nearing its end after OpenAI’s latest round fell short of its $30 billion target, landing at $11 billion, the Wall Street Journal reports. But even $30 billion would likely be a drop in the bucket when it comes to compute spending. The only option to fund the personal AI agent boom might be to go public, and even then success is not guaranteed.
This story was originally featured on Fortune.com