Running a large AI model at scale has become less a scientific feat than a brutal math problem about electricity bills a

Running a large AI model at scale has become less a scientific feat than a brutal math problem about electricity bills and datacenter rent. Every token Gemini generates costs Google something real—silicon cycles, memory bandwidth, and watts drawn from the wall. As query volume grows, fractions of a cent add up to sums that can swallow margins whole. That quiet urgency is behind Frozen v2, an internal server chip project now taking shape inside Google. Rather than refining its general-purpose Tensor Processing Units for another generation, the company is attempting something far more radical: casting the skeleton of the Gemini model directly into the silicon itself.

From Flexible Accelerators to Model-Specific Silicon

Google’s TPUs have been the workhorses of its infrastructure for nearly a decade. They train models, power search ranking algorithms, and even rent out by the hour to cloud customers including Meta and others looking for an alternative to Nvidia’s GPUs. That versatility is exactly what makes a TPU a TPU. It speaks a general vocabulary of matrix multiplication and memory movement, usable by almost any neural network you can describe in software.

Frozen v2 deliberately trades that flexibility away. The chip is being designed as a domain-specific accelerator whose circuits physically mirror portions of Gemini’s own architecture. Where a TPU fetches instructions and interprets them as software operations, Frozen v2 would burn the model’s structural blueprint—the arrangement of its layers and data paths—directly into the chip’s layout. Google expects this tight marriage of model and metal to make the chip six to ten times more efficient than current TPUs at serving AI responses. Fewer compute steps per query means less time waiting for a token to appear, and far less energy spent generating it.

This is not merely a faster version of the same idea. It is a different category of chip, one that trades generality for devotion to a single model family.

Why the First “Frozen” Melted

This approach has roots in an earlier concept attributed to Jeff Dean, Chief Scientist at Google DeepMind. The original “Frozen” proposal suggested pushing specialization even further by hardcoding not just the architecture, but the actual model weights—the billions of tuned parameters that constitute Gemini’s learned behavior—directly into the chip itself.

The logic was sound. If you know exactly which numbers the model will use, why bother fetching them from external memory? You could etch them into the transistors and eliminate entire categories of delay.

The problem was permanence. AI models do not stand still. Google updates Gemini continuously, retraining on new data, adjusting parameters, releasing improved versions. A chip with weights frozen in silicon would become a paperweight the moment a new model revision shipped. That lack of flexibility killed the original concept.

Architecture Without the Anchor

Frozen v2 solves the obsolescence trap by hardcoding the architecture while leaving the weights free to change. Think of it as pouring a custom racetrack instead of welding the car to the road. The shape of the circuit stays fixed, optimized for Gemini’s specific patterns of computation, but the contents flowing through those circuits can be refreshed by loading new weights from memory.

This distinction matters in practice. When engineers train a new Gemini checkpoint, they can deploy it to Frozen v2 hardware without fabricating a new chip. The exact degree of hardcoding is still an open question inside Google; teams must decide precisely which structural elements deserve silicon immortality and which should remain configurable. But the principle is settled. By freezing the shape and fluidly swapping the parameters, Google keeps the efficiency upside without sacrificing the ability to iterate.

The Economics of Keeping It In-House

There is another reason you will not see Frozen v2 listed on Google Cloud’s pricing sheet. Because the chip is so intimately shaped around Gemini’s internals, it would make little sense to outside developers running PyTorch or custom Transformer variants. Google has no plans to sell it as a general-purpose product. It will remain an internal tool, aimed squarely at the crushing demand for inference capacity inside Google’s own datacenters.

That choice reflects a blunt economic reality. In the current generative AI market, model capabilities are converging quickly. The gap between competitors often comes down to who can afford to run the largest model at the lowest cost per token. Inference is no longer an afterthought to training; for a widely used product like Gemini, it is the dominant expense. If Frozen v2 cuts that expense by a factor of six or more, Google gains headroom that competitors cannot easily match. It can either pocket the savings as margin or pass them along as lower prices for API consumers and product integrations, tightening the screws on OpenAI, Anthropic, and others.

What This Signals for the Industry

Google’s move also hints at where the broader hardware strategy is heading. For years, the standard playbook was to build the most flexible accelerator possible and let software handle the specialization. Nvidia’s GPUs dominate because they run everything from molecular dynamics to video games to large language models. Google’s own TPUs were conceived in that same spirit of broad utility.

Frozen v2 breaks from that tradition. It is an admission that when a single model family drives enough query volume, custom silicon tailored to that model can pay for itself many times over. Other hyperscalers have pursued similar logic—Amazon’s Trainium and Inferentia chips, for instance—but Google’s approach goes deeper by co-designing hardware around a specific model architecture rather than a general class of networks.

The risk, of course, is rigidity. If Gemini’s architecture evolves in a direction that the hardcoded circuits cannot accommodate, Google could find itself with expensive silicon that cannot run its newest ideas. That is precisely why the architecture-only compromise matters. It offers a middle path: enough specialization to capture dramatic efficiency gains, enough flexibility to avoid painting the company into a corner.

The Real Takeaway

Frozen v2 is best understood not as a chip announcement, but as a strategic bet on the future shape of AI competition. Google is wagering that the winners will not merely build the best models, but will own the entire stack—from the model’s blueprint down to the electrons pushing through the transistor. If the project succeeds, the payoff will not show up in benchmark scores. It will show up in the cost column of a quarterly earnings report, where a few cents saved per million tokens can redraw the boundaries of what is commercially possible in generative AI.