Can AI Stop Being a Space Heater?

Every serious conversation about AI hardware eventually collapses into thermodynamics, whether people admit it or not. A chip consumes electricity. That electricity becomes useful switching, leakage, signaling, and then, with brutal inevitability, heat. The more compute you pack into a rack, the more aggressively you must remove that heat or the rack throttles, fails, or both.

The International Energy Agency estimated that data centres consumed about 415 TWh of electricity in 2024, and projects roughly 945 TWh by 2030 in its base case. That is not a rounding error. It means AI is no longer just a software story or even a chip story. It is a building, utility, cooling, and transmission story.

People often talk as if cooling is a separate tax imposed on top of compute. Physically, it is the same tax viewed from another angle. If your hardware dissipates a megawatt, your facility now has a megawatt-shaped problem to get rid of. Fans, chillers, pumps, liquid loops, power delivery losses, and airflow design are not side quests. They are the invoice for your choice of computing substrate.

Thermodynamic computing only makes sense against that backdrop. If the problem were merely "we need a slightly better GPU," the entire field would sound eccentric. But once energy becomes the limiting resource, even odd ideas deserve a hearing. The question is no longer only "how many operations per second?" It is "how many useful answers per joule, per rack, per liter of cooling water, per megawatt of site capacity?"

GPUs are extraordinary machines. They also carry historical baggage. They were refined for graphics, then reused for machine learning because the math lined up beautifully with massively parallel linear algebra. That was a brilliant hack. It may not remain the optimal long-term substrate for every AI workload.

The central annoyance is the so-called memory wall. Arithmetic units are cheap only if data arrives exactly when needed. In large models, especially generative ones, moving activations, keys, values, and especially weights around the system becomes an energy problem in its own right. IBM's recent analog in-memory work phrases the issue plainly: every time you move weights on a traditional architecture, you spend time and energy just getting computation into position.

This is why so many "next-wave" hardware companies sound different but rhyme technically. Some want to compute inside memory. Some want to move information with photons. Some want to make randomness native instead of synthesizing it expensively with deterministic circuits. All of them are, in one way or another, rebelling against the same plumbing bill.

This is also why "AI without massive cooling" is a slightly misleading dream. You probably do not get there by abolishing heat. You get there by changing the ratio of useful reasoning to wasteful movement. That is a much harder, much more interesting target.

First, a defensive clarification. Thermodynamic computing does not mean you pour heat onto a chip until it becomes intelligent. The more sober idea is this: some physical systems naturally fluctuate, relax, and settle into patterns that correspond to probability distributions. If the computational problem you care about is itself about probabilities, then maybe you can let the physics do more of the work.

Classical digital chips work very hard to suppress ambiguity. A voltage must be cleanly interpreted as 0 or 1. Noise margins exist precisely so random fluctuations do not leak into the logic. Probabilistic hardware takes the opposite posture. It says: for certain tasks, especially sampling and uncertainty, that randomness is not contamination. It is raw material.

The cleanest entry point is the p-bit, or probabilistic bit. The literature describes it as a bit that flips between 0 and 1 with a controllable probability. Not a quantum state. Not a mysterious superposition marketing term. Think of a weighted coin that keeps tossing itself. Change its bias and it spends more time as heads than tails. Couple many such coins together and the network starts to represent a whole distribution rather than a single hard-coded answer.

How Do You Physically Build A P-bit?

The canonical hardware story in the p-bit literature is spintronic. A 2019 review by Çamsarı, Sutton, and Datta argues that low-barrier magnets are a natural physical representation of p-bits. In that picture, a stochastic magnetic tunnel junction, or sMTJ, fluctuates spontaneously because its magnetic free layer is intentionally made unstable enough to keep wandering. Attach standard transistor logic to that fluctuating element and you get a noisy bit whose mean value can be biased up or down by an input signal.

But that is not the only route. Early hardware demonstrations also emulated p-bits with microcontrollers, while newer work uses CMOS plus stochastic nanomagnets, oscillator-based p-bits, or other tunable noisy circuits. The common recipe stays the same even when the substrate changes: you need one element that naturally fluctuates, one mechanism that biases its average behavior, and a coupling scheme that lets many such units influence one another.

The brain comparison is useful here if handled carefully. A neuron is not literally a p-bit. It does not emit a perfectly memoryless random telegraph signal, and synapses are richer than a simple scalar coupling matrix. But as an analogy it is good: neurons integrate inputs, sit near thresholds, spike stochastically, and recurrent circuits can settle into attractors or clean up noisy evidence. So when people say "in brains this is neural spiking," the careful version is: brains are not p-bit chips, but noisy spiking neurons are the closest biological cousin to the basic computational idea.

This matters because many AI and scientific workloads are secretly sampling problems. Bayesian inference asks you to reason over distributions. Diffusion-style generation repeatedly denoises random states. Monte Carlo methods wander through state spaces. Optimization routines often benefit from controlled stochasticity because it helps them escape brittle local structure. A thermodynamic machine does not have to be good at everything to be economically relevant. It only has to be much better at the subset of workloads that are becoming expensive enough to hurt.

If that sounds suspiciously like "an Ising machine with better branding," you are not entirely wrong to notice the family resemblance. The important question is not the label. It is whether a given physical implementation can be manufactured, programmed, benchmarked, and integrated cheaply enough that the sampling advantage survives contact with the rest of the system.

Before talking about startups, it is fair to ask a more basic question: what kind of useful computation would a p-bit network even perform? The answer is not "instantly run GPT-7." The more honest answer is: p-bit networks are naturally good at problems where you want a system to settle into a self-consistent low-energy state. That includes toy versions of denoising, associative memory, error correction, optimization, and constraint satisfaction.

A classic mental model is a noisy memory. You store a few patterns in the couplings between bits. Then you present a corrupted clue. If the clue is not too damaged, and the network's dynamics are set up sensibly, the local stochastic updates can pull the state toward the nearest stored pattern. That is already a real computational act: the machine is cleaning up ambiguity by wandering through an energy landscape instead of following a fixed deterministic recipe.

The widget below is exactly that sort of toy. It is not modern machine learning. It is the conceptual bridge you need before Extropic or Normal start talking about denoising, diffusion-like models, or hardware-native sampling.

Scale this idea up and you can at least see why the AI connection exists. Diffusion models are denoisers. Bayesian inference is distributional. Many optimization problems want systems to explore, then settle. None of that proves a startup has solved the problem. But it does explain why people are chasing hardware whose native behavior looks less like rigid logic and more like guided probabilistic relaxation.

As of October 29, 2025, Extropic publicly laid out the cleanest startup version of the thermodynamic-computing thesis. The company describes its hardware as thermodynamic sampling units, or TSUs. Notice the naming move. Not processing units. Sampling units. Extropic's argument is that if generative AI ultimately needs samples from complicated probability distributions, then it is wasteful to route everything through ordinary deterministic processing first.

Its public materials say TSUs keep memory and compute distributed, and allow communication mostly between physically local circuits. That matters because the company explicitly identifies communication, not arithmetic alone, as the real energy sink in standard architectures. In its telling, TSUs do not merely run existing GPU-style models a little faster. They change the primitive from programmable deterministic compute to programmable probabilistic sampling.

Extropic also introduced a software layer, THRML, and a model family it calls Denoising Thermodynamic Models, or DTMs. The rhetorical intent is obvious and smart: show that the hardware is paired with an algorithmic stack rather than arriving as an isolated lab toy. DTMs are framed as diffusion-inspired generative models that use TSUs for the denoising process. That is a better story than "please rewrite AI from scratch for our exotic chip." It is still, importantly, a rewrite of meaningful parts of the stack.

The hardware roadmap on Extropic's site currently lists X0 as a Q1 2025 silicon prototype, XTR-0 as a Q3 2025 experimental platform, and Z1 for early access in 2026. The Z1 page describes hundreds of thousands of probabilistic circuits per chip and millions per card. That is meaningful because it tells you the company is thinking beyond toy boards and toward something card-shaped and rack-adjacent.

The charitable interpretation is that Extropic has a coherent thesis: sampling is native, locality matters, and probabilistic AI may deserve hardware built for it rather than inherited from graphics. The skeptical interpretation is equally obvious: the company must prove that a custom probabilistic stack can beat incredibly optimized GPU software once all the unglamorous system costs show up. Both interpretations can be true at once.

Normal Computing takes a related but stylistically different route. Its language is less about a sampling unit stack and more about physics-based ASICs. The company says the infrastructure behind today's AI was never designed for current scale or energy demands, and its research pages tie the program explicitly to probabilistic reasoning, diffusion-like models, and stochastic differential equation emulation.

There are two public milestones worth separating. First, on November 9, 2023, the company published a demonstration of thermodynamic matrix inversion using coupled electrical oscillators on a real board. That was not a production chip, but it was a concrete proof that physical equilibration could be used for a nontrivial numerical primitive. Second, on August 12, 2025, Normal announced tape-out of CN101, which it described as the world's first thermodynamic computing chip and as a validation step for its Carnot architecture.

Normal's framing differs slightly from Extropic's. Extropic emphasizes arrays of p-bits and energy-based generative models. Normal emphasizes fluctuations, dissipation, stochasticity, and hardware built for diffusion-like models and other probabilistic workloads. Underneath, though, the family resemblance is strong: both are trying to exploit local stochastic physics to do useful AI work more efficiently than a conventional deterministic stack.

Again, the number that grabs attention needs careful handling. Normal's CN101 announcement says the architecture could achieve up to 1000× energy-consumption efficiency on targeted AI and scientific workloads. The phrasing matters: targeted workloads, not all workloads; company announcement, not neutral production benchmark.

If you put Extropic and Normal side by side, the high-level pattern becomes visible. Both are betting that uncertainty is not a nuisance to be emulated on rigid hardware, but a computational primitive to be harnessed directly. Both also face the same brutal test: can that physical elegance survive the messiness of software stacks, I/O, manufacturing, economics, and customer adoption?

Thermodynamic computing is the flashy new entrant because it proposes a different primitive. But it is not the only serious attempt to reduce AI's power bill. The more general story is that the industry is attacking the same bottleneck from several angles at once: reduce data movement, shorten communication distance, lower numerical precision, push more work into memory, or use a different physical carrier entirely.

Analog in-memory computing is the most direct rebellion against the memory wall. IBM describes it succinctly: if traditional architectures burn time and energy every time they move weights to compute on them, then blend memory and compute until that movement shrinks. EnCharge frames its own technology the same way, emphasizing robust analog in-memory computing that reduces data movement. Mythic's pitch is similar again, but aimed especially at edge devices where power, cooling, and size are brutally constrained.

Photonics attacks a different slice of the problem. Lightmatter's public materials emphasize interconnect and bandwidth: if electrical signaling and chip-to-chip movement are becoming dominant costs, send information with light where that physics helps. Their recent public work also argues that lower-precision AI workloads make photonic compute more plausible than earlier generations of photonic hype did.

These are not mutually exclusive worlds. In fact, the most realistic future probably looks like a strange coalition: digital control logic, memory-centric compute for weight-heavy paths, photonic fabrics for bandwidth, and perhaps thermodynamic samplers where uncertainty and local stochastic relaxation are the natural fit.

The right mental model is not "who wins?" It is "which part of the energy bill is each approach trying to delete?" Thermodynamic chips want the sampling bill lower. In-memory chips want the weight-shuttling bill lower. Photonics wants the bandwidth bill lower. Edge accelerators want to avoid pushing everything into giant centralized cooling plants in the first place.

Here is the honest answer: not literally, no. If by "without significant energy or cooling costs" you mean a future where advanced AI somehow stops obeying physics, that future is not on the menu. A sufficiently valuable model running at sufficiently large scale will still draw serious power and will still need serious thermal management.

But the softer, engineering answer is much more encouraging. You can absolutely imagine a world where the joules per useful token, image, simulation, or control action fall dramatically. If that happens, the cooling problem also shrinks because there is less waste heat to remove per unit of useful work. The real win is not zero power. It is a much better ratio of intelligence to infrastructure.

The likely ingredients are boring in aggregate and radical in combination: lower precision, more sparsity, smarter routing, more work done inside or near memory, hardware tuned to dominant subroutines, more on-device inference when latency allows, and possibly stochastic hardware where the workload is naturally probabilistic anyway. No single ingredient is enough. Together they can bend the curve.

Training and inference do not have the same near-term future

Specialized hardware usually has an easier first life in inference than in frontier-scale training. Inference workloads repeat, costs are persistent, and customers tolerate more specialization if the cost per served result collapses. Training is harsher. It requires mature software, robust numerics, huge distributed coordination, reproducibility, and a developer ecosystem that does not break every time the model changes shape.

So yes: you can plausibly get much cheaper and cooler AI than today's trajectory suggests. No: you should not expect the energy problem to disappear. That distinction matters, because the most dangerous hype in this area usually begins by erasing it.

Interesting physics is cheap. Important computing platforms are expensive. There is a long, brutal funnel between the two. Thermodynamic computing becomes economically meaningful only if it survives every stage of that funnel without losing the very advantage that made it exciting in the first place.

Start with the algorithmic layer. The hardware must map to workloads that people genuinely want to run, not just to elegant toy problems. Then the accuracy story has to hold up: if energy savings buy you unusable outputs, nobody cares. Then the software story arrives with a baseball bat. If model developers cannot target the hardware without inventing a new religion, adoption slows to a crawl.

Then come the unromantic layers. How much host logic is needed? How expensive is calibration? How ugly is the compiler? How much of the promised efficiency vanishes once you include I/O, memory staging, packaging, monitoring, and orchestration? Does the foundry path look normal enough that customers can believe in supply? Do benchmark wins persist on economically relevant workloads rather than hand-picked demos? These are not gotchas. They are the actual job.

If you keep that filter in mind, the space becomes easier to reason about. You no longer have to choose between breathless belief and lazy cynicism. You can simply ask: where, exactly, is the claimed efficiency coming from, and what would have to remain true after packaging, software, and deployment for that advantage to survive?

The most plausible future is not a thermodynamic-computing monoculture. It is a rack full of specialists. CPUs still orchestrate. GPUs or digital tensor cores still dominate dense deterministic math. In-memory blocks handle weight-heavy inference paths. Photonic links relieve the bandwidth pain between chips and packages. And if thermodynamic hardware proves itself, it gets inserted where sampling, uncertainty, and local stochastic relaxation genuinely outperform the digital alternative.

This is a less cinematic future than "one weird chip kills Nvidia." It is also more believable. Hardware history usually advances by annexation, not revolution. New primitives enter the stack where they are obviously better, then expand outward if they earn the right.

The deepest lesson here is philosophical as much as technical. We have spent years treating AI as if every useful computation should be beaten into the shape of matrix multiply. That was a tremendously productive simplification. It may also be the simplification that starts failing first under energy pressure. Thermodynamic computing is one attempt to ask what comes after that simplification.

So, can AI stop being a space heater? Not entirely. But it can stop being needlessly space-heater-shaped. And that may turn out to matter more than any single miraculous chip.