thereâs a fierce scramble to secure every power contract still available for the rest of the decade, every voltage transformer that can possibly be procured.
By the end of the decade, American electricity production will have grown tens of percent;
The models, they just want to learn; you scale them up, and they learn more.
picking the many obvious low-hanging fruit on âun- hobblingâ gains should take us from chatbots to agents, from a tool to something that looks more like drop-in remote worker replacements.
âunhobblingâ gains: By default, models learn a lot of amazing raw capabilities, but they are hobbled in all sorts of dumb ways, limiting their practical value. With simple algorithmic improvements like reinforcement learning from human feedback (RLHF), chain-of-thought (CoT), tools, and scaffolding, we can unlock significant latent capabilities.
algorithmic efficiencies: Thereâs a continuous trend of algorithmic progress. Many of these act as âcompute multi- pliers,â and we can put them on a unified scale of growing effective compute.
compute: Weâre using much bigger computers to train these models.
We can decompose the progress in the four years from GPT-2 to GPT-4 into three categories of scaleups:
primarily because of broad scaleups in investment (and specializing chips for AI workloads in the form of GPUs and TPUs), the training com- pute used for frontier AI systems has grown at roughly ~0.5 OOMs/year.
1 OOM = 10x
2 OOM = 100x
3 OOM = 1000xâŚ
we seem to find new algorithmic improvements at a fairly consistent rate. Individual discoveries seem random, and at every turn, there seem insurmountable obstaclesâbut the long-run trendline is predictable, a straight line on a graph. Trust the trendline.
While compute efficiencies will become harder to find as we who would suggest it will all suddenly pick the low-hanging fruit, AI lab investments in money and come to a halt! talent to find new algorithmic improvements are growing rapidly.
What a modern LLM does during training is, essentially, very very quickly skim the textbook, the words just fly- ing by, not spending much brain power on it. ⢠Rather, when you or I read that math textbook, we read a couple pages slowly; then have an internal monologue about the material in our heads
âChain-of-thoughtâ prompting unlocked that for LLMs. Despite excellent raw ca- pabilities, they were much worse at math than they could be because they were hobbled in an obvious way, and it took a small algorithmic tweak to unlock much greater capabilities.
Tools: Imagine if humans werenât allowed to use calculators or computers. Weâre only at the beginning here, but Chat- GPT can now use a web browser, run some code, and so on.
posttraining improvements that unlocked latent model capability
A much smaller base model with, say, 100k tokens of relevant context can outperform a model that is much larger but only has, say, 4k relevant to- kens of contextâmore context is effectively a large compute efficiency gain.
GPT-4 has the raw smarts to do a decent chunk of many peopleâs jobs, but itâs sort of like a smart new hire that just showed up 5 minutes ago: it doesnât have any relevant con- text, hasnât read the company docs or Slack history or had conversations with members of the team
Most useful cognitive work hu- mans do is longer horizonâit doesnât just take 5 minutes, but hours, days, weeks, or months.
By 2027, rather than a chatbot, youâre going to have something that looks more like an agent, like a coworker.
f we could unlock âbeing able to think and work on something for months- equivalent, rather than a few-minutes-equivalentâ for mod- els, it would unlock an insane jump in capability.
Perhaps a small amount of RL helps a model learn to error correct (âhm, that doesnât look right, let me double check thatâ), make plans, search over possible solutions, and so on. In a sense, the model already has most of the raw capabilities, it just needs to learn a few extra skills on top to put it all together.
Intermediate models between now and the drop- in remote worker will require tons of schlep to change work- flows and build infrastructure to integrate and derive economic value from.
In the subsequent 4 years, we should expect 3â6 OOMs of base effective compute scaleup (physical compute al- gorithmic efficiencies)âwith perhaps a best guess of ~5 OOMsâplus step-changes in utility and applications un- locked by âunhobblingâ
But once models can automate AI research itself, thatâs enoughâenough to kick off intense feed- back loopsâand we could very quickly make further progress, the automated AI engineers themselves solving all the remain- ing bottlenecks to fully automating everything.
millions of automated researchers could very plausibly com- press a decade of further algorithmic progress into a year or less.
These AI systems will basi- cally be able to automate basically all cognitive jobs (think: all jobs that could be done remotely).
As with every generation before them, every new gen- eration of models will dumbfound most onlookers; theyâll be incredulous when, very soon, models solve incredibly difficult science problems that would take PhDs days, when theyâre whizzing around your computer doing your job, when theyâre writing codebases with millions of lines of code from scratch,
Scaling up simple deep learning techniques has just worked, the models just want to learn, and weâre about to do another 100,000x+ by the end of 2027.
weâre in the middle of a huge scaleup reap- ing one-time gains this decade, and progress through the OOMs will be multiples slower thereafter.
Spending scaleup: Spending a million dollars on a model used to be outrageous; by the end of the decade, we will likely have $100B or $1T clusters.
AI hardware has been improving much more quickly than Mooreâs law. Thatâs because weâve been specializing chips for AI workloads. For exam- ple, weâve gone from CPUs to GPUs; adapted chips for Transformers; and weâve gone down to much lower pre- cision number formats, from fp64/fp32 for traditional supercomputing to fp8 on H100s
Once we get AGI, we wonât just have one AGI. Iâll walk through the numbers later, but: given inference GPU fleets by then, weâll likely be able to run many millions of them (perhaps 100 million human-equivalents, and soon after at 10x+ human speed). Even if they canât yet walk around the office or make coffee, they will be able to do ML research on a computer.
the job of an AI researcher is fairly straightforward, in the grand scheme of things: read ML literature and come up with new questions or ideas, implement experiments to test those ideas, interpret the results, and repeat.
The job of an AI researcher is also a job that AI researchers at AI labs just, well, know really wellâso itâll be particularly intuitive to them to optimize models to be good at that job. And there will be huge incentives to do so to help them accelerate their research and their labsâ competitive edge.
Another way of thinking about it is that given inference fleets in 2027, we should be able to generate an entire internetâs worth of tokens, every single day.
Theyâll be able to read every single ML paper ever written, have been able to deeply think about every single previous experiment ever run at the lab, learn in parallel from each of their copies, and rapidly accumulate the equivalent of millennia of experience.
You wonât have to individually train up each automated AI researcher (indeed, training and onboarding 100 million new human hires would be difficult). Instead, you can just
A million times more research effort via automated research labor wonât mean a million times faster progress, because compute will still be limitedâand limited compute for experiments will be the bottleneck
if you can au- tomate, say, 70% of something, you get some gains but quickly the remaining 30% become your bottleneck. For any- thing that falls short of full automationâsay, really good copilotsâhuman AI researchers would remain a major bottleneck,
there are incredible returns to having great intuitions on the dozens of hyperparameters and details of an experi- ment,. Jason calls this ability to get things right on the first try based on intuition âyolo runsâ.
25 OOMs of algorithm progress seems impossible, since that would imply being able to train a GPT-4 level system in less than ~10 FLOPs
Factories would go from human-run, to AI- directed using human physical labor, to soon being fully run by swarms of robots
itâs clear that robots are an ML algorithms problem. LLMs had a much easier way to bootstrap: you had an entire internet to pretrain on. Thereâs no similarly large dataset for robot actions, and so it requires more nifty approaches (e.g. using multimodal models as a base, then using synthetic data/simulation/clever RL) to train them.
As AI revenue grows rapidly, many trillions of dollars will go into GPU, datacenter, and power buildout before the end of the decade.
AI is a massive industrial process: each new model requires a giant new cluster, soon giant new power plants, and eventually giant new chip fabs.
Trillions of dollars of capex will churn out 100s of millions of GPUs per year overall.
Nvidia shocked the world as its datacenter sales exploded from about $14B annualized to about $90B annualized in the last year.
What any com- pute guy is thinking about is securing power, land, permitting, and datacenter construction.4
in a post-AGI world, having the most compute will probably still really matter.
close to half of datacenter capex is on things other than the chips (site, building, cooling, power, etc.).
itâs plausible revenue is slowed because intermediate, pre-AGI models take a lot of âschlepâ to properly integrate into company workflows; historically, itâs taken a while to fully harvest the productivity gains from new general purpose technologies
Forecasts of overall revenue growth for these companies would skyrocket. Stock markets would follow
power has become the binding constraint: there simply isnât much spare capacity, and power contracts are usually long-term locked-in
The harder part would be building enough generators / turbines;
Permitting, utility regulation, FERC regulation of transmission lines, and NEPA environmental review makes things that should take a few years take a decade or more.
While chips are usually what comes to mind when people think about AI-supply-constraints, theyâre likely a smaller constraint than power.
From a pure logic fab standpoint ~100% of TSMCâs output for a year could already support the trillion-dollar cluster (again if all the chips went to one datacenter)
The AI revolution means working our transis- tors way harder, dedicating them all to constantly-running, high-performance AI datacenters instead of idle, battery- powered/energy-saving devices.
TSMC ~doubled66 in the past 5 years; theyâd likely need to
66 (Using revenue as a proxy.) go ~at least twice as fast on their pace of expansion to meet AI chip demand. Massive new fab investments would be neces- sary.
While onshoring more of AI chip production to the US would be nice, itâs less critical than hav- ing the actual datacenter (on which the AGI lives) in the US
Even if raw logic fabs wonât be the constraint, chip-on-wafer- on-substrate (CoWoS) advanced packaging (connecting chips to memory, also made by TSMC, Intel, and others) and HBM memory (for which demand is enormous) are already key bottlenecks for the current AI GPU scaleup; these are more specialized to AI, unlike the pure logic chips, so thereâs less pre-existing capacity.
irreversible security risk: it risks the AGI weights getting stolen6
we should prioritize datacenters in the US while betting more heavily on democratic allies like Japan and South Korea for fab projects
(What all of this means for NVDA/TSM/etc I leave as an exercise for the reader. Hint: Those with situational awareness bought much lower than you, but itâs still not even close to fully priced in.)
ânone of that matters if China or others can simply steal the model weights (all a finished AI model is, all AGI will be, is a large file on a computer) or key algorithmic secrets (the key technical break- throughs necessary to build AGI).
AGI-level security for algorithmic secrets is necessary years before AGI- level security for weights. These algorithmic breakthroughs will matter more than a 10x or 100x larger cluster in a few yearsâthis is a much bigger deal than export controls on com- pute, which the USG has been (presciently!) intensely pursu- ing
An AI model is just a large file of numbers on a server. This can be stolen. All it takes an adversary to match your trillions of dollars and your smartest minds and your decades of work is to steal this file.
Securing weights will require innovations in hardware and radically different cluster design; and security at this level canât be reached overnight, but requires cycles of iteration.
Between the labs, there are thousands of people with access to the most important secrets; there is basically no background-checking, siloâing, controls, basic infosec, etc. Things are stored on easily hackable SaaS services.
Anyone, with all the secrets in their head, could be offered $100M and recruited to a Chinese lab at any point.
Fully airgapped datacenters, with physical security on par with most secure military bases (cleared personnel, physical fortifications, onsite response team, extensive surveillance
The idea behind RLHF is simple: the AI system tries stuff, humans rate whether its behavior was good or bad, and then reinforce good behaviors and penalize bad behaviors. That way, it learns to follow human preferences.
What RL is doing is simply explor- ing strategies for succeeding at the objective. If a strategy works, it is rein- ⢠What we want is to add side-constraints: donât lie, donât forced in the model. So if lying, fraud, break the law, etc. power-seeking, etc. (or patterns of thinking that could lead to these sorts of behaviors in at least some situations) ⢠But here we come back to the fundamental issue of align- work, these will also be reinforced in the model.
we may well boot- strap our way to human-level or somewhat-superhuman AGI with systems that reason via chains of thoughts, i.e. via English tokens. This is extraordinarily helpful, because it means the models âthink out loudâ letting us catch ma- lign behavior (e.g., if itâs scheming against us). But surely having AI systems think in tokens is not the most efficient means to do it, surely thereâs something much better that does all of this thinking via internal statesâand so the model by the end of the intelligence explosion will almost certainly not think out loud, i.e. will have completely un- interpretable reasoning.
we might try to build an âAI lie detectorâ by identifying the parts of the neural net that âlight upâ when an AI system is lying.
China now seems to have demonstrated the ability to manufacture 7nm chips. While going beyond 7nm will be difficult (requiring EUV), 7nm is enough! For reference, 7nm is what Nvidia A100s used.
In the last decade, China has roughly built as much new elec- tricity capacity as the entire US capacity (while US capacity has remained basically flat). In the US, these things get stuck in environmental review, permitting, and regulation for a decade first. It thus seems quite plausible that China will be able to simply outbuild the US on the largest training clusters.
by late 26/27/28 it will be underway. The core AGI re- search team (a few hundred researchers) will move to a secure location; the trillion-dollar cluster will be built in record-speed;
whether we like it or not, superintelligence wonât look like an SF startup, and in some way will be primarily in the domain of national security.
Superintelligence is a matter of national security.
going all-in leveraged long Nvidia in early 2023 has been great and all,
$25k cost per H100, 10k H100s-equivalent, and Nvidia GPUs being around half the cost of a cluster (the rest being power, the physical datacen- ter, cooling, networking, maintenance personnel, etc.).