I challenge the claim that next-token prediction cannot surpass human performance. On the surface, it looks like it cannot. It looks like if you just learn to imitate, to predict what people do, it means that you can only copy people. But here is a counter argument for why it might not be quite so. If your base neural net is smart enough, you just ask it — What would a person with great insight, wisdom, and capability do? Maybe such a person doesn't exist, but there's a pretty good chance that the neural net will be able to extrapolate how such a person would behave.
Predicting the next token well means that you understand the underlying reality that led to the creation of that token.
Reinforcement Learning from Human Feedback (RLHF). The human feedback has been used to train the reward function and then the reward function is being used to create the data which trains the model.
Generally speaking, you'd like tokens which are speaking about smarter things, tokens which are more interesting.
Inference of better models will indeed become more expensive. But is it prohibitive? That depends on how useful it is. If it is more useful than it is expensive then it is not prohibitive.
To give you an analogy, suppose you want to talk to a lawyer. You have some case or need some advice or something, you're perfectly happy to spend $400 an hour. Right? So if your neural net could give you really reliable legal advice, you'd say — I'm happy to spend $400 for that advice. And suddenly inference becomes very much non-prohibitive. The question is, can a neural net produce an answer good enough at this cost?
What is the driving force behind the fact that the data exists, that the GPUs exist, and that the transformers exist? The data exists because computers became better and cheaper, we've got smaller and smaller transistors. And suddenly, at some point, it became economical for every person to have a personal computer. Once everyone has a personal computer, you really want to connect them to the network, you get the internet. Once you have the internet, you suddenly have data appearing in great quantities. The GPUs were improving concurrently because you have smaller and smaller transistors and you're looking for things to do with them.
There is an analogy where if you look at the size of a Tesla, and if you look at its self-driving behavior, it looks like it does everything. But it's also clear that there is still a long way to go in terms of reliability. And we might be in a similar place with respect to our models where it also looks like we can do everything, and at the same time, we will need to do some more work until we really iron out all the issues and make it really good and really reliable and robust and well behaved.
The only thing that matters about hardware is cost per flop and overall systems cost.
So let's say there's a tsunami in Taiwan or something, what happens to AI in general?
It's definitely going to be a significant setback. No one will be able to get more compute for a few years. But I expect compute will spring up.
The thing is that the scaling law tells you what happens to your log of your next word prediction accuracy, right? There is a whole separate challenge of linking next-word prediction accuracy to reasoning capability.
I don't think that there is a clean distinction between the world of bits and the world of atoms. Suppose the neural net tells you — hey here's something that you should do, and it's going to improve your life. But you need to rearrange your apartment in a certain way. And then you go and rearrange your apartment as a result. The neural net impacted the world of atoms.