This is my inaugural "take ai seriously" post.
I used to wonder if the years I would live through might be less exciting compared to other eras in human history, either past or future. It increasingly seems like I was mistaken, not having known about or been sufficiently interested in the developments in AI technology.
To be more specific, I was overlooking the following predictable developments:
- The continued trend of hardware improvements in GPUs.
- The relatively constant relationship between computational capacity, data input, and the performance of deep neural networks, known as scaling laws in the AI field.
- That these trendlines have been relatively constant, if not accelerating, for over a decade now.
An obvious picture emerges. Many tech-interested folks will likely have heard of this at some point, but taken it seriously?
Taking this seriously has been one of my main struggles over the past year, and I still feel like I'm not taking it seriously enough.
During all this, one has to be careful not to extrapolate trendlines to infinity; not all trendlines continue indefinitely, of course. But it is also possible to reason about the upper limits of these developments and how close we might be to them. One can argue the details over how long Moore's Law might last, what might be the solution to the lack of data, or if ultimately energy might be the limiting constraint on further AI development. No sober analysis suggests that we will hit any of these limits over the coming years, or even that any one of them is insurmountable. The truth is: whatever you thought about scaling, so far it holds, and it seems to hold for at least a few more years, as we are certainly far from upper bounds in multiple domains. A lot more chips can be built, data can be gathered, and algorithmic progress can be made.
Most of this confusion currently stems from the fact that we are in the break between GPT-4 level language models and their successor generation, which is likely ready right now or in the coming few months, at least internally at a few companies.
At that point, you will shift your attention to the new, increasingly small subset of tasks that the current generation of models is not yet capable of doing. How many more of these cycles do you want to repeat before realizing what kind of interesting situation you are living through?
So ask yourself then, what would your future self want you to give more consideration to now? Maybe start with asking yourself if you should spend a few hours getting up to speed on the basic trendlines and their outlook—you know, the big picture overview—to see if you might be missing something. Also, should you reconsider your expectations of the future in some way? What's your place in all this?
This will surely not apply to everyone, and many are already a step further than this basic point. At that point, you might ask yourself what kind of project you should choose to get some experience and if you have already priced the likeliest future into your actions and planning.
Taking this seriously also means embracing a lot of uncertainty, though. Wildly different outcomes are possible, ranging from utopic to dystopic.
Distinctly missing in this picture is a future that is more or less business as usual.
But is this really all? Will we all work in AI startups by day and spend our time in the metaverse by night then? Does anyone even want a "standard future," so seemingly coming our way? Do we want to lose our jobs, lose meaning, and the identity we get from our occupations, just because a machine can now perform the same tasks more efficiently? Do we trust the people who bring us this future? Or does this demand even more than our excitement from us, perhaps our consideration, even morality as well?
It is certainly easy to get lost in the hype. For the shiny and exciting to blind us to the very real risks that come our way. The personal, collective, or even existential risks that many of the people working the hardest to accelerate expect us to run into with near certainty. To think more than a few months ahead entails not just planning your career accordingly. Nothing personal, but maybe you and I are not the only ones whom this is about. More than about us, the coming decades will set the course for, and will ultimately be about, the generations that follow. Choose your stance wisely.