(Meta)*-Problems
My first programming job was at my college library, building web apps with Django. I had never actually programmed in Python or done anything resembling web development before, so the week between when I learned I'd gotten the job and when I was due to start was spent cramming CodingBat problems and the Django docs.
Because I went into this job with very few priors, I got differentially good at making Django apps - learning all the various conventions of the particular Django version I was using - to the point that six months into the job I called my dad and told him I wanted to drop out and work on a startup or at a tech company instead of finishing college (this was the end of my freshman year). In retrospect, it's probably a good thing that I didn't, because shortly after that conversation Facebook released React, and the demand for Django developers dried up precipitously.
The experience of learning how to code to make websites is illustrative of what you might call climbing the (meta)* ladder. At the top of the ladder lies true inductive reasoning. In my case:
- First rung: Learn how to make Django websites
- Second rung: Learn the principles of webdev, which simplifies learn particular web frameworks
- Third rung: Learn networking, which simplifies learning webdev
- Fourth rung: Learn computer science, which simplies learning networking
and so on.
There's a similar ladder for LLMs. Let's say you give your AI agent a command like /goal optimize this gpu kernel. It will probably go through a progression like this:
- First rung: Come up with modifications to the code to improve wall clock time
- Second rung: Build a better model of how CUDA works, to surface such modifications more quickly
- Third rung: Build a better model of how GPUs works, to better understand CUDA
- Fourth rung: Build a better model of how VLSI works, to better understand GPUs
and so on.
The boundary between whether we perceive some such system to be pattern matching verses inductive reasoning is likely just a function of the order that a given intelligence is operating at.
The problem is that climbing up this ladder gets less and less externally legible. So while it is easy to write down everything you might need to know if you want to make a Django application, it's slightly harder to write down the principles of web development, and slightly harder still to write down all the principles of networking.
The world is going to become less and less legible to more and more people over the next few years, as we collectively climb up this (meta)* ladder. It's already happening, but I expect that it will accelerate. A lot of people will make increasingly bad decisions, because they're operating on the wrong rung of the ladder. This is going to open up a lot of interesting opportunities for people who can climb quickly.