Something I’ve been thinking about lately: For ‘scarcity of compute’ reasons, I think it’s fairly likely we end up in a scaffolded AI world where one highly intelligent model (that requires much more compute) will essentially delegate tasks to weaker models as long as it knows that the weaker (maybe fine-tuned) model is capable of reliably doing that task.
Like, let’s say you have a weak doctor AI that can basically reliably answer most medical questions. However, it knows when it is less confident in a diagnosis, so it will reach out to the powerful AI when it needs a second opinion from the much more intelligent AI (that requires more compute).
Somewhat related, there’s a worldview that Noah Smith proposed, which is that maybe human jobs don’t actually end up automated because there’s an opportunity cost in giving up compute that a human can do (even if the AI can do it for cheaper) because you could instead use that compute for something much more important. Imagine, “Should I use the AI to build a Dyson sphere, or should I spread that compute across tasks humans can already do?”
This doesn’t really seem like a meaningful question. Of course “AI” will be “scaffolded”. But what is the “AI”? It’s not a natural kind. It’s just where you draw the boundaries for convenience.
An “AI” which “reaches out to a more powerful AI” is not meaningful—one could say the same thing of your brain! Or a Mixture-of-Experts model, or speculative decoding (both already in widespread use). Some tasks are harder than others, and different amounts of computation get brought to bear by the system as a whole, and that’s just part of the learned algorithms it embodies and where the smarts come from. Or one could say it of your computer: different things take different paths through your “computer”, ping-ponging through a bunch of chips and parts of chips as appropriate.
Do you muse about living in a world where for ‘scarcity of compute’ reasons your computer is a ‘scaffolded computer world’ where highly intelligent chips will essentially delegate tasks to weaker chips so long as it knows that the weaker (maybe highly specialized ASIC) chip is capable of reliably doing that task...? No. You don’t care about that. That’s just details of internal architecture which you treat as a black box.
(And that argument doesn’t protect humans for the same reason it didn’t protect, say, chimpanzees or Neanderthals or horses. Comparative advantage is extremely fragile.)
Thanks for the comment, makes sense. Applying the boundary to AI systems likely leads to erroneous thinking (though may be narrowly useful if you are careful, in my opinion).
It makes a lot of sense to imagine future AIs having learned behaviours for using their compute efficiently without relying on some outside entity.
Something I’ve been thinking about lately: For ‘scarcity of compute’ reasons, I think it’s fairly likely we end up in a scaffolded AI world where one highly intelligent model (that requires much more compute) will essentially delegate tasks to weaker models as long as it knows that the weaker (maybe fine-tuned) model is capable of reliably doing that task.
Like, let’s say you have a weak doctor AI that can basically reliably answer most medical questions. However, it knows when it is less confident in a diagnosis, so it will reach out to the powerful AI when it needs a second opinion from the much more intelligent AI (that requires more compute).
Somewhat related, there’s a worldview that Noah Smith proposed, which is that maybe human jobs don’t actually end up automated because there’s an opportunity cost in giving up compute that a human can do (even if the AI can do it for cheaper) because you could instead use that compute for something much more important. Imagine, “Should I use the AI to build a Dyson sphere, or should I spread that compute across tasks humans can already do?”
This doesn’t really seem like a meaningful question. Of course “AI” will be “scaffolded”. But what is the “AI”? It’s not a natural kind. It’s just where you draw the boundaries for convenience.
An “AI” which “reaches out to a more powerful AI” is not meaningful—one could say the same thing of your brain! Or a Mixture-of-Experts model, or speculative decoding (both already in widespread use). Some tasks are harder than others, and different amounts of computation get brought to bear by the system as a whole, and that’s just part of the learned algorithms it embodies and where the smarts come from. Or one could say it of your computer: different things take different paths through your “computer”, ping-ponging through a bunch of chips and parts of chips as appropriate.
Do you muse about living in a world where for ‘scarcity of compute’ reasons your computer is a ‘scaffolded computer world’ where highly intelligent chips will essentially delegate tasks to weaker chips so long as it knows that the weaker (maybe highly specialized ASIC) chip is capable of reliably doing that task...? No. You don’t care about that. That’s just details of internal architecture which you treat as a black box.
(And that argument doesn’t protect humans for the same reason it didn’t protect, say, chimpanzees or Neanderthals or horses. Comparative advantage is extremely fragile.)
Thanks for the comment, makes sense. Applying the boundary to AI systems likely leads to erroneous thinking (though may be narrowly useful if you are careful, in my opinion).
It makes a lot of sense to imagine future AIs having learned behaviours for using their compute efficiently without relying on some outside entity.
I agree with the fragility example.