So my model of progress has allowed me to observe our prosaic scaling without surprise, but it doesn’t allow me to make good predictions since the reason for my lack of surprise has been from Vingean prediction of the form “I don’t know what progress will look like and neither do you”.
This is indeed a locally valid way to escape one form of the claim—without any particular prediction carrying extra weight, and the fact that reality has to go some way, there isn’t much surprise in finding yourself in any given world.
I do think there’s value in another version of the word “surprise,” here, though. For example: the cross-entropy loss between the predicted distribution with respect to the observed distribution. Holding to a high uncertainty model of progress will result in continuously high “surprise” in this sense, because it struggles to narrow to a better distribution generator. It’s a sort of overdamped epistemological process.
I think we have enough information to make decent gearsy models of progress around AI. As a bit of evidence, some such models have already been exploited to make gobs of money. I’m also feeling pretty good[1] about many of my predictions (like this post) that contributed to me pivoting entirely into AI; there’s an underlying model that has a bunch of falsifiable consequences which has so far survived a number of iterations, and that model has implications through the development of extreme capability.
What I have been surprised about has been governmental reaction to AI...
Yup! That was a pretty major (and mostly positive) update for me. I didn’t have a strong model of government-level action in the space and I defaulted into something pretty pessimistic. My policy/governance model is still lacking the kind of nuance that you only get by being in the relevant rooms, but I’ve tried to update here as well. That’s also part of the reason why I’m doing what I’m doing now.
In any case, I’ve been hoping for the last few years I would have time to do my undergrad and start working on the alignment without a misaligned AI going RSI, and I’m still hoping for that. So that’s lucky I guess. 🍀🐛
🙋♂️
This is indeed a locally valid way to escape one form of the claim—without any particular prediction carrying extra weight, and the fact that reality has to go some way, there isn’t much surprise in finding yourself in any given world.
I do think there’s value in another version of the word “surprise,” here, though. For example: the cross-entropy loss between the predicted distribution with respect to the observed distribution. Holding to a high uncertainty model of progress will result in continuously high “surprise” in this sense, because it struggles to narrow to a better distribution generator. It’s a sort of overdamped epistemological process.
I think we have enough information to make decent gearsy models of progress around AI. As a bit of evidence, some such models have already been exploited to make gobs of money. I’m also feeling pretty good[1] about many of my predictions (like this post) that contributed to me pivoting entirely into AI; there’s an underlying model that has a bunch of falsifiable consequences which has so far survived a number of iterations, and that model has implications through the development of extreme capability.
Yup! That was a pretty major (and mostly positive) update for me. I didn’t have a strong model of government-level action in the space and I defaulted into something pretty pessimistic. My policy/governance model is still lacking the kind of nuance that you only get by being in the relevant rooms, but I’ve tried to update here as well. That’s also part of the reason why I’m doing what I’m doing now.
May you have the time to solve everything!
… epistemically