Right. For example, I think Stuart Armstrong is hitting something very important about AI alignment with his pursuit of the idea that there’s no free lunch in value learning. We only close the gap by making an “arbitrary” assumption, but it’s only arbitrary if you assume there’s some kind of context-free version of the truth. Instead we can choose in a non-arbitrary way based on what we care about and is useful to us.
I realize lots of people are bored by this point because they’re non-arbitrary solution that is useful is some version of rationality criteria since those are very useful for not getting Dutch booked, for example, but we could just as well choose something else and humans, for example, seem to do just that, even though so far we’d be hard pressed to very precisely say just what it is that humans do assume to ground things in, although we have some clues of things that seem important, like staying alive.
You’re talking about how we ground out our thinking in something that is true but is not just further conceptualization?
Look if we just make a choice about the truth by making an assumption then eventually the world really does “bite back”. It’s possible to try this out by just picking a certain fundamental orientation towards the world and sticking to it no matter what throughout your life for a little while. The more rigidly you adhere to it the more quickly the world will bite back. So I don’t think we can just pick a grounding.
But at the same time I very much agree that there is no concept that corresponds to the truth in a context-free or absolute way. The analogy I like the most is dance: imagine if I danced a dance that beautifully expressed what it’s like to walk in the forest at night. It might be an incredibly evocative dance and it might point towards a deep truth about the forest at night, but it would be strange to claim that a particular dance is the final, absolute, context-free truth. It would be strange to seek after a final, absolute, context-free dance that expresses what it’s like to walk in the forest at night in a way that finally captures the actual truth about the forest at night.
When we engage in conceptualization, we are engaging in something like a dance. It’s a dance with real consequence, real power, real impacts on the world, and real importance. It matters that we dance it and that we get it right. It’s hard to think of anything at this point that matters more. But its significance is not a function of its capturing the truth in a final or context-free way.
So when I consider “grounding out” my thinking in reality, I think of it in the same way that a dance should “ground out” in reality. That is: it should be about something real. It’s also possible to pick some idea about what it’s really like to walk in the forest at night and dance in a way that adheres to that idea but not to the reality of what it’s actually like to walk in the forest at night. And it’s possible to think in a way that is similarly not in accord with reality itself. But just as with dance, thinking in accord with reality is not at all about capturing reality in a final, absolute, or context-free way.
Right. For example, I think Stuart Armstrong is hitting something very important about AI alignment with his pursuit of the idea that there’s no free lunch in value learning. We only close the gap by making an “arbitrary” assumption, but it’s only arbitrary if you assume there’s some kind of context-free version of the truth. Instead we can choose in a non-arbitrary way based on what we care about and is useful to us.
I realize lots of people are bored by this point because they’re non-arbitrary solution that is useful is some version of rationality criteria since those are very useful for not getting Dutch booked, for example, but we could just as well choose something else and humans, for example, seem to do just that, even though so far we’d be hard pressed to very precisely say just what it is that humans do assume to ground things in, although we have some clues of things that seem important, like staying alive.
You’re talking about how we ground out our thinking in something that is true but is not just further conceptualization?
Look if we just make a choice about the truth by making an assumption then eventually the world really does “bite back”. It’s possible to try this out by just picking a certain fundamental orientation towards the world and sticking to it no matter what throughout your life for a little while. The more rigidly you adhere to it the more quickly the world will bite back. So I don’t think we can just pick a grounding.
But at the same time I very much agree that there is no concept that corresponds to the truth in a context-free or absolute way. The analogy I like the most is dance: imagine if I danced a dance that beautifully expressed what it’s like to walk in the forest at night. It might be an incredibly evocative dance and it might point towards a deep truth about the forest at night, but it would be strange to claim that a particular dance is the final, absolute, context-free truth. It would be strange to seek after a final, absolute, context-free dance that expresses what it’s like to walk in the forest at night in a way that finally captures the actual truth about the forest at night.
When we engage in conceptualization, we are engaging in something like a dance. It’s a dance with real consequence, real power, real impacts on the world, and real importance. It matters that we dance it and that we get it right. It’s hard to think of anything at this point that matters more. But its significance is not a function of its capturing the truth in a final or context-free way.
So when I consider “grounding out” my thinking in reality, I think of it in the same way that a dance should “ground out” in reality. That is: it should be about something real. It’s also possible to pick some idea about what it’s really like to walk in the forest at night and dance in a way that adheres to that idea but not to the reality of what it’s actually like to walk in the forest at night. And it’s possible to think in a way that is similarly not in accord with reality itself. But just as with dance, thinking in accord with reality is not at all about capturing reality in a final, absolute, or context-free way.
Is this how you see things too?
Yep, that accords well with my own current view.