Nice, thanks. It seems like the distinction the authors make between ‘building agents from the ground up’ and ‘understanding their behaviour and predicting roughly what they will do’ maps to the distinction I’m making, but I’m not convinced by the claim that the second one is a much stronger version of the first.
The argument in the paper is that the first requires an understanding of just one agent, while the second requires an understanding of all agents. But it seems like they require different kinds of understanding, especially if the agent being built is meant to be some theoretical ideal of rationality. Building a perfect chess algorithm is just a different task to summarising the way an arbitrary algorithm plays chess (which you could attempt without even knowing the rules).
Nice, thanks. It seems like the distinction the authors make between ‘building agents from the ground up’ and ‘understanding their behaviour and predicting roughly what they will do’ maps to the distinction I’m making, but I’m not convinced by the claim that the second one is a much stronger version of the first.
The argument in the paper is that the first requires an understanding of just one agent, while the second requires an understanding of all agents. But it seems like they require different kinds of understanding, especially if the agent being built is meant to be some theoretical ideal of rationality. Building a perfect chess algorithm is just a different task to summarising the way an arbitrary algorithm plays chess (which you could attempt without even knowing the rules).