I assumed as much, but my problem with this reasoning starts here:
To categorize something as an agent, you need to look at its internal dynamics. A planet is not an agent because it’s not doing any computation that could be interpreted as evaluating multiple possible choices, and it’s certainly not predicting their consequences.
Normally, I’d agree, but as I said, Eliezer_Yudkowsky claims that a pebble contains the laws of physics, which are nothing but a network of counterfactuals. So there necessarily is an isomorphism between a planet and “multiple possible consequences”.
This is why I say there must be a stronger sense in which you mean that the agent has computations that can be interpreted as evaluating multiple choices/consequences, because all of the universe is doing a sort of efficient version of that. And I don’t yet know what this stronger sense is.
I assumed as much, but my problem with this reasoning starts here:
Normally, I’d agree, but as I said, Eliezer_Yudkowsky claims that a pebble contains the laws of physics, which are nothing but a network of counterfactuals. So there necessarily is an isomorphism between a planet and “multiple possible consequences”.
This is why I say there must be a stronger sense in which you mean that the agent has computations that can be interpreted as evaluating multiple choices/consequences, because all of the universe is doing a sort of efficient version of that. And I don’t yet know what this stronger sense is.