Sure. Let’s adopt the “petrol/electric cars” thing from Holtman’s paper. In timestep 0, the agent has a choice: either create a machine which will create one petrol car every timestep indefinitely, or create a machine which will create one petrol car every timestep until the button is pressed and then switch to electric. The agent does not have any choices after that; its only choice is which successor agent to create at the start.
The utility functions are the same as in Holtman’s paper.
My main claim is that the π∗fcg0 agent is not indifferent between the two actions; it will actively prefer the one which ignores the button. I expect this also extends to the π∗fcgc agent, but am less confident in that claim.
Sure. Let’s adopt the “petrol/electric cars” thing from Holtman’s paper. In timestep 0, the agent has a choice: either create a machine which will create one petrol car every timestep indefinitely, or create a machine which will create one petrol car every timestep until the button is pressed and then switch to electric. The agent does not have any choices after that; its only choice is which successor agent to create at the start.
The utility functions are the same as in Holtman’s paper.
My main claim is that the π∗fcg0 agent is not indifferent between the two actions; it will actively prefer the one which ignores the button. I expect this also extends to the π∗fcgc agent, but am less confident in that claim.