Bostrom writes: “If an agent’s final goals concern the future, then in many scenarios there will be future actions it could perform to increase the probability of achieving its goals. This creates an instrumental reason for the agent to try to be around in the future—to help achieve its future-oriented goal.”
Similarly, the fact that it would be in the an AI’s interests to ensure it’s own survival doesn’t imply that *it* realises that, or that it has the ability to do so.
And this is the flaw and ironically something better SWEs know how to fix. Why does the model concern itself with the future? Can you think of a model where it doesn’t care?
Bostrom writes: “If an agent’s final goals concern the future, then in many scenarios there will be
future actions it could perform to increase the probability of achieving its goals.
This creates an instrumental reason for the agent to try to be around in the future—to
help achieve its future-oriented goal.”
Similarly, the fact that it would be in the an AI’s interests to ensure it’s own survival doesn’t imply that *it* realises that, or that it has the ability to do so.
And this is the flaw and ironically something better SWEs know how to fix. Why does the model concern itself with the future? Can you think of a model where it doesn’t care?