You’re saying AI will be much better than us at long-term planning?
It’s hard to train for tasks where the reward is only known after a long time (e.g. how would you train for climate prediction?)
No, I’m saying that AI might be much worse than us at long-term planning, because evolution has selected us on the basis of very long chains of causal effects, whereas we can’t train AIs on the basis of such long chains of causal effects.
You’re saying AI will be much better than us at long-term planning?
It’s hard to train for tasks where the reward is only known after a long time (e.g. how would you train for climate prediction?)
No, I’m saying that AI might be much worse than us at long-term planning, because evolution has selected us on the basis of very long chains of causal effects, whereas we can’t train AIs on the basis of such long chains of causal effects.