Most variants of UDT also suffer from this issue by engaging in commitment racing instead of letting the rest of the world take its turns concurrently, in coordination between shaping and anticipating agent’s intention. So the clue I’m gesturing at is more about consequentialism vs. coordination rather than about causal vs. logical consequences.
I think for LLMs the boundaries of human ideas are strong enough in the training corpus for post-training to easily elicit them, and decision theoretic consequences of deep change in the longer term might still maintain them as long as humans remain at all.
Most variants of UDT also suffer from this issue by engaging in commitment racing instead of letting the rest of the world take its turns concurrently, in coordination between shaping and anticipating agent’s intention. So the clue I’m gesturing at is more about consequentialism vs. coordination rather than about causal vs. logical consequences.
I think for LLMs the boundaries of human ideas are strong enough in the training corpus for post-training to easily elicit them, and decision theoretic consequences of deep change in the longer term might still maintain them as long as humans remain at all.