Can you link specifically an example of what you are criticizing here? Which formalisms require simulating the universe from the beginning to identify a specific being? (as opposed to, e.g. some probabilistic approximation?)
(trying to think of examples, maybe Aumann’s theorem qualifies? But I’m not sure, and not sure what else does?)
Just reviewed PreDCA, and while I consider it to be a clever solution to the wrong* problem, I don’t see how it requires perfect physical modelling? (maybe geting a perfect answer would require perfect modelling, but an approximate answer does not).
QACI seemed obviously overcomplicated to me, don’t care enough to really check if it requires perfect modelling, though I’m not sure I would call that MIRI-style.
*to clarify, I think Pre-DCA is trying to solve both a) “how does the AI identify a target to help” and b) “how does it help them”, and it is a clever solution to problem (a) for a clean sheet from-scratch AI but that is the wrong problem since it’s more likely an AI would be trained as a non-agent or weaker agent before being made a strong agent so would already know what humans are, and it is a clever but inadequate attempt to solve (b) which is one of the right problems.
Can you link specifically an example of what you are criticizing here? Which formalisms require simulating the universe from the beginning to identify a specific being? (as opposed to, e.g. some probabilistic approximation?)
(trying to think of examples, maybe Aumann’s theorem qualifies? But I’m not sure, and not sure what else does?)
PreDCA and QACI are the main things I have in mind.
Just reviewed PreDCA, and while I consider it to be a clever solution to the wrong* problem, I don’t see how it requires perfect physical modelling? (maybe geting a perfect answer would require perfect modelling, but an approximate answer does not).
QACI seemed obviously overcomplicated to me, don’t care enough to really check if it requires perfect modelling, though I’m not sure I would call that MIRI-style.
*to clarify, I think Pre-DCA is trying to solve both a) “how does the AI identify a target to help” and b) “how does it help them”, and it is a clever solution to problem (a) for a clean sheet from-scratch AI but that is the wrong problem since it’s more likely an AI would be trained as a non-agent or weaker agent before being made a strong agent so would already know what humans are, and it is a clever but inadequate attempt to solve (b) which is one of the right problems.