The above is the full Embedded Agency sequence, cross-posted from the MIRI website so that it’s easier to find the text version on AIAF/LW (via search, sequences, author pages, etc.).
Scott and Abram have added a new section on self-reference to the sequence since it was first posted, and slightly expanded the subsequent section on logical uncertainty and the start of the robust delegation section.
The above is the full Embedded Agency sequence, cross-posted from the MIRI website so that it’s easier to find the text version on AIAF/LW (via search, sequences, author pages, etc.).
Scott and Abram have added a new section on self-reference to the sequence since it was first posted, and slightly expanded the subsequent section on logical uncertainty and the start of the robust delegation section.
Abram added a lot of additional material to this today: https://www.lesswrong.com/posts/9vYg8MyLL4cMMaPQJ/updates-and-additions-to-embedded-agency.