(All of this is just based on my understanding, no guarantees.)
Miri is studying decision theory in the context of embedded agency. Embedded Agency is all about what happens if you stop having a clear boundary between the agent and the environment (and you instead have the agent as part of the environment, hence embedded). Decision problems where the outcome depends on your behavior in counter-factual situations are just one of several symptoms that come from being an embedded agent.
In this context, we care about things like “if an agent is made of parts, how can she ensure her parts are aligned” or “if an agent creates copies of herself, how can we make sure nothing goes wrong” or “if the agent creates a successor agent, how can we make sure the successor agent does what the original agent wants”.
I say this because (3) and (4) suddenly sound a lot more plausible when you’re talking about something like an embedded agent playing a newcomb-like game (or a counter-factual mugging type game or a prisonner-dilemma type game) with a copy of itself.
Also, I believe Timeless Decision Theory is outdated. The important decision theories are Updateless Decision Theory and Functional Decision Theory. Afaik, UDT is both better and better formalized than TDT.
I started reading the FDT paper and it seems to make a lot more sense than TDT. And most importantly does not fail like TDT did in regards to roko’s basilisk.
(All of this is just based on my understanding, no guarantees.)
Miri is studying decision theory in the context of embedded agency. Embedded Agency is all about what happens if you stop having a clear boundary between the agent and the environment (and you instead have the agent as part of the environment, hence embedded). Decision problems where the outcome depends on your behavior in counter-factual situations are just one of several symptoms that come from being an embedded agent.
In this context, we care about things like “if an agent is made of parts, how can she ensure her parts are aligned” or “if an agent creates copies of herself, how can we make sure nothing goes wrong” or “if the agent creates a successor agent, how can we make sure the successor agent does what the original agent wants”.
I say this because (3) and (4) suddenly sound a lot more plausible when you’re talking about something like an embedded agent playing a newcomb-like game (or a counter-factual mugging type game or a prisonner-dilemma type game) with a copy of itself.
Also, I believe Timeless Decision Theory is outdated. The important decision theories are Updateless Decision Theory and Functional Decision Theory. Afaik, UDT is both better and better formalized than TDT.
I started reading the FDT paper and it seems to make a lot more sense than TDT. And most importantly does not fail like TDT did in regards to roko’s basilisk.