1) Do you think that an opt-out clause is a useful-in-principle way to address your concerns?
Yes. In principle, you should better achieve your outcomes if you have more precise instructions. In principle, this seems sort of obvious.
In practice, I see two problems:
1) If the agency misinterprets your instructions.
2) Ambiguous instructions could facilitate corruption. In the same way that ambiguous laws do.
I’m not sure whether the upsides (more precise instructions allow you to better achieve your outcome) outweigh the downsides (1 and 2). My intuition is that I’m ~65% sure that the upsides outweigh the downsides, but that doesn’t reflect much thought.
Practical problem #3: The agency successfully understands your intentions, and is willing to implement them, but not able to implement them.
For example, a fast intelligence explosion removes their capability of doing so before they can pull the plug. Or a change in their legal environment makes it illegal for them to pull the plug (and they aren’t willing to put themselves at legal risk to do so).
Yes. In principle, you should better achieve your outcomes if you have more precise instructions. In principle, this seems sort of obvious.
In practice, I see two problems:
1) If the agency misinterprets your instructions.
2) Ambiguous instructions could facilitate corruption. In the same way that ambiguous laws do.
I’m not sure whether the upsides (more precise instructions allow you to better achieve your outcome) outweigh the downsides (1 and 2). My intuition is that I’m ~65% sure that the upsides outweigh the downsides, but that doesn’t reflect much thought.
Practical problem #3: The agency successfully understands your intentions, and is willing to implement them, but not able to implement them.
For example, a fast intelligence explosion removes their capability of doing so before they can pull the plug. Or a change in their legal environment makes it illegal for them to pull the plug (and they aren’t willing to put themselves at legal risk to do so).