Then it does seem that if Clippy had in fact been built to maximize paperclips by an agent with the right computational limitations, then Clippy would have been built to take the deal.
I’d say essentially the same thing about this problem as I said about the Counterfactual Mugging:
So, finding that you are declining Omega, contrary to UDT, isn’t even necessarily bad news. It probably just means that your designers lucked out more than they deserved, which would be good news.
But that doesn’t change the fact that your designers were stupid, and that you ought to want to do better. You ought to modify yourself and any of your progeny to act more in accord with UDT in the future.
I took it as being implied, yes. If Staply is an unknown algorithm, there’s no point in trading.
Then it does seem that if Clippy had in fact been built to maximize paperclips by an agent with the right computational limitations, then Clippy would have been built to take the deal.
I’d say essentially the same thing about this problem as I said about the Counterfactual Mugging: