This comment seems to imply that EY would prefer that such an optimizer be run, if the only other option was business-as-usual.
Okay, just to disclaim this clearly, I probably would press the button that instantly swaps us to this world—but that’s because right now people are dying, and this world implies a longer time to work on FAI 2.0.
But the Wrinkled Genie scenario is not supposed to be probable or attainable—most programmers this stupid just kill you, I think.
EDIT: that doesn’t imply it earns the label “Friendly” though.
This comment seems to imply that EY would prefer that such an optimizer be run, if the only other option was business-as-usual.
EDIT: that doesn’t imply it earns the label “Friendly” though.