Also, even if we accept that MIRI’s comparative advantage has to do with having a clearer view of the Friendliness vs. UnFriendliness distinction, why wouldn’t it be more effective for them to try to insure against an UnFriendly outcome by addressing the UnFriendliness already in the world today? For instance, corporate governance. Corporations’ optimization powers are a tremendous source of human happiness, but their UnFriendly tendencies are clear. For now, corporations have only parasitic intelligence, and don’t look particularly foomy, but if I had to bet on whether MIRI or Google/TenCent/Palantir/whatever was more likely to foom, there would be no contest.
[There are a bunch of assumptions embedded there. The principal ones are:
If a corporation, as currently constituted, somehow went foom it would be likely to be UnFriendly
If we were able to make it so corporations appeared more Friendly in their day-to-day actions, they would also become less likely to rush headlong into an UnFriendly foom.
I think 1 is pretty undeniable, but I could understand it if someone disagreed with 2.]
Also, even if we accept that MIRI’s comparative advantage has to do with having a clearer view of the Friendliness vs. UnFriendliness distinction, why wouldn’t it be more effective for them to try to insure against an UnFriendly outcome by addressing the UnFriendliness already in the world today? For instance, corporate governance. Corporations’ optimization powers are a tremendous source of human happiness, but their UnFriendly tendencies are clear. For now, corporations have only parasitic intelligence, and don’t look particularly foomy, but if I had to bet on whether MIRI or Google/TenCent/Palantir/whatever was more likely to foom, there would be no contest.
[There are a bunch of assumptions embedded there. The principal ones are:
If a corporation, as currently constituted, somehow went foom it would be likely to be UnFriendly
If we were able to make it so corporations appeared more Friendly in their day-to-day actions, they would also become less likely to rush headlong into an UnFriendly foom.
I think 1 is pretty undeniable, but I could understand it if someone disagreed with 2.]