by describing to each other efficiently enough how to prevent takeover by the power hungry at every level, anything less than a solution to loss of slack to power seekers is insufficient to prevent power seekers from suddenly eating up all of humanity’s slack.
or in other words, I don’t have a magic solution to post in this comment, but a solution has to be something that solves this above everything else. It is a challenge of hyper generalizing defense analysis because now we have to write down morality precisely enough that we don’t get an a war with a new species.
I genuinely think it might be harder than the actual technical problem of alignment, and we ought to look for any path which isn’t that hopelessly doomed.
by describing to each other efficiently enough how to prevent takeover by the power hungry at every level, anything less than a solution to loss of slack to power seekers is insufficient to prevent power seekers from suddenly eating up all of humanity’s slack.
or in other words, I don’t have a magic solution to post in this comment, but a solution has to be something that solves this above everything else. It is a challenge of hyper generalizing defense analysis because now we have to write down morality precisely enough that we don’t get an a war with a new species.
I genuinely think it might be harder than the actual technical problem of alignment, and we ought to look for any path which isn’t that hopelessly doomed.