Just going off my intuitive read, I like what you had to say about the doom memeplex. It does match my intuitive sense that “summoning a memeplex is easier than controlling it,” and there are a bunch of ways I expect the “we’re all going to die” belief system to have some bad side effects that would be better to avoid if possible.
I also liked this, for sort of sidestepping the “maybe they’ll keep us because they like us?” objection.
A colleague of mine describes it with the metaphor that we can find ourselves in the role of a herd of cows, whose fate is being determined by the farmer. And I don’t think we want to get to that stage. While there are many interesting questions about what might happen to such a herd afterwards, I think they are distracting. We need to focus on not losing control of the AI
Just going off my intuitive read, I like what you had to say about the doom memeplex. It does match my intuitive sense that “summoning a memeplex is easier than controlling it,” and there are a bunch of ways I expect the “we’re all going to die” belief system to have some bad side effects that would be better to avoid if possible.
I also liked this, for sort of sidestepping the “maybe they’ll keep us because they like us?” objection.