I don’t argue at any point that ASIs will have a single goal. The argument goes through equally well if it has many. The question is why some of those goals are of the form “tile the universe with squiggles” at all. That’s the part I’m addressing in this post.
I don’t argue at any point that ASIs will have a single goal. The argument goes through equally well if it has many. The question is why some of those goals are of the form “tile the universe with squiggles” at all. That’s the part I’m addressing in this post.