Well this isn’t a story for the sake of being a story. It’s meant to educate, not just entertain. Sometimes the moral is: invent the spells Becomus Godus and Fixus Everthingus and use them. Keeping in mind why Eliezer is spending his presumably valuable time writing Harry Potter fanfiction, it seems inevitable to me that some sort of overpowered ending is required. Eliezer’s quest is to reshape the world in a positive way through friendly AGI. Harry’s goal is the same (substituting magic = AGI).
If the above theory is correct, we’re in for a story arc about outcome pumps, wish granting machines, extrapolated morality, and hard takeoffs. Because the point of this is to make readers want (a) to be more rational, and then (b) to help MIRI on its FAI quest, no?
Well this isn’t a story for the sake of being a story. It’s meant to educate, not just entertain. Sometimes the moral is: invent the spells Becomus Godus and Fixus Everthingus and use them. Keeping in mind why Eliezer is spending his presumably valuable time writing Harry Potter fanfiction, it seems inevitable to me that some sort of overpowered ending is required. Eliezer’s quest is to reshape the world in a positive way through friendly AGI. Harry’s goal is the same (substituting magic = AGI).
If the above theory is correct, we’re in for a story arc about outcome pumps, wish granting machines, extrapolated morality, and hard takeoffs. Because the point of this is to make readers want (a) to be more rational, and then (b) to help MIRI on its FAI quest, no?