I’m not taking your post to be metaphorical, I’m taking your post to be literal and to be building up to a generalization step where you make an explicit comparison. My response is in the literal interpretation, under the assumption that AIs and humans are both population seen as needing to be aligned, and I am arguing that we need a perspective on inter-human alignment that generalizes to starkly superintelligent things without needing to consider aligning weak humans to strong humans forcibly. It seems like a local validity issue with the reasoning chain otherwise.
edit: but to be clear, I should say—this is an interesting approach and I’m excited about something like it. I habitually zoomed in on a local validity issue, but the step of generalizing this to ai seems like a natural and promising one. But I think the alignment issues between humans are at least as big of an issue and are made of the same thing; making one and only one human starkly superintelligent would be exactly as bad as making an ai starkly superintelligent.
but it is written in a way that applies literally as-is, and only works as a metaphor by nature of that literal description being somewhat accurate. so I feel it appropriate to comment on the literal interpretation. Do you have a response at that level?
I’m not taking your post to be metaphorical, I’m taking your post to be literal and to be building up to a generalization step where you make an explicit comparison. My response is in the literal interpretation, under the assumption that AIs and humans are both population seen as needing to be aligned, and I am arguing that we need a perspective on inter-human alignment that generalizes to starkly superintelligent things without needing to consider aligning weak humans to strong humans forcibly. It seems like a local validity issue with the reasoning chain otherwise.
edit: but to be clear, I should say—this is an interesting approach and I’m excited about something like it. I habitually zoomed in on a local validity issue, but the step of generalizing this to ai seems like a natural and promising one. But I think the alignment issues between humans are at least as big of an issue and are made of the same thing; making one and only one human starkly superintelligent would be exactly as bad as making an ai starkly superintelligent.
It is supposed to be taken as a metaphor
but it is written in a way that applies literally as-is, and only works as a metaphor by nature of that literal description being somewhat accurate. so I feel it appropriate to comment on the literal interpretation. Do you have a response at that level?
yes, from the point of view of the dictator, any changes to the dictator’s utility function (alignment with the population) are bad