>pragmatic AIs that would frame the world for the humans, in the ways that the humans would approve
The choice of how to do that is equivalent with choosing among the human values. That’s not to say that there are not better or worse ways of doing things, but as soon as human behaviour become legible to an AI, we have to be very specific about any squishiness we want to preserve, and encode those in AI values.
>pragmatic AIs that would frame the world for the humans, in the ways that the humans would approve
The choice of how to do that is equivalent with choosing among the human values. That’s not to say that there are not better or worse ways of doing things, but as soon as human behaviour become legible to an AI, we have to be very specific about any squishiness we want to preserve, and encode those in AI values.