The AI in that story actually seems to be surprisingly well done and does have an inherent goal to help humanity. It’s primary goal is to ‘satisfy human values through friendship and ponies’. That’s almost perfect, since here ‘satisfying human values’ seems to be based on humanity’s CEV.
It’s just that the added ‘through friendship and ponies’ turns it from a nigh-perfect friendly AI into something really weird.
The AI in that story actually seems to be surprisingly well done and does have an inherent goal to help humanity. It’s primary goal is to ‘satisfy human values through friendship and ponies’. That’s almost perfect, since here ‘satisfying human values’ seems to be based on humanity’s CEV.
It’s just that the added ‘through friendship and ponies’ turns it from a nigh-perfect friendly AI into something really weird.
I agree with your overall point, though.