It acts as though the “outer” alignment of humans is “wanting to have kids,” or something
That seems correct to me; to the extent that we can talk about evolution optimizing a species for something, I think it makes the most sense to talk about it as optimizing for the specific traits under selection. When the air in Manchester got more polluted and a dark color started conferring an advantage in hiding, dark-colored moths became more common; this obviously isn’t an instance of an inner optimizer within the moth’s brain since there’s no behavioral element involved, it’s just a physical change to the moth’s color. It’s just evolution directly selecting for a trait that had become useful in Manchester.
Likewise, if “wanting to have kids” is useful for having more surviving descendants, then “wanting to have more kids” becomes a trait that evolution is selecting for, analogously to evolution selecting for dark color in moths. There is an inner optimizer that is executing the trait that has been selected for, but it’s one that’s aligned with the original selection target.
The descriptions of the wanting in the OP seem to be about the inner optimizer doing the execution, though. That’s the distinction I want to make—confusing that for not-an-inner-optimizer seems importantly bad.
That seems correct to me; to the extent that we can talk about evolution optimizing a species for something, I think it makes the most sense to talk about it as optimizing for the specific traits under selection. When the air in Manchester got more polluted and a dark color started conferring an advantage in hiding, dark-colored moths became more common; this obviously isn’t an instance of an inner optimizer within the moth’s brain since there’s no behavioral element involved, it’s just a physical change to the moth’s color. It’s just evolution directly selecting for a trait that had become useful in Manchester.
Likewise, if “wanting to have kids” is useful for having more surviving descendants, then “wanting to have more kids” becomes a trait that evolution is selecting for, analogously to evolution selecting for dark color in moths. There is an inner optimizer that is executing the trait that has been selected for, but it’s one that’s aligned with the original selection target.
The descriptions of the wanting in the OP seem to be about the inner optimizer doing the execution, though. That’s the distinction I want to make—confusing that for not-an-inner-optimizer seems importantly bad.