One thing are good proxies for utility at the present time, another is coming up with a design-shaping process which generates utility out of a primitive earth across billions of years. Even looking forward, would you claim that a smiling or income maximizer would do better than evolution in the next 10,000 years? I highly doubt. Perhaps a better comparison, but still at fault, would be between our attempts of envisioning a FAI and evolution.
One thing are good proxies for utility at the present time, another is coming up with a design-shaping process which generates utility out of a primitive earth across billions of years. Even looking forward, would you claim that a smiling or income maximizer would do better than evolution in the next 10,000 years? I highly doubt. Perhaps a better comparison, but still at fault, would be between our attempts of envisioning a FAI and evolution.