Nitpick arguments about how you define this specifically are irrelevant and uninteresting.
Excuse me, what? This is not evolution’s utility function. It’s not optimizing for gene count. It does one thing, one thing only, and it does it well: it promotes genes that increase their RELATIVE FREQUENCY in the reproducing population.
It is completely irrelevant to my larger point unless you are claiming that changing out the specific detailed choice of evolutionary utility function would change the high level outcome of humans being successful (relatively high scoring) according to that utility function, or more importantly—simply not being a failure.
My analogy is between:
humans optimizing AI according to human utility function which actually results in extinction (0 score according to that utility function) due to inner misalignment
and
evolution optimizing brains according to some utility function which actually results in extinction (0 score according to that utility function) due to inner misalignment
No part of my argument depends on the specific nitpick detail of the evolution utility function, other than it outputs a non failure score for human history to date.
So are you actually arguing that humans are an evolutionary failure? Or that we simply got lucky as the development of early technoculture usually results in extinction? Any doomer predictions of our future extinction can’t be used as part of a doom argument—that’s just circular reasoning. The evidence to date is that brains were an enormous success according to any possible reasonable evolutionary utility function.
Or maybe you simply dislike the analogy because it doesn’t support your strongly held beliefs? Fine, but at least make that clear.
It is completely irrelevant to my larger point unless you are claiming that changing out the specific detailed choice of evolutionary utility function would change the high level outcome of humans being successful (relatively high scoring) according to that utility function, or more importantly—simply not being a failure.
My analogy is between:
humans optimizing AI according to human utility function which actually results in extinction (0 score according to that utility function) due to inner misalignment
and
evolution optimizing brains according to some utility function which actually results in extinction (0 score according to that utility function) due to inner misalignment
No part of my argument depends on the specific nitpick detail of the evolution utility function, other than it outputs a non failure score for human history to date.
So are you actually arguing that humans are an evolutionary failure? Or that we simply got lucky as the development of early technoculture usually results in extinction? Any doomer predictions of our future extinction can’t be used as part of a doom argument—that’s just circular reasoning. The evidence to date is that brains were an enormous success according to any possible reasonable evolutionary utility function.
Or maybe you simply dislike the analogy because it doesn’t support your strongly held beliefs? Fine, but at least make that clear.