Naturally the fact that bio evolution has been largely successful at alignment so far doesn’t mean that will continue indefinitely into the future after further metasystems transitions.
But that is future speculation, and its obviously tautological to argue “bio evolution will fail at alignment in the future” as part of your future model and argument for why alignment is hard in general—we haven’t collected that evidence yet!
Moreover, its not so much that we are misaligned with evolution, and more that bio evolution is being usurped by memetic/info evolution—the future replicator patterns of merit are increasingly no longer just physical genes.
I’m saying that the fact that you, an organism built by the evolutionary process, hope to step outside the evolutionary process and do stuff that the evolutionary process wouldn’t do, is misalignment with the evolutionary process.
I’m saying you didn’t seem to grasp my distinction between systemic vs bio evolution. I do not “hope to step outside the evolutionary process”. The decoupling is only with bio-evolution and genes. The posthuman goal is to move beyond biology, become substrate independent etc, but that is hardly the end of evolution.
A different way to maybe triangulate here: Is misalignment possible, on your view? Like does it ever make sense to say something like “A created B, but failed at alignment and B was misaligned with A”? I ask because I could imagine a position, that sort of sounds a little like what you’re saying, which goes:
There’s no such thing as misalignment. There’s one overarching process, call it evolution or whatever you like, and this process goes through stages of creating new things along new dimensions, but all the stages are part of the overall process. Anything called “misalignment” is describing the relationship of two parts or stages that are contained in the overarching process. The overarching process is at a higher level than that misalignment relationship, and the misalignment helps compute the overarching process.
I literally provided examples of what misalignment with bio-evolution would look like:
Due to observational selection effects, we naturally wouldn’t be here if mesaoptimization failure during brain evolution was too common across the multiverse.[3] But we could have found ourselves in a world with many archaeological examples of species achieving human general technocultural intelligence and then going extinct—not due to AGI of course, but simply due to becoming too intelligent to reproduce.
The original argument that your OP is responding to is about “bio evolution”. I understand the distinction, but why is it relevant? Indeed, in the OP you say:
For the evolution of human intelligence, the optimizer is just evolution: biological natural selection. The utility function is fitness: gene replication count (of the human defining genes).
The OP is talking about history and thus bio evolution, and this thread shifted into the future (where info-evolution dominates) here:
I think the jury is still out about what will happen in the future. Maybe future humans will all upload their brains into computers, which is presumably count as zero IGF, if we define IGF via literal DNA molecules (which we don’t have to, but that’s another story). Whatever, I dunno.
Of course—and we’d hope that there is some decoupling eventually! Otherwise it’s just be fruitful and multiply, forever.
I’m saying that you, a bio-evolved thing, are saying that you hope something happens, and that something is not what bio-evolution wants. So you’re a misaligned optimizer from bio-evolution’s perspective.
If you narrowly define the utility function as “IGF via literal DNA molecules” - (which obviously is the relevant context for my statement “hope that there is some decoupling eventually”) then obviously I’m somewhat misaligned to that util func (but not completely, I am having children). And i’m increasingly aligned with the more general utility functions.
None of this is especially relevant, because I am not a species.
Naturally the fact that bio evolution has been largely successful at alignment so far doesn’t mean that will continue indefinitely into the future after further metasystems transitions.
But that is future speculation, and its obviously tautological to argue “bio evolution will fail at alignment in the future” as part of your future model and argument for why alignment is hard in general—we haven’t collected that evidence yet!
Moreover, its not so much that we are misaligned with evolution, and more that bio evolution is being usurped by memetic/info evolution—the future replicator patterns of merit are increasingly no longer just physical genes.
I’m saying that the fact that you, an organism built by the evolutionary process, hope to step outside the evolutionary process and do stuff that the evolutionary process wouldn’t do, is misalignment with the evolutionary process.
I’m saying you didn’t seem to grasp my distinction between systemic vs bio evolution. I do not “hope to step outside the evolutionary process”. The decoupling is only with bio-evolution and genes. The posthuman goal is to move beyond biology, become substrate independent etc, but that is hardly the end of evolution.
A different way to maybe triangulate here: Is misalignment possible, on your view? Like does it ever make sense to say something like “A created B, but failed at alignment and B was misaligned with A”? I ask because I could imagine a position, that sort of sounds a little like what you’re saying, which goes:
I literally provided examples of what misalignment with bio-evolution would look like:
If we haven’t seen such an extinction in the archaeological record, it can mean one of several things:
misalignment is rare, or
misalignment is not rare once the species becomes intelligent, but intelligence is rare or
intelligence usually results in transcendence, so there’s only one transition before the bio becomes irrelevant in the lightcone (and we are it)
We don’t know which. I think it’s a combination of 2 and 3.
The original argument that your OP is responding to is about “bio evolution”. I understand the distinction, but why is it relevant? Indeed, in the OP you say:
So we’re talking about bio evolution, right?
The OP is talking about history and thus bio evolution, and this thread shifted into the future (where info-evolution dominates) here:
I’m saying that you, a bio-evolved thing, are saying that you hope something happens, and that something is not what bio-evolution wants. So you’re a misaligned optimizer from bio-evolution’s perspective.
If you narrowly define the utility function as “IGF via literal DNA molecules” - (which obviously is the relevant context for my statement “hope that there is some decoupling eventually”) then obviously I’m somewhat misaligned to that util func (but not completely, I am having children). And i’m increasingly aligned with the more general utility functions.
None of this is especially relevant, because I am not a species.