I wouldn’t go this far yet. E.g. I’ve been playing with the idea that the weighting where humans “win” evolution is something like adversarial robustness. This just wasn’t really a convincing enough weighting to be included in the OP. But if something like that turns out correct then one could imagine that e.g. humans get outcompeted by something that’s even more adversarially robust. Which is basically the standard alignment problem.
Like I did not in fact interject in response to Nate or Eliezer. Someone asked me what triggered my line of thought, and I explained that it came from their argument, but I also said that my point was currently too incomplete.
I wouldn’t go this far yet. E.g. I’ve been playing with the idea that the weighting where humans “win” evolution is something like adversarial robustness. This just wasn’t really a convincing enough weighting to be included in the OP. But if something like that turns out correct then one could imagine that e.g. humans get outcompeted by something that’s even more adversarially robust. Which is basically the standard alignment problem.
Like I did not in fact interject in response to Nate or Eliezer. Someone asked me what triggered my line of thought, and I explained that it came from their argument, but I also said that my point was currently too incomplete.