Try to come up with the best possible “unaligned” AGI. It’s better to be eaten by something that then goes out and explores a broad range of action in an interesting way, especially if you can arrange that it enjoy it, than it is to be eaten by Clippy.
“It’s better to be eaten by something … if you can arrange that it enjoy it”
Given my current state of knowledge, I’d advise against trying to do this because it might introduce s-risks. If you don’t know how to do alignment, then I’d guess it’s best to steer clear of conscious states if possible.
(Separately, I disagree because I don’t think our success odds are anywhere near low enough that we should give up on alignment!)
Point taken. Although, to be honest, I don’t think I can tell what would or would not be conscious anyway, and I haven’t heard anything to convince me that anybody else can either.
… and probably I shouldn’t have answered the headline question while ignoring the text’s points about delay and prevention...
I think we may have different terminal values. I would much rather live out my life in a technologically stagnant world than be eaten by a machine that comes up with interesting mathematical proofs all day.
Try to come up with the best possible “unaligned” AGI. It’s better to be eaten by something that then goes out and explores a broad range of action in an interesting way, especially if you can arrange that it enjoy it, than it is to be eaten by Clippy.
“It’s better to be eaten by something … if you can arrange that it enjoy it”
Given my current state of knowledge, I’d advise against trying to do this because it might introduce s-risks. If you don’t know how to do alignment, then I’d guess it’s best to steer clear of conscious states if possible.
(Separately, I disagree because I don’t think our success odds are anywhere near low enough that we should give up on alignment!)
Point taken. Although, to be honest, I don’t think I can tell what would or would not be conscious anyway, and I haven’t heard anything to convince me that anybody else can either.
… and probably I shouldn’t have answered the headline question while ignoring the text’s points about delay and prevention...
I think we may have different terminal values. I would much rather live out my life in a technologically stagnant world than be eaten by a machine that comes up with interesting mathematical proofs all day.