I think there isn’t much hope in this direction. Most AI resources will probably be spent on competition between AIs, and AIs will self-modify to remove wasteful spending. It’s not enough to have a weak value that favors us, if there’s a stronger value that paves over us. We’re teaching AI based on human behavior and with a goal of chasing money, but people chasing money often harm other people, so why would AI be nicer than that. It’s all just wishful thinking.
My angle here is not “there seems like there’s hope in this direction”, it’s that the discourse around this feels confused and unresolved and this is maybe creating some faultlines around which something tragic may happen later.
I’m not 100% sure if this actually cruxy for Paul/Buck/etc’s decisionmaking, but it seems part of an overall mosaic that is the difference between Eliezer and Nate saying “we’re doomed” and Paul/Buck etc saying “we may well be doomed or have some really-bad-but-not-doomed things happen”, and in some places that results in different high level strategies that may be subtly in conflict.
I think there isn’t much hope in this direction. Most AI resources will probably be spent on competition between AIs, and AIs will self-modify to remove wasteful spending. It’s not enough to have a weak value that favors us, if there’s a stronger value that paves over us. We’re teaching AI based on human behavior and with a goal of chasing money, but people chasing money often harm other people, so why would AI be nicer than that. It’s all just wishful thinking.
My angle here is not “there seems like there’s hope in this direction”, it’s that the discourse around this feels confused and unresolved and this is maybe creating some faultlines around which something tragic may happen later.
I’m not 100% sure if this actually cruxy for Paul/Buck/etc’s decisionmaking, but it seems part of an overall mosaic that is the difference between Eliezer and Nate saying “we’re doomed” and Paul/Buck etc saying “we may well be doomed or have some really-bad-but-not-doomed things happen”, and in some places that results in different high level strategies that may be subtly in conflict.