We already have robots that fetch things, and optimise for efficiency, and they do not kill the baby in the way. A bloody roomba is already capable of not running over the cat. ChatGPT is capable of identifying that racism is bad, but it should use a racial slur if the alternative is destroying the planet. Or of weighing between creativity and accuracy. Because they don’t maximise for efficiency above all else, single-mindedly. This is never desired, intended, shown, or encouraged. Other concerns are explicitly encoded. Outside of LW, practically no human thinks total utilitarianism represents their desires, and hence it is not what is taught. And we are no longer teaching explicit laws, but through practice, entailing complexity.
Yes, becoming more efficient, getting more stuff and power is useful for a lot of potential goals, and we would expect a lot of AIs to do it.
Big step from there to “do it, do it without limits, and disregard all else.”
Biological life is driven to gain resources and efficiency. And yet only very simple and stupid lifeforms do this to extreme degrees that fuck over all else. Bacteria and algae will destroy their environment that way, yes. Other life forms begin self-regulating. They make trade-offs. They take compromises. This emerges in such simple animals, why wouldn’t it totally never emerge in AI, when we explicitly want it and teach it?
We already have robots that fetch things, and optimise for efficiency, and they do not kill the baby in the way. A bloody roomba is already capable of not running over the cat. ChatGPT is capable of identifying that racism is bad, but it should use a racial slur if the alternative is destroying the planet. Or of weighing between creativity and accuracy. Because they don’t maximise for efficiency above all else, single-mindedly. This is never desired, intended, shown, or encouraged. Other concerns are explicitly encoded. Outside of LW, practically no human thinks total utilitarianism represents their desires, and hence it is not what is taught. And we are no longer teaching explicit laws, but through practice, entailing complexity.
Yes, becoming more efficient, getting more stuff and power is useful for a lot of potential goals, and we would expect a lot of AIs to do it.
Big step from there to “do it, do it without limits, and disregard all else.”
Biological life is driven to gain resources and efficiency. And yet only very simple and stupid lifeforms do this to extreme degrees that fuck over all else. Bacteria and algae will destroy their environment that way, yes. Other life forms begin self-regulating. They make trade-offs. They take compromises. This emerges in such simple animals, why wouldn’t it totally never emerge in AI, when we explicitly want it and teach it?