The power imbalance between modern civilization and the Sentinelese is so profound that one could easily imagine it being a crude imitation of what to expect from a superintelligence and humanity. The Sentinelese offer virtually no benefit to India or the West, and in fact occupy an island that could be used for other purposes. The Sentinelese have actively killed people who have gotten too close to them. They live pseudo-Paleolithic lives of naturalistic peril. They lack modern medicine and live on hunter-gatherer instincts. They are not even capable of creating fire by themselves. By all metrics, the Sentinelese serve no purpose and should either be wiped out or forcibly assimilated into modern culture.
And yet we don’t do this because we respect their autonomy and right to live as they so choose, even if it’s a less “civilized” way of life. If members of the Sentinelese choose to integrate into modern society, we’re more than open for them to join us. But the group is not completely uncontacted; they are aware we exist. They simply choose to stay insulated, even with the knowledge that we have advanced technology.
This is what alignment looks like. It may not be “pretty” or “benevolent” but we as a society are aligned to the values of the Sentinelese.
If we did have greater benevolence, we would never tolerate the Sentinelese living this way, but it would come at the cost of their autonomy and right to live as they wish. Indeed, we did have such a mindset once upon a time, and it is widely seen as “colonialist” and detrimental to the Sentinelese by modern standards. If the Sentinelese choose to live this way, who are we to decide for them, even if we know better?
It’s entirely possible that an aligned superintelligence would have very similar thoughts about humanity as a whole.
That’s certainly better than extinction. But Christlike benevolence is the thing to aim for. Call it colonialism if you want—I think the Sentinelese would be better off living more like the rest of us, too.
There is a difference between a truly benevolent superintelligence and an aligned superintelligence.
Alignment doesn’t necessarily mean Christlike benevolence.
Indeed, as I posited up above, we actually have a real-life analog for what “alignment” looks like: the Sentinelese
https://en.wikipedia.org/wiki/Sentinelese
The power imbalance between modern civilization and the Sentinelese is so profound that one could easily imagine it being a crude imitation of what to expect from a superintelligence and humanity. The Sentinelese offer virtually no benefit to India or the West, and in fact occupy an island that could be used for other purposes. The Sentinelese have actively killed people who have gotten too close to them. They live pseudo-Paleolithic lives of naturalistic peril. They lack modern medicine and live on hunter-gatherer instincts. They are not even capable of creating fire by themselves. By all metrics, the Sentinelese serve no purpose and should either be wiped out or forcibly assimilated into modern culture.
And yet we don’t do this because we respect their autonomy and right to live as they so choose, even if it’s a less “civilized” way of life. If members of the Sentinelese choose to integrate into modern society, we’re more than open for them to join us. But the group is not completely uncontacted; they are aware we exist. They simply choose to stay insulated, even with the knowledge that we have advanced technology.
This is what alignment looks like. It may not be “pretty” or “benevolent” but we as a society are aligned to the values of the Sentinelese.
If we did have greater benevolence, we would never tolerate the Sentinelese living this way, but it would come at the cost of their autonomy and right to live as they wish. Indeed, we did have such a mindset once upon a time, and it is widely seen as “colonialist” and detrimental to the Sentinelese by modern standards. If the Sentinelese choose to live this way, who are we to decide for them, even if we know better?
It’s entirely possible that an aligned superintelligence would have very similar thoughts about humanity as a whole.
That’s certainly better than extinction. But Christlike benevolence is the thing to aim for. Call it colonialism if you want—I think the Sentinelese would be better off living more like the rest of us, too.