This is basically misuse risk, which is not a weird problem that people need to be convinced even needs solving. To the extent AI appears likely to be powerful, society at large is already working on this. Of course, its efforts may be ineffective or even counterproductive.
They say power corrupts, but I’d say power opens up space to do what you were already inclined to do without constraints. Some billionaires, e.g. Bill Gates, seem to be sincerely trying to use their resources to help people. It isn’t hard for me to imagine that many people, if given power beyond what they can imagine, would attempt to use it to do helpful / altruistic things (at least, things they themselves considered helpful / altruistic).
I don’t in any sense think either of these are knockdowns, and I’m still pretty concerned about how controllable AI systems (whether that’s because they’re aligned, or just too weak and/or insufficiently agentic) may be used.
I’ve had similar thoughts. Two counterpoints:
This is basically misuse risk, which is not a weird problem that people need to be convinced even needs solving. To the extent AI appears likely to be powerful, society at large is already working on this. Of course, its efforts may be ineffective or even counterproductive.
They say power corrupts, but I’d say power opens up space to do what you were already inclined to do without constraints. Some billionaires, e.g. Bill Gates, seem to be sincerely trying to use their resources to help people. It isn’t hard for me to imagine that many people, if given power beyond what they can imagine, would attempt to use it to do helpful / altruistic things (at least, things they themselves considered helpful / altruistic).
I don’t in any sense think either of these are knockdowns, and I’m still pretty concerned about how controllable AI systems (whether that’s because they’re aligned, or just too weak and/or insufficiently agentic) may be used.