There are a lot of catastrophes that humans did or could do to themselves. In that regard, AI is like any multi-purpose tool, such as a hammer. We have to sort these out too earlier or later, but isn’t this orthogonal to the alignment question?
It is often considered as such, but my concern is less with “the alignment question” (how to build AI that values whatever its stakeholders value) and more with how to build transformative AI that probably does not lead to catastrophe. Misuse is one of the ways that it can lead to catastrophe. In fact, in practice, we have to sort misuse out sooner than accidents, because catastrophic misuses become viable at a lower tech level than catastrophic accidents.
There are a lot of catastrophes that humans did or could do to themselves. In that regard, AI is like any multi-purpose tool, such as a hammer. We have to sort these out too earlier or later, but isn’t this orthogonal to the alignment question?
In regards to safely-wieldable tool-AI versus ‘alignment’, I recommend thinking in terms of ‘intent alignment’ versus ‘values alignment’ as Seth Herd describes here: Conflating value alignment and intent alignment is causing confusion
It is often considered as such, but my concern is less with “the alignment question” (how to build AI that values whatever its stakeholders value) and more with how to build transformative AI that probably does not lead to catastrophe. Misuse is one of the ways that it can lead to catastrophe. In fact, in practice, we have to sort misuse out sooner than accidents, because catastrophic misuses become viable at a lower tech level than catastrophic accidents.