Who needs to put thought into the possibility of misbuilding an AI other than people who will themselves engage in AI research? (This is not a rhetorical question—again, I’m new to this.)
The people who do the real work. Utlimately it doesn’t matter if the people who do the AI research care about existential risk or not (if we make some rather absolute economic assumptions). But you’ve noticed this already and you are right about the ‘further alarm bells’.
Ultimately, the awareness of the layperson matters for the same reason that the awareness of the layperson matters for any other political issue. While with AI people can’t get their idealistic warm fuzzies out of barely relevant things like ‘turning off a light bulb’ things like ‘how they vote’ do matter. Even if it is at a lower level of ‘voting’ along the lines of ‘which institutions do you consider more prestigious’?
You can ramp up the variance of people’s opinions and come out better financially.
The people who do the real work. Utlimately it doesn’t matter if the people who do the AI research care about existential risk or not (if we make some rather absolute economic assumptions). But you’ve noticed this already and you are right about the ‘further alarm bells’.
Ultimately, the awareness of the layperson matters for the same reason that the awareness of the layperson matters for any other political issue. While with AI people can’t get their idealistic warm fuzzies out of barely relevant things like ‘turning off a light bulb’ things like ‘how they vote’ do matter. Even if it is at a lower level of ‘voting’ along the lines of ‘which institutions do you consider more prestigious’?
Good point!