I guess the way I look at it is that “alignment” means “an AI system whose terminal goal is to achieve your goals”. The distinction here is then whether the word ‘your’ means something closer to:
the current user making the current request
the current user making the current request, as long as the request is legal and inside the terms of service
the shareholders of the foundation lab that made the AI
all (righthinking) citizens of the country that foundation lab is in (and perhaps its allies)
all humans everywhere, now and in the future
all sapient living beings everywhere, now and in the future
something even more inclusive
Your first option would be somewhere around item 5. or 6. on this list, while your second option would be closer to items 1., 2. or 3.
If AI doesn’t kill or disenfranchise all of us, then which option on this spectrum of possibilities ends up being implemented is going to make a huge difference to how history will play out over the next few decades.
I guess the way I look at it is that “alignment” means “an AI system whose terminal goal is to achieve your goals”. The distinction here is then whether the word ‘your’ means something closer to:
the current user making the current request
the current user making the current request, as long as the request is legal and inside the terms of service
the shareholders of the foundation lab that made the AI
all (righthinking) citizens of the country that foundation lab is in (and perhaps its allies)
all humans everywhere, now and in the future
all sapient living beings everywhere, now and in the future
something even more inclusive
Your first option would be somewhere around item 5. or 6. on this list, while your second option would be closer to items 1., 2. or 3.
If AI doesn’t kill or disenfranchise all of us, then which option on this spectrum of possibilities ends up being implemented is going to make a huge difference to how history will play out over the next few decades.