Ah, I see. You are absolutely right. I unintentionally used two different meanings of the word “alignment” in problems 1 and 3.
If we define alignment as intent alignment (from my comment on problem 1), then humans don’t necessarily lose control over the economy in The Economic Transition Problem. The group of people to win the AI race will basically control the entire economy via controlling AI that’s controlling the world (and is intent aligned to them).
If we are lucky, they can create a democratic online council where each human gets a say in how the economy is run. The group will tell AI what to do based on how humanity voted.
Alternatively, with the help of their intent aligned AI, the group can try to build a value aligned AI. When they are confident that this AI is indeed value aligned, they can then release it and let it be the steward of humanity.
In this scenario, The Economic Transition Problem just becomes The Power Distribution Problem of ensuring that whoever wins the AI race will act in humanity’s best interests (or close enough).
Ah, I see. You are absolutely right. I unintentionally used two different meanings of the word “alignment” in problems 1 and 3.
If we define alignment as intent alignment (from my comment on problem 1), then humans don’t necessarily lose control over the economy in The Economic Transition Problem. The group of people to win the AI race will basically control the entire economy via controlling AI that’s controlling the world (and is intent aligned to them).
If we are lucky, they can create a democratic online council where each human gets a say in how the economy is run. The group will tell AI what to do based on how humanity voted.
Alternatively, with the help of their intent aligned AI, the group can try to build a value aligned AI. When they are confident that this AI is indeed value aligned, they can then release it and let it be the steward of humanity.
In this scenario, The Economic Transition Problem just becomes The Power Distribution Problem of ensuring that whoever wins the AI race will act in humanity’s best interests (or close enough).