This is quite well-written. The first point where I ran into a wall was the distinction you drew between short-term goals and long-term strategies as two categorically different things.
I note that AIs can trivially set long-term strategy. I just generated this on GPT-2:
“The President of the United States says, “my fellow Americans, the long-term strategy for our country is this:
Let’s get rid of the Bush tax rates
Cut income taxes for ordinary American workers (including retirees)
and for corporations”
The question is how to translate this strategy into action. It seems to me that a more realistic distinction is whether an AI can take over certain specific human jobs. An AI can do the job of a human calculator. Can an AI successfully take over the job of the President of the USA? No, not yet—but this is in large part because people wouldn’t listen to its directives, not because it couldn’t issue them.
But I am not sure why we’d think that short term plans are somehow easier to make than long term goals. I bet GPT-2 could give me an equally plausible explanation for what chores I should do from my todo list.
I do not claim that AI cannot set long-term strategies. My claim is that this is not where AI’s competitive advantages over humans will be. I could certainly imagine that a future AI would be 10 times better than me in proving mathematical theorems. I am not at all sure it would be 10 times better than Joe Biden in being a U.S. president, and mostly it is because I don’t think that the information-processing capabilities are really the bottleneck for that job. (Though certainly, the U.S. as a whole, including the president, would benefit greatly from future AI tools, and it is quite possible that some of Biden’s advisors would be replaced by AIs.)
This is quite well-written. The first point where I ran into a wall was the distinction you drew between short-term goals and long-term strategies as two categorically different things.
I note that AIs can trivially set long-term strategy. I just generated this on GPT-2:
“The President of the United States says, “my fellow Americans, the long-term strategy for our country is this:
Let’s get rid of the Bush tax rates
Cut income taxes for ordinary American workers (including retirees) and for corporations”
The question is how to translate this strategy into action. It seems to me that a more realistic distinction is whether an AI can take over certain specific human jobs. An AI can do the job of a human calculator. Can an AI successfully take over the job of the President of the USA? No, not yet—but this is in large part because people wouldn’t listen to its directives, not because it couldn’t issue them.
But I am not sure why we’d think that short term plans are somehow easier to make than long term goals. I bet GPT-2 could give me an equally plausible explanation for what chores I should do from my todo list.
I do not claim that AI cannot set long-term strategies. My claim is that this is not where AI’s competitive advantages over humans will be. I could certainly imagine that a future AI would be 10 times better than me in proving mathematical theorems. I am not at all sure it would be 10 times better than Joe Biden in being a U.S. president, and mostly it is because I don’t think that the information-processing capabilities are really the bottleneck for that job. (Though certainly, the U.S. as a whole, including the president, would benefit greatly from future AI tools, and it is quite possible that some of Biden’s advisors would be replaced by AIs.)