I’m not convinced AI researchers are the most relevant experts for predicting when human-level AI will occur, nor the circumstances and results of its arrival. Similarly, I’m not convinced that excellence in baking cakes coincides that well with expertise in predicting the future of the cake industry, nor the health consequences of one’s baking. Certainly both confer some knowledge, but I would expect someone with background in forecasting for instance to do better.
Relatedly, scientists involved in the Asilomar conference are sometimes criticized for failing to include people like epidemiologists, and other health experts at their conference to determine how recombinant DNA research should proceed (I think Krimsky makes this complaint). Almost all of the relevant experts were experts in recombinant DNA research, but the questions were often about how the research might harm society, for instance how diseases they accidentally produced might spread. They did include some lawyers, who apparently had a large effect.
Excellence at baking cakes is certainly helpful. I agree that there are other cake-experts who might be better poised to predict the future of the cake industry. I don’t know who their analogs are in the case of artificial intelligence. Certainly it seems like AI researchers have access to some important and distinctive information (contra the cake example).
The cake industry is responding to market conditions. Their success depends on the number of buyers.
AI technology advances are quite marketable. The level of R&D investment in AI will depend on the marketability of these advances, on government investment, and on regulation.
It is easier to predict the size of a market than to predict the R&D investment that companies will make to address the market, but there is a relationship.
Military successes may accelerate AI investment, or they may result in periods of disinclination that slow things down.
Nick Bostrum decided to draw an abstract picture. The reader is left on his or her own to find the players in the background. We have to look for their motives. Nobody is interested in HLMI except from universities. Companies want superhuman intelligence as fast as possible for the smallest budget. Any nice-to-have capability, making the AI more human-like, is causing delays and money.
Transparency and regulation are urgently needed. We should discuss it later.
Perhaps people who are a step removed from the actual AI research process? When I say that, I’m thinking of people like Robin Hanson and Nick Bostrom, whose work depends on AI but isn’t explicitly about it.
The experienced cake bakers aren’t the most relevant experts (within your imagination) only because you imagine that 1: there’s more relevant experts in general predicting, and 2: cake baking is sufficiently simple (in your imagination or the past, not in the real world where the cake baking got automated and the true “cake bakers” of today are experts in industrial automation, creating the future of cake baking just as we speak about it) that lack of expertise in cake baking is not a sufficient impediment for the former.
None of that is true for the AI—there seem to be no good experts on predicting that sort of thing in general, and a subject is difficult enough that such experts, if they existed, would be more than sufficiently handicapped by their lack of knowledge as to be worthless.
I’m not convinced AI researchers are the most relevant experts for predicting when human-level AI will occur, nor the circumstances and results of its arrival. Similarly, I’m not convinced that excellence in baking cakes coincides that well with expertise in predicting the future of the cake industry, nor the health consequences of one’s baking. Certainly both confer some knowledge, but I would expect someone with background in forecasting for instance to do better.
Relatedly, scientists involved in the Asilomar conference are sometimes criticized for failing to include people like epidemiologists, and other health experts at their conference to determine how recombinant DNA research should proceed (I think Krimsky makes this complaint). Almost all of the relevant experts were experts in recombinant DNA research, but the questions were often about how the research might harm society, for instance how diseases they accidentally produced might spread. They did include some lawyers, who apparently had a large effect.
Excellence at baking cakes is certainly helpful. I agree that there are other cake-experts who might be better poised to predict the future of the cake industry. I don’t know who their analogs are in the case of artificial intelligence. Certainly it seems like AI researchers have access to some important and distinctive information (contra the cake example).
The cake industry is responding to market conditions. Their success depends on the number of buyers.
AI technology advances are quite marketable. The level of R&D investment in AI will depend on the marketability of these advances, on government investment, and on regulation.
Investment levels will matter.
It is easier to predict the size of a market than to predict the R&D investment that companies will make to address the market, but there is a relationship.
Military successes may accelerate AI investment, or they may result in periods of disinclination that slow things down.
Follow the trail of money...
Nick Bostrum decided to draw an abstract picture. The reader is left on his or her own to find the players in the background. We have to look for their motives. Nobody is interested in HLMI except from universities. Companies want superhuman intelligence as fast as possible for the smallest budget. Any nice-to-have capability, making the AI more human-like, is causing delays and money.
Transparency and regulation are urgently needed. We should discuss it later.
Perhaps people who are a step removed from the actual AI research process? When I say that, I’m thinking of people like Robin Hanson and Nick Bostrom, whose work depends on AI but isn’t explicitly about it.
The experienced cake bakers aren’t the most relevant experts (within your imagination) only because you imagine that 1: there’s more relevant experts in general predicting, and 2: cake baking is sufficiently simple (in your imagination or the past, not in the real world where the cake baking got automated and the true “cake bakers” of today are experts in industrial automation, creating the future of cake baking just as we speak about it) that lack of expertise in cake baking is not a sufficient impediment for the former.
None of that is true for the AI—there seem to be no good experts on predicting that sort of thing in general, and a subject is difficult enough that such experts, if they existed, would be more than sufficiently handicapped by their lack of knowledge as to be worthless.