It’s very likely that the majority of ethical discussion in AI will become politicized and therefore develop a narrow Overton window, which won’t cover the actually important technical work that needs to be done.
The way that I see this happening currently is that ethics discussions have come to largely surround two issues: 1) Whether the AI system “works” at all, even in the mundane sense (could software bugs cause catastrophic outcomes?) and 2) Is it being used to do things we consider good?
The first one is largely just a question of implementing fairly standard testing protocols and developing refinements to existing systems, which is more along the lines of how to prevent errors in narrow AI systems. The same question can be applied to any software system at all, regardless of its status as actually being an “AI”. In AGI ethics you pretty much assume that lack of capability is not the issue.
The second question is much more likely to have political aspects, this could include things like “Is our ML system biased against this demographic?” or “Are governments using it to spy on people?” or “Are large corporations becoming incredibly wealthy because of AI and therefore creating more inequality?” I also see this question as applying to any technology whatsoever and not really specifically about AI. The same things could be asked about statistical models, cameras, or factories. Therefore, much of our current and near-future “AI ethics” discussions will take on a similar tack to historical discussions about the ethics of some new technology of the era, like more powerful weapons, nuclear power, faster communications, the spread of new media forms, genetic engineering and so on. I don’t see these discussions as even pertaining to AGI risk in a proper sense, which should be considered in its own class, but they are likely to be conflated with it. Insofar as people generally do not have concrete “data” in front of them detailing exactly how and why something can go wrong, these discussions will probably not have favorable results.
With nuclear weapons, there was some actual “data” available, and that may have been enough to move the Overton window in the right direction, but with AGI there is practically no way of obtaining that with a large enough time window to allow society to implement the correct response.
AI safety is already a pretty politicized topic. Unfortunately, the main dimension I see it politicized on is the degree to which it’s a useful line of research in the first place. (I think it’s possible that the way AI safety has historically been advocated for might have something to do with this.) Some have argued that “AI ethics” will help with this issue.
It’s very likely that the majority of ethical discussion in AI will become politicized and therefore develop a narrow Overton window, which won’t cover the actually important technical work that needs to be done.
The way that I see this happening currently is that ethics discussions have come to largely surround two issues: 1) Whether the AI system “works” at all, even in the mundane sense (could software bugs cause catastrophic outcomes?) and 2) Is it being used to do things we consider good?
The first one is largely just a question of implementing fairly standard testing protocols and developing refinements to existing systems, which is more along the lines of how to prevent errors in narrow AI systems. The same question can be applied to any software system at all, regardless of its status as actually being an “AI”. In AGI ethics you pretty much assume that lack of capability is not the issue.
The second question is much more likely to have political aspects, this could include things like “Is our ML system biased against this demographic?” or “Are governments using it to spy on people?” or “Are large corporations becoming incredibly wealthy because of AI and therefore creating more inequality?” I also see this question as applying to any technology whatsoever and not really specifically about AI. The same things could be asked about statistical models, cameras, or factories. Therefore, much of our current and near-future “AI ethics” discussions will take on a similar tack to historical discussions about the ethics of some new technology of the era, like more powerful weapons, nuclear power, faster communications, the spread of new media forms, genetic engineering and so on. I don’t see these discussions as even pertaining to AGI risk in a proper sense, which should be considered in its own class, but they are likely to be conflated with it. Insofar as people generally do not have concrete “data” in front of them detailing exactly how and why something can go wrong, these discussions will probably not have favorable results.
With nuclear weapons, there was some actual “data” available, and that may have been enough to move the Overton window in the right direction, but with AGI there is practically no way of obtaining that with a large enough time window to allow society to implement the correct response.
AI safety is already a pretty politicized topic. Unfortunately, the main dimension I see it politicized on is the degree to which it’s a useful line of research in the first place. (I think it’s possible that the way AI safety has historically been advocated for might have something to do with this.) Some have argued that “AI ethics” will help with this issue.