All attempts at making sure there will be only provably friendly AIs seem doomed though. Once somebody posts the recipe for practically feasible self-improving Gödel machines or AIs in form of code into which one can plug arbitrary utility functions, many users will equip such AIs with many different goals, often at least partially conflicting with those of humans. The laws of physics and the availability of physical resources will eventually determine which utility functions will help their AIs more than others to multiply and become dominant in competition with AIs driven by different utility functions. The survivors will define in hindsight what’s ‘moral’, since only survivors promote their values...
That seems more likely than a secretive monoplolistic agent keeping the technology for themselves from the beginning—and obliterating all potential rivals.
Keeping the technology of general-purpose inductive inference secret seems unlikely to happen in practice. It is going to go into embedded devices—from which it will inevitably be reverse engineered and made publicly accessible. Also, it’s likely to arise from a public collaborative development effort in the first place. I am inclined to doubt whether anyone can win while keeping their technology on a secure server—try to do that and you will just be overtaken—or rather, you will never be in the lead in the first place.
Not pessimism, realism, is my assessment. You have to apply your efforts where they will actually make a difference.
That seems more likely than a secretive monoplolistic agent keeping the technology for themselves from the beginning—and obliterating all potential rivals.
Keeping the technology of general-purpose inductive inference secret seems unlikely to happen in practice. It is going to go into embedded devices—from which it will inevitably be reverse engineered and made publicly accessible. Also, it’s likely to arise from a public collaborative development effort in the first place. I am inclined to doubt whether anyone can win while keeping their technology on a secure server—try to do that and you will just be overtaken—or rather, you will never be in the lead in the first place.
Not pessimism, realism, is my assessment. You have to apply your efforts where they will actually make a difference.