AI friendliness is an important goal and it would be insanely dangerous to build an AI without researching this issue first. I think this is pretty much the consensus view, and that is perfectly sensible.
However, I believe that we are making the wrong inferences from this.
The straightforward inference is “we should ensure that we completely understand AI friendliness before starting to build an AI”. This leads to a strongly negative view of AI researchers and scares them away. But unfortunately reality isn’t that simple. The goal isn’t “build a friendly AI”, but “make sure that whoever builds the first AI makes it friendly”.
It seems to me that it is vastly more likely that the first AI will be built by a large company, or as a large government project, than by a group of university researchers, who just don’t have the funding for that.
I therefore think that we should try to take a more pragmatic approach. The way to do this would be to focus more on outreach and less on research. It won’t do anyone any good if we find the perfect formula for AI friendliness on the same day that someone who has never heard of AI friendliness before finishes his paperclip maximizer.
LessWrong’s attitude towards AI research
AI friendliness is an important goal and it would be insanely dangerous to build an AI without researching this issue first. I think this is pretty much the consensus view, and that is perfectly sensible.
However, I believe that we are making the wrong inferences from this.
The straightforward inference is “we should ensure that we completely understand AI friendliness before starting to build an AI”. This leads to a strongly negative view of AI researchers and scares them away. But unfortunately reality isn’t that simple. The goal isn’t “build a friendly AI”, but “make sure that whoever builds the first AI makes it friendly”.
It seems to me that it is vastly more likely that the first AI will be built by a large company, or as a large government project, than by a group of university researchers, who just don’t have the funding for that.
I therefore think that we should try to take a more pragmatic approach. The way to do this would be to focus more on outreach and less on research. It won’t do anyone any good if we find the perfect formula for AI friendliness on the same day that someone who has never heard of AI friendliness before finishes his paperclip maximizer.
What is your opinion on this?