“I need to beat my competitors” could be used as a bad excuse for taking unnecessary risks. But it is pretty important. Given that an AI you coded right now with your current incomplete knowledge of Friendliness theory is already more likely to be Friendly than that of some competitor who’s never really considered the matter, you only have an incentive to keep researching Friendliness until the last possible moment when you’re confident that you could still beat your competitors.
The question then becomes: what is the minimum necessary amount of Friendliness research at which point going full speed ahead has a better expected result than continuing your research? Since you’ve been researching for several years and sound like you don’t have any plans to stop until you’re absolutely satisfied, you must have a lot of contempt for all your competitors who are going full-speed ahead and could therefore be expected to beat you if any were your intellectual equals. I don’t know your competitors and I wouldn’t know enough AI to be able to judge them if I did, but I hope you’re right.
“I need to beat my competitors” could be used as a bad excuse for taking unnecessary risks. But it is pretty important. Given that an AI you coded right now with your current incomplete knowledge of Friendliness theory is already more likely to be Friendly than that of some competitor who’s never really considered the matter, you only have an incentive to keep researching Friendliness until the last possible moment when you’re confident that you could still beat your competitors.
The question then becomes: what is the minimum necessary amount of Friendliness research at which point going full speed ahead has a better expected result than continuing your research? Since you’ve been researching for several years and sound like you don’t have any plans to stop until you’re absolutely satisfied, you must have a lot of contempt for all your competitors who are going full-speed ahead and could therefore be expected to beat you if any were your intellectual equals. I don’t know your competitors and I wouldn’t know enough AI to be able to judge them if I did, but I hope you’re right.