I made this a while back to make the math easier. The default values should be Salamon’s numbers. The calculations argue for more spending on AI research, but huge returns happen with almost any sort of existential risk research. Successful AI has the advantage of solving all other existential risk problems though.
Friendliness of conceptually clean AI and suppressing all work and discussion on messy AI hurts world 2 as it increases the chance we might end up with messy UFAI.
So don’t suppress that sort of research unless you’re sure it’s harmful. I don’t see why you’d want to suppress “messy” AI research in a world where “clean” AI is the solution (or vice-versa). If one type isn’t the solution, then it won’t work. You’re also neglecting that future-successful ideas can draw inspiration or useful information from research on unsuccessful ideas. Jürgen Schmidhuber says he gets ideas from both messy and clean AI.
If you look at the advice people are giving in the thread about advice to AI makers it mainly revolves around clean AI with most other research not encouraged. I believe this is indicative of the stance SIAI takes and promotes at the moment. It is not the only stance they could take.
There is no reason why both research projects shouldn’t be able to coexist (apart from the funding conflict which is a minor conflict).
I think SIAI is focusing on clean AI because most academic research is on messy AI. If you look at the margins, one more unit of research on clean AI is probably more beneficial than one more unit of research on messy AI.
Oh and there’s the selection effect of LW posters replying in an AI thread.
I made this a while back to make the math easier. The default values should be Salamon’s numbers. The calculations argue for more spending on AI research, but huge returns happen with almost any sort of existential risk research. Successful AI has the advantage of solving all other existential risk problems though.
So don’t suppress that sort of research unless you’re sure it’s harmful. I don’t see why you’d want to suppress “messy” AI research in a world where “clean” AI is the solution (or vice-versa). If one type isn’t the solution, then it won’t work. You’re also neglecting that future-successful ideas can draw inspiration or useful information from research on unsuccessful ideas. Jürgen Schmidhuber says he gets ideas from both messy and clean AI.
If you look at the advice people are giving in the thread about advice to AI makers it mainly revolves around clean AI with most other research not encouraged. I believe this is indicative of the stance SIAI takes and promotes at the moment. It is not the only stance they could take.
There is no reason why both research projects shouldn’t be able to coexist (apart from the funding conflict which is a minor conflict).
I think SIAI is focusing on clean AI because most academic research is on messy AI. If you look at the margins, one more unit of research on clean AI is probably more beneficial than one more unit of research on messy AI.
Oh and there’s the selection effect of LW posters replying in an AI thread.
There are very few people working on making messy AI Friendly, though. Most of it is charging forward recklessly.