If AGI alignment is possibly the most important problem ever, why don’t concerned rich people act like it? Why doesn’t Vitalik Buterin, for example, offer one billion dollars to the best alignment plan proposed by the end of 2023? Or why doesn’t he just pay AI researchers money to stop working on building AGI, in order to give alignment research more time?
Paying researchers to not work on AGI is a very bad idea, as it incentivizes pretending to work in order to not work. The general idea behind not paying Dane-geld is that just encourages more of it. You could sort of make it better by suggesting that they work on alignment rather than capabilities, but the problem with that is that they both often look the same from the outside. You’d end up with the equivalent of gain of function research.
If AGI alignment is possibly the most important problem ever, why don’t concerned rich people act like it? Why doesn’t Vitalik Buterin, for example, offer one billion dollars to the best alignment plan proposed by the end of 2023? Or why doesn’t he just pay AI researchers money to stop working on building AGI, in order to give alignment research more time?
There isn’t a best alignment plan yet—that’s part of the problem
The issue isn’t intuitive—you need quite a bit of context to even understand it. Like evolution 150 years ago
The best long term protection against unaligned AGI (or aliens, or whatever) is to have an aligned AGI, so the faster it appears, the better
People assume it will all work out somehow, as it always has in the past (basic black swan issues)
People are selfish and hope someone else will sort it out
People have other priorities which are more important for them to fund
Basically the whole suite of biases etc. (check Kahneman or the sequences for more)
Paying researchers to not work on AGI is a very bad idea, as it incentivizes pretending to work in order to not work. The general idea behind not paying Dane-geld is that just encourages more of it. You could sort of make it better by suggesting that they work on alignment rather than capabilities, but the problem with that is that they both often look the same from the outside. You’d end up with the equivalent of gain of function research.