OpenAI is an example of a project that threw a bunch of money at hiring people to do something about AI alignment. If you follow the perspective of Eliezer, they increased AI risk in the process.
You can easily go through LinkedIn and hire people to do alignment research. On the other hand, it’s not easy to make sure that you are producing a net benefit on the resulting risk and not just speeding up AI development.
Intuitively, I expect that gathering really talented people and telling them to do stuff related to X isn’t that bad of a mechanism for getting X done. The Manhattan Project springs to mind.
The Manhatten Project is not one that’s famous for reducing X-risk.
My assertion is that spending money hiring very high quality people will get you technical progress. Whether that’s technical progress on nuclear weapons, AI capabilities, or AI alignment is a function of what you incentivize the researchers to work on.
Yes, OpenAI didn’t effectively incentivize AI safety work. I don’t look at that and conclude that it’s impossible to incentivize AI safety work properly.
The take away isn’t that it “isn’t possible”, since one failure can’t possibly speak to all possible approaches. However, it does give you evidence about the kind of thing that’s likely to happen if you try—and apparently that’s “making things worse”. Maybe next time goes better, but it’s not likely to go better on it’s own, or by virtue of throwing more money at it.
The problem isn’t that you can’t buy things with money, it’s that you get what you pay for and not (necessarily) what you want. If you give a ten million dollar budget to someone to buy predictions, then they might use it to pay a superforcaster to invest time into thinking about their problem, they might use it to set up and subsidize a prediction market, or they might just use it to buy time from the “top of the line” psychics. They will get “predictions” regardless, but the latter predictions do not get better simply by throwing more money at the problem. If you do not know how to buy results themselves, then throwing more money at the problem is just going to get you more of whatever it is you are buying—which apparently wasn’t AI safety last time.
Nuclear weapon research is fairly measurable, and therefore fairly buyable. AI capability research is too, at least if you’re looking for marginal improvements. AI alignment research is much harder to measure, and so you’re going to have a much harder time buying what you actually want to buy.
If you think you know how to effectively incentivize people to work on AI safety in a way that produces AI alignment but does not increase AI capability buildup speed in a way that increases risk, why don’t you explicitly advocate for a way you think the money could be spent?
Generally, whenever you spent a lot of money you get a lot of side effects and not only that what you want to encourage.
OpenAI is an example of a project that threw a bunch of money at hiring people to do something about AI alignment. If you follow the perspective of Eliezer, they increased AI risk in the process.
You can easily go through LinkedIn and hire people to do alignment research. On the other hand, it’s not easy to make sure that you are producing a net benefit on the resulting risk and not just speeding up AI development.
The Manhatten Project is not one that’s famous for reducing X-risk.
My assertion is that spending money hiring very high quality people will get you technical progress. Whether that’s technical progress on nuclear weapons, AI capabilities, or AI alignment is a function of what you incentivize the researchers to work on.
Yes, OpenAI didn’t effectively incentivize AI safety work. I don’t look at that and conclude that it’s impossible to incentivize AI safety work properly.
The take away isn’t that it “isn’t possible”, since one failure can’t possibly speak to all possible approaches. However, it does give you evidence about the kind of thing that’s likely to happen if you try—and apparently that’s “making things worse”. Maybe next time goes better, but it’s not likely to go better on it’s own, or by virtue of throwing more money at it.
The problem isn’t that you can’t buy things with money, it’s that you get what you pay for and not (necessarily) what you want. If you give a ten million dollar budget to someone to buy predictions, then they might use it to pay a superforcaster to invest time into thinking about their problem, they might use it to set up and subsidize a prediction market, or they might just use it to buy time from the “top of the line” psychics. They will get “predictions” regardless, but the latter predictions do not get better simply by throwing more money at the problem. If you do not know how to buy results themselves, then throwing more money at the problem is just going to get you more of whatever it is you are buying—which apparently wasn’t AI safety last time.
Nuclear weapon research is fairly measurable, and therefore fairly buyable. AI capability research is too, at least if you’re looking for marginal improvements. AI alignment research is much harder to measure, and so you’re going to have a much harder time buying what you actually want to buy.
If you think you know how to effectively incentivize people to work on AI safety in a way that produces AI alignment but does not increase AI capability buildup speed in a way that increases risk, why don’t you explicitly advocate for a way you think the money could be spent?
Generally, whenever you spent a lot of money you get a lot of side effects and not only that what you want to encourage.