This post caught my eye as my background is in mathematics and I was, in the not-too-distant past, excited about the idea of rigorous mathematical AI alignment work. My mind is still open to such work but I’ll be honest, I’ve since become a bit less excited than I was. In particular, I definitely “bounced off” the existing write-ups on Infrabayesianism and now without already knowing what it’s all about, it’s not clear it’s worth one’s time. So, at the risk of making a basic or even cynical point: The remuneration of the proposed job could be important for getting attention/ incentivising people on-the-fence.
It could also work here. But I do feel like pointing out that the bounty format has other drawbacks. Maybe it works better when you want a variety of bitesize contributions, like various different proposals? I probably wouldn’t do work like Abram proposes—quite a long and difficult project, I expect—for the chance of winning a prize, particularly if the winner(s) were decided by someone’s subjective judgement.
My intuition is that you could break this task down into smaller chunks, like applications of Infra-bayes and musings on why Infra-bayes worked better than existing tools there (or worse!), which someone could do within a couple of weeks, and award bounties for those tasks. Then offer jobs to whomever seems like they could do good distillations.
I think that for a few 100 hour tasks, you might need to offer maybe $50k-$100k dollars. That sounds crazy high? Well AI safety is talent constrained, it doesn’t look like much is being done with the money, and MIRI seems to think there’s a high discount rate (doom within a decade or two) so money should be spent now on tasks that seem important.
This post caught my eye as my background is in mathematics and I was, in the not-too-distant past, excited about the idea of rigorous mathematical AI alignment work. My mind is still open to such work but I’ll be honest, I’ve since become a bit less excited than I was. In particular, I definitely “bounced off” the existing write-ups on Infrabayesianism and now without already knowing what it’s all about, it’s not clear it’s worth one’s time. So, at the risk of making a basic or even cynical point: The remuneration of the proposed job could be important for getting attention/ incentivising people on-the-fence.
Offering a bounty on what you want seems sensible here. It seemed like it worked OK for ELK proposals, so why not here?
It could also work here. But I do feel like pointing out that the bounty format has other drawbacks. Maybe it works better when you want a variety of bitesize contributions, like various different proposals? I probably wouldn’t do work like Abram proposes—quite a long and difficult project, I expect—for the chance of winning a prize, particularly if the winner(s) were decided by someone’s subjective judgement.
My intuition is that you could break this task down into smaller chunks, like applications of Infra-bayes and musings on why Infra-bayes worked better than existing tools there (or worse!), which someone could do within a couple of weeks, and award bounties for those tasks. Then offer jobs to whomever seems like they could do good distillations.
I think that for a few 100 hour tasks, you might need to offer maybe $50k-$100k dollars. That sounds crazy high? Well AI safety is talent constrained, it doesn’t look like much is being done with the money, and MIRI seems to think there’s a high discount rate (doom within a decade or two) so money should be spent now on tasks that seem important.