1) In the long run, for CFAR to succeed, it has to be supported by a CFAR donor base that doesn’t funge against SIAI money. I expect/hope that CFAR will have a substantially larger budget in the long run than SIAI. In the long run, then, marginal x-risk minimizers should be donating to SIAI.
2) But since CFAR is at a very young and very vital stage in its development and has very little funding, it needs money right now. And CFAR really really needs to succeed for SIAI to be viable in the long-term.
So my guess is that a given dollar is probably more valuable at CFAR right this instant, and we hope this changes very soon (due to CFAR having its own support base)...
...but...
...SIAI has previously supported CFAR, is probably going to make a loan to CFAR in the future, and therefore it doesn’t matter as much exactly which organization you give to right now, except that if one maxes out its matching funds you probably want to donate to the other until it also maxes...
...and...
...even the judgment about exactly where a marginal dollar spent is more valuable is, necessarily, extremely uncertain to me. My own judgment favors CFAR at the current margins, but it’s a very tough decision. Obviously! SIAI has given money to CFAR. If it had been obvious that this amount should’ve been shifted in direction A or direction B to minimize x-risk, we would’ve necessarily been organizationally irrational, or organizationally selfish, about the exact amount. SIAI has been giving CFAR amounts on the lower side of our error bounds because of the hope (uncertainty) that future-CFAR will prove effective at fundraising. Which rationally implies, and does actually imply, that an added dollar of marginal spending is more valuable at CFAR (in my estimates).
The upshot is that you should donate to whichever organization gets you more excited, like Luke said. SIAI is donating/loaning round-number amounts to CFAR, so where you donate $2K does change marginal spending at both organizations—we’re not going to be exactly re-fine-tuning the dollar amounts flowing from SIAI to CFAR based on donations of that magnitude. It’s a genuine decision on your part, and has a genuine effect. But from my own standpoint, “flip a coin to decide which one” is pretty close to my own current stance. For this to be false would imply that SIAI and I had a substantive x-risk-estimate disagreement which resulted in too much or too little funding (from my perspective) flowing to CFAR. Which is not the case, except insofar as we’ve been giving too little to CFAR in the uncertain hope that it can scale up fundraising faster than SIAI later. Taking this uncertainty into account, the margins balance. Leaving it out, a marginal absolute dollar of spending at CFAR does more good (somewhat) (in my estimation).
Thank you; that helps clarify the issue for me. Since people who know more seem to think it’s a tossup and SIAI motivates me more, I gave $250 to them.
And CFAR really really needs to succeed for SIAI to be viable in the long-term.
That’s an extremely strong claim. Is that actually your belief? Not merely that CFAR success would be useful to SIAI success? There is no alternate plan for SIAI to be successful that doesn’t rely on CFAR?
I have backup plans, but they tend to look a lot like “Try founding CFAR again.”
I don’t know of any good way to scale funding or core FAI researchers for SIAI without rationalists. There’s other things I could try, and would if necessary try, but I spent years trying various SIAI-things before LW started actually working. Just because I wouldn’t give up no matter what, doesn’t mean there wouldn’t be a fairly large chunk of success-probability sliced off if CFAR failed, and a larger chunk of probability sliced off if I couldn’t make any alternative to CFAR work.
I realize a lot of people think it shouldn’t be impossible to fund SIAI without all that rationality stuff. They haven’t tried it. Lots of stuff sounds easy if you haven’t tried it.
Thankyou Eliezer. I’m fascinated by the reasoning and analysis that you’re hinting at here. It helps puts the decisions you and SIAI have made in perspective.
Could you give a ballpark estimate of how much of the importance of successful rationality spin offs is based on expectations of producing core FAI researchers versus producing FAI funding?
I’ve tried less hard to get core FAI researchers than funding. I suspect that given sufficient funding produced by magic, it would be possible to solve the core-FAI-researchers issue by finding the people and talking to them directly—but I haven’t tried it!
How much money would you need magicked to allow you to shed fundraising and infrastructure, etc, and just hire and hole up with a dream team of hyper-competent maths wonks? Restated, at which set amount would SIAI be comfortably able to aggressively pursue its long-term research?
So my guess is that a given dollar is probably more valuable at CFAR right this instant, and we hope this changes very soon (due to CFAR having its own support base)...
an added dollar of marginal spending is more valuable at CFAR (in my estimates).
1) In the long run, for CFAR to succeed, it has to be supported by a CFAR donor base that doesn’t funge against SIAI money. I expect/hope that CFAR will have a substantially larger budget in the long run than SIAI. In the long run, then, marginal x-risk minimizers should be donating to SIAI.
2) But since CFAR is at a very young and very vital stage in its development and has very little funding, it needs money right now. And CFAR really really needs to succeed for SIAI to be viable in the long-term.
So my guess is that a given dollar is probably more valuable at CFAR right this instant, and we hope this changes very soon (due to CFAR having its own support base)...
...but...
...SIAI has previously supported CFAR, is probably going to make a loan to CFAR in the future, and therefore it doesn’t matter as much exactly which organization you give to right now, except that if one maxes out its matching funds you probably want to donate to the other until it also maxes...
...and...
...even the judgment about exactly where a marginal dollar spent is more valuable is, necessarily, extremely uncertain to me. My own judgment favors CFAR at the current margins, but it’s a very tough decision. Obviously! SIAI has given money to CFAR. If it had been obvious that this amount should’ve been shifted in direction A or direction B to minimize x-risk, we would’ve necessarily been organizationally irrational, or organizationally selfish, about the exact amount. SIAI has been giving CFAR amounts on the lower side of our error bounds because of the hope (uncertainty) that future-CFAR will prove effective at fundraising. Which rationally implies, and does actually imply, that an added dollar of marginal spending is more valuable at CFAR (in my estimates).
The upshot is that you should donate to whichever organization gets you more excited, like Luke said. SIAI is donating/loaning round-number amounts to CFAR, so where you donate $2K does change marginal spending at both organizations—we’re not going to be exactly re-fine-tuning the dollar amounts flowing from SIAI to CFAR based on donations of that magnitude. It’s a genuine decision on your part, and has a genuine effect. But from my own standpoint, “flip a coin to decide which one” is pretty close to my own current stance. For this to be false would imply that SIAI and I had a substantive x-risk-estimate disagreement which resulted in too much or too little funding (from my perspective) flowing to CFAR. Which is not the case, except insofar as we’ve been giving too little to CFAR in the uncertain hope that it can scale up fundraising faster than SIAI later. Taking this uncertainty into account, the margins balance. Leaving it out, a marginal absolute dollar of spending at CFAR does more good (somewhat) (in my estimation).
Thank you; that helps clarify the issue for me. Since people who know more seem to think it’s a tossup and SIAI motivates me more, I gave $250 to them.
That’s an extremely strong claim. Is that actually your belief? Not merely that CFAR success would be useful to SIAI success? There is no alternate plan for SIAI to be successful that doesn’t rely on CFAR?
I have backup plans, but they tend to look a lot like “Try founding CFAR again.”
I don’t know of any good way to scale funding or core FAI researchers for SIAI without rationalists. There’s other things I could try, and would if necessary try, but I spent years trying various SIAI-things before LW started actually working. Just because I wouldn’t give up no matter what, doesn’t mean there wouldn’t be a fairly large chunk of success-probability sliced off if CFAR failed, and a larger chunk of probability sliced off if I couldn’t make any alternative to CFAR work.
I realize a lot of people think it shouldn’t be impossible to fund SIAI without all that rationality stuff. They haven’t tried it. Lots of stuff sounds easy if you haven’t tried it.
Thankyou Eliezer. I’m fascinated by the reasoning and analysis that you’re hinting at here. It helps puts the decisions you and SIAI have made in perspective.
Could you give a ballpark estimate of how much of the importance of successful rationality spin offs is based on expectations of producing core FAI researchers versus producing FAI funding?
I’ve tried less hard to get core FAI researchers than funding. I suspect that given sufficient funding produced by magic, it would be possible to solve the core-FAI-researchers issue by finding the people and talking to them directly—but I haven’t tried it!
How much money would you need magicked to allow you to shed fundraising and infrastructure, etc, and just hire and hole up with a dream team of hyper-competent maths wonks? Restated, at which set amount would SIAI be comfortably able to aggressively pursue its long-term research?
He once mentioned a figure of US $10 million / year. Feels like he’s made a similar remark more recently, but it didn’t show in my brief search.
Is this still your view?