I don’t think cost had that much to do with the decision, I expect that Open Philanthropy thought it was worth the money and would have been willing to continue funding at this price point.
In general I think the correct response to uncertainty is not half-speed. In my opinion it was the right call to spend this amount of funding on the office for the last ~6 months of its existence even when we thought we’d likely do something quite different afterwards, because it was still marginally worth doing it and the cost-effectiveness calculations for the use of billions of dollars of x-risk money on the current margin are typically quite extreme.
You’re probably not the one to rant to about funding but I guess while the conversation is open I could use additional feedback and some reasons for why OpenPhil wouldn’t be irresponsible in spending the money that way. (I only talk about OpenPhil and not particularly Lightcone, maybe you couldn’t think of better ways to spend the money and didn’t have other options)
Cost effectiveness calculations for reducing x-risk kinda always favor x-risk reduction so looking at it in the absolute isn’t relevant. Currently AI x-risk reduction work is limited because of severe funding restrictions (there are many useful things to do that no one is doing for lack of money) which should warrant carefully done triage (and in particular considering the counterfactual).
I assume the average Lightcone office resident would be doing the same work with slightly reduced productivity (let’s say 1⁄3) if they didn’t have that office space (notably because many are rich enough to get other shared office space from their own pocket). Assuming 30 full time workers in the office, that’s 10 man months per month of extra x-risk reduction work.
For contrast, on the same time period, $185k/month could provide for salary, lodging and office space for 50 people in Europe, all who counterfactually would not be doing that work otherwise, for which I claim 50 man months per month of extra x-risk reduction work. The biggest difference I see is incubation time would be longer than for the Lightcone offices, but if I start now with $20k/month I’d find 5 people and scale it up to 50 by the end of the year.
For contrast, on the same time period, $185k/month could provide for salary, lodging and office space for 50 people in Europe, all who counterfactually would not be doing that work otherwise, for which I claim 50 man months per month of extra x-risk reduction work.
The default outcome of giving people money, is either nothing, noise, or the resources getting captured by existing incentive gradients. In my experience, if you give people free money, they will take it, and they will nominally try to please you with it, so it’s not that surprising if you can find 50 people to take your free money, but causing such people to do specific and hard things is a much higher level of challenge.
I had some hope that “just write good LessWrong posts” is sufficient incentive to get people to do useful stuff, but the SERI MATS scholars have tried this and only a few have produced great LessWrong posts, and otherwise there was a lot of noise. Perhaps it’s worth it in expected value but my guess is that you could do much more selection and save a lot of the money and still get 80% of the value.
I think free office spaces of the sort we offered are only worthwhile inside an ecosystem where there are teams already working on good projects, and already good incentive gradients to climb, such that pouring in resources get invested well even with little discernment from those providing them. In contrast, simply creating free resources and having people come for those with the label of your goal on them, sounds like a way to get all the benefits of goodharting and none of the benefits of the void.
You’re probably not the one to rant to about funding but I guess while the conversation is open I could use additional feedback and some reasons for why OpenPhil wouldn’t be irresponsible in spending the money that way. (I only talk about OpenPhil and not particularly Lightcone, maybe you couldn’t think of better ways to spend the money and didn’t have other options)
Cost effectiveness calculations for reducing x-risk kinda always favor x-risk reduction so looking at it in the absolute isn’t relevant. Currently AI x-risk reduction work is limited because of severe funding restrictions (there are many useful things to do that no one is doing for lack of money) which should warrant carefully done triage (and in particular considering the counterfactual).
I assume the average Lightcone office resident would be doing the same work with slightly reduced productivity (let’s say 1⁄3) if they didn’t have that office space (notably because many are rich enough to get other shared office space from their own pocket). Assuming 30 full time workers in the office, that’s 10 man months per month of extra x-risk reduction work.
For contrast, on the same time period, $185k/month could provide for salary, lodging and office space for 50 people in Europe, all who counterfactually would not be doing that work otherwise, for which I claim 50 man months per month of extra x-risk reduction work. The biggest difference I see is incubation time would be longer than for the Lightcone offices, but if I start now with $20k/month I’d find 5 people and scale it up to 50 by the end of the year.
The default outcome of giving people money, is either nothing, noise, or the resources getting captured by existing incentive gradients. In my experience, if you give people free money, they will take it, and they will nominally try to please you with it, so it’s not that surprising if you can find 50 people to take your free money, but causing such people to do specific and hard things is a much higher level of challenge.
I had some hope that “just write good LessWrong posts” is sufficient incentive to get people to do useful stuff, but the SERI MATS scholars have tried this and only a few have produced great LessWrong posts, and otherwise there was a lot of noise. Perhaps it’s worth it in expected value but my guess is that you could do much more selection and save a lot of the money and still get 80% of the value.
I think free office spaces of the sort we offered are only worthwhile inside an ecosystem where there are teams already working on good projects, and already good incentive gradients to climb, such that pouring in resources get invested well even with little discernment from those providing them. In contrast, simply creating free resources and having people come for those with the label of your goal on them, sounds like a way to get all the benefits of goodharting and none of the benefits of the void.