Lightcone Infrastructure/LessWrong is looking for funding
Lightcone Infrastructure is looking for funding and are working on the following projects:
We run LessWrong, the AI Alignment Forum, and have written a lot of the code behind the Effective Altruism Forum.
During 2022 and early 2023 we ran the Lightcone Offices, and are now building out a campus at the Rose Garden Inn in Berkeley, where we’ve been doing repairs and renovations for the past few months.
We’ve also been substantially involved in the Survival and Flourishing Fund’s S-Process (having written the app that runs the process) and are now running Lightspeed Grants.
We also pursue a wide range of other smaller projects in the space of “community infrastructure” and “community crisis management”. This includes running events, investigating harm caused by community institutions and actors, supporting programs like SERI MATS, and maintaining various small pieces of software infrastructure.
If you are interested in funding us, please shoot me an email at habryka@lesswrong.com (or if you want to give smaller amounts, you can donate directly via PayPal here).
Funding is quite tight since the collapse of FTX, and I do think we work on projects that have a decent chance of reducing existential risk and generally making humanity’s future go a lot better, though this kind of stuff sure is hard to tell. We are looking to raise around $3M to $6M for our operations in the next 12 months. [1]
Edit (June 23): I’ve now given a lot more details on how we operate and what we work on in the comments. I would recommend checking them out if you want to more context on our work.
- ^
Two draft readers of this post expressed confusion that Lightcone needs money, given that we just announced a funding process that is promising to give away $5M in the next two months. The answer to that is that we do not own the money moved via Lightspeed Grants and are only providing grant recommendations to Jaan Tallinn and other funders.
We do separately apply for funding from the Survival and Flourishing Fund, through which Jaan has been our second biggest funder. We also continue to actively fundraise from both SFF and Open Philanthropy (our largest funder).
I’m a fan of the Lightcone team & I think they’re one of the few orgs where I’d basically just say “yeah, they should probably just get as much funding as they want.”
With that in mind, I was surprised by the lack of information in this funding request. I feel mixed about this: high-status AIS orgs often (accurately) recognize that they don’t really need to spend time justifying their funding requests, but I think this often harms community epistemics (e.g., by leading to situations where everyone is like “oh X org is great—I totally support them” without actually knowing much about what work they’re planning to do, what models they have, etc.)
Here are some questions I’m curious about:
What are Lightcone’s plans for the next 3-6 months? (is it just going to involve continuing the projects that were listed?)
How is Lightcone orienting to the recent rise in interest in AI policy? Which policy/governance plans (if any) does Lightcone support?
What is Lightcone’s general worldview/vibe these days? (Is it pretty much covered in this post?) Where does Lightcone disagree with other folks who work on reducing existential risk?
What are Lightcone’s biggest “wins” and “losses” over the past ~3-6 months?
How much funding does Lightcone think it could usefully absorb, and what would the money be used for?
Sorry about that! I’ve drafted like 3-4 different fundraising posts over the last few months, most of which were much longer and had more detail, but I also repeatedly ran into problems that when I showed them to people, they ended up misunderstanding how Lightcone relates to our future work and thinking about our efforts in a very narrow way by overfitting to all the details I put into the post, while missing something important about the way we expect to approach things over the coming years.
I ended up deciding to instead publish a short post, expecting that people will write a lot of questions in the comments, and then to engage straightforwardly and transparently there, which felt like a way that was more likely to end up with shared understanding. Not sure whether this was the right call, and definitely a good chunk of my decisions here are also driven by finding fundraising to be the single most stressful aspect of running Lightcone, and I just find navigating the stress of that easier if I respond to things in a more reactive way.
We are going to be wrapping up renovations and construction for the next month, and will then host a bunch of programs over the summer (like SERI MATS and a bunch of workshops and conferences). During that time I hope to reconnect a bit more with the surrounding AI-Alignment/X-Risk/Rationality/EA/Longtermist-adjacent diaspora, which I intentionally took a pretty big step away from after the collapse of FTX.
I will also be putting a bunch of efforts into Lightspeed Grants. We will see how much traction we get here, but I definitely think there is a chance that blows up into the primary project we’ll be working on for a while, since I think there is a lot of value in diversifying and improving the funding ecosystem, which currently seems to drive a lot of crazy status-dynamics and crazy epistemics within people working on AI risk stuff.
After that, I expect to focus a lot more on online things. I probably want to do a major revamp of the AI Alignment Forum, as well as focus a lot of my attention more on LessWrong again. I am particularly excited about finally properly launching the dialogues feature and driving adoption of that, probably in parts by me and other Lightcone team members participating in a lot of dialogues while we also continue developing the technology on the backend.
I’ve been thinking a lot about this, and I don’t yet have a clear answer. My tentative guess is something like “with a lot of the best people that I know going into AI policy stuff and the hype/excitement around that increasing, the comparative advantage of the Lightcone team is actually even more strongly pointing in the direction of focusing on research forums that ground the epistemic health of the people jumping headfirst into policy stuff”. This means I currently expect to not get super deeply involved, but to interface a lot with people who are jumping into the policy fray, moving to DC, etc. and to figure out what infrastructure can allow those people to stay sane and grounded, since I do really expect that as we get more involved in advocacy, politics and policy-making that thinking clearly will become a lot harder.
But again, I don’t know yet, and at the meta-level I might just organize a bunch of events to help people orient to the shifting policy landscape while I am orienting myself to it as well.
This sure seems hard to answer concisely. Hopefully you can figure out our vibe from my comments and posts. I still endorse a lot of the post you linked, though also changed my mind on a bunch of stuff. I might write more here later. I think this is a valid question, this comment is just already getting very long and I don’t have an immediate good cached answer.
In my books I think by far the biggest win is that I think we relatively successfully handled a really major shift in relationship to the rest of our surrounding community, as an organization whose lifeblood is building infrastructure. My sense is that in all previous organization I’ve worked in, I would have rather left or shut the organization down when FTX collapsed, because I wouldn’t have been able to think clearly and see things with fresh eyes, and orient with my organization to the changing landscape. I think Lightcone successfully handled reorienting together, and I think this is really hard (and probably the result of us staying consistently very small in our permanent staff headcount, which is currently just 8 people).
We also built an IMO really amazing campus that I expect to utilize and get a lot of value out of for the next few years. I also think I am proud of me writing a lot of things publicly on the EA Forum during the time of the FTX collapse and afterwards. I think it also helped the rest of the ecosystem orient better, and a lot of the things were things that nobody else was saying and seemed quite important.
Thanks for this detailed response; I found it quite helpful. I maintain my “yeah, they should probably get as much funding as they want” stance. I’m especially glad to see that Lightcone might be interested in helping people stay sane/grounded as many people charge into the policy space.
This seems quite reasonable to me. I think it might’ve been useful to include something short in the original post that made this clear. I know you said “also feel free to ask any questions in the comments”; in an ideal world, this would probably be enough, but I’m guessing this isn’t enough given power/status dynamics.
For example, if ARC Evals released a post like this, I expect many people would experience friction that prevented them from asking (or even generating) questions that might (a) make ARC Evals look bad, (b) make the commenter seem dumb, or (c) potentially worsen the relationship between the commenter and ARC evals.
To Lightcone’s credit, I think Lightcone has maintained a (stronger) reputation of being fairly open to objections (and not penalizing people for asking “dumb questions” or something like that), but the Desire Not to Upset High-status People or Desire Not to Look Dumb In Front of Your Peers By Asking Things You’re Already Supposed to Know are strong.
I’m guessing that part of why I felt comfortable asking (and even going past the “yay, I like Lightcone and therefore I support this post” to the mental motion of “wait, am I actually satisfied with this post? What questions do I have”) is that I’ve had a chance to interact in-person with the Lightcone team on many occasions, so I felt considerably less psychological friction than most.
All things considered, perhaps an ideal version of the post would’ve said something short like “we understand we haven’t given any details about what we’re actually planning to do or how we’d use the funding. This is because Oli finds this stressful. But we actually really want you to ask questions, even “dumb questions”, in the comments.”
(To be clear I don’t think the lack of doing this was particularly harmful, and I think your comment definitely addresses this. I’m nit-picking because I think it’s an interesting microcosm of broader status/power dynamics that get in the way of discourse, and because I expect the Lightcone team to be unusually interested in this kind of thing.)
What is the Rose Garden Inn used for these days?
We are just wrapping up renovations so not much yet (though we are done very soon). This summer we are likely hosting a good chunk of the SERI MATS scholars, as well as providing space for various other retreats and events (like the Singular Value Learning Theory workshop and we are talking to Manifold about maybe running a 100+ person forecasting conference here).
In-parallel we are also providing office space to a small number of people that I expect to slowly grow over time, trying to build a tight-knit community of people working to reduce existential risk and develop an art of rationality. Currently this includes John Wentworth, Adam Scholl, Sydney von Arx and a few other people.
I am currently holding this part of our plan kind of lightly, and we might end up mostly just running events here, since I think a bunch of things didn’t work amazingly well with the way we ran the Lightcone Offices, and I don’t want the Rose Garden to become a place with similar dynamics where working here could end up feeling like some kind of status marker or like some necessary component to helping with existential risk when living in the Bay Area.
Donated! Could be useful to have the common donation options similar to https://intelligence.org/donate/ or through something like https://www.every.org/.
Thank you!
If I were near a computer any time soon I probably would have mindlessly e-mailed but on reflection posting is probably better in this culture anyway. I have a few questions and notes. For what it’s worth, some of these points would have me walk right on from an organization I’m moderately positive toward. LW exceeds that level for me
I’ve been a casual LW fan off and on for a while, probably over a decade (friend introduced me to The Sequences either shortly after or shortly before completion) and this is the first time I’ve heard of Lightcone. A link to your main page might have been useful in addition to (not instead of; the summary is good) the summary. Yes we all know where Google is, but it’s like not stamping a return envelope
Are you USA deductable?
How much are you asking per individual? I know it’s a broadcast but I still strongly advise organizations on whose boards I sit to include a number anyway. “Whatever you want” is a bad answer. Also if someone asks for $1k “no” sometimes looks like $100, but if I’m willing to give $5k to my local fire department and their top ask is $50 I’m not writing a check for more than $50. (actual situation. After a few years I asked and it turns out they just don’t need more money. Good on them.)
Are there any investor incentives? Naming opportunities are fun, raffles maybe aren’t a fit (though I was just talking with a friend about what an hour of EY answering a stupid question would auction for). MIRI’s leaderboard always struck me as cool.
Up and down. What would you do if your current operating budget doubled? How much of a cut/shortfall is probably game over?
Are your financials public, and if not could you generate some manner of redacted-as-you-like p&l and balance sheet (and a lot of folks care more than me about cash flow statements. I don’t, in most cases, but that’s a whole other conversation)?
Your bullet points are great. Better than half I work with easily
All just my opinion you do what you want. Thanks in advance for your answer and in retrospect for your work
We used to just be called the LessWrong team, but then we expanded our operations into more than just LW and changed our name.
Agree that a link to our website would have still been useful. Just for posterity, here is one: https://www.lightconeinfrastructure.com/
Yep! We are a fiscally sponsored project by the Center for Applied Rationality, which is a 501c3 organization in the US.
I think $1k seems like a reasonable ask for people who are heavy users of LessWrong and have benefitted a lot from it over the years. Though to keep existing I think some individuals/foundations would definitely have to give a bunch more than that.
Not currently, though I also like the leaderboard! We do have a physical campus now, and it somehow never occurred to me that we could name rooms or buildings or other structures after people.
My guess is if someone donates $10k+ there is probably an option of having a room or some structure on the campus be named after you or some other name of your choosing.
If our operating budget doubled we would probably hire slightly faster, though not enormously so (since I think hiring too fast is a very frequent way for organizations to fail). There are also a lot of projects that I would love to take on that would come with having access to more money. I’ve been wanting to run more AI Alignment events and conferences.
But I think mostly it would lead to us having more runway, which would open us more generically to taking risks, and what exact risks we would take is really very uncertain right now. I didn’t expect to be working on a hotel renovation for the last few months, and I didn’t expect I would be doing a bunch of mechanism design for grantmaking a few years ago.
Our expenses for the last year are really very heavily loaded towards renovation and construction (most of which ends up capitalized in the property value on which we have a mortgage on), and looks roughly like this:
Campus renovation
$6,250,000
($3,200,000)
Lightcone Offices & Lighthouses
LessWrong
Admin, overhead, misc. projects
Total
$10,500,000
($3,200,000)
However, this table is pretty misleading, in that it doesn’t amortize our renovation expenses (which I think you want to amortize over something like 5-7 years to get an accurate sense of costs), does not take into account property appreciation, and also is really not very representative of future spending (this was a year of spending heavily supported by FTX, and we’ve made massive cuts to reduce that spending).
See also this comment on my thinking on the economics of a large real estate project like this: https://www.lesswrong.com/posts/9iDw6ugMPk7pmXuyW/lightcone-infrastructure-lesswrong-is-looking-for-funding?commentId=ZijNLGtN9KAD2i8hi
Here is a rough breakdown of how I expect us to spend money over the coming year:
And then I expect those expenses to further go down next year, since we won’t have any renovation expenses left, and revenue will have gone up, since we’ll have more of a track record.
Thank you!
Thanks, just what I sought and quite timely. I appreciate that.
I’m good for the $1k. I’ll put a check in the mail (swipe fees are the devil) and keep an eye out for future opportunities
If we set any new concrete, I think it could be neat to write donors’ names into the concrete. Or maybe we can simply carve them into a wall somewhere.
Absolutely. Most common similar I’ve seen is buy-a-brick campaigns but I like the idea of doing things a little differently
Will much of that $3-6M go into renovating and managing the Rose Garden Inn, or to cover work that could have been covered by existing funding if the Inn wasn’t purchased?
If so, I’m curious to hear more about the strategy behind buying and renovating the space, since it seems like a substantial capital investment, and a divergence from LightCone Infrastructure’s previous work and areas of expertise. I’m aware that several (primarily social?) events were held there over the past year, and I see from an earlier comment that you’re planning to host SERI MATS scholars, and to continue providing space for events and retreats.
it seems valuable to have a central and optimized space for hosting people and events, but I’m curious how large the counterfactual benefit of the Inn is. If it didn’t exist, programs would have to use existing venues such as hotels, which would charge them more (I assume?) and presumably be somewhat less nice. How would you quantify the counterfactual benefit that the Inn has provided here? How does that compare to the expense of buying, renovating and managing it? If the costs exceed those benefits, what additional value do you plan to get out of the space?
Here are some things I like about owning this space:
You don’t have to ask anyone’s permission to make modifications to it or to use it in unusual ways. For instance, if we want to cover every wall with whiteboards (as we’ve done in many rooms), we don’t have to ask anyone’s permission, we just order the boards, hire a few laborers, and have them cut them to size and nail them all along the walls. If I want to turn a 4-person team office into a 4-person bedroom, I just get a few people, carry the furniture into storage, carry some beds in, and we’re done. This means that in the span of just a few days whole buildings can be turned from one use case into another (e.g. from housing into a conference venue), as demand comes up. This is not true of rented office spaces (where you certainly cannot reliably sleep many people or do surgery to the walls) or chain hotels (I cannot ask a hotel to use a room as a team office for 3 montsh), and most residential spaces are far too small. As another datapoint about the limitations of rented spaces, the last office building I was using wouldn’t even allow us to have a fully-dark nap room due to regulations about lighting visibility in the event of a fire.
This space is much more aesthetically beautiful and harmonious with our intentions than most alternatives we could find in Berkeley. It is relatively central in Berkeley, yet has a large, private outdoor area that you can work from in the sun all day. The rooms are not inhuman exact squares with sterile lighting, they’re all unique shapes, often with nice old woods and we can install our own lighting. We’ve had the space to build a number of wooden outdoor huts to work from with electricity and soon-to-be-heated cushions. In my model of the world the aesthetics of a space really affect the sorts of affordances and feelings and problem-solving-approaches that people will consider. For example I think this sort of natural beauty helps people be more willing to go deep on an idea or hypothesis for a longer amount of time.
The space is made of 5 separate buildings, which gives a little village-like feel, and (we’re hoping) can allow some fairly different things to be happening on-site at the same time. In one building you could have a self-contained retreat (CFAR workshop, AI alignment workshop, etc), in another building a few x-risk focused teams could be getting work done in their offices as they have ever day for the last 2 months, another building could be having a weekly communal lunch open to 100+ people as well as hosting a few visitors in rooms upstairs who have flown in from out of town for a few weeks, etc. My hope is that the separate buildings can allow a lot of different things to be happening concurrently, and have natural boundaries between them.
Perhaps of interest, when we were considering alternative locations, the main other places that had the properties of my first two bullets were educational religious spaces. The School of Religion, the School of Theology, and a strange surprise-Buddhist-temple that Habryka and I unexpectedly found ourselves in one evening (as the woman was showing us around the school-like building, she fully walked past the temple doors, until I politely asked to look inside, and she unlocked them to show us a ~7k square foot room with a 40-foot high ceiling, filled with golden statues and colorful ribbons hanging from the ceiling and ancient texts inscribed on rotating pillars and 300 folding chairs and a big stage). These places had a lot of beauty. But one of them basically wasn’t on sale, and the other two were only partially on sale (we couldn’t have owned the whole property and would have to share with some religious groups, which is not a total dealbreaker but I strongly prefer having full ownership).
We also considered renting solely office spaces, which would have been much faster to get started with, and were on the verge of going through on a deal last year. But then at the last minute they explained the elevator needed replacing and would be out of use for the first 2 months of us living there (which is a pretty big obstruction for moving in all of our heavy furniture up ~3 floors). They wouldn’t negotiate at all on this and we walked away. I actually heard (epistemic status: I assign 75% to this being true, I have pinged the person who said this to me to double-check) that the elevator only actually got fixed around a month or two ago. To me not having to deal with this sort of thing is part of the advantage of having full-ownership I describe in the first bullet above.
(I’d enjoy hearing more about the background/history of that temple.)
(I googled and found their name and website, which has a history section about them here. The rest of this comment is a list of other links and recollections from our interaction with them that I found as I looked back into our chat logs and did a little googling, in case anyone is interested.)
This was the LoopNet listing we responded to, where it advertises itself as a coworking space one block from the Downtown Berkeley BART station.
Habryka linked it to me, I phoned the number to set up a viewing, we visited it that evening (10th March, 2022, the day after I turned 25). After Habryka and I visited that night I don’t believe we ever talked with them again.
We thought it was a coworking space (“Coworking With Wisdom”, 4 reviews on yelp, 39 google reviews) that was open to selling, but it turned out to primarily be a Buddhist space called Dharma College (whose website I’ve just found now for the first time).
The Executive Director, Wangmo Dixey, walked us around. My vague recollection is that her father had brought the family to America to spread their tradition of Tibetan Buddhism and he bought the place (Wikipedia says the building was bought in ’09), but for whatever reason she didn’t have as much demand for the space these days (it’s 35k square feet) so she was trying out renting it as a coworking space, and also was quite open to us renting the majority of the building while she had daily meditation classes upstairs (and ran other classes). We told her a bit about our community’s cognitive science / reductionist / AI background, I recall her suggesting it could be exciting to have shared public dialogues with invited Buddhists and rationalists about the mind and consciousness.
The space had a lot of windows and high ceilings and multiple kitchens and more, and I think we could get a lot of use out of it if we were there. If we did co-own I thought many people would quite like the affordance of an upstairs daily meditation session at lunchtime.
I tried to find online photographs of the Temple inside, but there are none. This combined with her initial intent to not unlock the doors for us, makes me think they’ve intentionally made it a secret space and kept photographs off the internet. I think both Habryka and I were somewhat awed by the temple room and I remember positively reinforcing myself for overcoming the slight social awkwardness in asking to go back and open those doors. I recall using the word ‘glorious’ when describing the room as we walked out of it.
The Coworking with Wisdom website however links to a very faithful virtual tour of the rest of the building.
The Dharma College site has more about their traditions. Its history can be found here (including a photograph of Dixey and her father at a ceremony at the White House in 2021), and Dixey’s bio can be found here, where I’ve just learned that she has an honorary degree from Mahachulalongkornrajavidyalaya University in Thailand.
Interesting background. They sound like they are struggling and, like many organizations (not to mention, individuals), wound up buying more house than they could handle—I imagine that COVID was probably a bad thing for large urban temples like that, and it’s not like the Bay Area was undersupplied with various flavors of Buddhism AFAICT. If a Buddhist temple is trying to run a coworking space on the side, they must have a lot of excess room indeed. Might be worth keeping in touch for overflow or backup purposes? They sound well-equipped for conferences or other functions.
Coworking with Wisdom was around pre-pandemic, although that could mean they overbought years ago instead of recently.
I do agree it’s likely they’d be very open to a deal. In general office space in the Bay has taken a nosedive in terms of prices since many tech companies have stayed remote post-pandemic, and a lot of places will take any deal you offer them (I know places that previously would only consider 12-24 month leases will now jump at a month-to-month).
But we have (rough estimate) ~24k sq ft indoor space and ~20k sq ft outdoor space, the vast majority of which is not utilized and not got a set use planned, plus we’re renting a 9bdr house v nearby for overflow housing, so for now we’re certainly not looking for additional space!
These all sound like major benefits to owning the venue yourself!
To be clear, I don’t doubt at all that using the Inn for events is much better than non-purpose-built space. However, the Inn also has costs that renting existing spaces wouldn’t: I assume that purchasing and renovating it costs more than renting hotel spaces as-needed for events (though please correct me if I’m wrong!), and my impression is that it’s taken the Lightcone team a lot of time and effort over the past year+ to purchase and renovate, which naturally has opportunity costs.
I’m asking because my uninformed guess is that those financial and time costs outweigh the (very real) benefits of hosting events like you have been. I’m interested to hear if I’m just wrong about the costs, or if you have additional plans to make even more effective use of the space in the future, or if there’s additional context I’m missing.
ETA: Oli answered these questions below, so no need to respond to them unless you have something additional you’d like me to know.
Thinking about the exact financing of the Inn is a bit messy, especially if we compare it to doing something like running the Lightcone Offices, because of stuff like property appreciation, rental income from people hosting events here, and the hard-to-quantify costs of tying up capital in real estate as opposed to more liquid assets like stocks.
If you assume something like 5% property appreciation per year going forward, and you amortize the part of our construction costs that didn’t increase the property value over the next 7 years (which seems like a reasonable estimate for how long the venue is going to get used), I get an annual cost of running the Rose Garden of about $1.15M (property value of ~$19MM, ~$3.5M in uncapitalized construction costs amortized over 7 years, plus $700k in annual upkeep and maintenance costs).
This is pretty cost-effective compared to other options of even just providing office space, and I think is my preferred way of thinking about the cost of doing this construction project (most of which came from a loan that we took out specifically to finance the purchase and renovation of the place, which is insured against the Rose Garden property itself, so it didn’t straightforwardly funge against our other funding).
To compare this to other costs, renting two floors of the WeWork, which we did for most of the summer last year, cost around $1.2M/yr for 14,000 sq. ft. of office space. The Rose Garden has 20,000 sq. ft. of floor space and 20,000 additional sq. ft. of usable outdoor space for less implied annual cost than that.
In order to fund this initial capital investment, we did definitely cut into the funding we received in 2022, and we would have a bunch of money in our bank accounts right now if we didn’t do this construction project. On the other hand, our spending over the next few years would also be a bunch larger, since we would have to pay more in rent if we wanted to do anything in-person community shaped, for both events and office space, than we will have to pay in interest and upkeep minus appreciation for the Rose Garden Inn.
This doesn’t take into account the cost of having Jaan Tallinn’s capital tied up in real estate with 5% annual interest from us. In one sense it seems good for Jaan to diversify his funds away from crypto. On the other hand he has probably been making closer to 10%-20% annual returns over the last 10-15 years with his investments. My current guess is the diversification benefits here outweigh the higher potential returns (because man, x-risk efforts sure still are quite tech-stock and crypto loaded and I am very glad about more diversification), but I am really not confident, and this could easily dominate the relevant cost-calculations.
It’s also a bit unclear how to think about rental income from people working on existential risk stuff that we wouldn’t otherwise support, which I expect to also make us back some of these funds.
I could write more about how to think about financing the Rose Garden, but I’ll leave it at this. I think it it mostly makes sense for people to think that roughly $1.5M of our annual budget going forward will be used for our Rose Garden Inn, with ~$350k of that going into very illiquid long-term savings for the organization.
=====
By far the biggest source of uncertainty about our budget comes from the potential for FTX clawbacks, which explains most of the wide range of our budget. I currently think there are some good arguments that we owe back about $1.5M to FTX creditors, though the exact game theory here is a bit messy (and they might request up to $4M back from us), and I continue to feel quite confused about what the right thing to do here is. We don’t know if and when they will ask for that money.
The purchase of the Rose Garden Inn and really setting the whole renovation project in-motion was made before November last year, and as such before the collapse of FTX, and as is probably apparent from my comments and posts over the last few months (as well as our reasoning for closing down the Lightcone Offices), a lot of our strategy and perspective has shifted since then.
That said, I am actually still quite excited about our plans with the Rose Garden, but I do think it’s an important disclaimer to add that yeah, a lot of our plans for the Rose Garden were substantially changed when FTX collapsed, and our plans for it have become a bunch less straightforward (the original reasoning for it was something much closer to “this makes sense from a finance perspective even if we just compare it to renting office space at our current location”).
That said I am still quite excited about our plans. Some concrete things that continue to drive my excitement for our Rose Garden Inn investment:
I continue to think that in-person collaboration is really essential, especially for mentoring relationships. I think there continues to be a pretty substantial bottleneck in people getting oriented to working on AI Alignment that goes a lot better with in-person peers and via in-person mentoring from people who have made real contributions to the field, and investing in infrastructure for that is one of the best ways to drive progress on reducing existential risk from AI.
I really want to create a more distinct and intentionally separate culture both on LessWrong and at the Rose Garden Inn, and I think owning a physical space hugely helps with that. FTX, various experiences I’ve had in the EA space over the past few years, as well as a lot of safetywashing in AI Alignment in more recent years, have made me much more hesitant to build a community that can as easily get swept up in respectability cascades and get exploited as easily by bad actors, and I really want to develop a more intentional culture in what we are building here. Hopefully this will enable the people I am supporting to work on things like AI Alignment without making the world overall worse, or displaying low-integrity behavior, or get taken advantage of.
I am very excited about the setup where we get to have both long-term office space and short-term events in one location. I am hopeful that this will enable us to have a lot of people flowing through the space, and will allow us to interface and experiment with a lot of different approaches to AI alignment and rationality, and then slowly grow by picking a small number of the people coming through to work more long-term from our office space.
I’m sympathetic to the high-level claim that owning property usually beats renting if you’re committing for a long time period. But the comparison with WeWork seems odd: WeWork specializes in providing short-term, serviced office space and does so at a substantial premium to the more traditional long-term, unserviced commercial real estate contract. When we were looking for office space in Berkeley earlier this year we were seeing list price between $3.25-$3.75/month per square foot, or $780k-900k/year for 20,000 square feet. I’d expect with negotiation you could get somewhat better pricing than this implies, especially if committing to a longer time period.
Of course, the extra outdoor space, mixed-use zoning and ability to highly customize the space may well offset this. But it starts depending a lot more on the details (e.g. how often is the outdoor space used; how much more productive are people in a customized space vs a traditional office) than it might first seem.
Very small nitpick, but did you mean $3.25-$3.75? (This was the smallest diff I could think of to make your calculation clear).
Yes, thanks for spotting my typo! ($2.75 psf isn’t crazy for Berkeley after negotiation, but is not something I’ve ever seen as a list price.)
Yep, if you commit for longer time periods you can definitely get better deals, and there are definitely other ways to save on office space costs. I didn’t mean to imply this was the minimum you could rent office space for.
The $1.2M/yr estimate was roughly what we were paying at the time, and as such was the central comparison point we had. Comparing it to something more like $800k-$900k a year also seems reasonable to me, though I have less experience with the exact tradeoffs faced by doing that. One reason for that comparison is that the price estimate I did in the comment above included utilities and servicing the space, and I don’t have a ton of experience with how much cost that adds to an unserviced office lease, though I still expect it to be a bunch lower than the WeWork prices.
Thank you for such a detailed and thorough answer! This resolves a lot of my confusion.
Based on conversations around closing the wework Lightcone office, I had assumed that you didn’t want to continue hosting office space, and so hadn’t considered that counterfactual cost. But the Inn expenses you mention seem more reasonable if the alternative is continuing to rent wework space.
The FTX context also makes a lot of sense. I was confused how the purchase fit into your current strategy and funding situation, but I understand that both of those were quite different a year or two ago. Given how much things have changed, do you have conditions under which you would decide to sell the space and focus on other projects? Or are you planning to hold onto it no matter what, and decide how best to use it to support your current strategy as that develops?
I’m extremely excited by and supportive of this comment! An especially important related area I think is “solving the deference problem” or cascades of a sinking bar in forecasting and threatmodeling that I’ve felt over the last couple years.
LW has provided me a ton of value. The other endeavors are less obviously useful to me, but I’m certain they’re useful to many. I wonder if unbundling should be part of the strategy—generate a P&L for each, seek funding both overall and for each program, as a way of deciding what to prioritize if cuts or changes have to be made.
Donated!
There are things which I would be glad to support- like LW infra, and things that I’d prefer not to support, like SERI. What are my options?
For the record, our relationship to supporting events for this ecosystem is changing from something like “all of our resources are the same, here have my venue for free if you need it” to “markets and pricing are a great way for large masses of people to coordinate on the value of a good or service, let’s coordinate substantially via trade”.
For instance, during a previous cohort of SERI MATS scholars at the Lightcone Offices, I spent a couple of weeks of work adding a second floor and getting it furnished and doing interior design, hiring another support person to the office team, and then later on dealing with closing it down and downsizing when the demand went away. I did all of that for free, and was not paid salary or anything by MATS, it was part of my Lightcone work, because we wanted to support mentorship happening in the AI alignment ecosystem. It’s different this time around. They’re paying us a substantial amount of money (well over $100k) for the use of 2.5 of our nicely furnished and designed buildings for 2 months, an amount that makes the trade pretty good for Lightcone (and I hope+expect to work hard and make it worthwhile for SERI MATS too!). The other workshops Habryka has mentioned elsethread will also mostly be paying trade partners (general pricing TBD as we get a better sense of the demand).
I bring this up because the extent to which funds for Lightcone are spent supporting SERI MATS in particular (and other teams/orgs/events) is (I suspect) much less than you are thinking.
At least for the coming year, our expenses are pretty entangled between all the different projects in a way that makes differentially funding things hard. I do take preferences of our donors on how to focus our efforts into account, so donating and just telling us that you would prefer us to work more on one kind of thing vs. another will have some effect.
My guess is you will mostly just have to average our impact across different areas and decide whether the whole portfolio is above your bar.
I like LessWrong and I visit the site quite often. I would be willing to pay a monthly subscription for the site especially if the subscription included extra features.
Maybe LessWrong could raise money in a similar way to Twitter: by offering a paid premium version.
A Fermi estimate of how much revenue LessWrong could generate:
As far as I know, the site gets about 100,000 monthly visitors.
If 10,000 of them signed up for the premium subscription at $10 per month, then LessWrong could generate $100,000 in revenue per month or $1.2 million per year which would cover 20-40% of the $3-6 million funding gap mentioned.
Ehh… feels like your base rate of 10% for LW users who are willing to pay for a subscription is too high, especially seeing how the ‘free’ version would still offer everything I (and presumably others) care about. Generalizing to other platforms, this feels closest to Twitter’s situation with Twitter Blue, whose rates appear is far, far lower: if we be generous and say they have one million subscribers, then out of the 41.5 million monetizable daily active users they currently have, this would suggest a base rate of less than 3%.
ACX is probably a better reference class: https://astralcodexten.substack.com/p/2023-subscription-drive-free-unlocked. In Jan, ACX had 78.2k readers, of which 6.0k subscribers for a 7.7% subscription rate.
Strictly, the 78k is for subscribers (probably meaning people who get the emails), not readers; someone who uses RSS to keep up or just visits the site every now and then won’t be counted. I don’t have much intuition for how many readers will be subscribers.
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
can you also provide a crypto address for interested contributors?
btc eth etc
There is a Coinbase donate button on this page: https://rationality.org/donate
Let me know if that doesn’t work for you, and we can also figure out a direct ETH address.