Some context on why I wanted Jacob to post this...
A lot of new alignment/EA orgs and projects have been popping up lately. The majority of these are founded by people who are either fresh out of college, or researchers who have been in academia basically their whole lives. In other words, they’re founded by people who have minimal experience with or exposure to ops and management (and the ops/management they do interface with tends to be mediocre at best). Interfacing with such orgs inevitably means interfacing with people who are bad at ops, and are figuring it out as they go (insofar as they figure it out at all).
A couple weeks ago, Akash wrote up his “AI Safety field-building projects I’d like to see”. I was struck by visions of dozens of new projects attempting one or another thing on that list with terrible ops, all turning into mangled versions of themselves in which the organizers were constantly fighting fires from start to end with mixed success, with the things which actually matter most going unaddressed for lack of slack and time, until the projects eventually just kind of dissolved. In the end, I sighed, and accepted that somebody was going to have to metaphorically sit the kids down for a talk.
… of course this rarely works, and inevitably the kids will need to fail for a while before they figure things out. Hopefully reading some things by people who (in my judgement) are good at ops will at least speed it up. In the meantime, here’s the One Thing I’d tell someone starting a project/org: if I were starting an org, the very first thing I would do is hire an ops person, preferably someone who I’ve interfaced with before and know to be competent. I’m not saying your first hire needs to be an ops person; great ops people are scarce, and you’ll probably do more of your own ops than I would aim for and develop the skills yourself. The point is that getting ops right would literally be the first thing on my todo list for a new org. It might not be the Most Important Thing, but it’s the Most Foundational Thing, and everything else will constantly be slowed down and falling apart and generating inconveniences if the ops foundation isn’t there.
It would also be interesting to see examples of what terrible ops looks like. As one of the “kids”, here are some examples of things that were bad in previous projects I worked on (some mistakes for which I was responsible):
- getting obsessed with some bad metric (eg: number of people who come to an event) and spending lots of hours getting that number up instead of thinking about why I was doing that - so many meetings, calling a meeting if there’s any uncertainty about what to do next - there being uncertainty about what to do next because team members lack context, don’t know who’s responsible for what, who is working on what, etc— some people taking on too much responsibility and not being able to pass it on because having to explain how to do a task to someone else would itself take up too much time— very disorganised meetings where everyone wanted to have a say and it wasn’t clear at the end of it what the action steps were— an unwillingness for the person with the most context to take the role of explicitly telling others what concrete tasks to do (because the other team members were their friends and they didn’t want to be too bossy) - There not being enough structure for team members to give feedback if they thought an idea or project someone else was very excited about would be useless, as a result some mini-projects getting incubated that people weren’t excited about or projects that predictably failed because the person who wanted to do them did not have enough information to help them figure out how to avoid the failure modes other team members would have been concerned about (lack of sharing intuitions really well). also, people picking projects and tasks to do for bad reasons rather than via structured thinking about priorities - including too many people in meetings because “we’d love to get person X’s thoughts on our plans as well (and person Y and person Z...)” - it being hard to trust that other team members would actually get things assigned to them done on time, partly because it was difficult to see partial progress without having to ask the person how things were going - changing platforms and processes based on whims rather than figuring things out early and sticking with them
I think some of the things mentioned in the post are pretty helpful for avoiding some of these problems.
I think this is a great goal and that this post furthers it. I do hope someone from a more staid institution writes a parallel document about how they work. Lightcone runs on a very chaotic/heroic model that works well for them and has accomplished great stuff, but some projects and people do better with a slower and steadier approach.
Can confirm Lightcone is very chaotic and sometimes works heroic hours, and it seems tangled up in our way of working for reasons that are not super clear to me.
So when reading your comment I was asking myself why the above template couldn’t be run by a project that wanted to work closer to 40 hours rather than 80 hours per week? One answer is that “Well, if people are importantly blocking elements, they must be available to reply on slack and unblock other people whenever”, which is mostly true for us, except that 1) we almost never wake up people who are sleeping :) and 2) if people sign-post they are taking a rest day or going on vacation others usually try fairly hard to find a way to solve problems without contacting them.
I agree that most of these rules could smoothly transition to a less intense team, and nonetheless believe a less chaotic org would write a fairly different document, which is why I think it would be useful for them to do so.
One thing I can speak to a tiny bit is a software company I worked at that had a lot of the chaotic/hero-ness in certain roles, but absolutely had to be cross-continental, and thus was also remote and heavily asynchronous. It built up really great practices for documentation and async communication to make that work. Alas it’s been too long since I worked there for me remember specifics, so I can’t say anything useful.
I don’t feel that we’re especially “chaotic” or would describe us as a chaotic org. We have lots of structure and process and principles and intentionality in how we operate. Though I suspect there’s something real you’re thinking of.
Oh fwiw I think we’re quite chaotic. Like, amount of suddenly changing priority and balls sometimes getting dropped because we took on too many things.
(Not sure we’re that chaotic for a startup, but startups are already pretty chaotic)
For me, the interesting part is the transition from a fast-growing startup to a stable enterprise. I have been in transitional companies most of my professional life, and felt the growing pains. I have looked for resources to support this transition. One book provided a useful lens: Growing Pains: Transitioning from an Entrepreneurship to a Professionally Managed Firm by Eric G. Flamholtz and Yvonne Randle (Goodreads). It describes a number of stages or challenges that a maturing company has to tackle in a particular order of priority. It is mostly based on case studies and surveys.
In my experience, all the areas have to be advanced at the same time, but some more than others. For example, the clear instructions, mentioned above are a precursor to Corporate Culture. And it also makes sense to start with Operational Systems before you need them—at least to hire people with experience in these areas that will later be able to bring them about.
Pretty interesting, but I have a bunch of warning flags around the phrase “hire an ops person”, here’s a few thoughts:
Everyone on our team is a “generalist”, not an “ops person”. Anyone can do any task. When we run an event or space, responsibilities involve “venue setup” and also “preparing opening session”, and anyone should be able to be assigned to any task, or help with any task. (Of course, there’s explicitly one person owning each task, it’s fully clear who has final decision-making power, and who gets credit for failure/success.)
For new organizations I would bet on them much harder if all the cofounders could do all the work, than if the cofounders could each only do some of the work. And I’d bet against more strongly still if no single cofounder could do all of the work. I do think research orgs tend to have more of a split here, I’m not sure how good I think that is (I know multiple cases where it’s led to major dysfunction), I still think it’s best if there’s a person at the top who can do everything.
Historically when I’ve been told by a team-lead that “ops is not my responsibility, I’m not tightly in the loop on that” this has been closely correlated with there being an ops catastrophe somewhere that the team-lead is unable to fix, that’s been quite costly for the project.
Your point about giving space for newcomers to fail a while is a good one, and I agree. But I’m not of the same mind that “get better ops hires” is the method, as opposed to “relentlessly work until you yourself have the skill to ensure your project’s success”.
Especially if your ops hire doesn’t understand your work or your mission, and you aren’t likely to produce very legible signs of success (e.g. big funding, big publications, impressive product sales, etc), then it’s hard for that person to be motivated by the mission, and they may leave to do something for which they do have the generators. And then you’ll be left with a shell of an org built around them that nobody, not even you, really knows how it functions.
That said, I think if you’re weak in an area (e.g. in coding, or management, or other things), having a cofounder who is strong in that area is really excellent, and my best project collaborators have been people who, while in key ways are extremely similar to me, do have strengths where I have weaknesses and can pick up the slack there.
As a second example of Lightcone being generalists, when we’re evaluating how LessWrong/AF is doing, everyone should have an inside view and have engaged with the content, not only some of us e.g. at a recent team retreat, we spent an hour or two as a team debating whether the simulators post was right about the myopia of language models, and nobody wasn’t basically following the discussion.
Pardon the long scattershot of thoughts. I have a sense that people often try to ‘hire for ops roles’ poorly, and are confused about how to allocate work within a small team, so I wanted to say something about how it can go wrong and how else one can think about it.
Edit: I don’t think this applies for short-term contractors, it primarily applies to cofounders and other permanent hires.
given this notional use case (and the relative inexperience of the implied user), I think its even more important to (as Gunnar mentioned) contextualize this advice as to whom its for, and how they should use it.
doing that properly would take more than i have for this at the moment, but i’d appreciate epistemic tagging regarding things like; this only could work at a new/small scale (for reasons including because the cost of keeping everyone 100% context scales with org size and because benefits don’t) that strategy has to fit the employees you have, and this sort of strategy constrains the type of person you can hire to those who would fit it (which is a cost to be considered, not a fatal flaw).
For what it’s worth I don’t consider this essay to be about “ops” that much. Also lots of people keep calling much of what Lightcone does “ops”, but we often really don’t think of ourselves as doing ops that much :) Some words that sound to me more like the thing I think of Lightcone as doing, most of the time: “uncertainty reduction”, “product discovery”, “trying to doing things fast”.
Some context on why I wanted Jacob to post this...
A lot of new alignment/EA orgs and projects have been popping up lately. The majority of these are founded by people who are either fresh out of college, or researchers who have been in academia basically their whole lives. In other words, they’re founded by people who have minimal experience with or exposure to ops and management (and the ops/management they do interface with tends to be mediocre at best). Interfacing with such orgs inevitably means interfacing with people who are bad at ops, and are figuring it out as they go (insofar as they figure it out at all).
A couple weeks ago, Akash wrote up his “AI Safety field-building projects I’d like to see”. I was struck by visions of dozens of new projects attempting one or another thing on that list with terrible ops, all turning into mangled versions of themselves in which the organizers were constantly fighting fires from start to end with mixed success, with the things which actually matter most going unaddressed for lack of slack and time, until the projects eventually just kind of dissolved. In the end, I sighed, and accepted that somebody was going to have to metaphorically sit the kids down for a talk.
… of course this rarely works, and inevitably the kids will need to fail for a while before they figure things out. Hopefully reading some things by people who (in my judgement) are good at ops will at least speed it up. In the meantime, here’s the One Thing I’d tell someone starting a project/org: if I were starting an org, the very first thing I would do is hire an ops person, preferably someone who I’ve interfaced with before and know to be competent. I’m not saying your first hire needs to be an ops person; great ops people are scarce, and you’ll probably do more of your own ops than I would aim for and develop the skills yourself. The point is that getting ops right would literally be the first thing on my todo list for a new org. It might not be the Most Important Thing, but it’s the Most Foundational Thing, and everything else will constantly be slowed down and falling apart and generating inconveniences if the ops foundation isn’t there.
It would also be interesting to see examples of what terrible ops looks like. As one of the “kids”, here are some examples of things that were bad in previous projects I worked on (some mistakes for which I was responsible):
- getting obsessed with some bad metric (eg: number of people who come to an event) and spending lots of hours getting that number up instead of thinking about why I was doing that
- so many meetings, calling a meeting if there’s any uncertainty about what to do next
- there being uncertainty about what to do next because team members lack context, don’t know who’s responsible for what, who is working on what, etc—
some people taking on too much responsibility and not being able to pass it on because having to explain how to do a task to someone else would itself take up too much time—
very disorganised meetings where everyone wanted to have a say and it wasn’t clear at the end of it what the action steps were—
an unwillingness for the person with the most context to take the role of explicitly telling others what concrete tasks to do (because the other team members were their friends and they didn’t want to be too bossy)
- There not being enough structure for team members to give feedback if they thought an idea or project someone else was very excited about would be useless, as a result some mini-projects getting incubated that people weren’t excited about or projects that predictably failed because the person who wanted to do them did not have enough information to help them figure out how to avoid the failure modes other team members would have been concerned about (lack of sharing intuitions really well). also, people picking projects and tasks to do for bad reasons rather than via structured thinking about priorities
- including too many people in meetings because “we’d love to get person X’s thoughts on our plans as well (and person Y and person Z...)”
- it being hard to trust that other team members would actually get things assigned to them done on time, partly because it was difficult to see partial progress without having to ask the person how things were going
- changing platforms and processes based on whims rather than figuring things out early and sticking with them
I think some of the things mentioned in the post are pretty helpful for avoiding some of these problems.
This is a valuable list to have written up, thanks!
I think this is a great goal and that this post furthers it. I do hope someone from a more staid institution writes a parallel document about how they work. Lightcone runs on a very chaotic/heroic model that works well for them and has accomplished great stuff, but some projects and people do better with a slower and steadier approach.
Can confirm Lightcone is very chaotic and sometimes works heroic hours, and it seems tangled up in our way of working for reasons that are not super clear to me.
So when reading your comment I was asking myself why the above template couldn’t be run by a project that wanted to work closer to 40 hours rather than 80 hours per week? One answer is that “Well, if people are importantly blocking elements, they must be available to reply on slack and unblock other people whenever”, which is mostly true for us, except that 1) we almost never wake up people who are sleeping :) and 2) if people sign-post they are taking a rest day or going on vacation others usually try fairly hard to find a way to solve problems without contacting them.
I agree that most of these rules could smoothly transition to a less intense team, and nonetheless believe a less chaotic org would write a fairly different document, which is why I think it would be useful for them to do so.
One thing I can speak to a tiny bit is a software company I worked at that had a lot of the chaotic/hero-ness in certain roles, but absolutely had to be cross-continental, and thus was also remote and heavily asynchronous. It built up really great practices for documentation and async communication to make that work. Alas it’s been too long since I worked there for me remember specifics, so I can’t say anything useful.
I don’t feel that we’re especially “chaotic” or would describe us as a chaotic org. We have lots of structure and process and principles and intentionality in how we operate. Though I suspect there’s something real you’re thinking of.
Oh fwiw I think we’re quite chaotic. Like, amount of suddenly changing priority and balls sometimes getting dropped because we took on too many things.
(Not sure we’re that chaotic for a startup, but startups are already pretty chaotic)
(Campus team seems much more chaotic than LW team tho)
high frequency?
For me, the interesting part is the transition from a fast-growing startup to a stable enterprise. I have been in transitional companies most of my professional life, and felt the growing pains. I have looked for resources to support this transition. One book provided a useful lens: Growing Pains: Transitioning from an Entrepreneurship to a Professionally Managed Firm by Eric G. Flamholtz and Yvonne Randle (Goodreads). It describes a number of stages or challenges that a maturing company has to tackle in a particular order of priority. It is mostly based on case studies and surveys.
(picture from this summary)
The Growth Stages are
New Venture—Markets and Products
Expansion—Resources and Operational Systems
Professionalization—Management Systems
Consolidation—Corporate Culture
In my experience, all the areas have to be advanced at the same time, but some more than others. For example, the clear instructions, mentioned above are a precursor to Corporate Culture. And it also makes sense to start with Operational Systems before you need them—at least to hire people with experience in these areas that will later be able to bring them about.
Pretty interesting, but I have a bunch of warning flags around the phrase “hire an ops person”, here’s a few thoughts:
Everyone on our team is a “generalist”, not an “ops person”. Anyone can do any task. When we run an event or space, responsibilities involve “venue setup” and also “preparing opening session”, and anyone should be able to be assigned to any task, or help with any task. (Of course, there’s explicitly one person owning each task, it’s fully clear who has final decision-making power, and who gets credit for failure/success.)
For new organizations I would bet on them much harder if all the cofounders could do all the work, than if the cofounders could each only do some of the work. And I’d bet against more strongly still if no single cofounder could do all of the work. I do think research orgs tend to have more of a split here, I’m not sure how good I think that is (I know multiple cases where it’s led to major dysfunction), I still think it’s best if there’s a person at the top who can do everything.
Historically when I’ve been told by a team-lead that “ops is not my responsibility, I’m not tightly in the loop on that” this has been closely correlated with there being an ops catastrophe somewhere that the team-lead is unable to fix, that’s been quite costly for the project.
Your point about giving space for newcomers to fail a while is a good one, and I agree. But I’m not of the same mind that “get better ops hires” is the method, as opposed to “relentlessly work until you yourself have the skill to ensure your project’s success”.
Especially if your ops hire doesn’t understand your work or your mission, and you aren’t likely to produce very legible signs of success (e.g. big funding, big publications, impressive product sales, etc), then it’s hard for that person to be motivated by the mission, and they may leave to do something for which they do have the generators. And then you’ll be left with a shell of an org built around them that nobody, not even you, really knows how it functions.
That said, I think if you’re weak in an area (e.g. in coding, or management, or other things), having a cofounder who is strong in that area is really excellent, and my best project collaborators have been people who, while in key ways are extremely similar to me, do have strengths where I have weaknesses and can pick up the slack there.
As a second example of Lightcone being generalists, when we’re evaluating how LessWrong/AF is doing, everyone should have an inside view and have engaged with the content, not only some of us e.g. at a recent team retreat, we spent an hour or two as a team debating whether the simulators post was right about the myopia of language models, and nobody wasn’t basically following the discussion.
Pardon the long scattershot of thoughts. I have a sense that people often try to ‘hire for ops roles’ poorly, and are confused about how to allocate work within a small team, so I wanted to say something about how it can go wrong and how else one can think about it.
Edit: I don’t think this applies for short-term contractors, it primarily applies to cofounders and other permanent hires.
given this notional use case (and the relative inexperience of the implied user), I think its even more important to (as Gunnar mentioned) contextualize this advice as to whom its for, and how they should use it.
doing that properly would take more than i have for this at the moment, but i’d appreciate epistemic tagging regarding things like;
this only could work at a new/small scale (for reasons including because the cost of keeping everyone 100% context scales with org size and because benefits don’t)
that strategy has to fit the employees you have, and this sort of strategy constrains the type of person you can hire to those who would fit it (which is a cost to be considered, not a fatal flaw).
For what it’s worth I don’t consider this essay to be about “ops” that much. Also lots of people keep calling much of what Lightcone does “ops”, but we often really don’t think of ourselves as doing ops that much :) Some words that sound to me more like the thing I think of Lightcone as doing, most of the time: “uncertainty reduction”, “product discovery”, “trying to doing things fast”.
I nod along here, but I’m not sure what “getting ops right” actually looks like. Can someone explain or point me to something I should read?
+1 on many of the projects in my list requiring a team with really good ops and (ideally) people with experience beyond college/academia.
(I hope I didn’t give off the impression that anyone can or should start those.)