I’ll think more carefully about this later, but my instinctual response is that I don’t think this is nicely framed as a coordination problem. Basically, I don’t think you really lose that much when you “de-wheel,” in terms of actually effectiveness at EA-like goals. It’s worth noting that a major part of this article is that “wheeling” is incredibly costly (especially in opportunity).
What exactly do you think changes when everyone decides to de-wheel rather than just a few people? My read is that the biggest change is simply that de-wheeling becomes less stigmatized. There are cheaper ways to de-stigmatize de-wheeling than coordinating everyone to de-wheel at the same time.
My personal experience disagrees hard in magnitude but at a somewhat skew angle. I left the conventional track in some ways and seem to have mostly ruined my life as a result, but there are a lot of confounding forces there too; my goals are also more EA-like in feel than the most conventional goals I can think of, but I wouldn’t describe them centrally as that. One part that seems to hold up is that you lose a lot of ability to get into virtuous cycles that are dependent on material resources that are pinned to credential signaling by other market-equilibrium sorts of issues, to the point that I did a substantial amount of risky work on temporary self-modification to try to penetrate these cycles, which mostly failed. (I now notice that this is a possible strong subconscious bias which I should have revealed up front, and I apologize for not noticing that sooner.)
I specifically believe from the above experience that this is more true today than it was about fifteen years ago, but I’m not immediately sure how I’d justify it, and there’s a lot of potential bias in there too.
Some of Paul Graham’s essays exhibit a contrary viewpoint to mine which is more similar to yours, perhaps most centrally “After Credentials”. I notice that this is still kinda congruent with my (2), since that essay is from 2008. He describes it as being more true in 2008 than in ~1983, which I also think I agree with.
I think rank-biserial’s comment re “isn’t much of a middle path” points toward “there existing a recognized social role” as a sort-of coordination/common-knowledge aspect here.
Here’s a potential divergence I see. Do you believe the viability of skipping a conventionally credentialed path is more true for “EA-like” goals than for “mainstream” goals? If so, why is that, and are there some in-between goals that you could identify to make a spectrum out of?
Here’s a potential divergence I see. Do you believe the viability of skipping a conventionally credentialed path is more true for “EA-like” goals than for “mainstream” goals? If so, why is that, and are there some in-between goals that you could identify to make a spectrum out of?
This is slightly complicated.
If your goal is something like “become wealthy while having free time,” the Prep school->Fancy college->FIRE in finance or software path is actually pretty darn close to perfect.
If your goal is something like “make my parents proud” or “gain social validation,” you probably go down the same route too.
If your goal is something like “seek happiness” or “live a life where I am truly free” I think that the credentialing is something you probably need to get off ASAP. It confuses your reward mechanisms. There’s tons of warnings in pop culture about this.
If you have EA-like goals, you have a “maximize objective function”-type goal. It’s in the same shape as “become as rich as possible” or “make the world as horrible as possible.” Basically, the conventional path is highly highly unlikely to get you all the way there. In this case, you probably want to get into the
Get skills+resources
Use skills+resources to do impact
Repeat
Loop.
For a lot of important work, the resources required are minimal and you already have it. You only need skills. If you have skills, people will also give you resources.
It shouldn’t matter how much money you have.
Also, even if you were totally selfish, stopping apocalypse is better for you than earning extra money right now. If you believe the sort of AI arguments made on this forum, then it is probably directly irrational for you to optimize for things other than save the world.
So, do you think it’s instrumental to saving the world to focus on credentials? Perhaps it’s even a required part of this loop? (Perhaps you need credentials to get opportunities to get skills?)
I basically don’t think that is true. Even accepting colleges teach more effectively than people can learn auto-didactically, the amount of time wasted on bullshit and the amount of health wasted on stress probably makes this not true. It seems like you’d have to get very lucky for the credential mill to not be a significant skill cost.
--
I guess it’s worthwhile for me to reveal some weird personal biases too.
I’m personally a STEM student at a fancy college with a fancy (non-alignment) internship lined up. I actually love and am very excited about the internship (college somewhat less so. I might just be the wrong shape for college.), because I think it’ll give me a lot of extra skills.
My satisfaction with this doesn’t negate the fact that I mostly got those things by operating in a slightly different (more wheeled) mental model. A side effect of my former self being a little more wheeled is that I’d have to mess up even more badly to get into a seriously precarious situation. It’s probably easier for me to de-wheel at the current point, already having some signalling tools, then it is for the average person to de-wheel.
I’m not quite sure what cycles you were referring to (do you have examples?), but this might be me having a bad case of “this doesn’t apply to me so I will pay 0 thought to it,” and thus inadvertently burning a massive hole in my map.
Despite this though, that I probably mostly wish I de-wheeled earlier (middle school, when I first started thinking somewhat along these lines) rather than later. I’d be better at programming, better at math, and probably more likely to positively impact the world. At the expense of being less verbally eloquent and having less future money. I can’t say honestly that I would take the trade, but certainly a very large part of me wants to.
Certainly, I’d at least argue that Bob should de-wheel. The downside is quite limited.
--
There definitely is a middle path, though. Most of the AI alignment centers pay comparable salaries to top tech companies. You can start AI alignment companies and get funding, etc. There’s an entire gradient there. I also don’t entirely see how that is relevant.
ranked-biserial’s point was largely about Charlie, who wasn’t really a focus of the essay. What they said about Charlie might very much be correct. But it’s not given that Alice and Bob secretly want this. They may very well have done something else if not given the very conservative advice.
Interesting. I’ll want to look back at this later; it seems like I partially missed the point of your original post, but also it seems like there are potentially productive fuzzy conversations to be had more broadly?
To one aspect, and sorry in advance if I get rambly since I don’t have much time to edit down:
I’m not quite sure what cycles you were referring to (do you have examples?),
In short: the location/finance/legibility spiral, the employment/employment/legibility spiral, and the enormous energy needed to get back up if you fall down a class-marker level in enough ways. I don’t think I can expand on that without getting into the personal-ish version, so I’ll just go ahead and let you adjust/weaken for perspective. There’s a lot of potential fog of “which bits of world are accessible” here (but then, that’s to some degree part of the phenomenon I’m gesturing at, too).
Preamble: if you care about being influential then your problems skew a lot more toward a social-reality orientation than if you primarily care about doing good work in a more purely abstract sense. I decided long ago for myself that not caring enough about being influential in a more direct sense was likely to create problems with misinterpretation and value skew where even if I did work that had a shot at making an impact on the target, the effective result of any popularization of it might not be something I could meaningfully steer. In particular, this means I don’t expect the “live cheaply somewhere remote and put out papers while doing almost all my collaboration electronically” approach to work, at least at this point in my career.
Caveat: currently, I think I’ve likely overshot in terms of mindset for medium-term benefit even in terms of social reality (mostly due to risk aversion of the kind you disapprove of and due to the way a bunch of signaling is anti-inductive). I am deeply conflicted as to how much to backtrack on or abandon.
First cycle: Several social and career needs might be better met by moving to a specific place. That place has a high cost of living due to big-city amplification effects, which is a big obstacle in itself—but it’s not just the cost, but things like default tenant-landlord relationships and the signaling involved in that. It’s having the pay stub so you can qualify to rent housing, and having that pay stub come from the right place, and so on. Ability to work around this is limited; alternative documentation usually requires an order of magnitude longer of demonstrated, documented stability, and gaining access via local networks of people has a bootstrapping problem.
Second cycle: I see a lot of talk around some labor markets (especially in software work, which seems very common in this social sphere) currently being heavily worker-tilted, but I’ve still not seen much way to get in on skills alone, especially because it’s not just technical skill, it’s the remaining 90% of the work that involves having practiced collaborating and taking roles in an organization in the ‘right’ way so they don’t have to completely train you up for that. There’s plenty of market for people with three years of legible, verifiable, full-time experience, and almost nothing otherwise. This is classic “you need a job to get a job”, and if your existing role is of the wrong kind, you’re on the treadmill of that alternate track and need a massive pile of slack to switch around.
The above two amplify each other a lot, because the former of them gives you a lot of random-chance opportunity to try to get past barriers to the latter and the latter gets you the socioeconomic legibility for the former. For some corroboration, Patrick McKenzie talks about hiring in the software industry: (1), (2), (3) with some tactics for how to work within this. He specifically notes in (3) that “plausible” is essentially binary and recommends something congruent with your “It’s probably easier for me to de-wheel at the current point, already having some signalling tools, then it is for the average person to de-wheel.” in terms of getting past a threshold first (which is similar to the type of advice you get upset at in the OP).
Now, if you’re talking from purely an alignment perspective, and most work in alignment is currently theoretical and doesn’t benefit much from the above, and organizations funding it and people doing it manage to avoid adhesion to similar phenomena in selection, then you have a much better case for not caring much.
I’m personally a STEM student at a fancy college with a fancy (non-alignment) internship lined up.
Being a student is notably a special case that gets you a lot of passes, because that’s the perceived place in life where you’re ‘supposed to’ not have everything yet. Once you’re past the student phase, you get very little slack. This is especially true in terms of lack of external-system slack in mentoring/integration capacity—the above induction into the ‘right kind’ of experience is slack that is explicitly given to interns, but then selecting for anyone else alongside that is expensive, so if they can fill all their intake needs from student bodies, and you don’t have the “I am a student” pass yourself, you lose by default.
A recent BBC Worklife article loosely corroborates my impression of job market conditions and early-career mentorship bottlenecks.
Although a war for talent is certainly raging, employers aren’t fighting the same battles across the board. Only some candidates have power in the job market – typically experienced, mid-career employees. It means entry-level workers can still face difficulties finding employment – and this is especially the situation in certain sectors.
In many cases, labour shortages mean companies are offering flexible working arrangements to secure talent. Grace Lordan, director of the Inclusion Initiative at the London School of Economics, says this practice can further restrict opportunities for inexperienced candidates.
“If hybrid working is implemented, it makes more sense to hire someone with experience: an employee you know can just get on with the job working from home,” adds Lordan. “Managers need more time to train entry-level workers and show what good performance looks like. With employees often time-poor at the biggest firms, it’s not surprising that we’re seeing some inexperienced workers struggle in the job market.”
If you have EA-like goals, you have a “maximize objective function”-type goal. It’s in the same shape as “become as rich as possible” or “make the world as horrible as possible.” Basically, the conventional path is highly highly unlikely to get you all the way there. In this case, you probably want to get into the
Get skills+resources
Use skills+resources to do impact
Repeat
Loop.
I’m in a similar situation to yours. (I’m currently in the “Bob” stage of the Alice → Bob → Charlie pipeline.) How do you propose I, and those like us, go about doing step 1 without entering the “software/finance hamster wheel”? Are we supposed to found a dozen start-ups until one of them works? Are we supposed to find and exploit some massive inefficiency in crypto pricing, all by ourselves? Please, do tell.
True, if we’re talking solely about alignment. If we’re talking about the larger space of, as you put them, “maximize objective function”-type goals, then there’s plenty of demand for resources. Let’s say I wanna do (actually effective) longevity research. Since the competition for grant money is (like most things) Goodharted and broken, and because I don’t have enough biology credentials, I’m gonna need to self-fund in order to buy lab materials and grad student slaves.
Ok, sick. I largely agree with you btw (about the hamster wheel being corrosive). If I came off as agressive, fyi, I liked the spirit of your post a lot, and I strong-upvoted it.
I’ll think more carefully about this later, but my instinctual response is that I don’t think this is nicely framed as a coordination problem. Basically, I don’t think you really lose that much when you “de-wheel,” in terms of actually effectiveness at EA-like goals. It’s worth noting that a major part of this article is that “wheeling” is incredibly costly (especially in opportunity).
What exactly do you think changes when everyone decides to de-wheel rather than just a few people? My read is that the biggest change is simply that de-wheeling becomes less stigmatized. There are cheaper ways to de-stigmatize de-wheeling than coordinating everyone to de-wheel at the same time.
Edit: first sentence, for clarity
For a few other scattered reference points:
My personal experience disagrees hard in magnitude but at a somewhat skew angle. I left the conventional track in some ways and seem to have mostly ruined my life as a result, but there are a lot of confounding forces there too; my goals are also more EA-like in feel than the most conventional goals I can think of, but I wouldn’t describe them centrally as that. One part that seems to hold up is that you lose a lot of ability to get into virtuous cycles that are dependent on material resources that are pinned to credential signaling by other market-equilibrium sorts of issues, to the point that I did a substantial amount of risky work on temporary self-modification to try to penetrate these cycles, which mostly failed. (I now notice that this is a possible strong subconscious bias which I should have revealed up front, and I apologize for not noticing that sooner.)
I specifically believe from the above experience that this is more true today than it was about fifteen years ago, but I’m not immediately sure how I’d justify it, and there’s a lot of potential bias in there too.
Some of Paul Graham’s essays exhibit a contrary viewpoint to mine which is more similar to yours, perhaps most centrally “After Credentials”. I notice that this is still kinda congruent with my (2), since that essay is from 2008. He describes it as being more true in 2008 than in ~1983, which I also think I agree with.
I think rank-biserial’s comment re “isn’t much of a middle path” points toward “there existing a recognized social role” as a sort-of coordination/common-knowledge aspect here.
Here’s a potential divergence I see. Do you believe the viability of skipping a conventionally credentialed path is more true for “EA-like” goals than for “mainstream” goals? If so, why is that, and are there some in-between goals that you could identify to make a spectrum out of?
This is slightly complicated.
If your goal is something like “become wealthy while having free time,” the Prep school->Fancy college->FIRE in finance or software path is actually pretty darn close to perfect.
If your goal is something like “make my parents proud” or “gain social validation,” you probably go down the same route too.
If your goal is something like “seek happiness” or “live a life where I am truly free” I think that the credentialing is something you probably need to get off ASAP. It confuses your reward mechanisms. There’s tons of warnings in pop culture about this.
If you have EA-like goals, you have a “maximize objective function”-type goal. It’s in the same shape as “become as rich as possible” or “make the world as horrible as possible.” Basically, the conventional path is highly highly unlikely to get you all the way there. In this case, you probably want to get into the
Get skills+resources
Use skills+resources to do impact
Repeat
Loop.
For a lot of important work, the resources required are minimal and you already have it. You only need skills. If you have skills, people will also give you resources.
It shouldn’t matter how much money you have.
Also, even if you were totally selfish, stopping apocalypse is better for you than earning extra money right now. If you believe the sort of AI arguments made on this forum, then it is probably directly irrational for you to optimize for things other than save the world.
So, do you think it’s instrumental to saving the world to focus on credentials? Perhaps it’s even a required part of this loop? (Perhaps you need credentials to get opportunities to get skills?)
I basically don’t think that is true. Even accepting colleges teach more effectively than people can learn auto-didactically, the amount of time wasted on bullshit and the amount of health wasted on stress probably makes this not true. It seems like you’d have to get very lucky for the credential mill to not be a significant skill cost.
--
I guess it’s worthwhile for me to reveal some weird personal biases too.
I’m personally a STEM student at a fancy college with a fancy (non-alignment) internship lined up. I actually love and am very excited about the internship (college somewhat less so. I might just be the wrong shape for college.), because I think it’ll give me a lot of extra skills.
My satisfaction with this doesn’t negate the fact that I mostly got those things by operating in a slightly different (more wheeled) mental model. A side effect of my former self being a little more wheeled is that I’d have to mess up even more badly to get into a seriously precarious situation. It’s probably easier for me to de-wheel at the current point, already having some signalling tools, then it is for the average person to de-wheel.
I’m not quite sure what cycles you were referring to (do you have examples?), but this might be me having a bad case of “this doesn’t apply to me so I will pay 0 thought to it,” and thus inadvertently burning a massive hole in my map.
Despite this though, that I probably mostly wish I de-wheeled earlier (middle school, when I first started thinking somewhat along these lines) rather than later. I’d be better at programming, better at math, and probably more likely to positively impact the world. At the expense of being less verbally eloquent and having less future money. I can’t say honestly that I would take the trade, but certainly a very large part of me wants to.
Certainly, I’d at least argue that Bob should de-wheel. The downside is quite limited.
--
There definitely is a middle path, though. Most of the AI alignment centers pay comparable salaries to top tech companies. You can start AI alignment companies and get funding, etc. There’s an entire gradient there. I also don’t entirely see how that is relevant.
ranked-biserial’s point was largely about Charlie, who wasn’t really a focus of the essay. What they said about Charlie might very much be correct. But it’s not given that Alice and Bob secretly want this. They may very well have done something else if not given the very conservative advice.
I’ll reply to ranked-biserial later.
Edit: typo
Interesting. I’ll want to look back at this later; it seems like I partially missed the point of your original post, but also it seems like there are potentially productive fuzzy conversations to be had more broadly?
To one aspect, and sorry in advance if I get rambly since I don’t have much time to edit down:
In short: the location/finance/legibility spiral, the employment/employment/legibility spiral, and the enormous energy needed to get back up if you fall down a class-marker level in enough ways. I don’t think I can expand on that without getting into the personal-ish version, so I’ll just go ahead and let you adjust/weaken for perspective. There’s a lot of potential fog of “which bits of world are accessible” here (but then, that’s to some degree part of the phenomenon I’m gesturing at, too).
Preamble: if you care about being influential then your problems skew a lot more toward a social-reality orientation than if you primarily care about doing good work in a more purely abstract sense. I decided long ago for myself that not caring enough about being influential in a more direct sense was likely to create problems with misinterpretation and value skew where even if I did work that had a shot at making an impact on the target, the effective result of any popularization of it might not be something I could meaningfully steer. In particular, this means I don’t expect the “live cheaply somewhere remote and put out papers while doing almost all my collaboration electronically” approach to work, at least at this point in my career.
Caveat: currently, I think I’ve likely overshot in terms of mindset for medium-term benefit even in terms of social reality (mostly due to risk aversion of the kind you disapprove of and due to the way a bunch of signaling is anti-inductive). I am deeply conflicted as to how much to backtrack on or abandon.
First cycle: Several social and career needs might be better met by moving to a specific place. That place has a high cost of living due to big-city amplification effects, which is a big obstacle in itself—but it’s not just the cost, but things like default tenant-landlord relationships and the signaling involved in that. It’s having the pay stub so you can qualify to rent housing, and having that pay stub come from the right place, and so on. Ability to work around this is limited; alternative documentation usually requires an order of magnitude longer of demonstrated, documented stability, and gaining access via local networks of people has a bootstrapping problem.
Second cycle: I see a lot of talk around some labor markets (especially in software work, which seems very common in this social sphere) currently being heavily worker-tilted, but I’ve still not seen much way to get in on skills alone, especially because it’s not just technical skill, it’s the remaining 90% of the work that involves having practiced collaborating and taking roles in an organization in the ‘right’ way so they don’t have to completely train you up for that. There’s plenty of market for people with three years of legible, verifiable, full-time experience, and almost nothing otherwise. This is classic “you need a job to get a job”, and if your existing role is of the wrong kind, you’re on the treadmill of that alternate track and need a massive pile of slack to switch around.
The above two amplify each other a lot, because the former of them gives you a lot of random-chance opportunity to try to get past barriers to the latter and the latter gets you the socioeconomic legibility for the former. For some corroboration, Patrick McKenzie talks about hiring in the software industry: (1), (2), (3) with some tactics for how to work within this. He specifically notes in (3) that “plausible” is essentially binary and recommends something congruent with your “It’s probably easier for me to de-wheel at the current point, already having some signalling tools, then it is for the average person to de-wheel.” in terms of getting past a threshold first (which is similar to the type of advice you get upset at in the OP).
Now, if you’re talking from purely an alignment perspective, and most work in alignment is currently theoretical and doesn’t benefit much from the above, and organizations funding it and people doing it manage to avoid adhesion to similar phenomena in selection, then you have a much better case for not caring much.
Being a student is notably a special case that gets you a lot of passes, because that’s the perceived place in life where you’re ‘supposed to’ not have everything yet. Once you’re past the student phase, you get very little slack. This is especially true in terms of lack of external-system slack in mentoring/integration capacity—the above induction into the ‘right kind’ of experience is slack that is explicitly given to interns, but then selecting for anyone else alongside that is expensive, so if they can fill all their intake needs from student bodies, and you don’t have the “I am a student” pass yourself, you lose by default.
A recent BBC Worklife article loosely corroborates my impression of job market conditions and early-career mentorship bottlenecks.
I’m in a similar situation to yours. (I’m currently in the “Bob” stage of the Alice → Bob → Charlie pipeline.) How do you propose I, and those like us, go about doing step 1 without entering the “software/finance hamster wheel”? Are we supposed to found a dozen start-ups until one of them works? Are we supposed to find and exploit some massive inefficiency in crypto pricing, all by ourselves? Please, do tell.
My Solution
--
Optimize skill-building over resource-collection. You don’t need that many resources.
Ask:
What is the skill I’m most interested in building right now?
What’s the best way to get this skill?
A couple of things:
Open source libraries are free tutoring
Most alignment-relevant skills can be effectively self-taught
Projects are learning opportunities that demand mastery
Tutoring is cheaper and possibly more effective than college tuition
True, if we’re talking solely about alignment. If we’re talking about the larger space of, as you put them, “maximize objective function”-type goals, then there’s plenty of demand for resources. Let’s say I wanna do (actually effective) longevity research. Since the competition for grant money is (like most things) Goodharted and broken, and because I don’t have enough biology credentials, I’m gonna need to self-fund in order to buy lab materials and grad student slaves.
There is no universal advice that I can give.
The problem is that people are assuming that wheeling is correct without checking that it is.
I’m not proposing developing an allergic reaction to colleges or something.
Ok, sick. I largely agree with you btw (about the hamster wheel being corrosive). If I came off as agressive, fyi, I liked the spirit of your post a lot, and I strong-upvoted it.
Cool!