Hence, I would argue that “AGI-proof” jobs are unlikely to ever provide an income basis for a significant share of the human population.
For the categories of AGI-proof jobs that you discuss, I agree (and I much enjoyed your detailed exposition of some examples). However, in my post that you very you kindly cite, there is one AGI-proof job category that could be an exception to that, if there turned out to be sufficient demand from the AI side of the economy, my category 3:
”Giving human feedback/input/supervision to/of AI/robotic work/models/training data, in order to improve, check, or confirm its quality.”
Given the progress being made in synthetic training data, and that the AIs then being trained are likely to be far smarter than any human, the demand for this as training could drop, or increase, fairly rapidly. However, if we’re not actually extinct, presumably that means we solved the alignment problem, in which case AIs will be extremely interested in human values, and the only source of original new data about human values is from humans. So this is the one product that aligned AIs need that only humans can produce — and any human can produce it, not just a skilled expert. If AI demand for this was high enough, it could maintain full human employment, with basically everyone filling our surveys and being parts of focus groups, or whatever.
My guess is that this category of job is likely not to exist, both for alignment generalizing further than capabilities reasons, and the fact that I think the synthetic data pipeline for alignment is plausibly automatable as well, so I consider it a short-term solution at best.
I am very unsure about this category of job: I can see reasons for the demand for it to grow exponentially, or to go away, or to be perennial. Which of these effects dominates is on the far side of a Singularity. My prior here is to just use the uniform prior: any of these things could happen.
I had a non-uniform prior already on the category of job you’re suggesting, because I had learned things that were relevant to the category of job that made me update away from it being a full-time job for most humans.
For the categories of AGI-proof jobs that you discuss, I agree (and I much enjoyed your detailed exposition of some examples). However, in my post that you very you kindly cite, there is one AGI-proof job category that could be an exception to that, if there turned out to be sufficient demand from the AI side of the economy, my category 3:
”Giving human feedback/input/supervision to/of AI/robotic work/models/training data, in order to improve, check, or confirm its quality.”
Given the progress being made in synthetic training data, and that the AIs then being trained are likely to be far smarter than any human, the demand for this as training could drop, or increase, fairly rapidly. However, if we’re not actually extinct, presumably that means we solved the alignment problem, in which case AIs will be extremely interested in human values, and the only source of original new data about human values is from humans. So this is the one product that aligned AIs need that only humans can produce — and any human can produce it, not just a skilled expert. If AI demand for this was high enough, it could maintain full human employment, with basically everyone filling our surveys and being parts of focus groups, or whatever.
My guess is that this category of job is likely not to exist, both for alignment generalizing further than capabilities reasons, and the fact that I think the synthetic data pipeline for alignment is plausibly automatable as well, so I consider it a short-term solution at best.
I am very unsure about this category of job: I can see reasons for the demand for it to grow exponentially, or to go away, or to be perennial. Which of these effects dominates is on the far side of a Singularity. My prior here is to just use the uniform prior: any of these things could happen.
I had a non-uniform prior already on the category of job you’re suggesting, because I had learned things that were relevant to the category of job that made me update away from it being a full-time job for most humans.