My guess is that this category of job is likely not to exist, both for alignment generalizing further than capabilities reasons, and the fact that I think the synthetic data pipeline for alignment is plausibly automatable as well, so I consider it a short-term solution at best.
I am very unsure about this category of job: I can see reasons for the demand for it to grow exponentially, or to go away, or to be perennial. Which of these effects dominates is on the far side of a Singularity. My prior here is to just use the uniform prior: any of these things could happen.
I had a non-uniform prior already on the category of job you’re suggesting, because I had learned things that were relevant to the category of job that made me update away from it being a full-time job for most humans.
My guess is that this category of job is likely not to exist, both for alignment generalizing further than capabilities reasons, and the fact that I think the synthetic data pipeline for alignment is plausibly automatable as well, so I consider it a short-term solution at best.
I am very unsure about this category of job: I can see reasons for the demand for it to grow exponentially, or to go away, or to be perennial. Which of these effects dominates is on the far side of a Singularity. My prior here is to just use the uniform prior: any of these things could happen.
I had a non-uniform prior already on the category of job you’re suggesting, because I had learned things that were relevant to the category of job that made me update away from it being a full-time job for most humans.