My view that a cheap simulation of arbitrary human experts would be enough to end life as we know it one way or the other?
Just to add to this : many experts are just faking it. Simulating them is not helping. By faking it, because they are solving as humans an RL problem that can’t be solved, their learned policy is deeply suboptimal and in some cases simply wrong. Think expert positions like in social science, government, law, economics, business consulting, and possibly even professors who chair computer science departments but are not actually working on scaled cutting edge AI. Each of these “experts” cannot know a true policy that is effective, most of their status comes from various social proofs and finite Official Positions. The “cannot” because they will not in their lifespan receive enough objective feedback to learn a policy that is definitely correct. (they are more likely to be correct than non experts, however)
(In the subsequent text it seems like you are saying that you don’t need to match human experts in every domain in order to have a transformative impact, which I agree with. I set the TAI threshold as “economic impact as large as” but believe that this impact will be achieved by systems which are in some respects weaker than human experts and in other respects stronger/faster/cheaper than humans.)
I pointed out that you do not need to match human experts in any domain at all. Transformation depends on entirely different variables.
My view that a cheap simulation of arbitrary human experts would be enough to end life as we know it one way or the other?
Just to add to this : many experts are just faking it. Simulating them is not helping. By faking it, because they are solving as humans an RL problem that can’t be solved, their learned policy is deeply suboptimal and in some cases simply wrong. Think expert positions like in social science, government, law, economics, business consulting, and possibly even professors who chair computer science departments but are not actually working on scaled cutting edge AI. Each of these “experts” cannot know a true policy that is effective, most of their status comes from various social proofs and finite Official Positions. The “cannot” because they will not in their lifespan receive enough objective feedback to learn a policy that is definitely correct. (they are more likely to be correct than non experts, however)
(In the subsequent text it seems like you are saying that you don’t need to match human experts in every domain in order to have a transformative impact, which I agree with. I set the TAI threshold as “economic impact as large as” but believe that this impact will be achieved by systems which are in some respects weaker than human experts and in other respects stronger/faster/cheaper than humans.)
I pointed out that you do not need to match human experts in any domain at all. Transformation depends on entirely different variables.