You don’t necessarily need to know the correct answer beforehand to be able to validate whether or not an answer is correct. If we take Eliezer’s problem of generating text that matches a given hash value, it’s easy to validate whether an answer is true or not even if you don’t know the answer beforehand.
What’s important is that the AI is sometimes able to generate correct answers. If the criteria for a correct answer are well-defined enough it can go from solving a problem 1% of the time correctly to solving it 100% of the time correctly.
ChatGPT is used by millions of people and a good portion of that will click the feedback button, especially if they optimize their UI for that. It’s possible to build automated processes that will look at the problems where it currently frequently makes mistakes and learn to avoid them. It is possible to build a self-improving system around that.
If you let it do that for 10,000 different problems I would expect that it learns some reasoning habits that generalize and are useful for solving other problems as well.
You don’t necessarily need to know the correct answer beforehand to be able to validate whether or not an answer is correct. If we take Eliezer’s problem of generating text that matches a given hash value, it’s easy to validate whether an answer is true or not even if you don’t know the answer beforehand.
What’s important is that the AI is sometimes able to generate correct answers. If the criteria for a correct answer are well-defined enough it can go from solving a problem 1% of the time correctly to solving it 100% of the time correctly.
ChatGPT is used by millions of people and a good portion of that will click the feedback button, especially if they optimize their UI for that. It’s possible to build automated processes that will look at the problems where it currently frequently makes mistakes and learn to avoid them. It is possible to build a self-improving system around that.
If you let it do that for 10,000 different problems I would expect that it learns some reasoning habits that generalize and are useful for solving other problems as well.