The idea of having ChatGPT invent benchmarks can’t be tested by just asking it to, but I tried asking it to come up with a slightly more difficult intellectual challenge than writing easily debugged code. Its only two ideas seem to be:
Designing and implementing a new programming language that is easier to read and understand than existing languages, and has built-in features for debugging and error-checking.
Writing efficient and optimized algorithms for complex problems.
I don’t think either of these seem merely “slightly more difficult” than inventing easily debuggable code.
The idea of having ChatGPT invent benchmarks can’t be tested by just asking it to, but I tried asking it to come up with a slightly more difficult intellectual challenge than writing easily debugged code. Its only two ideas seem to be:
Designing and implementing a new programming language that is easier to read and understand than existing languages, and has built-in features for debugging and error-checking.
Writing efficient and optimized algorithms for complex problems.
I don’t think either of these seem merely “slightly more difficult” than inventing easily debuggable code.