Why didn’t GPT-3.5 also copy it if it was in the training data?
Two possible answers:
The quine wasn’t in the training data of GPT-3.5 but was in the training data of GPT-4
GPT-4 is better at “retrieving” answers from the training data
That being said, I also briefly tried to search for this quine online and couldn’t find anything. So I agree, it probably does exhibit this new ability. The reason I was suspicious at first is because the quine prompt seemed generic enough that it could have existed before, but I see that’s not the case.
Sure, but the point is that those theories are much less likely than if GPT-3.5 had done it too.
I too was a bit surprised. Critch should probably have emphasized the hello-world twist a bit more: I don’t spend much time reading quines or recreational programming, so I was assuming it could’ve been memorized and wasn’t sure that that was ‘novel’ (there are lots of quine ‘genres’, like multilingual quines or ‘radiation-hardened’ quines) until I’d look through a bunch of results and noticed none of them had that. So his point is not that quines are somehow incredibly amazing & impossible to write hitherto, but that it’s gotten good enough at code-writing that it can meaningful modify & adapt quines.
Surely one should look for ones that are like “Quine given argument 1, output [something else] given argument 2”. The presence or absence of this sort of already very modular template being in the data would give better context.
Two possible answers:
The quine wasn’t in the training data of GPT-3.5 but was in the training data of GPT-4
GPT-4 is better at “retrieving” answers from the training data
That being said, I also briefly tried to search for this quine online and couldn’t find anything. So I agree, it probably does exhibit this new ability. The reason I was suspicious at first is because the quine prompt seemed generic enough that it could have existed before, but I see that’s not the case.
Sure, but the point is that those theories are much less likely than if GPT-3.5 had done it too.
I too was a bit surprised. Critch should probably have emphasized the hello-world twist a bit more: I don’t spend much time reading quines or recreational programming, so I was assuming it could’ve been memorized and wasn’t sure that that was ‘novel’ (there are lots of quine ‘genres’, like multilingual quines or ‘radiation-hardened’ quines) until I’d look through a bunch of results and noticed none of them had that. So his point is not that quines are somehow incredibly amazing & impossible to write hitherto, but that it’s gotten good enough at code-writing that it can meaningful modify & adapt quines.
Surely one should look for ones that are like “Quine given argument 1, output [something else] given argument 2”. The presence or absence of this sort of already very modular template being in the data would give better context.
GPT-4 could also be trained for more epochs, letting it “see” this example multiple times.