It seems to just do really bad with parentheses on their own. It can fix them with like… f(f(f(x))) but not ‘((())’ type situations (I’m just using the beta).
It seems to just do really bad with parentheses on their own
Interesting. I’d think this might be due to the BPE issue Gwern has written about. But it looked like Arthur added spaces in order to get around that, and you’ve got spaces in some of your examples too, so that’s not a full explanation.
EDIT: Oh, lol, he cites you in that section:
And Matt Brockman has observed, testing thousands of examples over several orders of magnitude, that GPT-3’s arithmetic ability appears to dramatically improve several-fold if you merely format numbers with commas instead of being purely numeric
It seems to just do really bad with parentheses on their own. It can fix them with like… f(f(f(x))) but not ‘((())’ type situations (I’m just using the beta).
Code: https://gist.github.com/brockmanmatt/aea4fc4a962188f85d83db761bf0ac50
Interesting. I’d think this might be due to the BPE issue Gwern has written about. But it looked like Arthur added spaces in order to get around that, and you’ve got spaces in some of your examples too, so that’s not a full explanation.
EDIT: Oh, lol, he cites you in that section:
I don’t think it’s a BPE issue but not sure. I’d guess it’s closer to the parity issue. It has a hard time implicitly counting in general.
edit: thanks, i know how to link now.
If you select some text that you want to be a link, then a little UI element should pop up that allows you to paste in the URL.