nostalgebraist’s blog is a must-read regarding GPT-x, including GPT-3. Perhaps, start here (“the transformer… ‘explained’?”), which helps to contextualize GPT-x within the history of machine learning.
(Though, I should note that nostalgebraist holds a contrarian “bearish” position on GPT-3 in particular; for the “bullish” case instead, read Gwern.)
Thanks for the answer! I knew about the “transformer explained” post, but I was not aware of its author’s position on GPT-3.
nostalgebraist’s blog is a must-read regarding GPT-x, including GPT-3. Perhaps, start here (“the transformer… ‘explained’?”), which helps to contextualize GPT-x within the history of machine learning.
(Though, I should note that nostalgebraist holds a contrarian “bearish” position on GPT-3 in particular; for the “bullish” case instead, read Gwern.)
Thanks for the answer! I knew about the “transformer explained” post, but I was not aware of its author’s position on GPT-3.