Wow, it’s been 7 months since this discussion and we have a new version of GPT which has suddenly improved GPT’s abilities . . . . a lot. It has a much longer ‘short term memory’, but still no ability to adjust its weights-‘long term memory’ as I understand it.
“GPT-4 is amazing at incremental tasks but struggles with discontinuous tasks” resulting from its memory handicaps. But they intend to fix that and also give it “agency and intrinsic motivation”.
Dangerous!
Also, I have changed my mind on whether I call the old GPT-3 still ‘intelligent’ after training has ended without the ability to change its ANN weights. I’m now inclined to say . . . it’s a crippled intelligence.
Wow, it’s been 7 months since this discussion and we have a new version of GPT which has suddenly improved GPT’s abilities . . . . a lot. It has a much longer ‘short term memory’, but still no ability to adjust its weights-‘long term memory’ as I understand it.
“GPT-4 is amazing at incremental tasks but struggles with discontinuous tasks” resulting from its memory handicaps. But they intend to fix that and also give it “agency and intrinsic motivation”.
Dangerous!
Also, I have changed my mind on whether I call the old GPT-3 still ‘intelligent’ after training has ended without the ability to change its ANN weights. I’m now inclined to say . . . it’s a crippled intelligence.
154 page paper: https://arxiv.org/pdf/2303.12712.pdf
Youtube summary of paper: