Note that while he has amazing recall, to me, it seems that he has much less synthesis than you might expect. I remember that only later in life did he start to draw some connections between data he consumed. He doesn’t seem to form an integrated world model from all of the data.
In that sense, Kim Peek is more like a database with free lookup while GPT-3 is more like a human in that it provides integrated answers but can’t quote verbatim in general (or tell where its knowledge is coming from).
Maybe there is a trade-off between integration and recall.
Still mulling this over. I may end up revising the post and/or writing a follow-up. :)
Note that while he has amazing recall, to me, it seems that he has much less synthesis than you might expect. I remember that only later in life did he start to draw some connections between data he consumed. He doesn’t seem to form an integrated world model from all of the data.
In that sense, Kim Peek is more like a database with free lookup while GPT-3 is more like a human in that it provides integrated answers but can’t quote verbatim in general (or tell where its knowledge is coming from).
Maybe there is a trade-off between integration and recall.