This is probably obvious to you, but you can expand the working memory bottleneck by making lots of notes. You still need to store the “index” of the notes in your working memory though, to be able to get back to relevant ideas later. Making a good index includes compressing the ideas till you get the “core” insights into it.
Some part of what we consider intelligence is basically search and some part of what we consider faster search is basically compression.
Tbh you can also do multi-level indexing, the top-level index (crisp world model of everything) could be in working memory and it can point to indexes (crisp world model of a specific topic) actually written in your notes, which further point to more extensive notes on that topic.
As an aside, automated R&D using LLMs currently heavily relies on embedding search and RAG. AI’s context window is loosely analogous to human’s working memory in that way. AI knows millions of ideas but it can’t simulate pairwise interactions between all ideas as that would require too much GPU time. So it too needs to select some pairs or tuples of ideas (using embedding search or something similar) within which it can explore interactions.
The embedding dataset is a compressed version of the source dataset and the LLM itself is an even more compressed version of the source dataset. So there is interplay between data at different levels of compression.
(edited)
This is probably obvious to you, but you can expand the working memory bottleneck by making lots of notes. You still need to store the “index” of the notes in your working memory though, to be able to get back to relevant ideas later. Making a good index includes compressing the ideas till you get the “core” insights into it.
Some part of what we consider intelligence is basically search and some part of what we consider faster search is basically compression.
Tbh you can also do multi-level indexing, the top-level index (crisp world model of everything) could be in working memory and it can point to indexes (crisp world model of a specific topic) actually written in your notes, which further point to more extensive notes on that topic.
As an aside, automated R&D using LLMs currently heavily relies on embedding search and RAG. AI’s context window is loosely analogous to human’s working memory in that way. AI knows millions of ideas but it can’t simulate pairwise interactions between all ideas as that would require too much GPU time. So it too needs to select some pairs or tuples of ideas (using embedding search or something similar) within which it can explore interactions.
The embedding dataset is a compressed version of the source dataset and the LLM itself is an even more compressed version of the source dataset. So there is interplay between data at different levels of compression.