I generally agree with this thinking, although I’ll highlight that the brain and a hypothetical AI might not use the same primitives—they’re on very different hardware, after all. Certainly the general strategy of “start with a few primitives, and see if they can represent all these other things” is the sort of strategy I’m after. I currently consider causal DAGs with symmetry the most promising primitive. It directly handles causality and recursion, and alongside a suitable theory of abstraction, I expect it will allow us to represent things like spatial/temporal relations, hierarchies, analogies, composition, and many others, all in a unified framework.
On further reflection, the primitive of “temporal sequences” (more specifically high-order Markov chains) isn’t that different from cause-effect. High-order Markov chains are like “if A happens and then B and then C, probably D will happen next”. So if A and B and C are a person moving to kick a ball, and D is the ball flying up in the air...well I guess that’s at least partway to representing cause-effect...
(High-order Markov chains are more general than cause-effect because they can also represent non-causal things like the lyrics of a song. But in the opposite direction, I’m having trouble thinking of a cause-effect relation that can not be represented as a high-order Markov chain, at least at some appropriate level of abstraction, and perhaps with some context-dependence of the transitions.)
I have pretty high confidence that high-order Markov chains are one of the low-level primitives of the brain, based on both plausible neural mechanisms and common sense (e.g. it’s hard to say the letters of the alphabet in reverse order). I’m less confident about what exactly are the elements of those Markov chains, and what are the other low-level primitives, and what’s everything else that’s going on. :-)
I’ll highlight that the brain and a hypothetical AI might not use the same primitives—they’re on very different hardware, after all
Sure. There are a lot of levels at which algorithms can differ.
quicksort.c compiled by clang versus quicksort.c compiled by gcc
quicksort optimized to run on a CPU vs quicksort optimized to run on an FPGA
quicksort running on a CPU vs mergesort running on an FPGA
quicksort vs a different algorithm that doesn’t involve sorting the list at all
There are people working on neuromorphic hardware, but I don’t put much stock in anything coming of it in terms of AGI (the main thrust of that field is low-power sensors). So I generally think it’s very improbable that we would copy brain algorithms at the level of firing patterns and synapses (like the first bullet-point or less). I put much more weight on the possibility of “copying” brain algorithms at like vaguely the second or third bullet-point level. But, of course, it’s also entirely possible for an AGI to be radically different from brain algorithms in every way. :-)
My guess here is that there are some instrumentally convergent abstractions/algorithms which both a brain and a hypothetical AGI needs to use. But a brain will have implemented some of those as hacks on top of methods which evolved earlier, whereas an AI could implement those methods directly. So for instance, one could imagine the brain implementing simple causal reasoning as a hack on top of pre-existing temporal sequence capabilities. When designing an AI, it would probably make more sense to use causal DAGs as the fundamental, and then implement temporal sequences as abstract stick-dags which don’t support many (if any) counterfactuals.
Possibly better example: tree search and logic. Humans seem to handle these mostly as hacks on top of pattern-matchers and trigger-action pairs, but for an AI it makes more sense to implement tree search as a fundamental.
I generally agree with this thinking, although I’ll highlight that the brain and a hypothetical AI might not use the same primitives—they’re on very different hardware, after all. Certainly the general strategy of “start with a few primitives, and see if they can represent all these other things” is the sort of strategy I’m after. I currently consider causal DAGs with symmetry the most promising primitive. It directly handles causality and recursion, and alongside a suitable theory of abstraction, I expect it will allow us to represent things like spatial/temporal relations, hierarchies, analogies, composition, and many others, all in a unified framework.
On further reflection, the primitive of “temporal sequences” (more specifically high-order Markov chains) isn’t that different from cause-effect. High-order Markov chains are like “if A happens and then B and then C, probably D will happen next”. So if A and B and C are a person moving to kick a ball, and D is the ball flying up in the air...well I guess that’s at least partway to representing cause-effect...
(High-order Markov chains are more general than cause-effect because they can also represent non-causal things like the lyrics of a song. But in the opposite direction, I’m having trouble thinking of a cause-effect relation that can not be represented as a high-order Markov chain, at least at some appropriate level of abstraction, and perhaps with some context-dependence of the transitions.)
I have pretty high confidence that high-order Markov chains are one of the low-level primitives of the brain, based on both plausible neural mechanisms and common sense (e.g. it’s hard to say the letters of the alphabet in reverse order). I’m less confident about what exactly are the elements of those Markov chains, and what are the other low-level primitives, and what’s everything else that’s going on. :-)
Just thinking out loud :-)
Sure. There are a lot of levels at which algorithms can differ.
quicksort.c compiled by clang versus quicksort.c compiled by gcc
quicksort optimized to run on a CPU vs quicksort optimized to run on an FPGA
quicksort running on a CPU vs mergesort running on an FPGA
quicksort vs a different algorithm that doesn’t involve sorting the list at all
There are people working on neuromorphic hardware, but I don’t put much stock in anything coming of it in terms of AGI (the main thrust of that field is low-power sensors). So I generally think it’s very improbable that we would copy brain algorithms at the level of firing patterns and synapses (like the first bullet-point or less). I put much more weight on the possibility of “copying” brain algorithms at like vaguely the second or third bullet-point level. But, of course, it’s also entirely possible for an AGI to be radically different from brain algorithms in every way. :-)
My guess here is that there are some instrumentally convergent abstractions/algorithms which both a brain and a hypothetical AGI needs to use. But a brain will have implemented some of those as hacks on top of methods which evolved earlier, whereas an AI could implement those methods directly. So for instance, one could imagine the brain implementing simple causal reasoning as a hack on top of pre-existing temporal sequence capabilities. When designing an AI, it would probably make more sense to use causal DAGs as the fundamental, and then implement temporal sequences as abstract stick-dags which don’t support many (if any) counterfactuals.
Possibly better example: tree search and logic. Humans seem to handle these mostly as hacks on top of pattern-matchers and trigger-action pairs, but for an AI it makes more sense to implement tree search as a fundamental.
Sounds exciting, and I wish you luck and look forward to reading whatever you come up with! :-)