Thanks for the thoughtful response, that perspective makes sense. I take your point that ACT-R is unique in the ways you’re describing, and that most cognitive scientists are not working on overarching models of the mind like that. I think maybe our disagreement is about how good/useful of an overarching model ACT-R is? It’s definitely not like in physics, where some overarching theories are widely accepted (e.g. the standard model) even by people working on much more narrow topics—and many of the ones that aren’t (e.g. string theory) are still widely known about and commonly taught. The situation in cog sci (in my view, and I think in many people’s views?) is much more that we don’t have an overarching model of the mind in anywhere close to the level of detail/mechanistic specificity that ACT-R posits, and that any such attempt would be premature/foolish/not useful right now. Like, I think if you polled cognitive scientists, the vast majority would disagree with the title of your post—not because they think there’s a salient alternative, but because they think that there is no theory that even comes close to meriting the title of “best-validated theory of cognition” (even if technically one theory is ahead of the others). Do you know what I mean? Of course, even if most cognitive scientists don’t believe in ACT-R in that way, that alone doesn’t mean that ACT-R is wrong.. I’m curious about the evidence that Terry is talking about above. I just think the field would look really, really different if we actually had a halfway-decent paradigm/overarching model of the mind. And it’s not like ACT-R is some unknown idea that is poised to take over the field once people learn about it. Everyone knew about it in the 90s, and then it fell out of widespread use—and my prior on why that happened is that people weren’t finding it super useful. (Although like I said, I’m really curious to learn more about what Terry/other contemporary people are doing with it!)
I agree that there isn’t an overarching theory at the level of specificity of ACT-R that covers all the different aspects of the mind that cognitive science researchers wish it would cover. And so yes, I can see cognitive scientists saying that there is no such theory, or (more accurately) saying that even though ACT-R is the best-validated one, it’s not validated on the particular types of tasks that they’re interested in, so therefore they can ignore it.
However, I do think that there’s enough of a consensus about some aspects of ACT-R (and other theories) that there are some broader generalizations that all cognitive scientists should be aware of. That’s the point of the two papers listed in the original post on the “Common Model of Cognition”. They dig through a whole bunch of different cognitive architectures and ideas over the decades and point out that there are some pretty striking commonalities and similarities across these models. (ACT-R is just one of the theories that they look at, and they point out that there are a set of commonalities across all the theories, and that’s what they call the Common Model of Cognition). The Common Model of Cognition is much more loosely specified and is much more about structural organization rather than being about the particular equations used, though, so I’d still say that ACT-R is the best-validated model. But CMC is surprisingly consistent with a lot of models, and that’s why the community is getting together to write papers like that. The whole point is to try to show that there are some things that we can say right now about an overarching theory of the mind, even if people don’t want to buy into the particular details of ACT-R. And if people are trying to build overarching theories, they should at least be aware of what there is already.
(Full disclosure: I was at the 2017 meeting where this community came together on this topic and started the whole CMC thing. The papers from that meeting are at https://www.aaai.org/Library/Symposia/Fall/fs17-05.php and that’s a great collection of short papers of people talking about the various challenges of expanding the CMC. The general consensus from that meeting is that it was useful to at least have an explicit CMC to help frame that conversation, and it’s been great to see that conversation grow over the last few years. Note: at the time we were calling it the Standard Model of the Mind, but that got changed to Common Model of Cognition).
I think maybe our disagreement is about how good/useful of an overarching model ACT-R is? It’s definitely not like in physics, where some overarching theories are widely accepted (e.g. the standard model) even by people working on much more narrow topics—and many of the ones that aren’t (e.g. string theory) are still widely known about and commonly taught. The situation in cog sci (in my view, and I think in many people’s views?) is much more that we don’t have an overarching model of the mind in anywhere close to the level of detail/mechanistic specificity that ACT-R posits, and that any such attempt would be premature/foolish/not useful right now.
Makes some sense to me! This is part of why my post’s conclusion said stuff like this doesn’t mean you should believe in ACT-R. But yeah, I also think we have a disagreement somewhere around here.
I was trained in the cognitive architecture tradition, which tends to find this situation unfortunate. I have heard strong opinions, which I respect and generally believe, of the “we just don’t know enough” variety which you also espouse. However, I also buy Allen Newell’s famous argument in “you can’t play 20 questions with nature and win”, where he argues that we may never get there without focusing on that goal. From this perspective, it makes (some) sense to try to track a big picture anyway.
In some sense the grand goal of cognitive architecture is that it should eventually be seen as standard (almost required) for individual works of experimental psychology to contribute to a big picture in some way. Imagine for a moment if every paper had a section relating to ACT-R (or some other overarching model), either pointing out how it fits in (agreeing with and extending the overarching model) or pointing out how it doesn’t (revising the overarching model).
With the current state of things, it’s very unclear (as you highlighted in your original comment) what the status of overarching models like ACT-R even is. Is it an artifact from the 90s which is long-irrelevant? Is it the state of the art big-picture? Nobody knows and few care? Wouldn’t it be better if it were otherwise?
On the other hand, working with cognitive architectures like ACT-R can be frustrating and time consuming. In theory, they could be a time-saving tool (you start with all the power of ACT-R and can move forward from that!). In practice, my personal observation at least is that they add time and reduce other kinds of progress you can make. To caricaturize, a cog arch phd student spends their first 2 years learning the cognitive architecture they’ll work with, while a non-cog-arch cogsci student can hit the ground running instead. (This isn’t totally true of course; I’ve heard people say that most phd students are not really productive for their first year or two of grad school.) So I do not want to gloss over the downsides to a cog arch focus.
One big problem is what I’ll call the “task integration problem”. Let’s say you have 100 research psychologists who each spend a chunk of time doing “X in ACT-R” for many different values of X. Now you have lots of ACT-R models of lots of different cognitive phenomena. Can you mash them all together into one big model which does all 100 things?
I’m not totally sure about ACT-R, but I’ve heard that for most cognitive architectures, the answer is “no”. Despite existing in one cognitive architecture, the individual “X” models are sorta like standalone programs which don’t know how to talk to each other.
This undermines the premise of cog arch as helping us fit everything into one coherent picture. So, this is a hurdle which cog arch would have to get past in order to play the kind of role it wants to play.
Thanks for the thoughtful response, that perspective makes sense. I take your point that ACT-R is unique in the ways you’re describing, and that most cognitive scientists are not working on overarching models of the mind like that. I think maybe our disagreement is about how good/useful of an overarching model ACT-R is? It’s definitely not like in physics, where some overarching theories are widely accepted (e.g. the standard model) even by people working on much more narrow topics—and many of the ones that aren’t (e.g. string theory) are still widely known about and commonly taught. The situation in cog sci (in my view, and I think in many people’s views?) is much more that we don’t have an overarching model of the mind in anywhere close to the level of detail/mechanistic specificity that ACT-R posits, and that any such attempt would be premature/foolish/not useful right now. Like, I think if you polled cognitive scientists, the vast majority would disagree with the title of your post—not because they think there’s a salient alternative, but because they think that there is no theory that even comes close to meriting the title of “best-validated theory of cognition” (even if technically one theory is ahead of the others). Do you know what I mean? Of course, even if most cognitive scientists don’t believe in ACT-R in that way, that alone doesn’t mean that ACT-R is wrong.. I’m curious about the evidence that Terry is talking about above. I just think the field would look really, really different if we actually had a halfway-decent paradigm/overarching model of the mind. And it’s not like ACT-R is some unknown idea that is poised to take over the field once people learn about it. Everyone knew about it in the 90s, and then it fell out of widespread use—and my prior on why that happened is that people weren’t finding it super useful. (Although like I said, I’m really curious to learn more about what Terry/other contemporary people are doing with it!)
I agree that there isn’t an overarching theory at the level of specificity of ACT-R that covers all the different aspects of the mind that cognitive science researchers wish it would cover. And so yes, I can see cognitive scientists saying that there is no such theory, or (more accurately) saying that even though ACT-R is the best-validated one, it’s not validated on the particular types of tasks that they’re interested in, so therefore they can ignore it.
However, I do think that there’s enough of a consensus about some aspects of ACT-R (and other theories) that there are some broader generalizations that all cognitive scientists should be aware of. That’s the point of the two papers listed in the original post on the “Common Model of Cognition”. They dig through a whole bunch of different cognitive architectures and ideas over the decades and point out that there are some pretty striking commonalities and similarities across these models. (ACT-R is just one of the theories that they look at, and they point out that there are a set of commonalities across all the theories, and that’s what they call the Common Model of Cognition). The Common Model of Cognition is much more loosely specified and is much more about structural organization rather than being about the particular equations used, though, so I’d still say that ACT-R is the best-validated model. But CMC is surprisingly consistent with a lot of models, and that’s why the community is getting together to write papers like that. The whole point is to try to show that there are some things that we can say right now about an overarching theory of the mind, even if people don’t want to buy into the particular details of ACT-R. And if people are trying to build overarching theories, they should at least be aware of what there is already.
(Full disclosure: I was at the 2017 meeting where this community came together on this topic and started the whole CMC thing. The papers from that meeting are at https://www.aaai.org/Library/Symposia/Fall/fs17-05.php and that’s a great collection of short papers of people talking about the various challenges of expanding the CMC. The general consensus from that meeting is that it was useful to at least have an explicit CMC to help frame that conversation, and it’s been great to see that conversation grow over the last few years. Note: at the time we were calling it the Standard Model of the Mind, but that got changed to Common Model of Cognition).
Makes some sense to me! This is part of why my post’s conclusion said stuff like this doesn’t mean you should believe in ACT-R. But yeah, I also think we have a disagreement somewhere around here.
I was trained in the cognitive architecture tradition, which tends to find this situation unfortunate. I have heard strong opinions, which I respect and generally believe, of the “we just don’t know enough” variety which you also espouse. However, I also buy Allen Newell’s famous argument in “you can’t play 20 questions with nature and win”, where he argues that we may never get there without focusing on that goal. From this perspective, it makes (some) sense to try to track a big picture anyway.
In some sense the grand goal of cognitive architecture is that it should eventually be seen as standard (almost required) for individual works of experimental psychology to contribute to a big picture in some way. Imagine for a moment if every paper had a section relating to ACT-R (or some other overarching model), either pointing out how it fits in (agreeing with and extending the overarching model) or pointing out how it doesn’t (revising the overarching model).
With the current state of things, it’s very unclear (as you highlighted in your original comment) what the status of overarching models like ACT-R even is. Is it an artifact from the 90s which is long-irrelevant? Is it the state of the art big-picture? Nobody knows and few care? Wouldn’t it be better if it were otherwise?
On the other hand, working with cognitive architectures like ACT-R can be frustrating and time consuming. In theory, they could be a time-saving tool (you start with all the power of ACT-R and can move forward from that!). In practice, my personal observation at least is that they add time and reduce other kinds of progress you can make. To caricaturize, a cog arch phd student spends their first 2 years learning the cognitive architecture they’ll work with, while a non-cog-arch cogsci student can hit the ground running instead. (This isn’t totally true of course; I’ve heard people say that most phd students are not really productive for their first year or two of grad school.) So I do not want to gloss over the downsides to a cog arch focus.
One big problem is what I’ll call the “task integration problem”. Let’s say you have 100 research psychologists who each spend a chunk of time doing “X in ACT-R” for many different values of X. Now you have lots of ACT-R models of lots of different cognitive phenomena. Can you mash them all together into one big model which does all 100 things?
I’m not totally sure about ACT-R, but I’ve heard that for most cognitive architectures, the answer is “no”. Despite existing in one cognitive architecture, the individual “X” models are sorta like standalone programs which don’t know how to talk to each other.
This undermines the premise of cog arch as helping us fit everything into one coherent picture. So, this is a hurdle which cog arch would have to get past in order to play the kind of role it wants to play.