By “internal structure” or “cognitive terms” I also mean what’s inside the system, but usually at a higher level of abstraction than physical implementation. For instance, we can describe AlphaGo’s cognition as follows: it searches through a range of possible games, and selects moves that do well in a lot of those games. If we just take the value network by itself (which is still very good at Go) without MCTS, then it’s inaccurate to describe that network as searching over many possible games; it’s playing Go well using only a subset of the type of cognition the full system does.
This differs from the intentional stance by paying more attention to what’s going on inside the system, as opposed to just making inferences from behaviour. It’d be difficult to tell that the full AlphaGo system and the value network alone are doing different types of cognition, just from observing their behaviour—yet knowing that they do different types of cognition is very useful for making predictions about their behaviour on unobserved board positions.
What I was pointing at is that searching a definition (sorry, used the taboo word) of goal-directedness in terms of the internal structure (that is, the source code for example), is misguided.
You can probably guess what I’m going to say here: I still don’t know what you mean by “definition”, or why we want to search for it.
After talking with Evan, I think I understand your point better. What I didn’t understand was that you seemed to argue that there was something else than the behavior that mattered for goal-directedness. But as I understand it now, what you’re saying is that, yes, the behavior is what matters, but extracting the relevant information from the behavior is really hard. And thus you believe that computing goal-directedness in any meaningful way will require normative assumptions about the cognition of the system, at an abstract level.
If that’s right, then I would still disagree with you, but I think the case for my position is far less settled than I assumed. I believe there are lots of interesting parts of goal-directedness that can be extracted from the behavior only, while acknowledging that historically, it has been harder to compute most complex properties of a system from behavior alone.
If that’s not right, then I propose that we schedule a call sometime, to clarify the disagreement with more bandwidth. Actually, even if it’s right, I can call to update you on the research.
By “internal structure” or “cognitive terms” I also mean what’s inside the system, but usually at a higher level of abstraction than physical implementation. For instance, we can describe AlphaGo’s cognition as follows: it searches through a range of possible games, and selects moves that do well in a lot of those games. If we just take the value network by itself (which is still very good at Go) without MCTS, then it’s inaccurate to describe that network as searching over many possible games; it’s playing Go well using only a subset of the type of cognition the full system does.
This differs from the intentional stance by paying more attention to what’s going on inside the system, as opposed to just making inferences from behaviour. It’d be difficult to tell that the full AlphaGo system and the value network alone are doing different types of cognition, just from observing their behaviour—yet knowing that they do different types of cognition is very useful for making predictions about their behaviour on unobserved board positions.
You can probably guess what I’m going to say here: I still don’t know what you mean by “definition”, or why we want to search for it.
After talking with Evan, I think I understand your point better. What I didn’t understand was that you seemed to argue that there was something else than the behavior that mattered for goal-directedness. But as I understand it now, what you’re saying is that, yes, the behavior is what matters, but extracting the relevant information from the behavior is really hard. And thus you believe that computing goal-directedness in any meaningful way will require normative assumptions about the cognition of the system, at an abstract level.
If that’s right, then I would still disagree with you, but I think the case for my position is far less settled than I assumed. I believe there are lots of interesting parts of goal-directedness that can be extracted from the behavior only, while acknowledging that historically, it has been harder to compute most complex properties of a system from behavior alone.
If that’s not right, then I propose that we schedule a call sometime, to clarify the disagreement with more bandwidth. Actually, even if it’s right, I can call to update you on the research.