Reading this, I’m realizing again something I may have realized before and forgotten, but I think ideas about goal-directedness in AI have a lot of overlap with the philosophical topic of telos and Heideggerian care/concern.
The way I think about this is that ontological beings (that is, any process we can identify as producing information) have some ability to optimize (because information is produced by feedback) and must optimize for something rather than nothing (else they are not optimizers) or everything (in which case they are not finite, which they must be in our world). Thus we should expect anything we might think of as optimizing will have something it cares about, where here “caring about” is not the self-reflective way humans may knowingly care for something but in the implicit way that acts demonstrate care for something.
That something might not be very crisp and hard to specify or might be incoherent (or at least, incoherent when not conditioned on the entire state of the world), hence we might not be able to line it up perfectly with a notion like a utility function, although we could say a utility function is an attempt to represent the concern of an optimization process.
That optimization processes must care about something is, as I think underlies some of the discussion around Dennette’s position though not discussed here, similar to the way that the intentionality of thought/belief means that thoughts must be about something.
Yet I find myself lost when trying to find more information about this concept of care. It is mentioned in both the chapter on Heidegger in The History of Philosophy and the section on care in the SEP article on Heidegger, but I don’t get a single thing written there. I think the ideas of “thrownness” and “disposedness” are related?
Do you have specific pointers to deeper discussions of this concept? Specifically, I’m interested in new intuitions for how a goal is revealed by actions.
First, they consider systems that have observable behavior, i.e. systems that take inputs and produce outputs. Such systems can be either active, in that the system itself is the source of energy that produces the outputs, or passive, in that some outside source supplies the energy to power the mechanism. Compare an active plant or animal to something passive like a rock, though obviously whether or not something is active or passive depends a lot on where you draw the boundaries of its inside vs. its outside.
Active behavior is subdivided into two classes: purposeful and purposeless. They say that purposeful behavior is that which can be interpreted as directed to attaining a goal; purposeless behavior does not. They spend some time in the paper defending the idea of purposefulness, and I think it doesn’t go well. So I’d instead propose we think of these terms differently; I prefer to think of purposeful behavior as that which creates a reduction in entropy within the system and its outputs and purposeless behavior does not.
They then go on to divide purposeful behavior into teleological and non-teleological behavior, by which they simply mean behavior that’s the result of feedback (and they specify negative feedback) or not. In LessWrong terms, I’d say this is like the difference between optimizers (“fitness maximizers”) and adaptation executors.
They then go on to make a few additional distinctions that are not relevant to the present topic although do have some relevance to AI alignment relating to predictability of systems.
I’d say then that systems with active, purposeful, teleological behavior are the ones that “care”, and the teleological mechanism is the aspect of the system by which a system is made to care.
Doing a little digging, I realized that the idea of “teleological mechanism” from cybernetics is probably a better handle for the idea and will provide a more accessible presentation of the idea. Some decent references:
I don’t know of anywhere that presents the idea quite how I think of it, though. If you read Dreyfus on Heidegger you might manage to pick this out. Similarly I think this idea underlies Sartre’s talk about freedom, but I can’t recall that he explicitly makes the connection in the way I would. To the best of my knowledge philosophers have unfortunately not said enough about this topic because it’s omnipresent in humans and often something that comes up incidentally to considering other things but not something deeply explored for its own sake except when people are confused (cg. Hegel on teleology).
Reading this, I’m realizing again something I may have realized before and forgotten, but I think ideas about goal-directedness in AI have a lot of overlap with the philosophical topic of telos and Heideggerian care/concern.
The way I think about this is that ontological beings (that is, any process we can identify as producing information) have some ability to optimize (because information is produced by feedback) and must optimize for something rather than nothing (else they are not optimizers) or everything (in which case they are not finite, which they must be in our world). Thus we should expect anything we might think of as optimizing will have something it cares about, where here “caring about” is not the self-reflective way humans may knowingly care for something but in the implicit way that acts demonstrate care for something.
That something might not be very crisp and hard to specify or might be incoherent (or at least, incoherent when not conditioned on the entire state of the world), hence we might not be able to line it up perfectly with a notion like a utility function, although we could say a utility function is an attempt to represent the concern of an optimization process.
That optimization processes must care about something is, as I think underlies some of the discussion around Dennette’s position though not discussed here, similar to the way that the intentionality of thought/belief means that thoughts must be about something.
Thanks for the proposed idea!
Yet I find myself lost when trying to find more information about this concept of care. It is mentioned in both the chapter on Heidegger in The History of Philosophy and the section on care in the SEP article on Heidegger, but I don’t get a single thing written there. I think the ideas of “thrownness” and “disposedness” are related?
Do you have specific pointers to deeper discussions of this concept? Specifically, I’m interested in new intuitions for how a goal is revealed by actions.
Okay, so here’s a more adequate follow up.
In this seminal cybernetics essay a way of thinking about this is layed out.
First, they consider systems that have observable behavior, i.e. systems that take inputs and produce outputs. Such systems can be either active, in that the system itself is the source of energy that produces the outputs, or passive, in that some outside source supplies the energy to power the mechanism. Compare an active plant or animal to something passive like a rock, though obviously whether or not something is active or passive depends a lot on where you draw the boundaries of its inside vs. its outside.
Active behavior is subdivided into two classes: purposeful and purposeless. They say that purposeful behavior is that which can be interpreted as directed to attaining a goal; purposeless behavior does not. They spend some time in the paper defending the idea of purposefulness, and I think it doesn’t go well. So I’d instead propose we think of these terms differently; I prefer to think of purposeful behavior as that which creates a reduction in entropy within the system and its outputs and purposeless behavior does not.
They then go on to divide purposeful behavior into teleological and non-teleological behavior, by which they simply mean behavior that’s the result of feedback (and they specify negative feedback) or not. In LessWrong terms, I’d say this is like the difference between optimizers (“fitness maximizers”) and adaptation executors.
They then go on to make a few additional distinctions that are not relevant to the present topic although do have some relevance to AI alignment relating to predictability of systems.
I’d say then that systems with active, purposeful, teleological behavior are the ones that “care”, and the teleological mechanism is the aspect of the system by which a system is made to care.
Doing a little digging, I realized that the idea of “teleological mechanism” from cybernetics is probably a better handle for the idea and will provide a more accessible presentation of the idea. Some decent references:
https://www.jstor.org/stable/184878
https://www.jstor.org/stable/2103479
https://nyaspubs.onlinelibrary.wiley.com/toc/17496632/50/4
I don’t know of anywhere that presents the idea quite how I think of it, though. If you read Dreyfus on Heidegger you might manage to pick this out. Similarly I think this idea underlies Sartre’s talk about freedom, but I can’t recall that he explicitly makes the connection in the way I would. To the best of my knowledge philosophers have unfortunately not said enough about this topic because it’s omnipresent in humans and often something that comes up incidentally to considering other things but not something deeply explored for its own sake except when people are confused (cg. Hegel on teleology).