“I think there are a lot of things that are morally important that do seem like they require memory or involve memory. So having long term projects and long term goals, that’s something that human beings have. I wouldn’t be surprised if having memory versus not having memory is also just kind of a big determinant of what sorts of experiences you can have or affects what experiences you have in various ways. And yeah, it might be important for having an enduring self through time. So that’s one thing that people also say about large language models is they seem to have these short-lived identities that they spin up as required but nothing that lasts their time.”
There’s the interesting/tragic case of Clive Wearing, who has both retrograde and anterograde amnesia, causing him to experience consciousness only from one moment to the next. His brain is still able to construct internal narratives of his identity and experiences, which I would consider the definition of consciousness, but the lack of access to previously recorded narratives makes it seem to him that those experiences were unconscious and that he’s only just now attaining consciousness for the first time.
I would argue, as I’m sure most humans would agree, that he still has moral worth. So I’m not sure if lack of long-term memory should in itself exclude AIs from moral consideration.
Perhaps the moral worth of a system should be the product of sentience (capacity to experience suffering, at least) and consciousness (level of sophistication of the system’s internal self-narratives), where moral worth is defined as the weight we give to a system’s preferences when calculating trade-offs with other agents’ preferences in morally ambiguous situations. Of course, the problem with language models is, as you alluded to, that you can’t simply take their word for it when they declare their sentience and consciousness, even if that’s perfectly reasonable to do with humans. They’re only trained to predict what humans would say in the same context, after all. We will need to have some way of looking at their internal structures to gauge whether and to what extent they meet these criteria.
There’s the interesting/tragic case of Clive Wearing, who has both retrograde and anterograde amnesia, causing him to experience consciousness only from one moment to the next. His brain is still able to construct internal narratives of his identity and experiences, which I would consider the definition of consciousness, but the lack of access to previously recorded narratives makes it seem to him that those experiences were unconscious and that he’s only just now attaining consciousness for the first time.
I would argue, as I’m sure most humans would agree, that he still has moral worth. So I’m not sure if lack of long-term memory should in itself exclude AIs from moral consideration.
Perhaps the moral worth of a system should be the product of sentience (capacity to experience suffering, at least) and consciousness (level of sophistication of the system’s internal self-narratives), where moral worth is defined as the weight we give to a system’s preferences when calculating trade-offs with other agents’ preferences in morally ambiguous situations. Of course, the problem with language models is, as you alluded to, that you can’t simply take their word for it when they declare their sentience and consciousness, even if that’s perfectly reasonable to do with humans. They’re only trained to predict what humans would say in the same context, after all. We will need to have some way of looking at their internal structures to gauge whether and to what extent they meet these criteria.