That being said, I speculate that part of the disagreement I have with shard theorists is that I don’t think shard theory maps well onto my own internal experiences.
Could you expand on this? My own internal experience led me to believe in something like shard theory, though with less of a focus on values and more on cognitive patterns. Much of shard theory strikes me as so obvious that it might even be clouding my judgement as to how plausible it really is. Hearing an account of how your personal experience differs would be useful, I think.
As an example of what I’m looking for, consider the following fictional account.
As an example of what I mean, consider someone who reads about Cantor’s proof that |N|<|R|, followed by an exposition of its uses in elementary computability theory. As the person reads the proof, they learn a series of associations in their brain, going like “natural numbers aren’t equivalent to the reals --> can list all reals with integer index”, “take integer indexed reals and construct a new real”, “construct a new real from integer indexed list by making its first element not equal to the element before the decimal place in the first real, making its second elment” and so on. These associations are not entirely one way, and some of the associations may themselves be associated with a sequence of associations, or vice versa.
Regardless, they form corrseopond to something like a collection of contextually activated algorithms, composed together to make another contextually activated algorithm. These contexts get modified, activating on more abstract inputs as the person reads through the list of applications of the technique.
And depending on what, exactly, they think of whilst they’re reading the list of techniques—like whether they’re going through the sequence of associations they had for Cantor’s proof and generating a corresponding sequence of associations for the listed applications—they might modify their contextually activated pieces of cognition into something somewhat more general.
Regardless, they form corrseopond to something like a collection of contextually activated algorithms, composed together to make another contextually activated algorithm. These contexts get modified, activating on more abstract inputs as the person reads through the list of applications of the technique.
I agree that the part of shard theory that claims that agents can be thought of as consisting of policy fragments feels very obvious to me. I don’t think this is informative about shard theory vs other models of learned agents—it’s clear that you can almost always chunk up an agent’s policy into lots of policy fragments (and indeed, I can chunk up the motivation of many things I do into various learned heuristics). Even the rational agent model I presented above lets you chunk up the computation of the learned policy into policy fragments!
Shard theory does argue that the correct unit of analysis is a shard i.e. “a contextually activated computation that influences decisions”, but it also argues also that various shards should be modeled as caring about different things (that is, they are shards of value and not shards of cognition) and uses examples of shards like “juice-shard”, “ice-cream shard” and “candy-shard”. It’s the latter claim that I think doesn’t match my internal experience.
Shard theory does argue that the correct unit of analysis is a shard i.e. “a contextually activated computation that influences decisions”, but it also argues also that various shards should be modeled as caring about different things (that is, they are shards of value and not shards of cognition) and uses examples of shards like “juice-shard”, “ice-cream shard” and “candy-shard”. It’s the latter claim that I think doesn’t match my internal experience
My apologies, I didn’t convey what I was wanted clearly. Could you give a detailed example of your internal experiences as they conflict with shard theory, and perhaps what you think generates such experiences, so that we as readers can better grasp your crux with the theory. I should have tried to give such an example myself, instead of rambling on about shards of cognition.
Could you expand on this? My own internal experience led me to believe in something like shard theory, though with less of a focus on values and more on cognitive patterns. Much of shard theory strikes me as so obvious that it might even be clouding my judgement as to how plausible it really is. Hearing an account of how your personal experience differs would be useful, I think.
As an example of what I’m looking for, consider the following fictional account.
As an example of what I mean, consider someone who reads about Cantor’s proof that |N|<|R|, followed by an exposition of its uses in elementary computability theory. As the person reads the proof, they learn a series of associations in their brain, going like “natural numbers aren’t equivalent to the reals --> can list all reals with integer index”, “take integer indexed reals and construct a new real”, “construct a new real from integer indexed list by making its first element not equal to the element before the decimal place in the first real, making its second elment” and so on. These associations are not entirely one way, and some of the associations may themselves be associated with a sequence of associations, or vice versa.
Regardless, they form corrseopond to something like a collection of contextually activated algorithms, composed together to make another contextually activated algorithm. These contexts get modified, activating on more abstract inputs as the person reads through the list of applications of the technique.
And depending on what, exactly, they think of whilst they’re reading the list of techniques—like whether they’re going through the sequence of associations they had for Cantor’s proof and generating a corresponding sequence of associations for the listed applications—they might modify their contextually activated pieces of cognition into something somewhat more general.
I agree that the part of shard theory that claims that agents can be thought of as consisting of policy fragments feels very obvious to me. I don’t think this is informative about shard theory vs other models of learned agents—it’s clear that you can almost always chunk up an agent’s policy into lots of policy fragments (and indeed, I can chunk up the motivation of many things I do into various learned heuristics). Even the rational agent model I presented above lets you chunk up the computation of the learned policy into policy fragments!
Shard theory does argue that the correct unit of analysis is a shard i.e. “a contextually activated computation that influences decisions”, but it also argues also that various shards should be modeled as caring about different things (that is, they are shards of value and not shards of cognition) and uses examples of shards like “juice-shard”, “ice-cream shard” and “candy-shard”. It’s the latter claim that I think doesn’t match my internal experience.
My apologies, I didn’t convey what I was wanted clearly. Could you give a detailed example of your internal experiences as they conflict with shard theory, and perhaps what you think generates such experiences, so that we as readers can better grasp your crux with the theory. I should have tried to give such an example myself, instead of rambling on about shards of cognition.