I’m often asked, Why “shard theory”? I suggested this name to Quintin when realizing that human values have the type signature of contextually activated decision-making influences. The obvious choice, then, was to call these things “shards of value”—drawing inspiration from Eliezer’s Thou art godshatter, where he originally wrote “shard of desire.”
(Contrary to several jokes, the choice was not just “because ‘shard theory’ sounds sick.”)
This name has several advantages. Value-shards can have many subshards/facets which vary contextually (a real crystal may look slightly different along its faces or have an uneven growth pattern); value-shards grow in influence over time under repeated positive (just as real crystals can grow); value-shards imply a degree of rigidity, but also incompleteness—they are pieces of a whole (on my current guess, the eventual utility function which is the reflective equilibrium of value-handshakes between the set of endorsed shards which bid as a function of their own future prospects). Lastly, a set of initial shards will (I expect) generally steer the future towards growing themselves (e.g. banana-shard leads to more banana-consumption → more reward → the shard grows and becomes more sophisticated); similarly, given an initial start of a crystalline lattice which is growing, I’d imagine it becomes more possible to predict the later lattice configuration due to the nature of crystals.
I’m often asked, Why “shard theory”? I suggested this name to Quintin when realizing that human values have the type signature of contextually activated decision-making influences. The obvious choice, then, was to call these things “shards of value”—drawing inspiration from Eliezer’s Thou art godshatter, where he originally wrote “shard of desire.”
(Contrary to several jokes, the choice was not just “because ‘shard theory’ sounds sick.”)
This name has several advantages. Value-shards can have many subshards/facets which vary contextually (a real crystal may look slightly different along its faces or have an uneven growth pattern); value-shards grow in influence over time under repeated positive (just as real crystals can grow); value-shards imply a degree of rigidity, but also incompleteness—they are pieces of a whole (on my current guess, the eventual utility function which is the reflective equilibrium of value-handshakes between the set of endorsed shards which bid as a function of their own future prospects). Lastly, a set of initial shards will (I expect) generally steer the future towards growing themselves (e.g. banana-shard leads to more banana-consumption → more reward → the shard grows and becomes more sophisticated); similarly, given an initial start of a crystalline lattice which is growing, I’d imagine it becomes more possible to predict the later lattice configuration due to the nature of crystals.