How does representation interact with consciousness? Suppose you’re reasoning about the universe via a partially observable Markov decision process, and that your model is incredibly detailed and accurate. Further suppose you represent states as numbers, as their numeric labels.
To get a handle on what I mean, consider the game of Pac-Man, which can be represented as a
finite, deterministic, fully-observable MDP. Think about all possible game screens you can observe, and number them. Now get rid of the game screens. From the perspective of reinforcement learning, you haven’t lost anything—all policies yield the same return they did before, the transitions/rules of the game haven’t changed—in fact, there’s a pretty strong isomorphism I can show between these two MDPs. All you’ve done is changed the labels—representation means practically nothing to the mathematical object of the MDP, although many eg DRL algorithms should be able to exploit regularities in the representation to reduce sample complexity.
So what does this mean? If you model the world as a partially observable MDP whose states are single numbers… can you still commit mindcrime via your deliberations? Is the structure of the POMDP in your head somehow sufficient for consciousness to be accounted for (like how the theorems of complexity theory govern computers both of flesh and of silicon)? I’m confused.
I think a reasonable and related question we don’t have a solid answer for is if humans are already capable of mind crime.
For example, maybe Alice is mad at Bob and imagines causing harm to Bob. How well does Alice have to model Bob for her imaginings to be mind crime? If Alice has low cognitive empathy is it not mind crime but if her cognitive empathy is above some level is it then mind crime?
I think we’re currently confused enough about what mind crime is such that it’s hard to even begin to know how we could answer these questions based on more than gut feelings.
I suspect that it doesn’t matter how accurate or straightforward a predictor is in modeling people. What would make prediction morally irrelevant is that it’s not noticed by the predicted people, irrespective of whether this happens because it spreads the moral weight conferred to them over many possibilities (giving inaccurate prediction), keeps the representation sufficiently baroque, or for some other reason. In the case of inaccurate prediction or baroque representation, it probably does become harder for the predicted people to notice being predicted, and I think this is the actual source of moral irrelevance, not those things on their own. A more direct way of getting the same result is to predict counterfactuals where the people you reason about don’t notice the fact that you are observing them, which also gives a form of inaccuracy (imagine that your predicting them is part of their prior, that’ll drive the counterfactual further from reality).
How does representation interact with consciousness? Suppose you’re reasoning about the universe via a partially observable Markov decision process, and that your model is incredibly detailed and accurate. Further suppose you represent states as numbers, as their numeric labels.
To get a handle on what I mean, consider the game of Pac-Man, which can be represented as a finite, deterministic, fully-observable MDP. Think about all possible game screens you can observe, and number them. Now get rid of the game screens. From the perspective of reinforcement learning, you haven’t lost anything—all policies yield the same return they did before, the transitions/rules of the game haven’t changed—in fact, there’s a pretty strong isomorphism I can show between these two MDPs. All you’ve done is changed the labels—representation means practically nothing to the mathematical object of the MDP, although many eg DRL algorithms should be able to exploit regularities in the representation to reduce sample complexity.
So what does this mean? If you model the world as a partially observable MDP whose states are single numbers… can you still commit mindcrime via your deliberations? Is the structure of the POMDP in your head somehow sufficient for consciousness to be accounted for (like how the theorems of complexity theory govern computers both of flesh and of silicon)? I’m confused.
I think a reasonable and related question we don’t have a solid answer for is if humans are already capable of mind crime.
For example, maybe Alice is mad at Bob and imagines causing harm to Bob. How well does Alice have to model Bob for her imaginings to be mind crime? If Alice has low cognitive empathy is it not mind crime but if her cognitive empathy is above some level is it then mind crime?
I think we’re currently confused enough about what mind crime is such that it’s hard to even begin to know how we could answer these questions based on more than gut feelings.
I suspect that it doesn’t matter how accurate or straightforward a predictor is in modeling people. What would make prediction morally irrelevant is that it’s not noticed by the predicted people, irrespective of whether this happens because it spreads the moral weight conferred to them over many possibilities (giving inaccurate prediction), keeps the representation sufficiently baroque, or for some other reason. In the case of inaccurate prediction or baroque representation, it probably does become harder for the predicted people to notice being predicted, and I think this is the actual source of moral irrelevance, not those things on their own. A more direct way of getting the same result is to predict counterfactuals where the people you reason about don’t notice the fact that you are observing them, which also gives a form of inaccuracy (imagine that your predicting them is part of their prior, that’ll drive the counterfactual further from reality).