Consider a situation where Mary is so dexterous that she is able to perform fine-grained brain surgery on herself. In that case, she could look at what an example of a brain that has seen red looks like, and manually copy any relevant differences into her own brain. In that case, while she still never would have actually seen red through her eyes, it seems like she would know what it is like to see red as well as anyone else.
I think this demonstrates that the Mary’s room though experiment is about the limitations of human senses/means of learning, and that the apparent sense of mystery it has comes mainly from the vagueness of what it means to “know all about” something. (Not saying it was a useless idea—it can be quite valuable to be forced to break down some vague or ambiguous idea that we usually take for granted).
M’s R is about what it says its about, the existence of non physical facts. Finding a loophole where Mary can instantiate the brain state without having the perceptual stimulus doesn’t address that...indeed it assumes that an instantiation of the red-seeing is necessary, which is tantamount to conceding that something subectve is going on, which is tantamount to conceding the point.
What is a “non physical fact”? The experience of red seems to be physically encoded in the brain like anything else. It does seem clear that some knowledge exists which can’t be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design that doesn’t necessarily apply to eg AIs (which depending on design may be able to transmit and integrate snippets of their internal code and data), and I don’t think this thought experiment proves anything beyond that.
The argument treats physical knowledge as a subset of objective. kowledge. Subjective knowledge, which can only be known on a first person basis, automatically counts as non physical. That’s an epistemic definition.
The experience of red seems to be physically encoded in the brain like anything else.
If you have the expected intuition from M’s R, that Mary would be able to read cognitive information from brain scans, but not expetuental information. In that send, ‘red’ is not encoded in the same way as everything else, since it can not be decoded in the same way.
sIt does seem clear that some knowledge exists which can’t be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design
But noit super human design. The original paper (ave you read it?) avoids the issue of limited communication bandwidth by making Mary a super scientist who can examine brain scans of any level of detail.
Proves anything beyond that
What it proves to you depends on what intuitions you have about it . If you think Mary would know what red looks like while in the room, from reading brain scans, then it s going to prove anything to you.
A way to rephrase the question is, “is there any sequence of sensory inputs other than the stimulation of red cones by red light that will cause Mary to have comparable memories re: the color red as someone who has had their red cones stimulated at some point”. It’s possible that the answer is no, which says something interesting about the API of the human machine, but doesn’t seem necessarily fundamental to the concept of knowledge.
If physicalism is the claim that everything, has a physical explanation, then the inability to understand what pain is without being in pain is a contradiction to it. I don’ think anyone here believes that physicalism is an unmportamt issue.
I’m arguing that there’s no contradiction and that this inability is just a limit of humans/organic brains, not a fundamental fact about pain or information.
I think the argument is asserting that Mary post-brain surgery is a identical to Mary post-seeing-red. There is no difference; the two Mary’s would both attest to having access some ineffable quality of red-ness.
To put it bluntly, both Marys say the same things, think the same things, and generally are virtually indistinguishable. I don’t understand what disagreement is occurring here, hopefully I’ve given someone enough ammunition to explain.
I don’t understand what the point of that point is.
Do you think you argung against the intended conclusion of the Knowledge Argumemt in some way? If so, you are not...the loophole you have found s quite irrelevant,
I’m disagreeing that you have a valid refutation of the KA. However, I don’t know if you even think you have, since you haven’t responded to my hints that you should clarify.
They way you’re saying this, it makes it seem like we’re both in the same boat. I have no idea what position you’re even holding.
I feel like I’m doing the same thing over and over and nothing different is happening, but I’ll quote what I said in another place in this thread and hope I was a tiny bit clearer.
I think the distinction between ‘knowing all about’ and ‘seeing’ red is captured in my box analogy. The brain state is a box. There is another box inside it, call this ‘understanding’. We call something inside the first box ‘experienced’. So the paradox hear is the two distinct states [experiencing (red) ] and [experiencing ( [understanding (red) ] ) ] are both brought under the header [knowing (red)], and this is really confusing.
I have explained elsewhere why the loophole doesnt work:-
M’s R is about what it says its about, the existence of non physical facts. Finding a loophole where Mary can instantiate the brain state without having the perceptual stimulus doesn’t address that...indeed it assumes that an instantiation of the red-seeing is necessary, which is tantamount to conceding that something subectve is going on, which is tantamount to conceding the point
Moving on to your argument:-
I think the distinction between ‘knowing all about’ and ‘seeing’ red is captured in my box analogy. The brain state is a box. There is another box inside it, call this ‘understanding’. We call something inside the first box ‘experienced’. So the paradox hear is the two distinct states [experiencing (red) ] and [experiencing ( [understanding (red) ] ) ] are both brought under the header [knowing (red)], and this is really confusing
Confusing to whom?
Let’s suppose that person is Frank Jackson.
In the knowledge Argument, Jackson credits Mary with all objective knowledge, and only objective knowledge precisely because he is trying to establish the existence of subjective knowledge .. what Mary doesnt know must be subjective, if there is something Mary doesn’t know. So the eventual point s that there s more to knowledge than objective knowledge.
So you don’t show that Jackson is wrong by agreeing with him.
(translation: the experience of seeing red is no the experience of understanding how seeing red works)
This is true, when we take those statements literally. But it’s true in the same sense a Gödel encoding of statement in PA is not literally that statement. It is just a representation, but the representation is exactly homomorphic to its referent. Mary’s representation of reality is presumed complete ex hypothesi, therefore she will understand exactly what will happen in her brain after seeing color, and that is exactly what happens.
You wouldn’t call a statement of PA that isn’t a literally a Gödel encoding of a statement (for some fixed encoding) a non-mathematical statement. For one, because that statement has a Gödel encoding by necessity. But more importantly, even though the statement technically isn’t literally a Gödel-encoding, it’s still mathematical, regardless.
Mary’s know how she will respond to learning what red is like. Mary knows how others will respond. This exhausts the space of possible predictions that could be made on behalf of this subjective knowledge, and it can be done without it.
what Mary doesnt know must be subjective, if there is something Mary doesn’t know. So the eventual point s that there s more to knowledge than objective knowledge.
Tangentially to this discussion, but I don’t think that is a wise way of labeling that knowledge.
Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?
Mary has all objective knowledge, but certain facts about her own future behavior must escape her, because any certainty could trivially be negated.
Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?
There are three possibilities worth disambiguating here. 1) Mary predicts that she will do X given some assumed set S1 of knowledge, memories, experiences, etc., AND S1 includes Mary’s knowledge of this prediction. 2) Mary predicts that she will do X given some assumed set S2 of knowledge, memories, experiences, etc., AND S2 does not include Mary’s knowledge of this prediction. 3) Mary predicts that she will do X independent of her knowledge, memories, experiences, etc.
Mary’s representation of reality is presumed complete ex hypothesi, therefore she will understand exactly what will happen in her brain after seeing color, and that is exactly what happens
Mary is presumed to have all objective knowedge and only objectve knowledge, Your phrasing is ambiguous and therefire doesnt address the point. When you say Mary will know what happens when she sees red, do you mean she knows how red looks subjectively, or she knows something
objective like what her behaviour will be.....further on you mention predicting her reactions,
You wouldn’t call a statement of PA that isn’t a literally a Gödel encoding of a statement (for some fixed encoding) a non-mathematical statement. For one, because that statement has a Gödel encoding by necessity. But more importantly, even though the statement technically isn’t literally a Gödel-encoding, it’s still mathematical, regardless.
Is that supposed to relate to the objective/ subjective distinction somehow?
[Mary knows] how she will respond to learning what red is like. Mary knows how others will respond. This exhausts the space of possible predictions that could be made on behalf of this subjective knowledge, and it can be done without
So? The overall point is about physicalism, and to get to ‘physicalism is false’, all you need is the existence of subjective knowledge, not its usefulness in making prediction. So again I don’t see the relevance
Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?
Maybe. I don’t see the problem. There is still an unproblematic sense in which Mary has all objective knowledge, even if it doesn’t allow her to do certain things. If that was the point.
Mary is presumed to have all objective knowedge and only objectve knowledge, Your phrasing is ambiguous and therefire doesnt address the point.
The behavior of the neurons in her skull is an objective fact, and this is what I mean to referring to. Apologies for the ambiguity.
When you say Mary will know what happens when she sees red, do you mean she knows how red looks subjectively, or she knows something objective like what her behaviour will be
The latter. The former is purely experiential knowledge, and as I have repeatedly said is contained in a superset of verbal (what you call ‘objective’) knowledge, but is disjoint with the set of verbal (‘objective’) knowledge itself. This is my box metaphor.
Is that supposed to relate to the objective/ subjective distinction somehow?
Yes. Assuming the Godel encoding is fixed, [the metaphor is that] any and all statements of PA are experiential knowledge (an experience, in simple terms), non-Godel statements of PA are purely experiential knowledge; the redness of red, say, and finally the Godel statements of PA are verbal knowledge, or ‘objective knowledge’ in your terminology.
Despite not being Godel statements in the encoding, the second item in the above list is still mathematical, and redness of red is still physical.
So? The overall point is about physicalism, and to get to ‘physicalism is false’, all you need is the existence of subjective knowledge, not its usefulness in making prediction. So again I don’t see the relevance
What does this knowledge do? How do we tell the difference between someone with and without these ‘subjective experiences’? What definition of knowledge admits it as valid?
The latter. The former is purely experiential knowledge, and as I have repeatedly said is contained in a superset of verbal (what you call ’objectiive) knowledge, but is disjoint with the set of verbal (‘objective’) knowledge itself. This is my box metaphor.
You have said that according to you, stipulatively, subjective knowledge is a subset of objective knowledge. What we mean by objective knowledge is generally knowledge that can be understood at second hand, without being in a special state or having had particular experiences. You say that the subjective subset of objective knowledge is somehow opaque, so that it does not have the properties usually associated with objective knowledge..but why should anyone believe it is objective, when it lacks the usual properties, and is only asserted to be objective?
redness of red is still physical
I can’t see how that has been proven. You can’t prove that redness is physically encoded in the relevant sense just by noting that physical changes occur in brains, because
1 There’s no physical proof of physicalism
2 An assumption of physicalism is question begging
3 You need an absence of non physical proeties, states and processes, not just the presence of physical changes
4 Physicalism as a meaningful claim, and not just a stipulative label needs to pay its way in explanation...but its ability to explain subjective knowledge is just at is in question.
What does this knowledge do? How do we tell the difference between someone with and without these ‘subjective experiences’? What definition of knowledge admits it as valid
Its hard to prove the existence of subjective knowledge in an objective basis. What else would you expect.? There is a widespread belief in subjective, experiential knowledge and the evidence for it is subjective. The alternative is the sort of thing caricatured as ‘how was it for me, darling’.
If one of Mary’s predictions is, “When I see red, I will say, ’wow! I didn’t know that red looked like that!”, then the fact that she has predicted this in advance is hardly evidence that she does not learn anything by seeing red. If anything, it proves that she does.
The AI analogue would be: If the AI has the capacity to wirehead itself, it can make itself enter the color perception subroutines. Whether something new is learned depends on the remaining brain architecture. I would say, in the case of humans, it is clear that whenever something new is experienced, the human learns what that experience feels like. I reckon that for some people with strong visualization (in a broad sense) abilities it is possible to know what an experience feels like without experiencing first hand by synthesizing a new experience from previously known experiences. But in most cases there is a difference between imagining a sensation and experiencing it.
In the case of the AI, there could either be the case where no information is passed between the color perception subroutine and the main processing unit, in which case the AI may have a new experience, but not learn anything new. Or some representation of the experience of being in the subroutine is saved to memory, in which case something new is learned.
The stronger someones imaginative ability is, the more their imagining an experience is actually having it, in terms of brain states....and the less it s a counterexample to anything relevant.
If the knowedge the AI gets from the colour routine is unproblematically encoded in a string of bits, why can’t it just look at the string of bits...for that matter, why can’t Mary just look at the neural spike trains of someone seing red?
why can’t Mary just look at the neural spike trains of someone seing red?
Why can’t we just eat a picture of a plate of spaghetti instead of actual spaghetti? Because a representation of some thing is not the thing itself. Am I missing something?
The banal truth here is that knowing about a thing doesn’t turn you into it.
The significant and contentious claim is that here are certain kinds of knowledge that can only be accessed by instantiating a brain state. The existence of such subjective knowledge leads to a further argument against physicalism.
Consider a situation where Mary is so dexterous that she is able to perform fine-grained brain surgery on herself. In that case, she could look at what an example of a brain that has seen red looks like, and manually copy any relevant differences into her own brain. In that case, while she still never would have actually seen red through her eyes, it seems like she would know what it is like to see red as well as anyone else.
But in order to create a realistic experience she would have to create a false memory of having seen red, which is something that an agent (human or AI) that values epistemic rationality would not want to do.
Since you’d know it was a false memory, it doesn’t necessarily seem to be a problem, at least if you really need to know what red is like for some reason.
If you know that it is a false memory then the experience is not completely accurate, though it may be perhaps more accurate than what human imagination could produce.
Consider a situation where Mary is so dexterous that she is able to perform fine-grained brain surgery on herself. In that case, she could look at what an example of a brain that has seen red looks like, and manually copy any relevant differences into her own brain. In that case, while she still never would have actually seen red through her eyes, it seems like she would know what it is like to see red as well as anyone else.
I think this demonstrates that the Mary’s room though experiment is about the limitations of human senses/means of learning, and that the apparent sense of mystery it has comes mainly from the vagueness of what it means to “know all about” something. (Not saying it was a useless idea—it can be quite valuable to be forced to break down some vague or ambiguous idea that we usually take for granted).
M’s R is about what it says its about, the existence of non physical facts. Finding a loophole where Mary can instantiate the brain state without having the perceptual stimulus doesn’t address that...indeed it assumes that an instantiation of the red-seeing is necessary, which is tantamount to conceding that something subectve is going on, which is tantamount to conceding the point.
What is a “non physical fact”? The experience of red seems to be physically encoded in the brain like anything else. It does seem clear that some knowledge exists which can’t be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design that doesn’t necessarily apply to eg AIs (which depending on design may be able to transmit and integrate snippets of their internal code and data), and I don’t think this thought experiment proves anything beyond that.
The argument treats physical knowledge as a subset of objective. kowledge. Subjective knowledge, which can only be known on a first person basis, automatically counts as non physical. That’s an epistemic definition.
If you have the expected intuition from M’s R, that Mary would be able to read cognitive information from brain scans, but not expetuental information. In that send, ‘red’ is not encoded in the same way as everything else, since it can not be decoded in the same way.
But noit super human design. The original paper (ave you read it?) avoids the issue of limited communication bandwidth by making Mary a super scientist who can examine brain scans of any level of detail.
What it proves to you depends on what intuitions you have about it . If you think Mary would know what red looks like while in the room, from reading brain scans, then it s going to prove anything to you.
A way to rephrase the question is, “is there any sequence of sensory inputs other than the stimulation of red cones by red light that will cause Mary to have comparable memories re: the color red as someone who has had their red cones stimulated at some point”. It’s possible that the answer is no, which says something interesting about the API of the human machine, but doesn’t seem necessarily fundamental to the concept of knowledge.
The relevance is physicalism.
If physicalism is the claim that everything, has a physical explanation, then the inability to understand what pain is without being in pain is a contradiction to it. I don’ think anyone here believes that physicalism is an unmportamt issue.
I’m arguing that there’s no contradiction and that this inability is just a limit of humans/organic brains, not a fundamental fact about pain or information.
If you want to argue to that conclusion, then arge for it: What kind of limit? Where does it come from?
I think the argument is asserting that Mary post-brain surgery is a identical to Mary post-seeing-red. There is no difference; the two Mary’s would both attest to having access some ineffable quality of red-ness.
To put it bluntly, both Marys say the same things, think the same things, and generally are virtually indistinguishable. I don’t understand what disagreement is occurring here, hopefully I’ve given someone enough ammunition to explain.
I don’t understand what the point of that point is.
Do you think you argung against the intended conclusion of the Knowledge Argumemt in some way? If so, you are not...the loophole you have found s quite irrelevant,
I have no idea what your position even is and you are making no effort to elucidate it. I had hoped this line
Was enough to clue you in to the point of my post.
I’m disagreeing that you have a valid refutation of the KA. However, I don’t know if you even think you have, since you haven’t responded to my hints that you should clarify.
“I think you’re wrong” is not a position.
They way you’re saying this, it makes it seem like we’re both in the same boat. I have no idea what position you’re even holding.
I feel like I’m doing the same thing over and over and nothing different is happening, but I’ll quote what I said in another place in this thread and hope I was a tiny bit clearer.
http://lesswrong.com/lw/nnc/the_ai_in_marys_room/day2
I just explained the position I am holding.
I have explained elsewhere why the loophole doesnt work:-
Moving on to your argument:-
Confusing to whom?
Let’s suppose that person is Frank Jackson.
In the knowledge Argument, Jackson credits Mary with all objective knowledge, and only objective knowledge precisely because he is trying to establish the existence of subjective knowledge .. what Mary doesnt know must be subjective, if there is something Mary doesn’t know. So the eventual point s that there s more to knowledge than objective knowledge.
So you don’t show that Jackson is wrong by agreeing with him.
But I don’t know that you think Jackson is wrong,
Mary’s room seems to be arguing that,
(translation: the experience of seeing red is no the experience of understanding how seeing red works)
This is true, when we take those statements literally. But it’s true in the same sense a Gödel encoding of statement in PA is not literally that statement. It is just a representation, but the representation is exactly homomorphic to its referent. Mary’s representation of reality is presumed complete ex hypothesi, therefore she will understand exactly what will happen in her brain after seeing color, and that is exactly what happens.
You wouldn’t call a statement of PA that isn’t a literally a Gödel encoding of a statement (for some fixed encoding) a non-mathematical statement. For one, because that statement has a Gödel encoding by necessity. But more importantly, even though the statement technically isn’t literally a Gödel-encoding, it’s still mathematical, regardless.
Mary’s know how she will respond to learning what red is like. Mary knows how others will respond. This exhausts the space of possible predictions that could be made on behalf of this subjective knowledge, and it can be done without it.
Tangentially to this discussion, but I don’t think that is a wise way of labeling that knowledge.
Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?
Mary has all objective knowledge, but certain facts about her own future behavior must escape her, because any certainty could trivially be negated.
There are three possibilities worth disambiguating here.
1) Mary predicts that she will do X given some assumed set S1 of knowledge, memories, experiences, etc., AND S1 includes Mary’s knowledge of this prediction.
2) Mary predicts that she will do X given some assumed set S2 of knowledge, memories, experiences, etc., AND S2 does not include Mary’s knowledge of this prediction.
3) Mary predicts that she will do X independent of her knowledge, memories, experiences, etc.
Mary is presumed to have all objective knowedge and only objectve knowledge, Your phrasing is ambiguous and therefire doesnt address the point. When you say Mary will know what happens when she sees red, do you mean she knows how red looks subjectively, or she knows something objective like what her behaviour will be.....further on you mention predicting her reactions,
Is that supposed to relate to the objective/ subjective distinction somehow?
So? The overall point is about physicalism, and to get to ‘physicalism is false’, all you need is the existence of subjective knowledge, not its usefulness in making prediction. So again I don’t see the relevance
Maybe. I don’t see the problem. There is still an unproblematic sense in which Mary has all objective knowledge, even if it doesn’t allow her to do certain things. If that was the point.
The behavior of the neurons in her skull is an objective fact, and this is what I mean to referring to. Apologies for the ambiguity.
The latter. The former is purely experiential knowledge, and as I have repeatedly said is contained in a superset of verbal (what you call ‘objective’) knowledge, but is disjoint with the set of verbal (‘objective’) knowledge itself. This is my box metaphor.
Yes. Assuming the Godel encoding is fixed, [the metaphor is that] any and all statements of PA are experiential knowledge (an experience, in simple terms), non-Godel statements of PA are purely experiential knowledge; the redness of red, say, and finally the Godel statements of PA are verbal knowledge, or ‘objective knowledge’ in your terminology.
Despite not being Godel statements in the encoding, the second item in the above list is still mathematical, and redness of red is still physical.
What does this knowledge do? How do we tell the difference between someone with and without these ‘subjective experiences’? What definition of knowledge admits it as valid?
You have said that according to you, stipulatively, subjective knowledge is a subset of objective knowledge. What we mean by objective knowledge is generally knowledge that can be understood at second hand, without being in a special state or having had particular experiences. You say that the subjective subset of objective knowledge is somehow opaque, so that it does not have the properties usually associated with objective knowledge..but why should anyone believe it is objective, when it lacks the usual properties, and is only asserted to be objective?
I can’t see how that has been proven. You can’t prove that redness is physically encoded in the relevant sense just by noting that physical changes occur in brains, because
1 There’s no physical proof of physicalism
2 An assumption of physicalism is question begging
3 You need an absence of non physical proeties, states and processes, not just the presence of physical changes
4 Physicalism as a meaningful claim, and not just a stipulative label needs to pay its way in explanation...but its ability to explain subjective knowledge is just at is in question.
Its hard to prove the existence of subjective knowledge in an objective basis. What else would you expect.? There is a widespread belief in subjective, experiential knowledge and the evidence for it is subjective. The alternative is the sort of thing caricatured as ‘how was it for me, darling’.
If one of Mary’s predictions is, “When I see red, I will say, ’wow! I didn’t know that red looked like that!”, then the fact that she has predicted this in advance is hardly evidence that she does not learn anything by seeing red. If anything, it proves that she does.
I think your post seems to have been a reply to me. I’m the one who still accepts physicalism. AncientGreek is the one who rejects it.
I realize that. A reply doesn’t necessarily have to be an argument against the person it is a reply to.
The AI analogue would be: If the AI has the capacity to wirehead itself, it can make itself enter the color perception subroutines. Whether something new is learned depends on the remaining brain architecture. I would say, in the case of humans, it is clear that whenever something new is experienced, the human learns what that experience feels like. I reckon that for some people with strong visualization (in a broad sense) abilities it is possible to know what an experience feels like without experiencing first hand by synthesizing a new experience from previously known experiences. But in most cases there is a difference between imagining a sensation and experiencing it.
In the case of the AI, there could either be the case where no information is passed between the color perception subroutine and the main processing unit, in which case the AI may have a new experience, but not learn anything new. Or some representation of the experience of being in the subroutine is saved to memory, in which case something new is learned.
The stronger someones imaginative ability is, the more their imagining an experience is actually having it, in terms of brain states....and the less it s a counterexample to anything relevant.
If the knowedge the AI gets from the colour routine is unproblematically encoded in a string of bits, why can’t it just look at the string of bits...for that matter, why can’t Mary just look at the neural spike trains of someone seing red?
Why can’t we just eat a picture of a plate of spaghetti instead of actual spaghetti? Because a representation of some thing is not the thing itself. Am I missing something?
Yes: is about a kind of knowledge.
The banal truth here is that knowing about a thing doesn’t turn you into it.
The significant and contentious claim is that here are certain kinds of knowledge that can only be accessed by instantiating a brain state. The existence of such subjective knowledge leads to a further argument against physicalism.
But in order to create a realistic experience she would have to create a false memory of having seen red, which is something that an agent (human or AI) that values epistemic rationality would not want to do.
Since you’d know it was a false memory, it doesn’t necessarily seem to be a problem, at least if you really need to know what red is like for some reason.
If you know that it is a false memory then the experience is not completely accurate, though it may be perhaps more accurate than what human imagination could produce.