I was asking a disguised question. I really wanted to know: “which of the two future selfs do you identify with, and why?”
Oh, that’s easy. Both of them, equally. Assuming accurate enough simulations etc., of course.
ETA: Why? Well, they’ll both think that they’re me, and I can’t think of a way to disprove the claim of one without also disproving the claim of the other.
ETA: Why? Well, they’ll both think that they’re me, and I can’t think of a way to disprove the claim of one without also disproving the claim of the other.
Any of the models of consciousness-as-continuity would offer a definitive prediction.
Any of the models of consciousness-as-continuity would offer a definitive prediction.
IMO, there literally is no fact of the matter here, so I will bite the bullet and say that any model that supposes there is one is wrong. :) I’ll reconsider if you can point to an objective feature of reality that changes depending on the answer to this. (So-and-so will think it to be immoral doesn’t count!)
PS: The up/down karma vote isn’t a record of what you agree with, but whether a post has been reasonably argued.
It is neither of those things. This isn’t debate club. We don’t have to give people credit for finding the most clever arguments for a wrong position.
I make no comment about the subject of debate is in this context (I don’t know or care which party is saying crazy things about ‘conciousness’). I downvoted the parent specifically because it made a normative assertion about how people should use the karma mechanism which is neither something I support nor an accurate description of an accepted cultural norm. This is an example of voting being used legitimately in a way that is nothing to do with whether the post has been reasonably argued.
I did use the term “reasonably argued” but I didn’t mean clever. Maybe “rationally argued”? By my own algorithm a cleverly argued but clearly wrong argument would not garner an up vote.
I gave you an upvote for explaining your down vote.
I did use the term “reasonably argued” but I didn’t mean clever. Maybe “rationally argued”? By my own algorithm a cleverly argued but clearly wrong argument would not garner an up vote.
You are right, ‘clever’ contains connotations that you wouldn’t intend. I myself have used ‘clever’ as term of disdain and I don’t want to apply that to what you are talking about. Let’s say stick with either of the terms you used and agree that we are talking about arguments that are sound, cogent and reasonable rather than artful rhetoric that exploits known biases in human social behaviour to score persuasion points. I maintain that even then down-votes are sometimes appropriate. Allow me to illustrate.
There are two outwardly indistinguishable boxes with buttons that display heads or tails when pressed. You know that one of the boxes returns true 70% of the time, the other returns heads 40% of the time. A third party, Joe, has experimented with the first box three times and tells you that each time it returned true. This represents an argument that the first box is the “70%” box. Now, assume that I have observed the internals of the boxes and know that the first box is, in fact, the 40% box.
Whether I downvote Joe’s comment depends on many things. Obviously, tone matters a lot, as does my impression of whether Joe’s bias is based on dis-ingenuity or more innocent ignorance. But even in the case when Joe is arguing in good faith there are some cases where a policy attempting to improve the community will advocate downvoting the contribution. For example if there is a significant selection bias in what kind of evidence people like Joe have exposed themselves to then popular perception after such people share their opinions will tend to be even more biased than the individuals alone. In that case downvoting Joe’s comment improves the discussion. The ideal outcome would be for Joe to learn to stfu until he learns more.
More simply I observe that even the most ‘rational’ of arguments can be harmful if the selection process for the creation and repetition of those arguments is at all biased.
I won’t because that’s not what I’m arguing. My position is that subjective experience has moral consequences, and therefore matters.
OK, that’s fine, but I’m not convinced—I’m having trouble thinking of something that I consider to be a moral issue that doesn’t have a corresponding consequence in the territory.
PS: That downvote wasn’t me. I’m aware of how votes work around here. :)
Example: is it moral to power-cycle (hibernate, turn off, power on, restore) a computer running an self-aware AI? WIll future machine intelligences view any less-than-necessary AGI experiments I run the same way we do Josef Mengele’s work in Auschwitz? Is it a possible failure mode that an unfriendly/not-proovably-friendly AI that experiences routine power cycling might uncover this line of reasoning and decide it doesn’t want to “die” every night when the lights go off? What would it do then?
OK, in a hypothetical world where somehow pausing a conscious computation—maintaining all data such that it could be restarted losslessly—is murder, those are concerns. Agreed. I’m not arguing against that.
My position is that pausing a computation as above happens to not be murder/death, and that those who believe it is murder/death are mistaken. The example I’m looking for is something objective that would demonstrate this sort of pausing is murder/death. (In my view, the bad thing about death is its permanence, that’s most of why we care about murder and what makes it a moral issue.)
As Eliezer mentioned in his reply (in different words), if power cycling is death, what’s the shortest suspension time that isn’t? Currently most computers run synchronously off a common clock. The computation is completely suspended between clock cycles. Does this mean that an AI running on such a computer is murdered billions of times every second? If so, then morality leading to this absurd conclusion is not a useful one.
Edit: it’s actually worse than that: digital computation happens mostly within a short time of the clock level switch. The rest of the time between transitions is just to ensure that the electrical signals relax to within their tolerance levels. Which means that the AI in question is likely dead 90% of the time.
What Eliezer and you describe is more analogous to task switching on a timesharing system, and yes my understanding of computational continuity theory is that such a machine would not be sent to oblivion 120 times a second. No, such a computer would be strangely schizophrenic, but also completely self-consistent at any moment in time.
But computational continuity does have a different answer in the case of intermediate non-computational states. For example, saving the state of a whole brain emulation to magnetic disk, shutting off the machine, and restarting it sometime later. In the mean time, shutting off the machine resulted in decoupling/decoherence of state between the computational elements of the machine, and general reversion back to a state of thermal noise. This does equal death-of-identity, and is similar to the transporter thought experiment. The relevance may be more obvious when you think about taking the drive out and loading it in another machine, copying the contents of the disk, or running multiple simulations from a single checkpoint (none of these change the facts, however).
In the mean time, shutting off the machine resulted in decoupling/decoherence of state between the computational elements of the machine, and general reversion back to a state of thermal noise.
It is probably best for you to stay away from the physics/QM point of view on this, since you will lose: the states “between the computational elements”, whatever you may mean by that, decohere and relax to “thermal noise” much quicker than the time between clock transitions, so there no difference between a nanosecond an an hour.
Maybe what you mean is more logic-related? For example, when a self-aware algorithm (including a human) expects one second to pass and instead measures a full hour (because it was suspended), it interprets that discrepancy of inputs as death? If so, shouldn’t any unexpected discrepancy, like sleeping past your alarm clock, or day-dreaming in class, be treated the same way?
This does equal death-of-identity, and is similar to the transporter thought experiment.
I agree that forking a consciousness is not a morally trivial issue, but that’s different from temporary suspension and restarting, which happens all the time to people and machines. I don’t think that conflating the two is helpful.
It is probably best for you to stay away from the physics/QM point of view on this, since you will lose: the states “between the computational elements”, whatever you may mean by that, decohere and relax to “thermal noise” much quicker than the time between clock transitions, so there no difference between a nanosecond an an hour.
Maybe what you mean is more logic-related?...
No, I meant the physical explanation (I am a physicist, btw). It is possible for a system to exhibit features at certain frequencies, whilst only showing noise at others. Think standing waves, for example.
I agree that forking a consciousness is not a morally trivial issue, but that’s different from temporary suspension and restarting, which happens all the time to people and machines. I don’t think that conflating the two is helpful.
When does it ever happen to people? When does your brain, even just regions ever stop functioning, entirely? You do not remember deep sleep because you are not forming memories, not because your brain has stopped functioning. What else could you be talking about?
Hmm, I get a feeling that none of these are your true objections and that, for some reason, you want to equate suspension to death. I should have stayed disengaged from this conversation. I’ll try to do so now. Hope you get your doubts resolved to your satisfaction eventually.
I don’t want to, I just think that the alternatives lead to absurd outcomes that can’t possibly be correct (see my analysis of the teleporter scenario).
I really have a hard time imagining a universe where there exists a thing that is preserved when 10^-9 seconds pass between computational steps but not when 10^3 pass between steps (while I move the harddrive to another box).
Yes. Probabilities aside, this is what I was asking.
I was asking a disguised question. I really wanted to know: “which of the two future selfs do you identify with, and why?”
Oh, that’s easy. Both of them, equally. Assuming accurate enough simulations etc., of course.
ETA: Why? Well, they’ll both think that they’re me, and I can’t think of a way to disprove the claim of one without also disproving the claim of the other.
Any of the models of consciousness-as-continuity would offer a definitive prediction.
IMO, there literally is no fact of the matter here, so I will bite the bullet and say that any model that supposes there is one is wrong. :) I’ll reconsider if you can point to an objective feature of reality that changes depending on the answer to this. (So-and-so will think it to be immoral doesn’t count!)
I won’t because that’s not what I’m arguing. My position is that subjective experience has moral consequences, and therefore matters.
PS: The up/down karma vote isn’t a record of what you agree with, but whether a post has been reasonably argued.
For many people, the up/down karma vote is a record of what we want more/less of.
It is neither of those things. This isn’t debate club. We don’t have to give people credit for finding the most clever arguments for a wrong position.
I make no comment about the subject of debate is in this context (I don’t know or care which party is saying crazy things about ‘conciousness’). I downvoted the parent specifically because it made a normative assertion about how people should use the karma mechanism which is neither something I support nor an accurate description of an accepted cultural norm. This is an example of voting being used legitimately in a way that is nothing to do with whether the post has been reasonably argued.
I did use the term “reasonably argued” but I didn’t mean clever. Maybe “rationally argued”? By my own algorithm a cleverly argued but clearly wrong argument would not garner an up vote.
I gave you an upvote for explaining your down vote.
You are right, ‘clever’ contains connotations that you wouldn’t intend. I myself have used ‘clever’ as term of disdain and I don’t want to apply that to what you are talking about. Let’s say stick with either of the terms you used and agree that we are talking about arguments that are sound, cogent and reasonable rather than artful rhetoric that exploits known biases in human social behaviour to score persuasion points. I maintain that even then down-votes are sometimes appropriate. Allow me to illustrate.
There are two outwardly indistinguishable boxes with buttons that display heads or tails when pressed. You know that one of the boxes returns true 70% of the time, the other returns heads 40% of the time. A third party, Joe, has experimented with the first box three times and tells you that each time it returned true. This represents an argument that the first box is the “70%” box. Now, assume that I have observed the internals of the boxes and know that the first box is, in fact, the 40% box.
Whether I downvote Joe’s comment depends on many things. Obviously, tone matters a lot, as does my impression of whether Joe’s bias is based on dis-ingenuity or more innocent ignorance. But even in the case when Joe is arguing in good faith there are some cases where a policy attempting to improve the community will advocate downvoting the contribution. For example if there is a significant selection bias in what kind of evidence people like Joe have exposed themselves to then popular perception after such people share their opinions will tend to be even more biased than the individuals alone. In that case downvoting Joe’s comment improves the discussion. The ideal outcome would be for Joe to learn to stfu until he learns more.
More simply I observe that even the most ‘rational’ of arguments can be harmful if the selection process for the creation and repetition of those arguments is at all biased.
OK, that’s fine, but I’m not convinced—I’m having trouble thinking of something that I consider to be a moral issue that doesn’t have a corresponding consequence in the territory.
PS: That downvote wasn’t me. I’m aware of how votes work around here. :)
Example: is it moral to power-cycle (hibernate, turn off, power on, restore) a computer running an self-aware AI? WIll future machine intelligences view any less-than-necessary AGI experiments I run the same way we do Josef Mengele’s work in Auschwitz? Is it a possible failure mode that an unfriendly/not-proovably-friendly AI that experiences routine power cycling might uncover this line of reasoning and decide it doesn’t want to “die” every night when the lights go off? What would it do then?
OK, in a hypothetical world where somehow pausing a conscious computation—maintaining all data such that it could be restarted losslessly—is murder, those are concerns. Agreed. I’m not arguing against that.
My position is that pausing a computation as above happens to not be murder/death, and that those who believe it is murder/death are mistaken. The example I’m looking for is something objective that would demonstrate this sort of pausing is murder/death. (In my view, the bad thing about death is its permanence, that’s most of why we care about murder and what makes it a moral issue.)
As Eliezer mentioned in his reply (in different words), if power cycling is death, what’s the shortest suspension time that isn’t? Currently most computers run synchronously off a common clock. The computation is completely suspended between clock cycles. Does this mean that an AI running on such a computer is murdered billions of times every second? If so, then morality leading to this absurd conclusion is not a useful one.
Edit: it’s actually worse than that: digital computation happens mostly within a short time of the clock level switch. The rest of the time between transitions is just to ensure that the electrical signals relax to within their tolerance levels. Which means that the AI in question is likely dead 90% of the time.
What Eliezer and you describe is more analogous to task switching on a timesharing system, and yes my understanding of computational continuity theory is that such a machine would not be sent to oblivion 120 times a second. No, such a computer would be strangely schizophrenic, but also completely self-consistent at any moment in time.
But computational continuity does have a different answer in the case of intermediate non-computational states. For example, saving the state of a whole brain emulation to magnetic disk, shutting off the machine, and restarting it sometime later. In the mean time, shutting off the machine resulted in decoupling/decoherence of state between the computational elements of the machine, and general reversion back to a state of thermal noise. This does equal death-of-identity, and is similar to the transporter thought experiment. The relevance may be more obvious when you think about taking the drive out and loading it in another machine, copying the contents of the disk, or running multiple simulations from a single checkpoint (none of these change the facts, however).
It is probably best for you to stay away from the physics/QM point of view on this, since you will lose: the states “between the computational elements”, whatever you may mean by that, decohere and relax to “thermal noise” much quicker than the time between clock transitions, so there no difference between a nanosecond an an hour.
Maybe what you mean is more logic-related? For example, when a self-aware algorithm (including a human) expects one second to pass and instead measures a full hour (because it was suspended), it interprets that discrepancy of inputs as death? If so, shouldn’t any unexpected discrepancy, like sleeping past your alarm clock, or day-dreaming in class, be treated the same way?
I agree that forking a consciousness is not a morally trivial issue, but that’s different from temporary suspension and restarting, which happens all the time to people and machines. I don’t think that conflating the two is helpful.
No, I meant the physical explanation (I am a physicist, btw). It is possible for a system to exhibit features at certain frequencies, whilst only showing noise at others. Think standing waves, for example.
When does it ever happen to people? When does your brain, even just regions ever stop functioning, entirely? You do not remember deep sleep because you are not forming memories, not because your brain has stopped functioning. What else could you be talking about?
Hmm, I get a feeling that none of these are your true objections and that, for some reason, you want to equate suspension to death. I should have stayed disengaged from this conversation. I’ll try to do so now. Hope you get your doubts resolved to your satisfaction eventually.
I don’t want to, I just think that the alternatives lead to absurd outcomes that can’t possibly be correct (see my analysis of the teleporter scenario).
I really have a hard time imagining a universe where there exists a thing that is preserved when 10^-9 seconds pass between computational steps but not when 10^3 pass between steps (while I move the harddrive to another box).