I think it is factually correct that we get Alzheimer’s and dementia at old age because the brain gets worn out. Whether it is because of failing to keep up with all the memory accumulation could be more speculative. So I admit that I shouldn’t have made that claim.
But the brain gets worn out from what? Doing its job. And what’s its job...?
Anyway, I think it would be more productive to at least present an explanation in a couple of lines rather than only saying that I’m wrong.
“So, odds are, you would not need to explicitly delete anything, it fades away with disuse.”
I don’t know. Even some old people feel overwhelmed with so many memories. The brain does some clean-up, for sure. But I doubt whether it would work for really long timelines.
“So, I don’t expect memory accumulation to be an obstacle to eternal youth. Also, plenty of time to work on brain augmentations and memory offloading to external storage :)”
Mind you that your personal identity is dependent on your “story”, which has to encompass all your life, even if only the very key moments. My concern is that, as time tends to infinite, so does the “backbone” of memory, i.e. the minimum necessary, after “trimming” with the best technology possible, to maintain personal identity which should include memories from all along the timeline. So the question is whether a finite apparatus, the brain, can keep up with infinite data. Is it possible to remain you while not remembering all your story? And I don’t even care about not remembering all my story. I just care about remaining me.
We know that a computer can theoretically function forever. Just keep repairing its parts (same could be done with the brain, no doubt) and deleting excess data. But the computer doesn’t have a consciousness / personal identity which is dependent on its data. So computationalism might be leading us astray here. (Yes, I don’t like computationalism/functionalism.)
Note: this is all speculation, of which I’m quite uncertain of. Before all the downvotes come (not that I care, but just to make it clear anyway).
The concern here is not boredom. I even believe that boredom could be solved with some perfect drug or whatever. The concern here is whether a consciousness identity can properly exist forever without inevitably degrading.
“No. There is nothing I find inherently scary or unpleasant about nonexistence.”
Would you agree that you’re perhaps a minority? That most people are scared/depressed about their own mortality?
“I’m just confused about the details of why that would happen. I mean, it would be sad if some future utopia didn’t have a better solution for insanity or for having too many memories, than nonexistence.
Insanity: Look at the algorithm of my mind and see how it’s malfunctioning? If nothing else works, revert my mindstate back a few months/years?
Memories: offload into long-term storage?”
On insanity, computationalism might be false. Consciousness might not be algorithmic. If it is, you’re right, it’s probably easy to deal with.
But I suspect that excess memories might always remain a problem. Is it really possible to off-load them while maintaining personal identity? That’s an open question in my view.
Specially when, like me, you don’t really buy into computationalism.
“Do I choose between being forced to exist forever, or to die after less than 100 years of existence? Neither. I’d like to have the option to keep living for as long as I want.”
I didn’t mean being forced to exist forever, or pre-commiting to anything. I meant that I really do WANT to exist forever, yet I can’t see a way that it can work. That’s the dilemma that I mentioned: to die, ever, even after a gazillion years, feels horrible, because YOU will cease to exist, no matter after how much time. To never die feels just as horrible because I can’t see a way to remain sane after a very long time.
Who can guarantee that after x years you would feel satisfied and ready to die? I believe that as long as the brain remains healthy, it really doesn’t wanna die. And even if it could reach the state of accepting death, the current me just don’t ever wanna cease to exist. Doesn’t the idea of inevitably eventually ceasing to exist feel absolutely horrible to you?
You’re coming from a psychologic/spiritual point of view, which is valid. But I think you should perhaps consider a bit more the scientific perspective. Why do people get Alzheimer’s and dementia at old age? Because the brain fails to keep up with all the experience/memory accumulation. The brain gets worn out, basically. My concern is more scientific than anything. Even with the best psychotherapy or the best meditation or even the best brain tinkering possible, as time tends to infinite so do memories and so does “work” for the brain to do, and unfortunately the brain is finite, so it will invariably get overwhelmed eventually.
Like, I don’t doubt that in a few centuries or millenia we would have invented the technology to no longer get our brains worn out at age 100 but only at 1000 or 5000, but I don’t think we’ll ever invent the technology to avoid it past age 1 billion (just a gross estimate of course).
Personally, I’m only 30 and I don’t feel tired of living at all, fortunately. Like I said, I wanna live forever. But both my intuition and these scientific considerations tell me that it can’t remain like that indefinitely.
Thanks for the feedback (and the back-up). Well, I’d say that half of what I write on Lesswrong is downvoted and 40% is ignored, so I don’t really care at this point. I don’t think (most of) my opinions are outlandish. I never downvote anything I disagree with myself, and there’s plenty that I disagree with. I only downvote absurd or rude comments. I think that’s the way it should be, so I’m in full agreement on that.
You also got it right on my main point. That’s precisely it. Mind you that “ending” consciousness also feels horrid to me! That’s the dilemma. Would be great if we could find a way to achieve neverending consciousness without it being horrid.
″ I can’t imagine getting bored with life even after a few centuries.”
Ok, but that’s not a lot of time, is it? Furthermore, this isn’t even a question of time. For me, no finite time is enough. It’s the mere fact of ceasing to exist. Isn’t it simply horrifying? Even if you live a million healthy years, no matter, the fact is that you will cease to exist one day. And then there will be two options. Either your brain will be “healthy” and therefore will dread death as much as it does now, or it will be “unhealthy” and welcome death to relieve it’s poor condition. To me both options seem horrifying—what matters more is that the present me wants to live forever with a healthy, not tired of life, not with dementia or Alzheimer’s, sane brain forever.
Like I said in my post, as time tends to infinite, so do memories, so there will inevitably be a time where the brain no longer holds up. Even the best brain that science could buy in a far future, so to speak. It seems that we will have to cease to exist inevitably.
“I’m sure that there will be people that choose to die, and others that choose to deadhead, and those that choose to live with only a rolling century of memories. None of those sound appealing to my imagination at this point.”
The latter actually sounds kinda appealing to me, if only I was more convinced that it was possible...
“I don’t think we need to answer these questions to agree that many people would prefer to live longer than they currently are able.”
Certainly, but that’s not the issue here. The issue here is immortality. Many transhumanists desire to live forever, literally. Myself included. In fact I believe that many people in general do. Extending the human lifespan to thousands of years would be a great victory already, but that doesn’t invalidate the search for true immortality, if people are interested in such which I’m sure some are.
“I have no idea what problems need to be solved to enable people to happily and productively live thousands of years, but I also have no reason to believe they’re insurmountable.”
Once again you’re deviating from the question. I haven’t questioned the possibility of living a few thousand years, but forever properly.
1.) Suffering seems to need a lot of complexity, because it demands consciousness, which is the most complex thing that we know of.
2.) I personally suspect that the biological substrate is necessary (of course that I can’t be sure.) For reasons, like I mentioned, like sleep and death. I can’t imagine a computer that doesn’t sleep and can operate for trillions of years as being conscious, at least in any way that resembles an animal. It may be superintelligent but not conscious. Again, just my suspicion.
3.) I think it’s obvious—it means that we are trying to recreate something that biological systems do (arithmetics, imagine recognition, playing games, etc) on these electronic systems called computers or AI. Just like we try to recreate a murder scene with pencils and paper. But the murder drawing isn’t remotely a murder, it’s only a basic representation of a person’s idea of a murder.
4.) Correct. I’m not completely excluding that possibility, but like I said, it would be a great luck to get there not on purpose. Maybe not “winning the lottery” luck as I’ve mentioned, but maybe 1 to 5% probability.
We must understand that suffering takes consciousness, and consciousness takes a nervous system. Animals without one aren’t conscious. The nature of computers is so drastically different from that of a biological nervous system (and, at least until now, much less complex) that I think it would be quite unlikely that we eventually unwillingly generate this very complex and unique and unknown property of biological systems that we call consciousness. I think it would be a great coincidence.
I never said you claimed such either, but Charbel did.
“It is possible that this [consciousness] is the only way to effectively process information”
I was replying to his reply to my comment, hence I mentioned it.
Consciousness definitely serves a purpose, from an evolutionary perspective. It’s definitely an adaptation to the environment, by offering a great advantage, a great leap, in information processing.
But from there to say that it is the only way to process information goes a long way. I mean, once again, just think of the pocket calculator. Is it conscious? I’m quite sure that it isn’t.
I think that consciousness is a very biological thing. The thing that makes me doubt the most about consciousness in non-biological systems (let alone in the current ones which are still very simple) is that they don’t need to sleep and they can function indefinitely. Consciousness seems to have these limits. Can you imagine not ever sleeping? Not ever dying? I don’t think such would be possible for any conscious being, at least one remotely similar to us.
I highly doubt this on an intuitive level. If a draw a picture of a man being shot, is it suffering? Naturally not, since those are just ink pigments in a sheet of cellulose. Suffering seems to need a lot of complexity and also seems deeply connected to biological systems. AI/computers are just a “picture” of these biological systems. A pocket calculator appears to do something similar to the brain but in reality it’s much less complex and much different, and it’s doing something completely different. In reality it’s just an electric circuit. Are lightbulbs moral patients?
Now, we could someday crack consciousness in electronic systems, but I think it would be winning the lottery to get there not on purpose.
Everyone knows that the Holocaust wasn’t just genocide. It was also torture, evil medical experiments, etc. But you’re right, I should have used a better example. Not that I think that anyone really misunderstood what I meant.
“That said, I’m not convinced that permanent Holocaust is worse than permanent extinction, but that’s irrelevant to my point anyway.”
Maybe it’s not. What we guess are other people’s values is heavily influenced by our own values. And if you are not convinced that permanent Holocaust is worse than permanent extinction, then, no offense, but you have a very scary value system.
“If someone isn’t convinced by the risk of permanent extinction, are you likely to convince them by the (almost certainly smaller) risk of permanent Holocaust instead?”
Naturally, because the latter is orders of magnitude worse than the former. But again, if you don’t share this view, I can’t see myself convincing you.
And we also have no idea if it really is smaller. But even a small risk of an extremely bad outcome is reason for high alarm.
“I love violence and would hope that Mars is an utter bloodbath.”
The problem is that biological violence hurts like hell. Even most athletes live with chronic pain, imagine most warriors. Naturally we could solve the pain part, but then it wouldn’t be the violence I’m referring to. It would be videogame violence, which I’m ok with since it doesn’t cause pain or injury or death. But don’t worry, I still got the joke!
“”don’t kill anyone and don’t cause harm/suffering to anyone”
The problem with this one is that the AI’s optimal move is to cease to exist.”
I’ve thought about it as well. Big brain idea: perhaps the first AGI’s utility function would be to act in the real world as minimally as possible, maybe only with the goal of preventing other people from developing AGI, keeping like this until we solve alignment? Of course this latter part of policing the world would be already prone to a lot of ambiguity and sophism, but again, if we program do not’s (do not let anyone else build AGI, plus do not kill anyone, plus do not cause suffering, etc) instead of do’s, it could lead to a lot less ambiguity and sophism, by drastically curtailing the maneuver space. (Not that I’m saying that it would be easy.) As opposed to like “cure cancer”, or “build an Eiffel tower”.
“And that’s already relying on being able to say what ‘kill someone’ means in a sufficiently clear way that it will satisfy computer programmers”
I don’t think so. When the brain irreversibly stops you’re dead. It’s clear. This plays into my doubt that perhaps we keep underestimating the intelligence of a superintelligence. I think that even current AIs could be made to discern when a person is dead or alive, perhaps even better than us already.
“For instance, when Captain Kirk transports down to the Planet-of-Hats, did he just die when he was disassembled, and then get reborn? Do we need to know how the transporter works to say?”
Maybe don’t teletransport anyone until we’ve figured that out? There the problem is teletransportation itself, not AGI efficiently recognizing what is death at least as well as we do. (But I’d venture saying that it could even solve that philosophical problem, since it’s smarter than us.)
“Stuart Russell is a very clever man, and if his approach to finessing the alignment problem can be made to work then that’s the best news ever, go Stuart!
But I am a little sceptical because it does seem short on details, and the main worry is that before he can get anywhere, some fool is going to create an unaligned AI, and then we are all dead.”
I gotta admit that I completely agree.
“Whereas alignment looks harder and harder the more we learn about it.”
I’ll say that I’m not that much convinced about most of this that I’ve said. I’m still way more to the side that “control is super difficult and we’re all gonna die (or worse)”. But I keep thinking about these things, to see if maybe there’s a “way out”. Maybe we in this community have built a bias that “only the mega difficult value alignment will work”, when it could be false. Maybe it’s not just “clever hacks”, maybe there are simply more efficient and tractable ways to control advanced AI than the intractable value alignment. But again, I’m not even that much convinced myself.
“and I am pretty sure that the author of ‘Failed Utopia 4-2’ has at least considered the possibility that it might not be so bad if we only get it 99%-right.”
Exactly. Again, perhaps there are much more tractable ways than 100% alignment. OR, if we could at least solve worst-case AI safety (that is, prevent s-risk) it would already be a massive win.
“Plenty of ‘failed utopia’-type outcomes that aren’t exactly what we would ideally want would still be pretty great, but the chances of hitting them by accident are very low.”
I’m assuming you’ve read Eliezer’s post “Failed Utopia 4-2”, since you use the expression? I’ve actually been thinking a lot about that, how that specific “failed utopia” wasn’t really that bad. In fact it was even much better than the current world, as disease and aging and I’m assuming violence too got all solved at the cost of all families being separated for a few decades, which is a pretty good trade if you ask me. It makes me think if there’s some utility function with unaligned AI that could lead to some kind of nice future, like “don’t kill anyone and don’t cause harm/suffering to anyone”. The truth is that in stories of genies the wishes are always very ambiguous, so a “wish” stated negatively (don’t do this) might lead to less ambiguity than one stated positively (do that).
But this is even assuming that it will be possible to give utility functions to advanced AI, which I’ve heard some people say it won’t.
This also plays into Stuart Russell’s view. His approach seems much more simple than alignment, it’s just in short not letting the advanced AI know its final objective. It makes me think whether there could be solutions to the advanced AI problem that would be more tractable than the intractable alignment.
Perhaps it’s not that difficult after all.
“Significant chances of Hell, maybe I take the nice safe extinction option if available.”
The problem is that it isn’t available. Plus realistically speaking the good outcome percentage is way below 50% without alignment solved.
“From the point of view of most humans, there are few outcomes worse than extinction of humanity (x-risk).”
That’s obviously not true. What would you prefer: extinction of humanity, or permanent Holocaust?
“Are you implying that most leaders would prefer extinction of humanity to some other likely outcome, and could be persuaded if we focused on that instead?”
Anyone would prefer extinction to say a permanent Holocaust. Anyone sane at least. But I’m not implying that they would prefer extinction to a positive outcome.
“but I also think that those unpersuaded by the risk of extinction wouldn’t be persuaded by any other argument anyway”
I’ll ask you again: which is worse, extinction or permanent Holocaust?
Perhaps our main difference is that you seem to believe in computationalism, while I don’t. I think consciousness is something fundamentally different from a computer program or any other kind of information. It’s experience, which is beyond information.