Just by the by, it might be a good party piece for you, but it would be a truly horrible party piece for half the people you performed it to.
UnholySmoke
Karma: 204
I, Eliezer Yudkowsky, do now publicly announce that I am not planning to commit suicide, at any time ever, but particularly not in the next couple of weeks
ROFLcopters.
18 months too late, but http://xkcd.com/505/
By Eliezer’s line of reasoning above—that the subjective experience is in the causal change between one state and the ‘next’ then yes, symbols are as good a substrate as any. FWIW, this is how I see things too.
Ha, never noticed this. What I meant was ‘Stupid me forgetting to log in.’ So yes, we’re worried! ;)
Ben
I think you are right that paperclip maximizers would not care at all about ethics.
Correct. But neither would they ‘care’ about paperclips, under the way Eliezer’s pushing this idea. They would flarb about paperclips, and caring would be as alien to them as flarbing is to you.
Seconded. One of the many modern connotations of ‘Singularity’ is ‘Geek Apocalypse’.
Which is happening, like, a good couple of years afterwards.
Intelligence explosion does away with that, and seems to nail the concept much better anyway.
How about the middle ground—“If constant PR consideration stops you from expressing yourself all the time, maybe it’s time to reconsider your priorities”?
Posting stuff on Facebook that might get you in trouble is the archetype these day I suppose, but I really can’t bring myself to care about things like that.
Maybe I just don’t have a strong enough terminal value to protect right now, but I find it easier to imagine myself thinking, 50 years hence, “I wish I’d just decided ‘to hell with it’ and said what I thought” than “I wish I’d shut up, gone with the flow and eased my path.”
I’ll hit you up in late 2059 and let you know how that went.
There’s a thesis in there somewhere.
We all know what’s really going down. The Dark Lords of the Matrix are currently cacking themselves and coming up with semi-plausible reasons to break the thing until they can decide on a long-term strategy.
Favourite album post-1960?
Who actually gets off on earning loads of karma across multiple accounts with no-one knowing?
Please stop allowing your practical considerations get in the way of the pure, beautiful counterfactual!
Seriously though, either you allow yourself to suspend practicalities and consider pure decision theory, or you don’t. This is a pure maths problem, you can’t equate it to ‘John has 4 apples.’ John has 3^^^3 apples here, causing your mind to break. Forget the apples and years, consider utility!
My commiserations, to the extent that you seem to need them.
I’d like to imagine I’d have a similar reaction, this is an inspiring post. All the best.
Cracking idea, like it a lot. Hofstadter would jump for joy, and in his honour:
Beware of generalising across people you haven’t spent much time around, however tempting the hypothesis. Drawing a map of the city from your living room etc.
My first 18 years were spent attending a Catholic church once a week. To the extent that we can ever know what other people actually believe (whatever that means), most of them have genuinely internalised the bits they understand. Like, really.
We can call into question what we mean by ‘believe’, but I can’t agree that a majority of the world population is just cynically going with the flow. Finally, my parish priest is one of the most intelligent people I’ve ever met, and he believed in his god harder/faster/whatever than I currently believe anything. Scary thought, right?
Also upvoted, and very succintly put.
Rationality is a tool we use get to our terminal value. And what do we do when that tool tells us our terminal value is irrational?
Never ask that question.
I wonder whether you can hold to any meaningful ‘individual’, whether the difference be bit-wise or no.
Indeed, that’s what I’m driving at.
Harking back to my earlier comment, changing a single bit and suddenly having a whole new person is where my problem arises. If you change that bit back, are you back to one person? I might not be thinking hard enough, but my intuition doesn’t accept that. With that in mind, I prefer to bite that bullet than talk about degrees of person-hood.
At some point you will surely admit that we now have 2 people and not just 1
Actually I won’t. While I grok your approach completely, I’d rather say my concept of ‘an individual’ breaks down once I have two minds with one bit’s difference, or two identical minds, or any of these borderline cases we’re so fond of.
Say I have two optimisers with one bit’s difference. If that bit means one copy converts to Sufism and the other to Mennonism, then sure, two different people. If that one bit is swallowed up in later neural computations due to the coarse-grained-ness of the wetware, then we’re back to one person since the two are, once again, functionally identical. Faced with contradictions like that, I’m expecting our idea of personal identity to go out the window pretty fast once tech like this actually arrives. Greg Egan’s Diaspora pretty much nails this for me, have a look.
All your ‘contradictions’ go out the window once you let go of the idea of a mind as an indivisible unit. If our concept of identity is to have any value (and it really has to) then we need to learn to think more like reality, which doesn’t care about things like ‘one bit’s difference’.
Voted this down, then changed my mind and undid it. This is a genuine question, the answer to which was graciously accepted. Downvoting people who need guidance to understand a concept and are ready to learn is exactly what we don’t want to do.
Thanks for the link ;).
OK, on the one hand we have many-worlds. As you say, no direct subjective corroborating evidence (it’s what we’d see either way). What’s more, it’s the simplest explanation of what we see around us.
On the other hand, we have one-world. Again, ‘it’s what we’d see either way’. However, we now have to postulate an extra mechanism that causes the ‘collapse’.
I know which of these feels more like a privileged complex hypothesis pulled out of thin air, like a dragon.
Could whomever downvoted me above let me know where I’m going wrong here?
On what are you basing your assumption that the world should have whatever you mean by ‘meaning’?