I’ve read a fair number of x-risk related news pieces, and this was by far the most positive and non-sensationalist coverage that I’ve seen by someone who was neither a scientist nor involved with x-risk organisations.
The previous two articles I’d seen on the topic were about 30% Terminator references. This article, while not necessarily a 100% accurate account, at least takes the topic seriously.
This is a summary reasonably close to my opinion.
In particular, outright denouncement of ordinary social norms of the sort used by (and wired into) most flesh people, and endorsement of an alternative system involving much more mental exhaustion for the likes of people like me, feels so much like defecting that I would avoid interacting with any person signalling such opinions.
Actually I don’t think you’re right. I don’t think there’s much consensus on the issue within the community, so there’s not much of a conclusion to draw:
Last year’s survey answer to “which disaster do you think is most likely to wipe out greater than 90% of humanity before the year 2100?” was as follows:
Pandemic (bioengineered): 272, 23% Environmental collapse: 171, 14.5% Unfriendly AI: 160, 13.5% Nuclear war: 155, 13.1% Economic/Political collapse: 137, 11.6% Pandemic (natural): 99, 8.4% Nanotech: 49, 4.1% Asteroid: 43, 3.6%
I’m pretty sure this is one of the main areas Prof David Spiegelhalter is trying to cover with experiments like this one. He advises the British government on presenting medical statistics, and his work is worth a read if you want to know about how to phrase statistical questions so people get them more right.
This post reminded me of a conversation I was having the other day, where I noted that I commit the planning fallacy far less than average because I rarely even model myself as an agent.
A non-exhaustive list of them in very approximate descending order of average loudness:
Offspring (optimising for existence, health and status thereof. This is my most motivating goal right now and most of my actions are towards optimising for this, in more or less direct ways.)
Learning interesting things
Sex (and related brain chemistry feelings)
Love (and related brain chemistry feelings)
Empathy and care for other humans
Prestige and status
Epistemic rationality
Material comfort
I notice the problem mainly as the loudness of “Offspring” varies based on hormone levels, whereas “Learning new things” doesn’t. In particular when I optimise almost entirely for offspring, cryonics is a waste of time and money, but on days where “learning new things” gets up there it isn’t.
As an “INFJ” who has learned to think in an “INTJ” way through doing a maths degree and hanging out with INTJs, I also agree that different ways of problem solving can be learned. What I tend to find is that my intuitive way of thinking gets me a less accurate, faster answer, which is in keeping with what everyone else has suggested.
However, with my intuitive thinking, there is also an unusual quirk that although my strongly intuitive responses are fairly inaccurate (correct about half the time), this is much more accurate than they have any right to be given the precision of the ones that are correct. My intuitive thinking usually applies to people and their emotions, and I frequently get very specific hypotheses about the relationships between a set of people. Learning logical thinking has allowed me to first highlight hypotheses with intuition, then slowly go through and falsify the wrong ones, which leads me to an answer that I think I couldn’t possibly get with logic alone, since my intuition uses things like facial expressions and body languages and voice inflections to gather much more data than I could consciously.
“Whichever subagent currently talks in the “loudest” voice in my head” seems to be the only way I could describe it. However, “volume” doesn’t lend to a consistent weighting because it varies, and I’m pretty sure varies depending on hormone levels amongst many things, making me easily dutch-bookable based on e.g. time of month.
I’m not entirely sure. What questions could I ask myself to figure this out? (I suspect figuring this out is equivalent to answering my original question)
My other subagents consider that such an appalling outcome that my processor agent refuses to even consider the possibility...
Though given this, it seems likely that I do have some degree of built-in weighting, I just don’t realise what it is yet. That’s quite reassuring.
Edit: More clarification in case my situation is different from yours: my 3 main subagents have such different aims that each of them evokes a “paper-clipper” sense of confusion in the others. Also, a likely reason why I refuse to consider it is because all of them are hard-wired into my emotions, and my emotions are one of the inputs my processing takes. This doesn’t bode well for my current weighting being consistent (and Dutch-book-proof).
So I started reading this, but it seems a bit excessively presumptuous about what the different parts of me are like. It’s really not that complicated: I just have multiple terminal values which don’t come with a natural weighting, and I find balancing them against each other hard.
In the process of trying to pin down my terminal values, I’ve discovered at least 3 subagents of myself with different desires, as well as my conscious one which doesn’t have its own terminal values, and just listens to theirs and calculates the relevant instrumental values. Does LW have a way for the conscious me to weight those (sometimes contradictory) desires?
What I’m currently using is “the one who yells the loudest wins”, but that doesn’t seem entirely satisfactory.
I can’t upvote this point enough.
And more worryingly, with the Christians I have spoken to, those who are more consistent in their beliefs and actually update the rest of their beliefs on them (and don’t just have “Christianity” as a little disconnected bubble in their beliefs) are overwhelmingly in this category, and those who believe that most Christians will go to heaven usually haven’t thought very hard about the issue.
Surely a more obvious cost is the vast number of people who like tigers and would be sad if they all died?
I’d be alarmed if anyone claimed to accurately numerically update their priors. Non-parametric Bayesian statistics is HARD and not the kind of thing I can do in my head.
I second what gothgirl said; but in case you were looking for more concrete advice:
Exchange compliments. Accept compliments graciously but modestly (e.g. “Thanks, that’s kind of you”).
Increase your sense of humour (watching comedy, reading jokes) until it’s at population average levels, if it’s not there.
Practise considering other people’s point of view.
Do those three things consciously for long enough that you start doing them automatically.
At least, that’s what worked for me when I was younger. Especially 1 actually, I think it helped with 3.
Thank you. This is appreciated. I know it’s hard work, but from our point of view we can’t take your word that you’re not just making most of it up off the top of your head. (Also a lot of people like to independently assess the reliability of sources.)
I approve of this post, everything in it seems pretty reasonable (my current OH did about 80% of the long-terming male list), though I do wish you could’ve added a list of citations; this is quite a lot of content to just pull out of the hat.
I wouldn’t worry too much about the comments. Even Guardian readers don’t hold the online commentariat of the Guardian in very high esteem, and it’s reader opinion, not commenter opinion, that matters the most.
It seems like the most highly upvoted comments are pretty sane anyway!