I’m Michael “Valentine” Smith. Cofounder & senior instructor at CFAR back in the day. I’ve been in the rationalist scene since 2011 but mostly left in late 2018. To the extent that “post-rationalist” means anything, the term should probably apply to me.
You can find my non-LW writing on my Substack. You can also find my social media profiles via my Linktree.
In broad strokes I agree with you. Here I was sharing my observation of four cases where a friend was involved this way. One case might have been miscommunication but it doesn’t seem likely to me. The other three definitely weren’t. In one of those I personally knew the guy; I liked him, but he was also emotionally very unstable and definitely not a safe father. I don’t think the abuse was physical in any of those four cases.
! I’m genuinely impressed if you wrote this post without having a mental frame for the concepts drawn from LDT.
Thanks. :)
And thanks for explaining. I’m not sure what “quasi-Kantian” or “quasi-Rawlsian” mean, and I’m not sure which piece of Eliezer’s material you’re gesturing toward, so I think I’m missing some key steps of reasoning.
But on the whole, yeah, I mean defensive power rather than offensive. The offensive stuff is relevant only to the extent that it works for defense. At least that’s how it seems to me! I haven’t thought about it very carefully. But the whole point is, what could make me safe if a hostile telepath discovers a truth in me? The “build power” family of solutions is based on neutralizing the relevance of the “hostile” part.
I think you’re saying something more sophisticated than this. I’m not entirely sure what it is. Like here you say:
Basically, you have to control things orthogonal to your position in the lineup, to robustly improve your algorithm for negotiating with others.
I’m not sure what “the lineup” refers to, so I don’t know what it means for something to be orthogonal to my position in it.
I think I follow and agree with what you’re saying if I just reason in terms of “setting up arms races is bad, all else being equal”.
Or to be more precise, if I take the dangers of adaptive entropy seriously and I view “create adaptive entropy to get ahead” as a confused pseudo-solution. It might be that that’s my LDT-like framework.
I like this way of expressing it. Thanks for sharing.
I think it’s the same core thing I was pointing at in “We’re already in AI takeoff”, only it goes in the opposite direction for metaphors. I was arguing that it’s right to view memes as alive for the same reason we view trees and cats as alive. Grey seems to be arguing to set aside the question and just look at the function. Same intent, opposite approaches.
I think David Deutsch’s article “The Evolution of Culture” is masterful at describing this approach to memetics.
(Though maybe I should say that the therapist needs to either experience unconditional positive regard toward the client, or successfully deceive themselves and the client into thinking that they do. Heh.)
I mean, technically they don’t even need to deceive themselves. They can be consciously judgy as f**k as long as they can mask it effectively. Psychopaths might make for amazing therapists in this one way!
I think the word “power” might be creating some confusion here.
I mean something pretty specific and very practical. I’m not sure how to precisely define it, but here are some examples:
If someone threatens to freak out at you if you disagree with them, and you tend to get overwhelmed and panic when the freak out at you, then they have a kind of power over you. Building power here probably looks like learning to experience them freaking out without you getting overwhelmed.
If someone pays for your rent and food but might stop if they get any hint that you’re gay, it might not be safe to even ask yourself honestly whether you are. You build power here by getting an income, or a source of rent and food, that doesn’t depend on the hostile telepathic benefactor.
If your lover gets turned on by you politically agreeing with them and turned off by disagreement, you might find your political views drifting toward theirs for “unrelated” reasons. One way to build power here is to get other access to sex. Another is to diminish your libido. Another is to break up with them. (Not saying any of these are a great idea. I’m just naming what the solution of “building power” might look like here.)
I’m not familiar with LDT. I can’t comment on that part. Sorry if that means what I just said misses your point.
The fact that Bob has this policy in the first place is more likely when he’s being self-deceptive.
I don’t know if that’s true. It might be. But some possible counterpoints:
People can distrust systems that demand they check. “You have nothing to fear if you have nothing to hide” can get a response of “No” even from people who don’t have anything to hide.
If someone subconsciously thinks they can pull off the illusion of honestly looking while in fact finding nothing, they become more likely to choose to look because they’re self-deceiving.
Someone with a policy of not looking might be better at making their own self-deception unnecessary.
…more often it will be the result of Bob noticing that he’s the sort of person who might have something to hide.
Sure, that way of deciding doesn’t work.
Likewise, if you’re inclined to decide you’re going to dig into possible sources of self-deception because you think it’s unlikely that you have any, then you can’t do this trick.
The hypothetical respect for any self-deception that might be there needs to be unconditional on its existence. Otherwise, for the reason you say, it doesn’t work as well.
(…with some caveats about how people are imperfect telepaths, so some fuzz in implementation here is in practice fine.)
That said, I think you’re right in that if Omega-C is looking only at the choice of whether to look or not, then yes, Omega-C would be right to take the choice as evidence of a deception.
But the whole point is that Omega-C can read what conscious processes you’re using, and can see that you’re deciding for a glomerizing reason.
That’s why why you choose what you do matters so much here. Not just what you choose.
It’s a general rule that if E is strong evidence for X, then ~E is at least weak evidence for ~X.
Conservation of expected evidence is what makes looking relevant. It’s not what makes deciding to look relevant.
If I decide to appease Omega-C by looking, and then I find that I’m self-deceiving, the fact that I chose to look gets filtered. The fact that this is possible is why not finding evidence can matter at all. Otherwise it’d just be a charade.
Relatedly: I have a coin in my pocket. I don’t feel like checking it for bias. Does that make it more likely that the coin is biased? Maybe. But if I could magically show you that I’m not looking because I honestly do not care one way or the other and don’t want to waste the effort, and it doesn’t affect me whether it’s biased or not… then you can’t use my disinterest in checking the coin for bias as evidence of some kind of subconscious deception about the coin’s bias. I’m just refusing to do things that would inform you of the coin’s possible bias.
If this kind of reasoning weren’t possible, then it seems to me that glomerization wouldn’t be possible.
It’s not very hard to detect when someone’s deceiving them self…
A few notes:
Sometimes this is obviously true. I agree.
It’s a curious question why many folk turn their attention away from someone else’s self-deception when it’s obvious. Often they don’t, but sometimes they do. Why they (we) do that is an interesting question worthy of some sincere curiosity.
Confirmation bias. You don’t notice the cases where you don’t pick up on someone else’s self-deception.
…people should notice more and disincentivise that
Boy oh boy do I disagree.
If someone’s only option for dealing with a hostile telepath is self-deception, and then you come in and punish them for using it, thou art a dick.
Like, do you think it helps the abused mothers I named if you punish them somehow for not acknowledging their partners’ abuse? Does it even help the social circle around them?
Even if the “hostile telepath” model is wrong or doesn’t apply in some cases, people self-deceive for some reason. If you don’t dialogue with that reason at all and just create pain and misery for people who use it, you’re making some situation you don’t understand worse.
I agree that getting self-deception out of a culture is a great idea. I want less of it in general.
But we don’t get there by disincentivizing it.
…I went in the other direction: trying to self-deceive little, and instead be self-honest about my real motivations, even if they are “bad PR”.
Yep. I’m not sure why you think this is a “very different” conclusion. I’d say the same thing about myself. The key question is how to handle the cases where becoming conscious of a “bad PR” motivation means it might get exposed.
And you answer that! In part at least. You divide people into three categories based on (a) whether you need occlumency with them at all and (b) whether you need to use occlumency on the fact that you’re using occlumency.
I don’t think of it in terms this explicit, but it’s pretty close to what I do now. People get to see me to the extent that I trust them with what I show them. And that’s conscious.
Am I misunderstanding you somehow?
Moreover, having an extremely difficult high-stakes problem is not just a strong reason to self-deceive less, it’s also strong reason to become more truth-oriented as a community. This means that people with such a common cause should strive to put each other at least in category 2 above, tentatively moving towards 3 (with the caveat of watching out for bad actors trying to exploit that).
I both agree and partly disagree. I tagged your comment with where.
Totally, yes, having a real and meaningful shared problem means we want a truth-seeking community. Strong agreement.
But I think how we “strive” to be truth-seeking might be extremely important. If it’s a virtue instead of an engineering consideration, and if people are shamed or punished for having non-truth-seeking behaviors, then the collective “striving” being talked about will encourage individual self-deception and collective untalkaboutability. It’s an example of inducing adaptive entropy.
Relatedly: mathematicians don’t have truth-seeking collaboration because they’re trying hard to be truth-seeking. They’re trying to solve problems, and they can verify whether their proposed solutions actually solve the problems they’re working on. That means truth-seeking is more useful for what they’re doing than any alternatives are. There’s no need for focusing on the Virtue of Seeking Truth as a culture.
Likewise, there’s no Virtue of Using a Hammer in carpentry.
What puts someone in category 2 or 3 for me isn’t something I can strive for. It’s more like, I can be open to the possibility and be willing to look for how they and I interact. Then I discover how my trust of them shifts. If I try to trust people more than I do, I end up in more adaptive entropic confusion. I’m pretty sure this is lawful on par with thermodynamics.
This might be what you meant. If so, sorry to set up and take a swing at a strawman of what you were saying.
I think I disagree. I’ll add some precision to point out how. Happy to hear if I’m missing something.
E is Bayesian evidence of X if E is more likely to happen when X is true than when it’s not.
If Bob says “As a policy, I’m not going to check whether I’m running an Omega-C deception”, that’s equally likely whether Bob is running a deception or not. (Hence the “as a policy” part.) It just fully happens in both cases. So from Omega-C’s point of view, it’s not Bayesian evidence that distinguishes between the two versions of Bob.
It would be evidence if the choice were made from a stance of “Oh shoot, that might be self-deception! Well, I’m now going to adopt the no-looking policy so that I don’t have to check it!” Then yeah, sure, that’s clearly evidence — which is precisely why that method of deciding not to look isn’t what can work.
The policy of always deeply investigating oneself can produce evidence for Omega-C, but the act of choosing that policy might not. Choosing the policy not to look just doesn’t produce evidence.
Or at least that’s how it seems to me.
I can secondhand lend some affirmation to the newcomb case.
Oh yeah, that’s a cool example.
Another solution is illegible-ization/orthogonalization of preferences to the hostile telepath so that you don’t overlap in anything they might care about or overpower you with.
You mean something like, look boring to them? Like, I don’t care how good Putin is at reading people, I just don’t have anything he wants, so I’m safe as long as I keep (apparently) not having anything he wants?
Cool. I knew there at least used to be “antisocial personality disorder”, which I thought was under cluster B along with narcissism and borderline. And I thought “psychopathy” was a different term for APD. Thanks for the correction.
The main thing I wanted to gesture at there is that I wasn’t using “psychopath” as something derogatory. I didn’t mean “bad guys”. I meant something more like “people who are naturally unconstrained by social pressures and have no qualms breaking even profound taboos if they think it’ll benefit them”. (I just now made that up.) It seems to me that it’s a pretty specifically different mental/emotional architecture.
Some cultures used to, and maybe still do, have a solution to the hostile telepaths problem you didn’t list: perform rituals even if you don’t mean them.
Ah, yep! True that!
Your point relates more directly to my main interest, memetics. I bet there are memes that encourage both (a) these rituals and (b) the telepathic attacks that make those rituals necessary.
Oh huh. Yeah. It’s not a solution by itself since there are lots of other cues hostile telepaths can use. But rigidity might dampen what they can read for sure!
This is testable. It predicts that improved skill with occlumency and/or gaining power should sometimes cause a release of chronic tension.
I think breastfeeding is different because… public health people decided it should be, and we’ve internalized their messaging.
I haven’t gone around and checked much, but my gut impression isn’t that this is about public health people. I think it’s more like a Chesterton’s Fence backlash against previous generations’ experts claiming that formula was obviously better. IIRC, mothers were warned against using breastmilk and told to go to formula instead, because it’s Scientific™. So it took some cultural pushback to reclaim evolution’s solution to feeding newborns.
I haven’t read OP yet, just a quick translation note:
The Sanskrit word “tanha” shares an etymology with English words like “tenacious”, “tendency”, and “tenet”. The PIE root means “grip” or “hold”.
I think most folk in my social circles who use “tanha” these days are referencing Romeo’s “(mis)Translating the Buddha”:
Tanha is usually translated as desire or craving but this is wrong and misleading. Tanha is more literally translated as ‘fused to’ or ‘welded to’. It immediately follows the mental moment that you zoom in with the attentional aperture on something. It could be that a flower or an item on the shelf at the supermarket captures your attention, or you turn your head to catch more detail as you pass by an accident on the road. Many hundreds of thousands of such events take place in the course of a single day. With most of them attention then relaxes and makes space for the next thing. But with some small proportion you find the mind doesn’t quite ‘unclench’ from the object or some aspect of the object. This tension aspect is why it is sometimes translated as ‘grasping’ which is closer. Imagine something you aren’t finished with being pulled out of your hand and you tensing your fingers to resist.
That seems maybe true. What’s the problem you see with that?
I consider ultra-BS a primarily ‘central route’ argument, as the practitioner uses explicit reasoning to support explicit narrative arguments. […]
Putting someone off balance, on the other hand, is more ‘peripheral route’ persuasion. There’s far more emphasis on the implicit messaging.
Ah! This distinction helped clarify a fair bit for me. Thank you!
…I think I might conclude that your implicit primers and vibes are very good at detecting implicit persuasion, which typically but not always has a correlation with dark artsy techniques.
I agree on all accounts here. I think I dumped most of my DADA skill points into implicit detection. And yes, the vibes thing isn’t a perfect correlation to Dark stuff, I totally agree.
Is this example satisfying?
It’s definitely helpful! The category still isn’t crisp in my mind, but it’s a lot clearer. Thank you!
Thanks for the response in any case, I really enjoy these discussions! Would you like to do a dialogue sometime?
I’ve really enjoyed this exchange too. Thank you!
And sure, I’d be up for a dialogue sometime. I don’t have a good intuition for what kind of thing goes well in dialogues yet, so maybe take the lead if & when you feel inspired to invite me into one?
A few examples:
Framing kids as “disruptive” or “inattentive” or otherwise having the wrong nature if they feel disengaged. This is after informing them what they’re going to study without consulting what’s relevant or interesting to them, and then using social power to require them to study those things. But the problem is supposedly the student, not the system.
Claiming that they’ll need these math tools later in life, and that this justifies adults pressuring the kids to learn those skills now. (This is more bullshit-flavored than gaslight-flavored, but I think they’re psychological neighbors.)
Pretending that because a word problem touches on a topic kids care about, the math is relevant to what the kids like about that topic.
Insisting that forcing kids to take math classes is for their own good, and if the kids don’t see why or don’t agree, then they should believe the adults over their own sense of things.
It makes me so angry. It’s perfectly antithetical to the essence of math as I see it.