The details reminds me a lot of hypnosis, with thoughts about thoughts, instead of just thinking things directly.
Breath. Body attention. Meta. Listen to the voice. Respond and recieve. Be open to the update. Body attention. Meta. Listen to the voice. Everyone trancing themselves and everyone else in a fuzzy haze...
Or how about, actually, NO!
How about instead we try to ramp up our critical faculties and talk about models and evidence?
I do not trust casual hypnosis because hypnosis can become “not casual” very fast.
Hypnosis is a power tool and basically it is one of those “things I won’t work with” unless it is wartime and my side is losing and it seems highly relevant to victory. And it probably wouldn’t be my side I’d be hypnotizing, it would be the bad guys.
“I...” Harry swallowed. “I still don’t see how, I’ve been thinking but—”
“I asked if the Transfiguration was safe and you answered me! ”
There was a pause...
“Right...” Harry said slowly. “That’s probably one of those things they don’t even bother telling you not to do because it’s too obvious. Don’t test brilliant new ideas for Transfiguration by yourselves in an unused classroom without consulting any professors.”
My own experience with circling is much more like hypnosis than it is like the more cognitive/alert state described in this post. On the other hand, this may be because ordinary social charisma has a strong hypnotic effect on me.
My experience with being in circles is that the closer they are to the original “genuine article” (with The extreme end being me being birthday circled by Guy Sengstock, the founder of circling) the more it feels like plain old hypnosis: a rapid induction of an emotional catharsis, tears, gratitude, a sort of suspended or awed and highly receptive state of mind...
My other personal observation of circling is that it makes men hotter, by making them more hypnotic.
I think hypnosis is nothing more than a mental state in which one is more disposed to play along with suggestions. It’s not inherently bad — you may choose to induce a suggestible state in order to learn faster, or to be more spontaneously creative. You can induce it by doing perfectly “normal“ things like looking deep into someone’s eyes and breathing deeply. I don’t think it’s an especially dangerous tool, especially given how common it is and how many people use hypnotic techniques without knowing it.
I do think that hypnosis is kind of boring. If you offer something as a technique for leveling up my skill, and all I get out of it is being hypnotized, then I haven’t really leveled up at all. I’ve heard meditation teachers describe trance states as a trap to avoid while meditating, not to be confused with the actual goals of meditation.
I think circling can be hypnotic, at least for some people. It is for me, pretty consistently. That doesn’t mean that other people may not be getting something else out of it, something more interesting. The author of this post is definitely describing a more interesting effect.
I’ll add that I don’t expect the people most effective at inducing highly suggestible trance states to identify as hypnotists. For example, you enter such a state every time you’re totally absorbed in a movie, and we call the people who caused that effect “filmmakers” and “actors”, not hypnotists.
To clarify, my claim below is that the practice of Circling does not include in its philosophy or its intended practice to involve hypnosis (the intentional induction of a trance state where a person loses their full agency and becomes highly receptive to suggestion, including to the point where their perceptions can be rewritten or their TAPs can be altered to, e.g. stop smoking cigarettes).
If you’re talking about something weaker than that, something that happens all the time by accident, even in normal, everyday conversation, then … that makes sense it happens.
I don’t think Circling is supposed to be for this. If it were me, I would hopefully notice myself in it and be like, “I seem to automatically want to do whatever it is you suggest.” (And I’ve had that level of noticing before.)
…
I’m confused why “a rapid induction of an emotional catharsis, tears, gratitude, a sort of suspended or awed and highly receptive state of mind...” would be considered reminiscent of hypnosis except for the ‘highly receptive state of mind’ part. ?
The cluster of things I’m calling hypnotism involves pretty much any ”guided meditation”, the opening ritual/warmup to most physical classes like martial arts or dance, some kinds of teaching, flirting of the kind where one partner “leads” the other, political rallies, movies, etc. It’s not so universal as to be meaningless, but it’s really really common.
It doesn’t permanently remove one’s agency, but literal hypnotists and cult leaders don’t do that either. Suggestible states are usually temporary and you don’t totally lose your preexisting personality. See Gwern’s research on “brainwashing” being mostly a myth. I think being influenced hypnotically to some extent is common, and as we go through our days we’re affected by a lot of influences, but total mind control is probably impossible.
I would totally buy that circling can have cool epistemic and relational properties beyond the hypnotic effect it has on some people. I’m just reporting that the hypnotic effect does exist.
It sounds like you don’t identify as the source of these feelings when you have them, hence your framing of other people suggesting the feelings to you. Is that an accurate description of your position?
Let me offer an alternative frame for what I think is going on when I see similar strong-feelings-as-a-result-of-circling in myself and others (although it’s certainly possible that your experience is quite different from the experiences I’m using as a reference): there are some parts of you (the generic you) that have strong feelings about things for a variety of reasons, and for a variety of reasons your response to this is often to shut those parts up and stuff them in a corner in the back of your mind. It generally doesn’t feel safe to let these parts out, so you don’t.
Circling can offer an environment in which things feel safe enough in some emotional sense (the term of art is that people are “holding space” for you) that these parts temporarily get let out, and the result can be surprisingly strong displays of emotion, crying, screaming, shuddering, etc. I have personally had this effect on people without any explicit suggestion on my part that they have strong feelings about anything; I “hold space” for them (whatever that means, I don’t have gears around it yet) and they start crying. This has been done for me at least twice and I’ve done it for others at least four times now.
When this has happened to me it has not felt even slightly hypnotic; I strongly identified as the parts that were having the emotions (although I think I identify as my S1 in general much more than most rationalists), and it never felt like the emotions were coming from anywhere other than me.
Yeah, to me it feels like “sure, you can do ‘magic’ and make me cry and hug and shudder, but that has very little to do with my long-term behavior patterns, it’s just a transient effect.” It feels like being flipped onto the mat by a skilled martial artist; I’m being a guinea pig for someone to demonstrate a cool trick.
My experience is that the cluster of experiences around “cry and hug and shudder” are what it feels like to become aware of something that’s important to my system 1, and that those moments are intervention points for shifting system 1′s heuristics. Progress on reducing akrasia, unendorsed social anxiety, etc. has often come from moments like that.
I don’t know you well, but I model you as someone with strong willpower and a general “mind over matter” attitude. This may make it less salient what your system 1 is up to?
Yeah. With these recent discussions I’m not sure instrumental rationality deserves the name anymore. It’s too welcoming of woo. “You are the easiest person to fool” (Feynman).
I mean, I get why instrumental rationality exists. People noticed that epistemic rationality doesn’t lead to success in life. But neither does science, look at all these starving postdocs. Neither does art, look at all these starving artists. Clearly we need “instrumental science” that makes you Tony Stark, and “instrumental art” that makes you Ron Hubbard. Or not.
To me, epistemic rationality is a great idea that solves the problem it sets out to solve. “Always try to find the simplest alternative explanation for the same data, compared to your idea.” But when people mix it with self-help, it just creeps me out.
I am fairly confident in this: Circling does not involve hypnosis and does not borrow from hypnosis.
I have had lots of exposure to Circling, its leadership, and have passed a training course in it. If Circling had anything to do with hypnosis, I would expect at least some of its curricula to recommend hypnosis-centered books or mention hypnosis techniques in their teachings. Or I would expect their trainings to include lessons on how to cause people to go into trance states or anything resembling this. Or I would have found some instances of people trying to use rhythm or patterned behaviors or “giving directions” or something like you’re describing.
I have been exposed to all the main schools of Circling, and I haven’t found anything remotely like this.
I want to be clear on this point, because I do think there are real risks and pitfalls of Circling, and conflating Circling with hypnosis is likely to muddy the waters, rather than bringing clarity.
I’m pretty sure these people don’t think that what they are doing “borrows from” hypnosis or trance or suggestibility hacking or mesmerism or whatever words you want to use for it.
Their emotions are high, caused by skillful intentional actions, and involves a general dynamic of “playing along” with numerous secondary “critical cognitive faculties” seemingly disengaged. Their focus is on their own feelings, and how their feelings feel, and so on. It isn’t that they don’t notice what’s directly happening to (and inside) them, it is that they notice very little else.
Maybe that’s great. Being in religions seems empirically to be somewhat positive for people?
Maybe the preacher there has studied hypnosis and optimized things for trance states… but I don’t think that would been required for him to be interacting with more or less the same basic mechanisms in people’s cognitive machinery.
Those mechanisms are not particularly exotic or hard to mess with, but they cut directly to “goal-content integrity” and so caution is appropriate.
Willing to update that ‘hyponosis’ is actually more common and easy to do than I previously imagined.
That said, my point still stands.
If Circling involved intentionally trying to hypnotize people, even without calling it hypnosis, I would expect its training to include SOME kind of incentive toward “you’re doing good Circling IF the people look a certain way, or are taking your suggestions, or are feeling lots of emotion.” (When I see someone making this assumption—that good Circling is about feeling a certain way or avoiding negative emotion—I often try to correct them.)
There is no directive toward getting people to cry, getting people to play along, getting people to amplify their emotions, etc.
The directive is often to become more aware of your surroundings and the other people, as well as your feelings, thoughts, sensations.
I have seen Circling leaders—if someone is going into overwhelming emotion or getting sucked into something—pause and try to keep the level of awareness high, so the person doesn’t just automatically get sucked in without making a choice.
I want to flag that I am pretty confident that I’ve heard circling facilitators boasting about having had people cry during circling. I don’t think it’s an explicit directive, but it does seem to be something that at least some value or interpret as a sign of deepness.
There is no directive toward getting people to … amplify their emotions
The directive is often to become more aware of … your feelings
This seems like a pretty subtle distinction.
Won’t “And how does that make you feel?” or “And where do you feel that in your body?” frequently amplify the feeling?
Like, maybe something was below the level of my conscious awareness (or was on the edge of my awareness), but now my attention has been directed towards it, so it’s been ‘amplified’ to take up more of my awareness.
Not saying that’s a bad thing, just that it does seem to me like it would fit the description of amplifying a feeling or emotion. Curious whether what I described matches your model of what’s going on.
There are lots of dials you can play with, basically.
One of the dials is moving your awareness around.
One of them is being close/face-deep in a felt sense or distancing from a felt sense. Gendlin calls this smelling the soup—putting your face in the soup is very close, and not being able to smell it is far.
Another is deliberately amplifying an emotion. It involves, e.g. playing a memory in your mind so that the emotions triggered by the memory increase. With corresponding amplification in body reaction (faster breathing, etc).
My model is something like, moving my awareness around can ‘open the door’ for an emotion to come through (like I’m inviting it to speak). And sometimes I open the door, and the emotion is loud (I start crying, say). Sometimes I open it, and it is quiet (I don’t feel much, even when my awareness is on it). I don’t see this as the same as amplifying the emotion.
I can successfully be aware of my feelings and NOT feel them very much. Whereas, if I am aiming to amplify my feelings, I will know I’m succeeding if I feel them more. Different success criteria.
If I’ve been ignoring my emotions or pushing them down for a while, I imagine they’ll be louder when I open the door. This feels like a common dynamic.
I think I see you calling explicit attention attention to your model of cognition, and how your own volitional mental moves interact with seemingly non-volitional mental observations you become aware of.
Then you’re integrating this micro-experimental data into an explanatory framework that implicitly acknowledges the possibility that your own model of yourself might be wrong, and even if it is right other people might work differently or have different observations.
I think that to get any sort of genuine, reproducible, safe, inter-subjectively validated meditative science that knows general laws of subjective psychology, it will involve conversations in this mode :-)
Etymologically, “meditation” comes from the latin meditari, “to study”.
To make a “science word” we switch to ancient greek, where “meletan” means “to study or meditate”. The three original “Boetian muses” were memory (Mnemosyne, who often is considered the mother of them all), song (Aoede), and meditation (Melete)… so if a science existed here it might be called “meletology”?
A few times I’ve playfully used the term “meletonaut” to describe someone whose approach to the field is more exploratory than scholarly or experimental.
If I hear you correctly, in your cognitive explorations, you find that you can page through memories while watching yourself for symptoms of high “adrenaline” (by which I mean often actual adrenaline, but also the general constellation of “arousal” including heart rate and sweaty skin and probably cortisol and so on).
And then maybe when you think of yourself as “aware of your feelings” that phrase could be unpacked to say that you have a basically accurate metacognitive awareness of which memories or images cause adrenaline spikes, without the active metacognitive awareness itself causing an adrenaline spike.
So if someone accuses you of “causing feelings” you can defend yourself by saying the goal is actually to help people non-emotionally know what “causes them to have emotions” without actually “experiencing the feelings directly” except as a means of gathering emotional data.
I think I understand the basis of such defense, and the validity of the defense in terms of the real value of using this technique for some people.
My personal pet name for specifically this exploratory technique (which can be performed alone and appears to occur in numerous sociological and religious contexts) is “engram dousing”.
The same basic process happens in the neuro lingusitic programming (NLP) community as one step of a process they might call something like “memory reconsolidation”.
It also happens in Scientology, where instead of self reported adrenaline symptoms they use an “e-meter” (to measure sweaty palms electronically) and instead of a two person birthday circle they formalize the process quite a bit and call it an “audit”. In scientology it is pretty clear they noticed how great this is as an introductory step in acquiring blackmail material and gaining the unjustified trust of marks (prior to headfucking them) and optimized it for that purpose.
Which is not to say that circling is as bad as scientology!
Also, apostate scientologists regularly report that “the tech” of scientology (which is scientology’s jargon term for all their early well scripted psychological manipulations of new members) does in fact work and gives life benefits.
With dynamite, construction workers could suddenly build tunnels through mountains remarkably fast so that trains and roads could go places that would otherwise have been economically impossible. Dynamite used towards good ends, with decent safety engineering and skill, is great!
But if someone wants to turn a garbage can upside down, strap a chair to it, and have me sit in the chair while they put a smallish, roughly measured quantity of dynamite under it… even if the last person in the chair survived and thought it was a wild ride and wants to do it again… uh… yeah… I would love to watch from a safe distance, but I think I’d pass on sitting in the chair.
And more generally, as an aspiring meletologist and hobbyist in the sociology of religion, all I’m trying to say is that engram dousing (along with some other mental techniques) is like “cognitive nuclear technology”, and circling might not be literally playing with refined uranium, but “the circling community in general” appears to have some cognitive uranium ore, and they’ve independently refined it a bit, and they’re doing tricks with it.
That’s all more or less great :-)
But it sounds like they are not being particularly careful, and many of them might not realize their magic rocks are powered by more than normal levels of uranium decay, and if they have even heard of Louis Slotin then they don’t think he has anything to do with their toy (uranium) pellets.
Scientology is basically about telling people not to react to stimuli and suppressing them. It’s about exercises like being able to say the same sentence for an hour at the same tone of voice without having emotional triggers that disturb the voice.
Practices that are about connecting with the felt sense and giving that felt sense the space to show up and bring any emotional trigger to the forefront do on an important dimension the opposite of what the Scientology tech does.
And then maybe when you think of yourself as “aware of your feelings” that phrase could be unpacked to say that you have a basically accurate metacognitive awareness of which memories or images cause adrenaline spikes, without the active metacognitive awareness itself causing an adrenaline spike.
Being aware of your feelings and being aware of what causes them are two different issues.
As far as Circling goes Circling (as I was taught it) is mostly not about having feeling because of memories or images but feelings caused by the interaction with the other people in the Circle.
Modeling feelings as adrenaline spikes is a bad model and would be a different one. Feelings are not represented by people as high/low adrenaline states. Fear of excitement are both states with with adrenaline but they feel quite different and a person who confuses them isn’t “aware of their feelings” and profits from a practice like Circling or Focusing to learn to get better in touch with their emotions.
That video triggers some strong emotional response in me (disgust, aversion), and it seems that video is likely to feel weird to most LW viewers, one way or another. As in, the video’s perspective is one of a voyeur or an observer to something strange, alienating, and possibly bad.
I don’t want to fight this battle if it involves bringing in videos that engender strong positive or negative reactions. (Especially given we’re not Circling here, which is an arena I’d feel more comfortable bringing in emotions. As it is, I find that video manipulative in the current context.)
I would have much less objection if the video were replaced with a verbal phrase or description.
For whatever it’s worth, I would not describe my experiences with circling as “a fuzzy haze”. Closer to the opposite, with a pretty sharp awareness of my own thoughts and cognition, and of the cognition of the people around me. But then, I have mostly been in circles with other rationalists, so experiences might vary.
Circling seems like one of those things where both its promoters and detractors vastly overestimate its effects, either positive or negative. Like a lot the responses to this are either “it’s pretty cool” or “it’s pretty creepy.” What about “meh”? The most likely outcome is that Circling does something extremely negligible if it does anything at all, or if it does seem to have some benefit it’s because of the extra hour you set aside to think about things without many other distractions. In which case, a question I’d ask is “What was inadequate about the boring technique that doesn’t have a name because it’s so obvious?” Or, if you want to make a comparison to meditation / hypnosis, other stuff: When you stumbled across a Chesterton’s Fence, what made you go “Hey, let’s try moving this fence 50 feet to the left!”? You’ll either get a) something that works pretty much the same as some more traditional practice or b) something that doesn’t work at all.
Circles can vary extremely widely based on who’s in them and how skilled the facilitators are, so it’s not surprising that people have both widely varying experiences and widely varying senses of the possible range of experiences. (Again, the analogy to sex is helpful here.) I want to generally caution everyone in this discussion, both promoters and detractors, to avoid updating too strongly based only on their own circles.
I can repeat from my other comment that circling has been extremely helpful for me personally and also that this is probably in large part because I’ve gotten to work with unusually skilled facilitators. I’m not surprised to hear that other people have very different and neutral or even much more negative experiences. A facilitator who’s Goodharting on the wrong thing can be very bad, especially if no one else in the circle is experienced enough to notice and call them out on it.
I think I understand the disconnect here, so let me try and describe it.
Suppose I have certain values, and preferences, which I endorse upon reflection; I am satisfied with what I value, in other words. Say that I enjoy physical activity, especially rock climbing and hiking; and I enjoy listening to [what I consider to be] good music; and I like writing poetry; and I enjoy fine dining (in particularly, exploring new cuisines); and say that I especially like doing this together with my friends, whom I respect and whose company I enjoy. I endorse these values; I take them to be part of who I am, and to develop the virtues I consider important.
Suppose that I go on a hike with a good friend of mine. I will enjoy this activity, yes? I will think that it’s really great, won’t I? Suppose we schedule the hike and my friend has to cancel—wouldn’t I be disappointed? That sounds like a “strong emotional attachment”… likewise if I were working on some verse which wasn’t coming together, etc. And is this bad? It doesn’t seem bad; after all, these really are my values; these are my true preferences; I endorse them; thus my “strong emotional attachment” to these activities, my judgment of them as being really great, is true.
Now suppose I go and engage in some activity which has nothing to do with my values and preferences, and is, perhaps, even anti-endorsed. Maybe I take some drugs. Maybe I get hypnotized. Whatever it is, I have no reason to endorse it; it forms no part of my identity, nor do I wish it to; it develops no virtues; were I to meet someone else who did this thing, I would not respect them more for it (in fact I’d probably respect them less).
And yet, the activity feels good; it produces a strong emotional attachment; I come away thinking that it’s really great. In this case, that feeling, that attachment, that evaluation, is false.
In short: the idea is that Circling is wireheading.
(Of course, I don’t speak for PDV, so maybe what I say is not descriptive of his reasons; but it does describe, to a large extent, my views on the matter.)
I think that I’m missing some of the anti-wireheading genes; not that there wouldn’t exist behaviors that I’d classify as wireheading and recoil from, but they tend to be things like rewriting your brain in a way that causes a permanent loss of agency, or hypnotizing yourself to believe that your child is happy and well when they are in fact starving and would need your help. But for the most part, I operate on a kind of implicit assumption that if something feels great, then that feeling of greatness is something intrinsically valuable itself. My wireheading revulsion only seems to kick in if the thing actually does active damage… and even then, I’m not sure if it’s so much the wireheading aspect that I’m recoiling from, but rather the damage aspect.
It is good to have great things in your life. It is not necessarily good to have things you feel are great in your life; those feelings are not necessarily accurate. Many things that feel really good are metaphorical junk food. They are the Symbolic Representation of The Thing. Anything that quickly generates emotional attachment is most likely to be Goodharting, optimizing for feeling great and generating attachment, rather than being great.
This reads to me as a problem with System1-System 2 alignment / integration. You can Interal Double Crux about your feelings such that they start to align “great feelings” with actual greatness.
Goodharting will always be an issue, but if System 1 & 2 actually talk to each other (and have a trusting, we’re-in-it-together relationship), it’s much easier to at least notice.
If System 1 doesn’t trust System 2, it’s more likely System 1 will try to hide information, self-sabotage, and otherwise do more backstabby things, making it hard to strive for goals.
Okay. I don’t seem to distinguish between “things that feel great” and “things that are great” in the same way as you do. (Obviously, there are things that are great despite not feeling great; e.g. helping someone else can be great even if it makes you feel bad at the time. But something feeling great is by itself a type of greatness to me, even though it shouldn’t be the only type of greatness in one’s life.)
I consider this a factual dispute about minds and Goodhart’s Law, rather than a difference of subjective categorization, so this response is a non sequitur to me.
Your comment used terms like “good” and “great”, which I interpret as subjective valuations, or preferences. I don’t know how to translate a question about subjective valuations into one of factual claims.
I claim that as a general principle, “something feeling great is by itself a type of greatness to me” is a category error. What feels great is a map, and being great is the territory. There is a fact of the matter with regards to what is great for PDV, and what is great for Kaj. They are not identical, and they are not directly queriable, but there is a fact of the matter. Something great is something that increases your utility significantly. (Non-utilitarian ethics: translate that into language your system permits.)
What feels great is a separate fact. It is directly queriable, and correlates with being great, but it is only an approximation, and can therefore be Goodharted. The distinction between the true utility and the approximation is a general property of human minds, with some regularities (superstimuli), but also not identical between people.
So when you say “for me that’s a subcategory”, I conclude that you have a) misunderstood my claim, and b) mistaken the map for the territory.
Like, if we are talking about a claim like “is it raining outside”, then the territory is made up of whether it actually is raining outside or not. It’s a concrete physical event.
For “is something great”, the nearest physical referent that I could think of is “does a person’s brain make the evaluation that this is great”. Which would make it into a question of subjective valuation, but you seem to have some more objective criteria in mind.
I said that already? “Something great is something that increases your utility significantly.” This is a property of timelines, not of world-states, and so can’t be directly queried, but better approximations can be built up by retrospecting on which times feeling great was accurate and which times it was not.
Unreal, in a subthread above, claims that it is possible to realign System 1 such that feeling great coincides with being great. This seems wrong to me, but is the kind of thing that could be right. Your description does not seem to be the kind of thing that could be right.
I’d like to try to explain and see if I’m pointing at the right thing.
I might value being loved. (This thing has utility to me.)
However, I do not actually have neurons that connect to the territory such that my neurons fire If and Only If I am being loved. My neurons are not magic.
So instead they use proxy measures. Like looking at the person’s face and seeing it smiling at me. Or seeing their body language and noticing it is relaxed and open. Or feeling their gentle touch. Etc.
All these proxy measures add up to something that feels good. However, it is NEVER certain that it’s measuring the thing I ultimately want (being loved). I’m just going off a guess. A pretty good guess, sometimes. But still.
This is Goodhart’s dilemma here.
When I have a measure of a good thing (someone smiling at me), I will try to optimize for the measure, which is not necessarily the thing I was originally wanting to track (being loved).
So at some point I may try to optimize for smiles, even when they’re not out of love. And whatever those behaviors are, we call pica.
Right, I agree that there can be things which I value, and for which I can mistaken about whether or not I have them / they exist / etc.
But PDV didn’t seem to be just saying that “you can be mistaken about whether you actually have the thing that you think you have”. They said that it’s a category error for me to say that something feeling good is by itself something that I value, and that there’s a factual dispute about minds here, rather than a dispute of subjective categorization.
Your example doesn’t feel like it helps me understand those claims. I can have a subjective categorization that being loved is something that I value, and I can be correct or mistaken about whether or not I’m actually loved. And I can indeed end up optimizing for something like smiles, which I think indicates being-lovedness, even when it’s only weakly correlated.
But that doesn’t seem to be like a reason for why something feeling good couldn’t also be something that I value for its own sake.
Wait… are you just trying to say that you can, in theory, value “positive feelings” like joy, delight, etc. in themselves? That seems unobjectionable.
I thought PDV was saying that if you mistake “good feelings” for “good things” in general, that this was a category error. Like, if you always just think, “I feel good when the sun shines on me! It must BE good that the sun is shining on me.” Then THAT is an error.
Wait… are you just trying to say that you can, in theory, value “positive feelings” like joy, delight, etc. in themselves?
Yes. And not just in theory, I would expect that this is what many if not most people do: see e.g. all the advice about how to be happy, or the fact that many people take something like classical utilitarianism seriously as a moral theory.
I thought PDV was saying that if you mistake “good feelings” for “good things” in general, that this was a category error.
Oh. I thought that I already mentioned much earlier that I didn’t mean that, when I said that things can be great despite not feeling great, and that “good feelings” are just one of the possible types of good things you can have in your life, and they shouldn’t be the only ones.
Many if not most people are Goodharting in most aspects of their lives. Why not this one?
I acknowledge your claim that you value feeling good over and above the things that cause you to feel good. I agree that many people implicitly endorse this claim about themselves. I think you and they are very likely mistaken about this preference, and that ceasing to optimize for it would improve your life significantly according to your other preferences.
was hoping you’d validate whether my “I thought PDV was saying” one way or another, above …
also, it seems like an important milestone if you guys actually sussed out where the actual disagreement is. and it seems like it isn’t what either of you previously thought it was. so i want that to be made clear.
Kaj wasn’t saying ‘a thing that couldn’t be right’. Kaj was describing a totally realistic thing to do. which is to value feeling good itself.
i think conversational milestones in arguments are important places to stop and orient, and i was worried this milestone would be quickly passed over.
and NOW the disagreement is about a preference / why aren’t you worried about Goodharting, whereas before it wasn’t clear. is this actually agreed now by both parties?
FWIW, I think ‘valuing positive feelings in themselves’ is a bad idea. It’s theoretically possible to do it, but I wouldn’t recommend it as part of one’s final evolutionary form.
Symmetrically, I think ‘equating negative feelings with badness’ or believing ‘feeling bad is bad’ is also not recommended.
I have both hypnosis and Circling skills. I understand hypnosis to be about having a specific behavior change that you want as outcome.
When doing Circling your focus is not on specific behavior change. It’s more like meditation about being in a certain state together.
How about instead we try to ramp up our critical faculties and talk about models and evidence?
I consider Circling to have been very useful at getting some models of other people.
After doing Circling at the LessWrong community weekend, one person said that they discovered in both Circlings in which they are that something they thought was unique about their own experience was also shared by other people.
Many models aren’t shared when you ramp up the critical nature of an exchange. Circling is about sharing of models about how people relate to their own experience that’s valuable both for understanding oneselves and understanding other people. It’s not about changing the models that are there even when they can change when you give them explicit attention.
As far as the state goes, after a hypnotic trance state it makes sense to say “Now come back” to get people to be “awake” again. That’s not the state in which Circling goes.
I frequently have meditations seen to go into that space, but when meditating or guiding a meditation I usually intent to lead to a more present state as well.
Back when I was first getting into hypnosis, we talked about my experiments with hypnosis and all the terrifying possibilities that they implied. Even though I’d expect you’d have taken basically the same stance even without those conversations, I imagine it is still a significant contributing factor towards your take on hypnosis, and so I feel compelled to note that I no longer feel this way about it.
To be clear, I don’t think anything we talked about is “wrong”, and the fact that the uncertainty mostly resolved on the “less scary” side isn’t very reassuring. I still can’t think of any circumstance with any hypnotist that I would allow them to “hypnotize” me, in the central meaning of the word, and I do still think people are insufficiently afraid of being hypnotized. That stuff is all more or less the same.
The big difference is that now recognize more of how “responding hypnotically” is a really important part of both learning and relating to people, and that it’s possible to do it without risking falling into any of the obvious traps that enable the scary bad possibilities. “Engage critical faculties, keep in mind evidence, develop models, etc” yes. Do that and “Listen to the voice. Respond and receive. Be open to the update, etc”—to the extent that you can do that without losing track of the former (and work to increase this extent as much as you can).
I don’t even think it’s always crazy to trade off some control for quicker learning, so long as this decision itself is made very carefully with full input of critical faculties, you understand the potential traps, and the person guiding you really can be verified to be worthy of the required trust, etc.
However it’s not necessary either. I’ve gotten better at it myself without sacrificing my need for control, and I have a very “control freaky” friend who is also figuring out how to respond hypnotically without giving up any control, and has gotten some really cool results from it. It’s taken her four years to be able to accept half the suggestions a good hypnotic subject can do in five minutes, but on the upside since she is deciding for herself which things to accept hypnotically, not only does she not expose herself to unnecessary risk, she’s able to more efficiently spot what would be useful to her in a normal conversation without anyone having to lean on it as if it were an actual hypnotic suggestion.
I guess it’s kinda like exploring caves that have a lot of goodies. Just make sure you know your way out.
The details reminds me a lot of hypnosis, with thoughts about thoughts, instead of just thinking things directly.
Breath. Body attention. Meta. Listen to the voice. Respond and recieve. Be open to the update. Body attention. Meta. Listen to the voice. Everyone trancing themselves and everyone else in a fuzzy haze...
Or how about, actually, NO!
How about instead we try to ramp up our critical faculties and talk about models and evidence?
I do not trust casual hypnosis because hypnosis can become “not casual” very fast.
Hypnosis is a power tool and basically it is one of those “things I won’t work with” unless it is wartime and my side is losing and it seems highly relevant to victory. And it probably wouldn’t be my side I’d be hypnotizing, it would be the bad guys.
“We broke the rules, Harry,” she said in a hoarse voice. “We broke the rules.”
“I...” Harry swallowed. “I still don’t see how, I’ve been thinking but—”
“I asked if the Transfiguration was safe and you answered me! ”
There was a pause...
“Right...” Harry said slowly. “That’s probably one of those things they don’t even bother telling you not to do because it’s too obvious. Don’t test brilliant new ideas for Transfiguration by yourselves in an unused classroom without consulting any professors.”
Except there are no decent professors in this subject. (There were crazy CIA mind control experiments, but instead of publishing their results, the records were mostly purged in 1973.)
My own experience with circling is much more like hypnosis than it is like the more cognitive/alert state described in this post. On the other hand, this may be because ordinary social charisma has a strong hypnotic effect on me.
My experience with being in circles is that the closer they are to the original “genuine article” (with The extreme end being me being birthday circled by Guy Sengstock, the founder of circling) the more it feels like plain old hypnosis: a rapid induction of an emotional catharsis, tears, gratitude, a sort of suspended or awed and highly receptive state of mind...
My other personal observation of circling is that it makes men hotter, by making them more hypnotic.
I think hypnosis is nothing more than a mental state in which one is more disposed to play along with suggestions. It’s not inherently bad — you may choose to induce a suggestible state in order to learn faster, or to be more spontaneously creative. You can induce it by doing perfectly “normal“ things like looking deep into someone’s eyes and breathing deeply. I don’t think it’s an especially dangerous tool, especially given how common it is and how many people use hypnotic techniques without knowing it.
I do think that hypnosis is kind of boring. If you offer something as a technique for leveling up my skill, and all I get out of it is being hypnotized, then I haven’t really leveled up at all. I’ve heard meditation teachers describe trance states as a trap to avoid while meditating, not to be confused with the actual goals of meditation.
I think circling can be hypnotic, at least for some people. It is for me, pretty consistently. That doesn’t mean that other people may not be getting something else out of it, something more interesting. The author of this post is definitely describing a more interesting effect.
I’ll add that I don’t expect the people most effective at inducing highly suggestible trance states to identify as hypnotists. For example, you enter such a state every time you’re totally absorbed in a movie, and we call the people who caused that effect “filmmakers” and “actors”, not hypnotists.
To clarify, my claim below is that the practice of Circling does not include in its philosophy or its intended practice to involve hypnosis (the intentional induction of a trance state where a person loses their full agency and becomes highly receptive to suggestion, including to the point where their perceptions can be rewritten or their TAPs can be altered to, e.g. stop smoking cigarettes).
If you’re talking about something weaker than that, something that happens all the time by accident, even in normal, everyday conversation, then … that makes sense it happens.
I don’t think Circling is supposed to be for this. If it were me, I would hopefully notice myself in it and be like, “I seem to automatically want to do whatever it is you suggest.” (And I’ve had that level of noticing before.)
…
I’m confused why “a rapid induction of an emotional catharsis, tears, gratitude, a sort of suspended or awed and highly receptive state of mind...” would be considered reminiscent of hypnosis except for the ‘highly receptive state of mind’ part. ?
The cluster of things I’m calling hypnotism involves pretty much any ”guided meditation”, the opening ritual/warmup to most physical classes like martial arts or dance, some kinds of teaching, flirting of the kind where one partner “leads” the other, political rallies, movies, etc. It’s not so universal as to be meaningless, but it’s really really common.
It doesn’t permanently remove one’s agency, but literal hypnotists and cult leaders don’t do that either. Suggestible states are usually temporary and you don’t totally lose your preexisting personality. See Gwern’s research on “brainwashing” being mostly a myth. I think being influenced hypnotically to some extent is common, and as we go through our days we’re affected by a lot of influences, but total mind control is probably impossible.
I would totally buy that circling can have cool epistemic and relational properties beyond the hypnotic effect it has on some people. I’m just reporting that the hypnotic effect does exist.
Because suggestibility + being prompted to have strong vulnerable feelings results in actually having said strong vulnerable feelings.
It sounds like you don’t identify as the source of these feelings when you have them, hence your framing
of other people suggesting the feelings to you. Is that an accurate description of your position?
Let me offer an alternative frame for what I think is going on when I see similar strong-feelings-as-a-result-of-circling in myself and others (although it’s certainly possible that your experience is quite different from the experiences I’m using as a reference): there are some parts of you (the generic you) that have strong feelings about things for a variety of reasons, and for a variety of reasons your response to this is often to shut those parts up and stuff them in a corner in the back of your mind. It generally doesn’t feel safe to let these parts out, so you don’t.
Circling can offer an environment in which things feel safe enough in some emotional sense (the term of art is that people are “holding space” for you) that these parts temporarily get let out, and the result can be surprisingly strong displays of emotion, crying, screaming, shuddering, etc. I have personally had this effect on people without any explicit suggestion on my part that they have strong feelings about anything; I “hold space” for them (whatever that means, I don’t have gears around it yet) and they start crying. This has been done for me at least twice and I’ve done it for others at least four times now.
When this has happened to me it has not felt even slightly hypnotic; I strongly identified as the parts that were having the emotions (although I think I identify as my S1 in general much more than most rationalists), and it never felt like the emotions were coming from anywhere other than me.
Yeah, to me it feels like “sure, you can do ‘magic’ and make me cry and hug and shudder, but that has very little to do with my long-term behavior patterns, it’s just a transient effect.” It feels like being flipped onto the mat by a skilled martial artist; I’m being a guinea pig for someone to demonstrate a cool trick.
My experience is that the cluster of experiences around “cry and hug and shudder” are what it feels like to become aware of something that’s important to my system 1, and that those moments are intervention points for shifting system 1′s heuristics. Progress on reducing akrasia, unendorsed social anxiety, etc. has often come from moments like that.
I don’t know you well, but I model you as someone with strong willpower and a general “mind over matter” attitude. This may make it less salient what your system 1 is up to?
Thanks for posting this. Strongly agree with learning to notice trance states and not confusing them for things they aren’t.
Yeah. With these recent discussions I’m not sure instrumental rationality deserves the name anymore. It’s too welcoming of woo. “You are the easiest person to fool” (Feynman).
I mean, I get why instrumental rationality exists. People noticed that epistemic rationality doesn’t lead to success in life. But neither does science, look at all these starving postdocs. Neither does art, look at all these starving artists. Clearly we need “instrumental science” that makes you Tony Stark, and “instrumental art” that makes you Ron Hubbard. Or not.
To me, epistemic rationality is a great idea that solves the problem it sets out to solve. “Always try to find the simplest alternative explanation for the same data, compared to your idea.” But when people mix it with self-help, it just creeps me out.
I consider Circling to be about epistemic rationality, and that’s a big chunk of why it’s interesting to me.
Are you objecting to circling-type instrumental rationality techniques, or to instrumental rationality in general?
Good job, well predicted. Even CFAR has degenerated into woo now.
I am fairly confident in this: Circling does not involve hypnosis and does not borrow from hypnosis.
I have had lots of exposure to Circling, its leadership, and have passed a training course in it. If Circling had anything to do with hypnosis, I would expect at least some of its curricula to recommend hypnosis-centered books or mention hypnosis techniques in their teachings. Or I would expect their trainings to include lessons on how to cause people to go into trance states or anything resembling this. Or I would have found some instances of people trying to use rhythm or patterned behaviors or “giving directions” or something like you’re describing.
I have been exposed to all the main schools of Circling, and I haven’t found anything remotely like this.
I want to be clear on this point, because I do think there are real risks and pitfalls of Circling, and conflating Circling with hypnosis is likely to muddy the waters, rather than bringing clarity.
I’m pretty sure these people don’t think that what they are doing “borrows from” hypnosis or trance or suggestibility hacking or mesmerism or whatever words you want to use for it.
Their emotions are high, caused by skillful intentional actions, and involves a general dynamic of “playing along” with numerous secondary “critical cognitive faculties” seemingly disengaged. Their focus is on their own feelings, and how their feelings feel, and so on. It isn’t that they don’t notice what’s directly happening to (and inside) them, it is that they notice very little else.
Maybe that’s great. Being in religions seems empirically to be somewhat positive for people?
Maybe the preacher there has studied hypnosis and optimized things for trance states… but I don’t think that would been required for him to be interacting with more or less the same basic mechanisms in people’s cognitive machinery.
Those mechanisms are not particularly exotic or hard to mess with, but they cut directly to “goal-content integrity” and so caution is appropriate.
Willing to update that ‘hyponosis’ is actually more common and easy to do than I previously imagined.
That said, my point still stands.
If Circling involved intentionally trying to hypnotize people, even without calling it hypnosis, I would expect its training to include SOME kind of incentive toward “you’re doing good Circling IF the people look a certain way, or are taking your suggestions, or are feeling lots of emotion.” (When I see someone making this assumption—that good Circling is about feeling a certain way or avoiding negative emotion—I often try to correct them.)
There is no directive toward getting people to cry, getting people to play along, getting people to amplify their emotions, etc.
The directive is often to become more aware of your surroundings and the other people, as well as your feelings, thoughts, sensations.
I have seen Circling leaders—if someone is going into overwhelming emotion or getting sucked into something—pause and try to keep the level of awareness high, so the person doesn’t just automatically get sucked in without making a choice.
I want to flag that I am pretty confident that I’ve heard circling facilitators boasting about having had people cry during circling. I don’t think it’s an explicit directive, but it does seem to be something that at least some value or interpret as a sign of deepness.
It’s evidence that something interesting is happening, and like most such things, is tempting and dangerous to Goodhart on.
This seems like a pretty subtle distinction.
Won’t “And how does that make you feel?” or “And where do you feel that in your body?” frequently amplify the feeling?
Like, maybe something was below the level of my conscious awareness (or was on the edge of my awareness), but now my attention has been directed towards it, so it’s been ‘amplified’ to take up more of my awareness.
Not saying that’s a bad thing, just that it does seem to me like it would fit the description of amplifying a feeling or emotion. Curious whether what I described matches your model of what’s going on.
There are lots of dials you can play with, basically.
One of the dials is moving your awareness around.
One of them is being close/face-deep in a felt sense or distancing from a felt sense. Gendlin calls this smelling the soup—putting your face in the soup is very close, and not being able to smell it is far.
Another is deliberately amplifying an emotion. It involves, e.g. playing a memory in your mind so that the emotions triggered by the memory increase. With corresponding amplification in body reaction (faster breathing, etc).
My model is something like, moving my awareness around can ‘open the door’ for an emotion to come through (like I’m inviting it to speak). And sometimes I open the door, and the emotion is loud (I start crying, say). Sometimes I open it, and it is quiet (I don’t feel much, even when my awareness is on it). I don’t see this as the same as amplifying the emotion.
I can successfully be aware of my feelings and NOT feel them very much. Whereas, if I am aiming to amplify my feelings, I will know I’m succeeding if I feel them more. Different success criteria.
If I’ve been ignoring my emotions or pushing them down for a while, I imagine they’ll be louder when I open the door. This feels like a common dynamic.
I really like this comment!
I think I see you calling explicit attention attention to your model of cognition, and how your own volitional mental moves interact with seemingly non-volitional mental observations you become aware of.
Then you’re integrating this micro-experimental data into an explanatory framework that implicitly acknowledges the possibility that your own model of yourself might be wrong, and even if it is right other people might work differently or have different observations.
I think that to get any sort of genuine, reproducible, safe, inter-subjectively validated meditative science that knows general laws of subjective psychology, it will involve conversations in this mode :-)
Etymologically, “meditation” comes from the latin meditari, “to study”.
To make a “science word” we switch to ancient greek, where “meletan” means “to study or meditate”. The three original “Boetian muses” were memory (Mnemosyne, who often is considered the mother of them all), song (Aoede), and meditation (Melete)… so if a science existed here it might be called “meletology”?
A few times I’ve playfully used the term “meletonaut” to describe someone whose approach to the field is more exploratory than scholarly or experimental.
If I hear you correctly, in your cognitive explorations, you find that you can page through memories while watching yourself for symptoms of high “adrenaline” (by which I mean often actual adrenaline, but also the general constellation of “arousal” including heart rate and sweaty skin and probably cortisol and so on).
And then maybe when you think of yourself as “aware of your feelings” that phrase could be unpacked to say that you have a basically accurate metacognitive awareness of which memories or images cause adrenaline spikes, without the active metacognitive awareness itself causing an adrenaline spike.
So if someone accuses you of “causing feelings” you can defend yourself by saying the goal is actually to help people non-emotionally know what “causes them to have emotions” without actually “experiencing the feelings directly” except as a means of gathering emotional data.
I think I understand the basis of such defense, and the validity of the defense in terms of the real value of using this technique for some people.
My personal pet name for specifically this exploratory technique (which can be performed alone and appears to occur in numerous sociological and religious contexts) is “engram dousing”.
The same basic process happens in the neuro lingusitic programming (NLP) community as one step of a process they might call something like “memory reconsolidation”.
It also happens in Scientology, where instead of self reported adrenaline symptoms they use an “e-meter” (to measure sweaty palms electronically) and instead of a two person birthday circle they formalize the process quite a bit and call it an “audit”. In scientology it is pretty clear they noticed how great this is as an introductory step in acquiring blackmail material and gaining the unjustified trust of marks (prior to headfucking them) and optimized it for that purpose.
Which is not to say that circling is as bad as scientology!
Also, apostate scientologists regularly report that “the tech” of scientology (which is scientology’s jargon term for all their early well scripted psychological manipulations of new members) does in fact work and gives life benefits.
With dynamite, construction workers could suddenly build tunnels through mountains remarkably fast so that trains and roads could go places that would otherwise have been economically impossible. Dynamite used towards good ends, with decent safety engineering and skill, is great!
But if someone wants to turn a garbage can upside down, strap a chair to it, and have me sit in the chair while they put a smallish, roughly measured quantity of dynamite under it… even if the last person in the chair survived and thought it was a wild ride and wants to do it again… uh… yeah… I would love to watch from a safe distance, but I think I’d pass on sitting in the chair.
And more generally, as an aspiring meletologist and hobbyist in the sociology of religion, all I’m trying to say is that engram dousing (along with some other mental techniques) is like “cognitive nuclear technology”, and circling might not be literally playing with refined uranium, but “the circling community in general” appears to have some cognitive uranium ore, and they’ve independently refined it a bit, and they’re doing tricks with it.
That’s all more or less great :-)
But it sounds like they are not being particularly careful, and many of them might not realize their magic rocks are powered by more than normal levels of uranium decay, and if they have even heard of Louis Slotin then they don’t think he has anything to do with their toy (uranium) pellets.
Scientology is basically about telling people not to react to stimuli and suppressing them. It’s about exercises like being able to say the same sentence for an hour at the same tone of voice without having emotional triggers that disturb the voice.
Practices that are about connecting with the felt sense and giving that felt sense the space to show up and bring any emotional trigger to the forefront do on an important dimension the opposite of what the Scientology tech does.
Being aware of your feelings and being aware of what causes them are two different issues.
As far as Circling goes Circling (as I was taught it) is mostly not about having feeling because of memories or images but feelings caused by the interaction with the other people in the Circle.
Modeling feelings as adrenaline spikes is a bad model and would be a different one. Feelings are not represented by people as high/low adrenaline states. Fear of excitement are both states with with adrenaline but they feel quite different and a person who confuses them isn’t “aware of their feelings” and profits from a practice like Circling or Focusing to learn to get better in touch with their emotions.
In the state in which Circling happens, people do notice what the other people in the circle are doing and what they are experiencing.
In addition to the primary perception that you listed there’s a lot of attention on the secondary cognitive faculty of relating in Circling.
That video triggers some strong emotional response in me (disgust, aversion), and it seems that video is likely to feel weird to most LW viewers, one way or another. As in, the video’s perspective is one of a voyeur or an observer to something strange, alienating, and possibly bad.
I don’t want to fight this battle if it involves bringing in videos that engender strong positive or negative reactions. (Especially given we’re not Circling here, which is an arena I’d feel more comfortable bringing in emotions. As it is, I find that video manipulative in the current context.)
I would have much less objection if the video were replaced with a verbal phrase or description.
For whatever it’s worth, I would not describe my experiences with circling as “a fuzzy haze”. Closer to the opposite, with a pretty sharp awareness of my own thoughts and cognition, and of the cognition of the people around me. But then, I have mostly been in circles with other rationalists, so experiences might vary.
This was my experience as well, though I’ve only done Circling once (it was with rationalists).
Circling seems like one of those things where both its promoters and detractors vastly overestimate its effects, either positive or negative. Like a lot the responses to this are either “it’s pretty cool” or “it’s pretty creepy.” What about “meh”? The most likely outcome is that Circling does something extremely negligible if it does anything at all, or if it does seem to have some benefit it’s because of the extra hour you set aside to think about things without many other distractions. In which case, a question I’d ask is “What was inadequate about the boring technique that doesn’t have a name because it’s so obvious?” Or, if you want to make a comparison to meditation / hypnosis, other stuff: When you stumbled across a Chesterton’s Fence, what made you go “Hey, let’s try moving this fence 50 feet to the left!”? You’ll either get a) something that works pretty much the same as some more traditional practice or b) something that doesn’t work at all.
Circles can vary extremely widely based on who’s in them and how skilled the facilitators are, so it’s not surprising that people have both widely varying experiences and widely varying senses of the possible range of experiences. (Again, the analogy to sex is helpful here.) I want to generally caution everyone in this discussion, both promoters and detractors, to avoid updating too strongly based only on their own circles.
I can repeat from my other comment that circling has been extremely helpful for me personally and also that this is probably in large part because I’ve gotten to work with unusually skilled facilitators. I’m not surprised to hear that other people have very different and neutral or even much more negative experiences. A facilitator who’s Goodharting on the wrong thing can be very bad, especially if no one else in the circle is experienced enough to notice and call them out on it.
Circling was “meh” for me. Maybe people who find it “meh” aren’t as motivated to talk about it, so we get selection effects.
Producing a strong emotional attachment to the activity and thinking it’s really great, is itself a significant, negative effect.
Having things in your life that you feel are great, feels like a positive thing to me. (I have too few of them.)
I think I understand the disconnect here, so let me try and describe it.
Suppose I have certain values, and preferences, which I endorse upon reflection; I am satisfied with what I value, in other words. Say that I enjoy physical activity, especially rock climbing and hiking; and I enjoy listening to [what I consider to be] good music; and I like writing poetry; and I enjoy fine dining (in particularly, exploring new cuisines); and say that I especially like doing this together with my friends, whom I respect and whose company I enjoy. I endorse these values; I take them to be part of who I am, and to develop the virtues I consider important.
Suppose that I go on a hike with a good friend of mine. I will enjoy this activity, yes? I will think that it’s really great, won’t I? Suppose we schedule the hike and my friend has to cancel—wouldn’t I be disappointed? That sounds like a “strong emotional attachment”… likewise if I were working on some verse which wasn’t coming together, etc. And is this bad? It doesn’t seem bad; after all, these really are my values; these are my true preferences; I endorse them; thus my “strong emotional attachment” to these activities, my judgment of them as being really great, is true.
Now suppose I go and engage in some activity which has nothing to do with my values and preferences, and is, perhaps, even anti-endorsed. Maybe I take some drugs. Maybe I get hypnotized. Whatever it is, I have no reason to endorse it; it forms no part of my identity, nor do I wish it to; it develops no virtues; were I to meet someone else who did this thing, I would not respect them more for it (in fact I’d probably respect them less).
And yet, the activity feels good; it produces a strong emotional attachment; I come away thinking that it’s really great. In this case, that feeling, that attachment, that evaluation, is false.
In short: the idea is that Circling is wireheading.
(Of course, I don’t speak for PDV, so maybe what I say is not descriptive of his reasons; but it does describe, to a large extent, my views on the matter.)
Thank you for the explanation.
I think that I’m missing some of the anti-wireheading genes; not that there wouldn’t exist behaviors that I’d classify as wireheading and recoil from, but they tend to be things like rewriting your brain in a way that causes a permanent loss of agency, or hypnotizing yourself to believe that your child is happy and well when they are in fact starving and would need your help. But for the most part, I operate on a kind of implicit assumption that if something feels great, then that feeling of greatness is something intrinsically valuable itself. My wireheading revulsion only seems to kick in if the thing actually does active damage… and even then, I’m not sure if it’s so much the wireheading aspect that I’m recoiling from, but rather the damage aspect.
Why do you enjoy rock climbing? Do you think that’s independent of your experiences of rock climbing having produced adrenaline rushs?
It is good to have great things in your life. It is not necessarily good to have things you feel are great in your life; those feelings are not necessarily accurate. Many things that feel really good are metaphorical junk food. They are the Symbolic Representation of The Thing. Anything that quickly generates emotional attachment is most likely to be Goodharting, optimizing for feeling great and generating attachment, rather than being great.
This reads to me as a problem with System1-System 2 alignment / integration. You can Interal Double Crux about your feelings such that they start to align “great feelings” with actual greatness.
Goodharting will always be an issue, but if System 1 & 2 actually talk to each other (and have a trusting, we’re-in-it-together relationship), it’s much easier to at least notice.
If System 1 doesn’t trust System 2, it’s more likely System 1 will try to hide information, self-sabotage, and otherwise do more backstabby things, making it hard to strive for goals.
Okay. I don’t seem to distinguish between “things that feel great” and “things that are great” in the same way as you do. (Obviously, there are things that are great despite not feeling great; e.g. helping someone else can be great even if it makes you feel bad at the time. But something feeling great is by itself a type of greatness to me, even though it shouldn’t be the only type of greatness in one’s life.)
I consider this a factual dispute about minds and Goodhart’s Law, rather than a difference of subjective categorization, so this response is a non sequitur to me.
Your comment used terms like “good” and “great”, which I interpret as subjective valuations, or preferences. I don’t know how to translate a question about subjective valuations into one of factual claims.
I claim that as a general principle, “something feeling great is by itself a type of greatness to me” is a category error. What feels great is a map, and being great is the territory. There is a fact of the matter with regards to what is great for PDV, and what is great for Kaj. They are not identical, and they are not directly queriable, but there is a fact of the matter. Something great is something that increases your utility significantly. (Non-utilitarian ethics: translate that into language your system permits.)
What feels great is a separate fact. It is directly queriable, and correlates with being great, but it is only an approximation, and can therefore be Goodharted. The distinction between the true utility and the approximation is a general property of human minds, with some regularities (superstimuli), but also not identical between people.
So when you say “for me that’s a subcategory”, I conclude that you have a) misunderstood my claim, and b) mistaken the map for the territory.
So what makes up the territory?
Like, if we are talking about a claim like “is it raining outside”, then the territory is made up of whether it actually is raining outside or not. It’s a concrete physical event.
For “is something great”, the nearest physical referent that I could think of is “does a person’s brain make the evaluation that this is great”. Which would make it into a question of subjective valuation, but you seem to have some more objective criteria in mind.
I said that already? “Something great is something that increases your utility significantly.” This is a property of timelines, not of world-states, and so can’t be directly queried, but better approximations can be built up by retrospecting on which times feeling great was accurate and which times it was not.
Unreal, in a subthread above, claims that it is possible to realign System 1 such that feeling great coincides with being great. This seems wrong to me, but is the kind of thing that could be right. Your description does not seem to be the kind of thing that could be right.
Taboo “utility”? To me it’s again just another word for personal preferences.
I’d like to try to explain and see if I’m pointing at the right thing.
I might value being loved. (This thing has utility to me.)
However, I do not actually have neurons that connect to the territory such that my neurons fire If and Only If I am being loved. My neurons are not magic.
So instead they use proxy measures. Like looking at the person’s face and seeing it smiling at me. Or seeing their body language and noticing it is relaxed and open. Or feeling their gentle touch. Etc.
All these proxy measures add up to something that feels good. However, it is NEVER certain that it’s measuring the thing I ultimately want (being loved). I’m just going off a guess. A pretty good guess, sometimes. But still.
This is Goodhart’s dilemma here.
When I have a measure of a good thing (someone smiling at me), I will try to optimize for the measure, which is not necessarily the thing I was originally wanting to track (being loved).
So at some point I may try to optimize for smiles, even when they’re not out of love. And whatever those behaviors are, we call pica.
Right, I agree that there can be things which I value, and for which I can mistaken about whether or not I have them / they exist / etc.
But PDV didn’t seem to be just saying that “you can be mistaken about whether you actually have the thing that you think you have”. They said that it’s a category error for me to say that something feeling good is by itself something that I value, and that there’s a factual dispute about minds here, rather than a dispute of subjective categorization.
Your example doesn’t feel like it helps me understand those claims. I can have a subjective categorization that being loved is something that I value, and I can be correct or mistaken about whether or not I’m actually loved. And I can indeed end up optimizing for something like smiles, which I think indicates being-lovedness, even when it’s only weakly correlated.
But that doesn’t seem to be like a reason for why something feeling good couldn’t also be something that I value for its own sake.
Wait… are you just trying to say that you can, in theory, value “positive feelings” like joy, delight, etc. in themselves? That seems unobjectionable.
I thought PDV was saying that if you mistake “good feelings” for “good things” in general, that this was a category error. Like, if you always just think, “I feel good when the sun shines on me! It must BE good that the sun is shining on me.” Then THAT is an error.
Wait… are you just trying to say that you can, in theory, value “positive feelings” like joy, delight, etc. in themselves?
Yes. And not just in theory, I would expect that this is what many if not most people do: see e.g. all the advice about how to be happy, or the fact that many people take something like classical utilitarianism seriously as a moral theory.
I thought PDV was saying that if you mistake “good feelings” for “good things” in general, that this was a category error.
Oh. I thought that I already mentioned much earlier that I didn’t mean that, when I said that things can be great despite not feeling great, and that “good feelings” are just one of the possible types of good things you can have in your life, and they shouldn’t be the only ones.
Many if not most people are Goodharting in most aspects of their lives. Why not this one?
I acknowledge your claim that you value feeling good over and above the things that cause you to feel good. I agree that many people implicitly endorse this claim about themselves. I think you and they are very likely mistaken about this preference, and that ceasing to optimize for it would improve your life significantly according to your other preferences.
was hoping you’d validate whether my “I thought PDV was saying” one way or another, above …
also, it seems like an important milestone if you guys actually sussed out where the actual disagreement is. and it seems like it isn’t what either of you previously thought it was. so i want that to be made clear.
Kaj wasn’t saying ‘a thing that couldn’t be right’. Kaj was describing a totally realistic thing to do. which is to value feeling good itself.
i think conversational milestones in arguments are important places to stop and orient, and i was worried this milestone would be quickly passed over.
and NOW the disagreement is about a preference / why aren’t you worried about Goodharting, whereas before it wasn’t clear. is this actually agreed now by both parties?
(I greatly appreciate your attempt to clarify/improve the quality of the conversation.)
FWIW, I think ‘valuing positive feelings in themselves’ is a bad idea. It’s theoretically possible to do it, but I wouldn’t recommend it as part of one’s final evolutionary form.
Symmetrically, I think ‘equating negative feelings with badness’ or believing ‘feeling bad is bad’ is also not recommended.
People who don’t have things that feel great in their life are likely to be depressed. Do you think that’s a desireable state to be in?
I have both hypnosis and Circling skills. I understand hypnosis to be about having a specific behavior change that you want as outcome.
When doing Circling your focus is not on specific behavior change. It’s more like meditation about being in a certain state together.
I consider Circling to have been very useful at getting some models of other people.
After doing Circling at the LessWrong community weekend, one person said that they discovered in both Circlings in which they are that something they thought was unique about their own experience was also shared by other people.
Many models aren’t shared when you ramp up the critical nature of an exchange. Circling is about sharing of models about how people relate to their own experience that’s valuable both for understanding oneselves and understanding other people. It’s not about changing the models that are there even when they can change when you give them explicit attention.
As far as the state goes, after a hypnotic trance state it makes sense to say “Now come back” to get people to be “awake” again. That’s not the state in which Circling goes.
I frequently have meditations seen to go into that space, but when meditating or guiding a meditation I usually intent to lead to a more present state as well.
Back when I was first getting into hypnosis, we talked about my experiments with hypnosis and all the terrifying possibilities that they implied. Even though I’d expect you’d have taken basically the same stance even without those conversations, I imagine it is still a significant contributing factor towards your take on hypnosis, and so I feel compelled to note that I no longer feel this way about it.
To be clear, I don’t think anything we talked about is “wrong”, and the fact that the uncertainty mostly resolved on the “less scary” side isn’t very reassuring. I still can’t think of any circumstance with any hypnotist that I would allow them to “hypnotize” me, in the central meaning of the word, and I do still think people are insufficiently afraid of being hypnotized. That stuff is all more or less the same.
The big difference is that now recognize more of how “responding hypnotically” is a really important part of both learning and relating to people, and that it’s possible to do it without risking falling into any of the obvious traps that enable the scary bad possibilities. “Engage critical faculties, keep in mind evidence, develop models, etc” yes. Do that and “Listen to the voice. Respond and receive. Be open to the update, etc”—to the extent that you can do that without losing track of the former (and work to increase this extent as much as you can).
I don’t even think it’s always crazy to trade off some control for quicker learning, so long as this decision itself is made very carefully with full input of critical faculties, you understand the potential traps, and the person guiding you really can be verified to be worthy of the required trust, etc.
However it’s not necessary either. I’ve gotten better at it myself without sacrificing my need for control, and I have a very “control freaky” friend who is also figuring out how to respond hypnotically without giving up any control, and has gotten some really cool results from it. It’s taken her four years to be able to accept half the suggestions a good hypnotic subject can do in five minutes, but on the upside since she is deciding for herself which things to accept hypnotically, not only does she not expose herself to unnecessary risk, she’s able to more efficiently spot what would be useful to her in a normal conversation without anyone having to lean on it as if it were an actual hypnotic suggestion.
I guess it’s kinda like exploring caves that have a lot of goodies. Just make sure you know your way out.