Kaj_Sotala

You can val­idly be seen and val­i­dated by a chatbot

• Kaj_Sotala 19 Dec 2024 12:44 UTC

  7 points

  0

  in reply to: Viliam’s comment on: Don’t As­so­ci­ate AI Safety With Activism

  I feel like activists are generally seen like this when one disagrees with their cause, and seen as brave people doing an important thing when one agrees with their cause. If one doesn’t have an opinion, could go either way, depending on how much they seem to violate generally-accepted norms and how strongly the person in question feels about those norms.

• Kaj_Sotala 19 Dec 2024 10:40 UTC

  2 points

  0

  in reply to: Kaj_Sotala’s comment on: Try­ing to trans­late when peo­ple talk past each other

  Okay, going through the messages in detail, the best account of what I can reconstruct of what actually happened is:

  • The mechanics in this particular game involved 1) a choice of what kind of an action to play 2) once the action had been chosen, a choice of where exactly to play it. Person A had previously agreed to make certain plays.

  • For one of the plays (call this “action 1”), communication had been ambiguous. A had ended up thinking that we’d agreed on the action to play but left the choice of where to play it up to him, whereas person B had ended up with the belief that we’d decided both on the action and the location.

  • We had also agreed on A doing another thing, call it “action 2a”.

  • When the time came, A noticed that if he played action 1 in a particular location, he could do another thing (“action 2b”) that would still lead to the same outcome that 2a would have led to.

  • Person A now said that it didn’t matter whether he played action 2a or action 2b, since by that point either one would lead to the same outcome.

  • However person B objected that A’s claim of “both 2a and 2b lead to the same result” was only true given that A had already decided to play that action in a different location than had already been decided, while B held that the choice of where to play it was part of what needed to be decided together.

  And the specific thing that B found triggering was that (in her view) A didn’t even acknowledge that he was deviating from something that had already been agreed upon (the choice of where to play action 1), and instead that gave (what seemed to B like an) excuse for why it was okay to unilaterally change action 2.

  That seems complex enough that I’m not sure how to rewrite the post to take it into account while also keeping it clear.

  What links here?

  • Try­ing to trans­late when peo­ple talk past each other by Kaj_Sotala (17 Dec 2024 9:40 UTC; 41 points)

• Kaj_Sotala 18 Dec 2024 8:13 UTC

  2 points

  0

  in reply to: Ben Pace’s comment on: Habryka’s Short­form Feed

  I don’t think alternative stories have negligible probability

  Okay! Good clarification.

  I think it’s good to discuss norms about how appropriate it is to bring up cynical hypotheses about someone during a discussion in which they’re present.

  To clarify, my comment wasn’t specific to the case where the person is present. There are obvious reasons why the consideration should get extra weight when the person is present, but there’s also a reason to give it extra weight if none of the people discussed are present—namely that they won’t be able to correct any incorrect claims if they’re not around.

  so I think it went fine

  Agree.

  (As I mentioned in the original comment, the point I made was not specific to the details of this case, but noted as a general policy. But yes, in this specific case it went fine.)

• Kaj_Sotala 18 Dec 2024 5:54 UTC

  8 points

  0

  on: Be­ing Pre­sent is Not a Skill

  I think it’s both true what you say, that removing blocks can give you instant improvements that no amount of practice ever would, and also that one can make progress with practice in the right conditions.

• Kaj_Sotala 18 Dec 2024 5:22 UTC

  4 points

  0

  on: Try­ing to trans­late when peo­ple talk past each other

  Oh. This discussion got me to go back and review some messages written in the aftermath of this, when I was trying to explain things to A… and I noticed a key thing I’d misremembered. (I should have reviewed those messages before posting this, but I thought that they only contained the same things that I already covered here.)

  It wasn’t that A was making a different play that was getting the game into a better state; it was that he was doing a slightly different sequence of moves that nevertheless brought the game into exactly the same state as the originally agreed upon moves would have. That was what the “it doesn’t matter” was referring to.

  Well that explains much better why this felt so confusing for the rest of us. I’ll rewrite this to make it more accurate shortly. Thanks for the comments on this version for making me look that up!

• Kaj_Sotala 18 Dec 2024 4:25 UTC

  4 points

  0

  in reply to: deepthoughtlife’s comment on: Try­ing to trans­late when peo­ple talk past each other

  but there doesn’t have to be any past betrayal to object to betrayal in the present; people don’t need to have ever been betrayed in the past to be against it as a matter of principle.

  True, but that is assuming that everyone was perceiving this as a betrayal. A relevant question is also, what made A experience this as a betrayal, when there were four people present and none of the other three did? (It wasn’t even B’s own plan that was being affected by the changed move, it was my plan—but I was totally fine with that, and certainly didn’t experience that as a betrayal.)

  Betrayal usually means “violating an agreement in a way that hurts one person so that another person can benefit”—it doesn’t usually mean “doing something differently than agreed in order to get a result that’s better for everyone involved”. In fact, there are plenty of situations where I would prefer someone to not do something that we agreed upon, if the circumstances suddenly change or there is new information that we weren’t aware of before.

  Suppose that I’m a vegetarian and strongly opposed to buying meat. I ask my friend to bring me a particular food from the store, mistakenly thinking it’s vegetarian. At the store, my friend realizes that the food contains meat and that I would be unhappy if they followed my earlier request. They bring me something else, despite having previously agreed to bring the food that I requested. I do not perceive this as a betrayal, I perceive this as following my wishes. While my friend may not be following our literal agreement, they are following my actual goals that gave rise to that agreement, and that’s the most important thing.

  In the board game, three of us (A, me, and a fourth person who I haven’t mentioned) were perceiving the situation in those terms: that yes, A was doing something differently than we’d agreed originally. But that was because he had noticed something that actually got the game into a better state, and “getting the game into as good of a state as possible” was the purpose of the agreement.

  Besides, once B objected, A was entirely willing to go back to the original plan. Someone saying “I’m going to do things differently” but then agreeing to do things the way that were originally agreed upon as soon as the other person objects isn’t usually what people mean by betrayal, either.

  And yet B was experiencing this as a betrayal. Why was that?

  I would strongly caution against assuming mindreading is correct.

  I definitely agree! At the same time, I don’t think one should take this far as never having hypotheses about the behavior of other people. If a person is acting differently than everyone else in the situation is, and thing X about them would explain that difference, then it seem irrational not to at least consider that hypothesis.

  But of course one shouldn’t just assume themselves to be correct without checking. Which I did do, by (tentatively) suggesting that hypothesis out loud and letting B confirm or disconfirm it. And it seemed to me that this was actually a good thing, in that a significant chunk of B’s experience of being understood came from me having correctly intuited that. Afterward she explicitly and profusely thanked me for having spoken up and figured it out.

• Kaj_Sotala 18 Dec 2024 3:23 UTC

  2 points

  0

  in reply to: Said Achmiz’s comment on: Try­ing to trans­late when peo­ple talk past each other

  Also, as I mentioned, this is a slightly fictionalized account that I wrote based on my recollection of the essence of what happened. But the exact details of what was actually said were messier than this, and the logic of exactly what was going on didn’t seem as clear as it does in this narrative. Regenerating the events based on my memory of the essence of the issue makes things seem clearer than they actually were, because that generator doesn’t contain any of the details that made the essence of the issue harder to see at the time.

  So if this conversation had actually taken place literally as I described it, then the hypothesis that you object to would have been more redundant. In the actual conversation that happened, things were less clear, and quite possibly the core of the issue may actually have been slightly different from what seems to make sense to me in retrospect when I try to recall it.

• Kaj_Sotala 18 Dec 2024 2:25 UTC

  2 points

  0

  in reply to: Said Achmiz’s comment on: Try­ing to trans­late when peo­ple talk past each other

  My read was that one might certainly just object to the thing on those grounds alone, but that the intensity of B’s objection was such that it seemed unlikely without some painful experience behind it. B also seemed to become especially agitated by some phrases (“it doesn’t matter”) in particular, in a way that looked to me like she was being reminded of some earlier experience where similar words had been used.

  And then when I tried to explain things to A and suggested that there was about something like that going on, B confirmed this.

• Kaj_Sotala 17 Dec 2024 20:02 UTC

  2 points

  0

  in reply to: habryka’s comment on: Habryka’s Short­form Feed

  (I read

  I think many well-intentioned people will say something like this, and that this is probably because of two reasons

  as implying that the list of reasons is considered to exhaustive, such that any reasons besides those two have negligible probability.)

• Kaj_Sotala 17 Dec 2024 17:06 UTC

  2 points

  0

  in reply to: xpym’s comment on: Cir­cling as prac­tice for “just be your­self”

  The truth of that literal statement depends on exactly how much trust someone would need in somebody else before having sex with them—e.g. to my knowledge, studies tend to find that most single men but very few if any women would be willing to have sex with a total stranger. Though I’ve certainly also known women who have had a relatively low bar of getting into bed with someone, even if they wouldn’t quite do it with a total stranger.

  But more relevantly, even if that statement was correct, I don’t think it’d be a particularly good analogy to Circling. It seems to involve the “obligatory openness” fallacy that I mentioned before. I’m not sure why some people with Circling experience seemed to endorse it, but I’m guessing it has to do with some Circling groups being more into intimacy than others. (At the time of that discussion, I had only Circled once or twice, so probably didn’t feel like I had enough experience to dispute claims by more experienced people.)

  My own experience with Circling is that it’s more like meeting a stranger for coffee. If both (all) of you feel like you want to take it all the way to having sex, you certainly can. But if you want to keep it to relatively shallow and guarded conversation because you don’t feel like you trust the other person enough for anything else, you can do that too. Or you can go back and forth in the level of intimacy, depending on how the conversation feels to you and what topics it touches on. In my experience of Circling, I definitely wouldn’t say that it feeling anywhere near as intimate as sex would be the norm.

  You can also build up that trust over time. I think Circling is best when done with people who you already have some pre-existing reason to trust, or in a long-term group where you can get to know the people involved. That way, even if you start at a relatively shallow level, you can go deeper over time if (and only if) that feels right.

Try­ing to trans­late when peo­ple talk past each other

• Kaj_Sotala 17 Dec 2024 7:23 UTC

  7 points

  0

  in reply to: hamiltonianurst’s comment on: Cir­cling as prac­tice for “just be your­self”

  I don’t know the details. The official explanation is this:

  When individuals with little training attempt to facilitate Circling, or teach/​train others their arbitrarily altered versions and still call it Circling, then consumers and students – at best – receive a sub-standard experience and the reputation of Circling suffers greatly, along with its impact in the world.

  Between the three schools there are hundreds of accounts of:

  • People taking one or two 3-hour workshops, or merely experiencing Circling at a drop in event or festival, and then advertising that they are leading their own Circling workshops

  • People coming to a few drop in events & turning around and offer “Circling” to large corporations for corporate culture training.

  • People claiming they were emotionally abused by facilitators at an event that advertised itself as “Circling” but had no ties to any of the 3 Certified Circling Schools

  In order to protect the public consumer and the legacy of Circling, we need to use the term “Circling” consistently and limit the use of the term to those who are actually using and teaching the authentic communication and relating tools taught by the Certified Circling Schools.

  … but then I also heard it claimed that Circling Europe, previously one of the main Circling schools in very good standing, ended up not having a permission to use the trademark because the licensing fees for it would have been so exorbitant that CE found it better to use a different name than to pay them. So maybe it was more of a cash grab? (Or just a combination of several different motives.)

Cir­cling as prac­tice for “just be your­self”

• Kaj_Sotala 14 Dec 2024 13:03 UTC

  3 points

  0

  in reply to: AnthonyC’s comment on: Just one more ex­po­sure bro

  What’s the long version of the professional’s standard advice?

• Kaj_Sotala 10 Dec 2024 19:50 UTC

  6 points

  6

  in reply to: KvmanThinking’s comment on: [Fic­tion] Lena (MMAcevedo)

  Historically there were plenty of rationalizations for slavery, including ones holding that slaves weren’t really people and were on par with animals. Such an argument would be much easier for a mind running on a computer and with no physical body—“oh it just copies the appearance of suffering but it doesn’t really suffer”.

• Kaj_Sotala 10 Dec 2024 12:02 UTC

  11 points

  9

  in reply to: Ben Pace’s comment on: Habryka’s Short­form Feed

  I think many people have learned to believe the reasoning step “If people believe bad things about my team I think are mistaken with the information I’ve given them, then I am responsible for not misinforming people, so I should take the information away, because it is irresponsible to cause people to have false beliefs”. I think many well-intentioned people will say something like this, and that this is probably because of two reasons (borrowing from The Gervais Principle):

  (Comment not specific to the particulars of this issue but noted as a general policy:) I think that as a general rule, if you are hypothesizing reasons for why somebody might say a thing, you should always also include the hypothesis that “people say a thing because they actually believe in it”. This is especially so if you are hypothesizing bad reasons for why people might say it.

  It’s very annoying when someone hypothesizes various psychological reasons for your behavior and beliefs but never even considers as a possibility the idea that maybe you might have good reasons to believe in it. Compare e.g. “rationalists seem to believe that superintelligence is imminent; I think this is probably because that lets them avoid taking responsibility about their current problems if AI will make those irrelevant anyway, or possibly because they come from religious backgrounds and can’t get over their subconscious longing for a god-like figure”.

• Kaj_Sotala 10 Dec 2024 11:34 UTC

  LW: 9 AF: 3

  1

  AF

  on: o1: A Tech­ni­cal Primer

  We can also learn something about how o1 was trained from the capabilities it exhibits. Any proposed training procedure must be compatible with the following capabilities:

  1. Error Correction: “[o1] learns to recognize and correct its mistakes.”

  2. Factoring: “[o1] learns to break down tricky steps into simpler ones.”

  3. Backtracking: “[o1] learns to try a different approach when the current one isn’t working.”

  I would be cautious of drawing particularly strong conclusions from isolated sentences in an announcement post. The purpose of the post is marketing, not technical accuracy. It wouldn’t be unusual for engineers at a company to object to technical inaccuracies in marketing material and have their complaints ignored.

  There probably aren’t going to be any blatant lies in the post, but something like “It’d sound cool if we said that the system learns to recognize and correct its mistakes, would there be a way of interpreting the results like that if you squinted the right way? You’re saying that in principle yes, but yes in a way that would also apply to every LLM since GPT-2? Good enough, let’s throw that in” seems very plausible.

• Kaj_Sotala 10 Dec 2024 8:20 UTC

  8 points

  0

  in reply to: KvmanThinking’s comment on: [Fic­tion] Lena (MMAcevedo)

  Compare to e.g. factory farming today, which also persists despite a lot of people thinking it not okay (while others don’t care).

• Kaj_Sotala 6 Dec 2024 19:38 UTC

  LW: 8 AF: 2

  4

  AF

  in reply to: Marius Hobbhahn’s comment on: Fron­tier Models are Ca­pable of In-con­text Scheming

  I didn’t say that roleplaying-derived scheming would be less concerning, to be clear. Quite the opposite, since that means that there now two independent sources of scheming rather than just one. (Also, what Mikita said.)