Tragedy of the commons, the shared resource being mutual trust. The first one to defect reaps the rewards of his faux signals being taken at face value (“I don’t mind at all sticking around”, wow, such pleasantness, many social laurels, wow), degrading the network of trust a “tell culture” relies upon.
It’s like saying “wouldn’t we as a society benefit overall if hidden negative externalities were internalized”, yea well, first one to secretly pollute the river gets some bonus shares next quarter (wow, such money, many boni, wow)! Same with a trust culture ending in a race to the bottom.
I’m not suggesting all of society is ready for this. I’m suggesting we work toward it among highly rational peers and allies. This is how, and much of why, my close social circles work. Now that I’m used to it, I’d have it no other way.
I wouldn’t say “among rational peers” so much as “among EA-oriented peers”. For our specific community, there is significant overlap in the Venn diagram depicting those two qualities, but those two are very much distinct qualities nonetheless.
A community of HPMOR!Quirrell variations would have your very post in main, with plenty of upvotes, all the while secretly whetting their blades. Perfectly rational.
The more established the trust culture, the more vulnerable it would be to a traitor, a cunning red-pill bastard who plays the trust-network like a fiddle to the tune of his/her egotistical agenda.
Trust—the quintessential element of your so-called “tell culture”—and vulnerability are two sides of the same coin.
When the social circle is small enough as to resemble an expanded family unit, a clan, it may work. A strong sense of ties that bind to keep the commitment to honesty honest would tend to keep a “tell culture”′ social circle’s cardinality well below Dunbar’s number.
Trust—the quintessential element of your so-called “tell culture”—and vulnerability are two sides of the same coin.
That’s true in general. In network security circles, a trusted party is one with the explicit ability to compromise you, and that’s really the operational meaning of the term in any context.
A community of HPMOR!Quirrell variations would have your very post in main, with plenty of upvotes, all the while secretly whetting their blades. Perfectly rational.
I really don’t think so. A community of Briennes, which is not a community of HPMOR!Quirrells but shares some relevant features, would recognize the overwhelming benefit of coordination. Any given individual would be much stronger if she had the knowledge of all the other individuals, or if she could count on them as external memory. And because she would be stronger that way, she knows that they would be stronger if she also remains trustworthy. Her being trustworthy allows her to derive greater benefit from the rest of the community. Other people are useful, you see. With Tell culture in place, you can do things like feed your model of the world into someone else’s truth-checker and get back a more info-rich version. You only defect if the expected utility of doing so outweighs the expected utility of the entire community to your future plans.
I’d love to hear what culture Eliezer thinks an entire community of Quirrells would create.
If they all started off in a symmetrical position, they’d use Unbreakable Vows to keep from killing each other and then proceed to further affairs, not necessarily cooperatively.
Wouldn’t this require one Quirrell to agree to sacrifice a part of his power before any other Quirrell does? (Assuming that all of the vow rituals taking place at the same time would require each Quirrell to take part in more than one ritual simultaneously, which doesn’t seem possible.) It seems to me that a Quirrell wouldn’t agree to this.
You don’t have to sacrifice your own power for that, the bonder sacrifices power. And the Unbreakable Vow could be worded to only come into force once all Vows were taken.
But, in this case, the bonder is another Quirrell picked from this all-Quirrel community, right?
Of course, if we allow the ritual to depend on the completion of other rituals, then the problem is moot.
It strikes me that this conversation really hinges on just how evil HPMOR’s Quirrell turns out to be, which is problematic since you know a few chapters more plot than I do...
(Also, since I find myself having a conversation with you, might I say that I very much like HPMOR, and that I would like it even more if you were to amend chapter 19 so that Quirrell didn’t perpetuate one or two myths about martial arts, a subject on which I focus a certain amount of my own nerdly attentions? I posted a review under “James”, but the short version is that (1) martial arts aren’t particularly Asian, and (2) “I’m a sixth dan” means no more than “I once got a B- in a class whose subject I won’t divulge except to say that it was ‘Math’.”)
The great-grandparent comment did make me consider unbreakable vows as a theory of what happened on Halloween. E.g. to prevent one of his Horcruxes from later killing him, Voldemort made an unbreakable vow not to magically interact with his alter egos (this causing Harry’s sense of Doom around Quirrell). Doesn’t seem necessary, though.
A community of Briennes, [...], would recognize the overwhelming benefit of coordination.
But it would pay the price Tell comes with. And the Briennes wouldn’t need it because they know all their rules and could easily use the more efficient Guess.
“You only defect if the expected utility of doing so outweighs the expected utility of the entire community to your future plans.” These aren’t the two options available, though: you’d take into account the risk of other people defecting and thus reducing the expected utility of the entire community by an appreciable amount. Your argument only works if you can trust everyone else not to defect, too—in a homogenous community of Briennes, for instance. In a heterogenous community, whatever spooky coordination your clones would use won’t work, and cooperation is a much less desirable option.
Signaling pleasantness is sometimes near to signalling low status. In some situations it will give you benefits in others it might not.
One the web you find plenty of material that recommends that you will get more social success by being more confident. One way to be confident is to go the ask road instead of the guess road.
I have read a lot of self help and didn’t come across one that substantially focuses on acting in a guess culture way.
Is the Prisoners’ Dilemma really the right metaphor here? I don’t really get what the defector gains. Sure, I like them better for being so accommodating, but meanwhile they’re paying the costs of giving me what I want, and if they try to invoke some kind of quid pro quo than all the positive feelings go out the window when I find out they were misleading me.
Think of it as having an additional tool in your shed, a really important one: it confers unto you an additional degree of freedom: You can manipulate someone else’s state of mind by signalling various faux states of mind of your own (no longer are social signals a tell-culture mandated 1-to-1 mapping, but you can choose whatever input leads to the desired reaction). Social signals and the benefits they confer are sufficiently vague that often you won’t find out they were misleading you. Or you may find out (“The last years that person X worked for me I always thought she looked up to and admired me, turns out she always just pretended so she could keep the job!”), but the defector already reaped the (transient in time) rewards. Nothing is forever, the traitor can milk you like a gullible cow (or a gullicalf, living in California) then leave, harm done.
Tragedy of the commons, the shared resource being mutual trust. The first one to defect reaps the rewards of his faux signals being taken at face value (“I don’t mind at all sticking around”, wow, such pleasantness, many social laurels, wow), degrading the network of trust a “tell culture” relies upon.
It’s like saying “wouldn’t we as a society benefit overall if hidden negative externalities were internalized”, yea well, first one to secretly pollute the river gets some bonus shares next quarter (wow, such money, many boni, wow)! Same with a trust culture ending in a race to the bottom.
I’m not suggesting all of society is ready for this. I’m suggesting we work toward it among highly rational peers and allies. This is how, and much of why, my close social circles work. Now that I’m used to it, I’d have it no other way.
Tricky (like most anything).
I wouldn’t say “among rational peers” so much as “among EA-oriented peers”. For our specific community, there is significant overlap in the Venn diagram depicting those two qualities, but those two are very much distinct qualities nonetheless.
A community of HPMOR!Quirrell variations would have your very post in main, with plenty of upvotes, all the while secretly whetting their blades. Perfectly rational.
The more established the trust culture, the more vulnerable it would be to a traitor, a cunning red-pill bastard who plays the trust-network like a fiddle to the tune of his/her egotistical agenda.
Trust—the quintessential element of your so-called “tell culture”—and vulnerability are two sides of the same coin.
When the social circle is small enough as to resemble an expanded family unit, a clan, it may work. A strong sense of ties that bind to keep the commitment to honesty honest would tend to keep a “tell culture”′ social circle’s cardinality well below Dunbar’s number.
That’s true in general. In network security circles, a trusted party is one with the explicit ability to compromise you, and that’s really the operational meaning of the term in any context.
I really don’t think so. A community of Briennes, which is not a community of HPMOR!Quirrells but shares some relevant features, would recognize the overwhelming benefit of coordination. Any given individual would be much stronger if she had the knowledge of all the other individuals, or if she could count on them as external memory. And because she would be stronger that way, she knows that they would be stronger if she also remains trustworthy. Her being trustworthy allows her to derive greater benefit from the rest of the community. Other people are useful, you see. With Tell culture in place, you can do things like feed your model of the world into someone else’s truth-checker and get back a more info-rich version. You only defect if the expected utility of doing so outweighs the expected utility of the entire community to your future plans.
I’d love to hear what culture Eliezer thinks an entire community of Quirrells would create.
If they all started off in a symmetrical position, they’d use Unbreakable Vows to keep from killing each other and then proceed to further affairs, not necessarily cooperatively.
Wouldn’t this require one Quirrell to agree to sacrifice a part of his power before any other Quirrell does? (Assuming that all of the vow rituals taking place at the same time would require each Quirrell to take part in more than one ritual simultaneously, which doesn’t seem possible.) It seems to me that a Quirrell wouldn’t agree to this.
You don’t have to sacrifice your own power for that, the bonder sacrifices power. And the Unbreakable Vow could be worded to only come into force once all Vows were taken.
But, in this case, the bonder is another Quirrell picked from this all-Quirrel community, right?
Of course, if we allow the ritual to depend on the completion of other rituals, then the problem is moot.
It strikes me that this conversation really hinges on just how evil HPMOR’s Quirrell turns out to be, which is problematic since you know a few chapters more plot than I do...
(Also, since I find myself having a conversation with you, might I say that I very much like HPMOR, and that I would like it even more if you were to amend chapter 19 so that Quirrell didn’t perpetuate one or two myths about martial arts, a subject on which I focus a certain amount of my own nerdly attentions? I posted a review under “James”, but the short version is that (1) martial arts aren’t particularly Asian, and (2) “I’m a sixth dan” means no more than “I once got a B- in a class whose subject I won’t divulge except to say that it was ‘Math’.”)
The great-grandparent comment did make me consider unbreakable vows as a theory of what happened on Halloween. E.g. to prevent one of his Horcruxes from later killing him, Voldemort made an unbreakable vow not to magically interact with his alter egos (this causing Harry’s sense of Doom around Quirrell). Doesn’t seem necessary, though.
But it would pay the price Tell comes with. And the Briennes wouldn’t need it because they know all their rules and could easily use the more efficient Guess.
“You only defect if the expected utility of doing so outweighs the expected utility of the entire community to your future plans.” These aren’t the two options available, though: you’d take into account the risk of other people defecting and thus reducing the expected utility of the entire community by an appreciable amount. Your argument only works if you can trust everyone else not to defect, too—in a homogenous community of Briennes, for instance. In a heterogenous community, whatever spooky coordination your clones would use won’t work, and cooperation is a much less desirable option.
Signaling pleasantness is sometimes near to signalling low status. In some situations it will give you benefits in others it might not.
One the web you find plenty of material that recommends that you will get more social success by being more confident. One way to be confident is to go the ask road instead of the guess road.
I have read a lot of self help and didn’t come across one that substantially focuses on acting in a guess culture way.
Quite so. Which is why constraining yourself to honesty precludes you from always choosing the personally beneficial path.
Is the Prisoners’ Dilemma really the right metaphor here? I don’t really get what the defector gains. Sure, I like them better for being so accommodating, but meanwhile they’re paying the costs of giving me what I want, and if they try to invoke some kind of quid pro quo than all the positive feelings go out the window when I find out they were misleading me.
Think of it as having an additional tool in your shed, a really important one: it confers unto you an additional degree of freedom: You can manipulate someone else’s state of mind by signalling various faux states of mind of your own (no longer are social signals a tell-culture mandated 1-to-1 mapping, but you can choose whatever input leads to the desired reaction). Social signals and the benefits they confer are sufficiently vague that often you won’t find out they were misleading you. Or you may find out (“The last years that person X worked for me I always thought she looked up to and admired me, turns out she always just pretended so she could keep the job!”), but the defector already reaped the (transient in time) rewards. Nothing is forever, the traitor can milk you like a gullible cow (or a gullicalf, living in California) then leave, harm done.